We now find ourselves already arrived in August – and that time to report on API ratings over the past seven days. We launched API.expert earlier in the year as a simple way to provide everybody with insights into the API economy and the leading APIs in different categories in particular.
The basic service is free and will remain free, but other, related services will be built on top of it, including your very own API.expert dashboards if you want them!
If something is missing in our API ratings that you think should be here, don’t hesitate to drop us a line!
Now, onto what the past week tells us about the state of API ratings as we dive into the swansong August 2020.
API Performance Headlines
We look at over 200 APIs, but pull all the metrics together to give you a general feeling for the service quality for an organization’s APIs in a particular category.
We do see that certain providers consistently vie for the top spot in their API ratings category, including GitHub and Google, with others doing less well, such as Halifax and the NHS.
Top Performers
API Ratings Week Ending 3 August 2020
| Category | Organization | CASC score |
| Corporate Infrastructure | Pivotal Tracker | 9.55 |
| Covid-19 | The COVID Tracking Project | 9.17 |
| Cryptocurrency Exchanges | Coinpaprika | 9.36 |
| PSD2 Banks | Nordea Bank | 9.87 |
| Search | 9.46 | |
| Social Networks | 9.70 | |
| UK Open Banking (Open Data) | Bank of Ireland | 9.78 |
| UK Government | GOV.UK | 9.79 |
| US Government | Department of Justice | 9.77 |
A fairly volatile week this week:
- Pivotal Tracker replacing GitHub in Corporate Infrastructure
- The COVID Tracking Project replacing Coronavirus COVID19 API (covid19.api.com) in Covid-19
- Bank of Ireland replacing Danske Bank
- GOV.UK replacing Police.UK in UK Government
- Department of Justice replacing DFEC in US Government
Nordea Bank retains the crown for the private sector with a fine CASC score of 9.87. A CASC score of over 9.00 is very good and one of 9.50 or more exceptional.
Six of the nine categories are headed by an organization with a CASC score of 9.50 or more this week up from five last week. Sustaining a CASC score of >9.25 over a period of several weeks is a good showing and congratulations to those organizations that achieved it.
Top Performers by latency
API Ratings Week Ending 3 August 2020
| Category | Organization | Median latency |
| Corporate Infrastructure | Microsoft Office | 228 ms |
| Covid-19 | CDC tools (Centers for Disease Control and Prevention) | 169 ms |
| Cryptocurrency Exchanges | FTX | 216 ms |
| PSD2 Banks | Nordea Bank | 169 ms |
| Search | 289 ms | |
| Social Networks | 59 ms | |
| UK Open Banking (Open Data) | HSBC | 79 ms |
| UK Government | Police.UK | 90 ms |
| US Government | Department of Justice | 69 ms |
Yet another stable week this week in this stable category. No changes again to the top performers. Google takes the overall title again with a median latency of 59 ms in Social Networks.
An important caveat: medians in API ratings can be misleading! An API might have a fast median latency but produce many slow outliers. These won’t affect the median, but they mean that users can experience many calls that were unacceptably slow.
So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score!
As so often, FTX tops the Cryptocurrency Exchange category and this week are up to eleventh out of 18 overall with a CASC score of 8.39, placing them in the Green Zone. But it is still worth remembering that it’s no good just being fast if you are flaky, although being fast does help!
Worst quality across all categories
API Ratings Week Ending 3 August 2020
| Category | Organization | CASC score |
| Corporate Infrastructure | SendGrid | 6.88 |
| Covid-19 | Coronavirus Data APIU (thevirustracker.com) | 7.19 |
| Cryptocurrency Exchanges | RadarRelay | 6.30 |
| PSD2 Banks | Open Bank Project | 7.44 |
| Search | Nobody below 8.00! | |
| Social Networks | Nobody below 8.00! | |
| UK Government | NHS | 7.02 |
| UK Open Banking (Open Data) | Halifax | 7.92 |
| US Government | Federal Communications Commission | 5.37 |
A few changes this week:
- SendGrid a new entrant in Corporate Infrastructure
- Coronavirus Data APIU (thevirustracker.com) replacing CDC tools (Centers for Disease Control and Prevention) in Covid-19
- Nobody below 8.00 in Social Networks,
- Halifax replacing Bank of Ireland in UK Open Banking (Open Data)
- NHS replacing GOV.UK in UK Government
Federal Communications Commission takes the overall big fat loser prize with a pretty bad CASC score of 5.37, placing it firmly in the Red Zone.
Something of interest
SendGrid had a bad week ending up with a CASC score of 6.88, putting it in the Amber Zone. Its pass rate was by far the worst in its category. What went wrong?
In this case, we know, like Bank of Ireland, that is was an outage. A big difference is that the Bank of Ireland open data APIs are probably not for most users mission-critical. That’s not true for SendGrid and Twitter lit up on 28 July with irate users who couldn’t access the service.
The outage began around midday UTC on 28 July. That’s four o’clock in the morning on the West Coast of the US, just about the kind of time that you might cut over to a new software build.
The outage lasted until 19:21 UTC. That’s a 7-hour outage. A lot of people at SendGrid had a bad day that day and quite possibly a lot of people who are SendGrid customers. Interestingly, we observed an eclectic mix of HTTP status code errors:
- 500 Internal Server Error
- 502 Bad Gateway
- 503 Service Unavailable
- 504 Gateway Timeout
We also saw some 401 Unauthorized warnings, which since the calls were authorized means that there were even more problems because the backend was returning an inappropriate status code.
To make things worse, we saw a number of fails on 29 July with no HTTP status code returned with the client-side message “Unable to connect to endpoint”. This is one reason why it is absolutely vital to actively monitor from endpoints from external locations.
Your users might not even be able to access your service because of network issues. Often they will be nothing to do with you but want to know about them so you don’t get the blame, but sometimes it can be because you have dropped off the internet for some reason.
So drop us a line to get the conversation started about how APImetrics can help you understand how your APIs are really behaving right now.
SendGrid knew about the outage straightaway because people were tweeting them. But you shouldn’t rely on your customers to tell you about outages. You should be actively monitoring your APIs from a variety of cloud locations and depending solely on internal monitoring or gateways logs.
That way, when your users tweet you to say that the service is down, you can “Yes, we know, it will be back up as soon as possible.” And know exactly how the outage evolves and we really useful in finding and fixing the bugs in your APIs and backend software including the one that caused that 401 Unauthorized.
See you again in a week as we look once more at the State of the APIs as we plunge deeper into August! And look out for our monthly State of the APIs report for July coming soon.
