We are now well into March – and that time of the week when it is our duty to discourse on the State of the APIs.
We recently launched API.expert as a simple way to provide everybody with insights into the API economy and the leading APIs in different categories in particular. The basic service is free and will remain free. But related services will be built on top of it, including your very own API.expert dashboards!
Each week, month and year we look at the APIs we track in a variety of sectors for their quality (using our patent pending CASC score) and other factors like uptime and availability.
If something is missing that you think should be here, don’t hesitate to drop us a line!
Now, onto what the past week tells us about the state of the world of APIs in the first full week of March 2020.
API Performance Headlines
We look at over 200 APIs and pull all the metrics together to give you a general feeling for the service quality for an organization’s APIs in a particular category.
We do see that certain providers consistently vie for top spot in their category including Twitter, Slack and Github with others doing less well such as Cisco’s Spark services.
Top API performers based on overall quality:
Week ending March 9, 2020
| Category | Organization | CASC score |
|---|---|---|
| Corporate Infrastructure | GitHub | 968 |
| Cryptocurrency Exchanges | LATOKEN | 926 |
| PSD2 Banks | ABN AMRO |
971 |
| Search | 944 | |
| Social Networks | Google (including YouTube) |
960 |
| UK Government | GOV.UK | 941 |
| UK Open Banking (Open Data) | Bank of Ireland | 972 |
| US Government | Department of Justice | 977 |
Just one change this week with Google replacing Twitter at the top of Social Networks. The Department of Justice once again takes the overall title keeping the it in the public sector. A CASC score of over 900 is very good and one of 950 or more exceptional. Five of the eight categories are headed by organization with a CASC score of 950 or more this week, which is a good showing. Sustaining a CASC score of >925 over a period of several weeks is a good showing and congratulations to those organizations that have achieved it.
Top API performers by latency
Week Ending March 2, 2020
| Category | Organization | Median Latency |
|---|---|---|
| Corporate Infrastructure | Slack | 176 ms |
| Cryptocurrency Exchanges | FTX | 233 ms |
| PSD2 Banks | Open Bank Project |
208 ms |
| Search | 391 ms | |
| Social Networks | Google |
121 ms |
| UK Government | Police.UK | 75 ms |
| UK Open Banking (Open Data) | HSBC | 86 ms |
| US Government | Department of Justice | 73 ms |
No changes at all again this week, which shows that APIs are more consistent in latency than they are in overall performance. An important caveat: medians can be misleading! An API might have a fast median latency but produce many slow outliers. These won’t affect the median, but they mean that users can experience many calls that were unacceptably slow. So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score! As so often, FTX might have a low enough latency to top the Cryptocurrency Exchange category, but they also have a lot of outliers and a high variance on their latency, which is why they come out 9th out of 18 in terms of the CASC score this week, up from 15th last week. They do have a CASC score of 845, which is a reasonable CASC score. It’s no good being fast if you are flakey, although being fast does help.
Top API performers by latency
Week Ending March 9, 2020
| Category | Organization | Median Latency |
|---|---|---|
| Corporate Infrastructure | Slack | 176 ms |
| Cryptocurrency Exchanges | FTX | 234 ms |
| PSD2 Banks | Open Bank Project |
202 ms |
| Search | 399 ms | |
| Social Networks | Google |
120 ms |
| UK Government | Police.UK | 76 ms |
| UK Open Banking (Open Data) | HSBC | 84 ms |
| US Government | Department of Justice | 68 ms |
Just one change this week, with ABN AMRO replacing BNP Paribas.
Worst API quality across all categories
Week Ending March 2, 2020
| Category | Organization | CASC score |
|---|---|---|
| Corporate Infrastructure | Cisco Spark | 663 |
| Cryptocurrency Exchanges | Oasis Dex | 289 |
| PSD2 Banks | Visa |
607 |
| Search | Nobody below 900! | |
| Social Networks | Nobody below 800! |
|
| UK Government | NHS | 773 |
| UK Open Banking (Open Data) | HSBC | 433 |
| US Government | Bureau of Labor Statistics | 677 |
Two changes this week with HSBC dropping to the bottom of UK Open Banking (Open Data) replacing Halifax and the Bureau of Labor Statistics replacing the Department of Commerce. Oasis Dex takes the overall Wooden Spoon for the week from the NHS and is definitely in need of some attention with a truly terrible CASC score of just 289.
Something of interest
Oasis Dex is a perennial under-performer, but last week it really fell off the cliff. What went wrong?
The main problem are the pass rate of just 85.42% and a outlier rate of 15.03%. These are both indicative of very poor performance and are going to really tank the CASC score.
The median latency is not worst in class. CryptoCompare is much worse at 1021 ms, but places 9th out of 18 with a CASC score of 837. It’s the pass rate/outlier combo that is killing Oasis Dex here. But where are those outliers. They are in Process Time. The times from East Asia and Oceania are much slower than for Europe and North America.
But why should Process Time be affected by location?
We are clearly seeing a geographical lag here. All the download times are 0 ms, so the actual payload must be being returned in the Process Time. This suggests a design issue with the back-end architecture relating to how Oasis Dex are retuning the data. The Process Time should be the similar for different locations typically and not have such an obvious geographical time lag, whereas it might be expected for download time.
Fixing this issue with the Process Time might well be key to improving pass rate, reducing outliers and increasing the CASC score.
See you again in a week to see how things are going as the evenings get longer by the day.
