API Ratings: Who won the week? | March 16-23, 2020

We have now passed the spring equinox (in the Northern hemisphere anyway) (and sadly are still in the Age of COVID-19 everywhere) and once more that time of the week when we talk of the State of the APIs has heaved into view.

We recently launched API.expert as a simple way to provide everybody with insights into the API economy and the leading APIs in different categories in particular. The basic service is free and will remain free, but other, related services will be built on top of it, including your very own API.expert dashboards if you want them! 

Each week, month and year we’ll look at the APIs we track in a variety of sectors for their quality (using our patent pending CASC score) and other factors like uptime and availability. 

If something is missing that you think should be here, don’t hesitate to drop us a line! 

Now, onto what the last seven days and last month can tells us about where the world of APIs is in early 2020.

API Performance Headlines 

We look at over 200 APIs and pull all the metrics together to give you a general feeling for the service quality for an organization’s APIs in a particular category.  

We do see that certain providers consistently vie for top spot in their category including Google, LATOKEN and GitHub with others doing less well such as Cisco’s Spark services.

Top Performers based on overall quality in each category were: 

Two changes this week, with ABN AMRO swapping places with Railsbank at the top of PSD2 Banks and Police.UK replacing GOV.UK in UK Government. The Department of Justice once again takes the overall title in the public sector.

A CASC score of over 900 is very good and one of 950 or more exceptional. Six of the eight categories are headed by organization with a CASC score of 950 or more this week, which is a good showing. Sustaining a CASC score of >925 over a period of several weeks is an excellent effort and congratulations to those organizations that have achieved it.

No changes at all yet again this week, which shows that APIs are more consistent in latency than they are in overall performance.

An important caveat: medians can be misleading!

An API might have a fast median latency but produce many slow outliers. These won’t affect the median, but they mean that users can experience many calls that were unacceptably slow. So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score!

As so often, FTX might have a low enough latency to top the Cryptocurrency Exchange category, but they also have a lot of outliers and a high variance on their latency, which is why they come out 8th out of 18 in terms of the CASC score this week, up from 11^th last week. They do though have a CASC score of 874, which is a reasonable CASC score. It’s no good being fast if you are flakey, although being fast does help.

Two changes this week with the Bureau of Labor Statistics swapping places with the Department of Commerce in US Government and Tumblr a new entrant in Social Networks with a CASC score of 772 putting it in the Amer Zone. We also have APIs with CASC scores of less than 900.

Oasis Dex takes the overall WORST for the third week in a row and has somehow managed to “top” last week’s appallingly CASC score of 241 with one of 187. They have a pass rate of 69.30%, whilst 16 of their 17 peers manage 100% (Coinpaprika at No. 2 has a pass rate of 99.85%). They also 34.28% outliers (next worse CoinMarketCap at No. 16 with 6.98%; LATOKEN at No. 1 has 0.0%).

It is hard to believe that Oasis Dex’s performance hasn’t been having a serious impact on its users over the last few weeks.

Something of interest

The perennial under-performer in Corporate Infrastructure is Cisco Spark. With us all being encouraged to work from home as much as possible in this period of pandemic, conferencing/collaboration services like Cisco Spark are increasingly critical to the smooth functioning of the global economy.

So just where is that Cisco Spark is falling down?  

It’s pretty slow with a median latency of 1106 ms. This is the slowest in Corporate Infrastructure. It also has a lot of outliers (8.75%).  What we also see are some pretty big variations between cloud and region for DNS Lookup Time and Download Time.

Azure South America has a DNS Lookup Time of 509 ms, which indicates something pretty seriously out of kilter. That 125 ms for IBM Cloud East Asia could also do with being looked at.

The Download Times are also a cause for concern. 219 ms for Google in East Asia? 0 ms for AWS in Oceania IBM Cloud in Oceania compared to 2 ms for Azure and 334 ms for IBM Cloud and 338 ms for Google.

We see these types of variation in every region expect Europe. This suggests that Cisco Spark is hosted in every region except Europe, but there are huge differences in download time for calls from agents in at cloud locations that are not physically proximate to the location at which the service is hosted. It is often the case the data centers for different locations might be next to one another on the same technology park (or even in the same building).

If clients are consuming Cisco Spark services via apps that are hosted at the “wrong” cloud locations, it is possible that the service they are receiving is significantly impaired compared to users using apps hosted at the “right” locations.

See you again in a week to look at the State of the APIs as we stagger towards the end of March.