API Ratings: Who won the week? | Feb 24 – Mar 4, 2020

We have entered March like a lion and point of the week when it is time to talk of the State of the APIs has rolled into view once more, not just over the last seven days, but the past month. 

We recently launched API.expert as a simple way to provide everybody with insights into the API economy and the leading APIs in different categories in particular. The basic service is free and will remain free, but other, related services will be built on top of it, including your very own API.expert dashboards if you want them! 

Each week, month and year we’ll look at the APIs we track in a variety of sectors for their quality (using our patent pending CASC score) and other factors like uptime and availability. 

If something is missing that you think should be here, don’t hesitate to drop us a line! 

Now, onto what the last seven days and last month can tells us about where the world of APIs is in early 2020.

API Performance Headlines 

We look at over 200 APIs and pull all the metrics together to give you a general feeling for the service quality for an organization’s APIs in a particular category.  

We do see that certain providers consistently vie for top spot in their category including Twitter, Slack and Github with others doing less well such as Cisco’s Spark services. 

Top Performers based on overall quality in each category were: 

A fair amount of movement over the last month with changes in five of the eight categories (GitHub replacing Slack, BNP Paribas Railsbank, Twitter Google (in Social Networks), GOV.UK Police.UK and the Department of Justice the General Service Administration. All of these APIs have maintained an excellent level of performance over the month, but the amount of turnover indicates that it is tough at the top. It will be interesting to see how many APIs can maintain their position in March. 

Top performers by latency 

Week Ending March 2 2020 

No changes at all again this week, which shows that APIs are more consistent in latency than they are in overall performance. 

An important caveat: medians can be misleading! An API might have a fast median latency but produce many slow outliers. These won’t affect the median, but they mean that users can experience many calls that were unacceptably slow.

So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score! As so often, FTX might have a low enough latency to top the Cryptocurrency Exchange category, but they also have a lot of outliers and a high variance on their latency, which is why they come out 9th out of 18 in terms of the CASC score this week, up from 15th last week. 

They do have a CASC score of 845, which is a reasonable CASC score. It’s no good being fast if you are flaky, although being fast does help. 

Top performers by latency 

Month of February 2020 

Two changes since January with Slack replacing GitHub and Google Foursquare. It is easier to be fast than to be good! FTX are first again for latency, but 11th out of 18 overall with a respectable CASC score of 812. 

Worst quality across all categories 

Two changes this week with Halifax dropping to the bottom of UK Open Banking (Open Data) replacing Santander UK and the Department of Commerce replacing the Federal Communications Commission. The perennial under-performer NHS retains the overall Wooden Spoon for the week, in the Red Zone again and definitely in need of some attention. 

Only one change from last month with Santander (UK) replacing Halifax at the bottom of UK Open Banking (Open Banking). NHS are bottom overall replacing Oasis Dex. It’s clear that there tends to less competition at the bottom. Good APIs – and there are a lot of them – tend to stay good whilst bad ones stay horrid. 

Something of interest 

The overall best performer in February was BNP Paribas with an excellent CASC score of 972. What made BNP Paribas so good?  It has a 100% pass rate, tops its category for fewest outliers (0.15%) and the fast median latency (136 ms). If you win at everything, you are going to win overall! 

We get a 18 ms difference between the best cloud for BNP Paribas (Azure) and the worst (AWS). That’s 14% slower, although given the size of differences we often see between clouds, probably not something to worry too much about.

We can see that a lot of the difference is down to handshake time with AWS 12 ms slower than Azure or more than 50%.  If you want to get the best possible performance out of your APIs for all your users no matter which cloud they are using, it often comes down to the details of network configuration. That’s why it is important to actively monitor your APIs, so that you can discover the rate-determining steps and squeeze out all the improvements in speed.

See you again in a week to see how things are shaping up as head towards the spring equinox.