API Ratings: Who won the week? | February 10-17, 2020

We are now more than halfway through February and it’s time once more to look at the State of the APIs over the past week. We recently launched API.expert as a simple way to provide everybody with insights into the API economy and the leading APIs in different categories in particular. The basic service is free and will remain free, but other, related services will be built on top of it, including your very own API.expert dashboards if you want them! 

Each week, month and year we’ll look at the APIs we track in a variety of sectors for their quality (using our patent-pending CASC score, which allows you at a glance to see the performance of an API and compare it to other ones) as well as factors like uptime and availability. 

If something is missing that you think should be here, don’t hesitate to drop us a line! 

Now, onto what the last seven days can tell us about where the world of APIs is in early February 2020.

API Performance Headlines 

We look at over 200 APIs but pull all the metrics together to give you a general feeling for the service quality for an organization’s APIs in a particular category.  

We do see that certain providers consistently vie for top spot in their category including Twitter, Slack and Github with others doing less well such as Cisco’s Spark services. 

Top Performers based on overall quality in each category were: 

Two changes again this week with ABN AMRO Bank replacing BNP Paribas in PSD2 Banks and the Department of Justice the General Services Administration in US government. Police.UK in UK Government.  The Department of Justice is best overall this week with an excellent CASC score of 972 replacing GOV.UK, which falls from 982 to 931, but at least the Department of Justice is keeping things up for the public sector. A CASC score of over 900 is very good and one of 950 or more exceptional. Six of the nine categories are headed by organization with a CASC score of 950 or more this week, which is a very good showing. Sustaining a CASC score of >925 over a month is a good showing and congratulations to those organizations that achieved it. 

Top performers by latency 

Week Ending February 17, 2020 

Just a single change this week with Open Bank Project replacing BNP Paribas in PSD2 Banks. An important caveat: medians can be misleading! An API might have a fast median latency but produce many slow outliers. These won’t affect the median, but they mean that users can experience many calls that were unacceptably slow. So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score! 

As so often, FTX might have a low enough latency to top the Cryptocurrency Exchange category, but they also have more outliers than any of their competitors (7.29% compared to 7.44% last week), which is why they come out 10th out of 18 in terms of the CASC score this week. It’s no good being fast if you are flaky, and FTX are consistently fast and flaky. 

Worst quality across all categories 

Again only a single change, this week with Halifax dropping to the bottom of  UK Open Banking (Open Data) replacing HSBC. The NHS retains the overall Wooden Spoon for the week, in the Red Zone again and in need of some attention. 

A couple things of interest 

Why is the NHS struggling so badly. They have 100% availability and a respectable enough median latency of 347 ms. Their outliers though are shocking at 8.93%. And looking at median latency per cloud, we see something fascinating (note that we only run the NHS test calls from locations in the British Isles). 

This is total time. Everything looks fine for AWS and IBM Cloud. But what the heck is going on with Azure? Looking at the process time, we can see that this is the latency component that is doing the damage.  

Even the fact that IBM Cloud is more than twice as fast as AWS is noteworthy, but what the heck is going on with Azure. Why is the processing time 40 times slower than IBM Cloud and 16 times slower than AWS? Answers on a postcard, please. What could be going on?

Differences in performance between clouds are often related to network configuration. We have seen in the past cases in which performance is fine expect from the location at which an API is hosted. This kind of behavior is generally down to some kind of internal routing issue. This is that the case here? But why would we see it with processing time and not the other components?

Remember we are making the consistently slow calls from an APImetrics agent located at Azure locations. If the NHS endpoints are also hosted at the same location this might explain the issue if there is a very odd routing problem. (DNS lookup time is a common culprit in slow latency because configuration issues are often seen with that component, but this is not the case here.)

Certainly, some investigation is needed here on the NHS’s part. But the moral of this story is plainly seen never trust your cloud location. Performance can vary wildly between clouds and even if things look fine from your end, your clients, located at some particular cloud location might be having a torrid time and blaming it on you even the fundamental cause is something outside your immediate control “in the cloud.”

See you again in a week to see how things are shaping up as the days get longer each day.