Ware now more than halfway through February and it’s time once more to look at the State of the APIs over the past week. We recently launched API.expert as a simple way to provide everybody with insights into the API economy and the leading APIs in different categories in particular. The basic service is free and will remain free, but other, related services will be built on top of it, including your very own API.expert dashboards if you want them! 

Each week, month and year we’ll look at the APIs we track in a variety of sectors for their quality (using our patent-pending CASC score, which allows you at a glance to see the performance of an API and compare it to other onesas well as factors like uptime and availability. 

If something is missing that you think should be here, don’t hesitate to drop us a line! 

Now, onto what the last seven days can tell us about where the world of APIs is in early February 2020.

API Performance Headlines 

We look at over 200 APIs but pull all the metrics together to give you a general feeling for the service quality for an organization’s APIs in a particular category.  

We do see that certain providers consistently vie for top spot in their category including Twitter, Slack and Github with others doing less well such as Cisco’s Spark services. 

Top Performers based on overall quality in each category were: 

CASC score

Week ending February 17, 2020

CategoryOrganizationCASC score
Corporate InfrastructureGitHub967
Cryptocurrency ExchangesLATOKEN937
PSD2 BanksABN AMRO Bank
967
SearchGoogle949
Social NetworksTwitter
957
UK GovernmentGOV.UK931
UK Open Banking (Open Data)Bank of Ireland969
US GovernmentDepartment of Justice972

Two changes again this week with ABN AMRO Bank replacing BNP Paribas in PSD2 Banks and the Department of Justice the General Services Administration in US government. Police.UK in UK Government.  The Department of Justice is best overall this week with an excellent CASC score of 972 replacing GOV.UK, which falls from 982 to 931, but at least the Department of Justice is keeping things up for the public sector. A CASC score of over 900 is very good and one of 950 or more exceptional. Six of the nine categories are headed by organization with a CASC score of 950 or more this week, which is a very good showing. Sustaining a CASC score of >925 over a month is a good showing and congratulations to those organizations that achieved it. 

Top performers by latency 

Week Ending February 17, 2020 

CategoryOrganizationMedian latency
Corporate InfrastrutureSlack186 ms
Cryptocurrency ExchangesFTX248 ms
PSD2 banksOpen Bank Project193 ms
SearchGoogle386 ms
Social NetworksGoogle121 ms
UK GovernmentPolice.UK83 ms
UK Open banking (Open Data)HSBC80 ms
US GovernmentDepartment of Justice67 ms

Just a single change this week with Open Bank Project replacing BNP Paribas in PSD2 Banks. An important caveat: medians can be misleading! An API might have a fast median latency but produce many slow outliers. These won’t affect the median, but they mean that users can experience many calls that were unacceptably slow. So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score! 

As so often, FTX might have a low enough latency to top the Cryptocurrency Exchange category, but they also have more outliers than any of their competitors (7.29% compared to 7.44% last week), which is why they come out 10th out of 18 in terms of the CASC score this week. It’s no good being fast if you are flaky, and FTX are consistently fast and flaky. 

Worst quality across all categories 

 

CategoryOrganizationCASC Score
Corporate InfrastrutureCisco Spark720
Cryptocurrency ExchangesOasis Dex756
PSD2 banksVisa704
SearchNobody below 800! 
Social NetworksNobody below 800! 
UK GovernmentNHS475
UK Open banking (Open Data)Halifax775
US GovernmentDepartment of Commerce645

Again only a single change, this week with Halifax dropping to the bottom of  UK Open Banking (Open Data) replacing HSBC. The NHS retains the overall Wooden Spoon for the week, in the Red Zone again and in need of some attention. 

A couple things of interest 

Why is the NHS struggling so badly. They have 100% availability and a respectable enough median latency of 347 ms. Their outliers though are shocking at 8.93%. And looking at median latency per cloud, we see something fascinating (note that we only run the NHS test calls from locations in the British Isles). 

API Performance API Expert APImetrics

This is total time. Everything looks fine for AWS and IBM Cloud. But what the heck is going on with Azure? Looking at the process time, we can see that this is the latency component that is doing the damage.  

API Performance API Expert APImetrics

Even the fact that IBM Cloud is more than twice as fast as AWS is noteworthy, but what the heck is going on with Azure. Why is the processing time 40 times slower than IBM Cloud and 16 times slower than AWS? Answers on a postcard, please. What could be going on?

Differences in performance between clouds are often related to network configuration. We have seen in the past cases in which performance is fine expect from the location at which an API is hosted. This kind of behavior is generally down to some kind of internal routing issue. This is that the case here? But why would we see it with processing time and not the other components?

Remember we are making the consistently slow calls from an APImetrics agent located at Azure locations. If the NHS endpoints are also hosted at the same location this might explain the issue if there is a very odd routing problem. (DNS lookup time is a common culprit in slow latency because configuration issues are often seen with that component, but this is not the case here.)

Certainly, some investigation is needed here on the NHS’s part. But the moral of this story is plainly seen never trust your cloud location. Performance can vary wildly between clouds and even if things look fine from your end, your clients, located at some particular cloud location might be having a torrid time and blaming it on you even the fundamental cause is something outside your immediate control “in the cloud.”

See you again in a week to see how things are shaping up as the days get longer each day.