API Ratings: Performance

For our API ratings, we look at over 100 APIs and pull all the metrics together to give you a general feeling for the service quality for an organization’s APIs in a particular API ratings category with our patented CASC score.

Something of interest

Pager Duty has a lot of outliers – 7.10% – more than twice the number of Slack (3.38%). If we look at Total Time API ratings, we can see there is a lot of variation between locations.
api ratings outliers 1

AWS North America is more than four times quicker than Azure South America and we see that the 95th percent time is nearly three times faster. This indicates that we may get a lot of the outliers from South America.

api ratings outliers 2

We can see that it is the PageDuty List Incidents endpoint that is pulling down PagerDuty overall in terms of outliers. This endpoint has 18.4% outliers. Digging down we can that there is quite a big variation between clouds in terms of percentage of outliers with IBM Cloud being nearly three times more likely to produce an outlier than Azure.

That’s potentially going to have a big impact on user experience. This is why you need to actively monitor your APIs from a range of clouds and locations. Just because an API works fine from one cloud location does not mean that your users who are making calls from other ones are going to be getting the same performance.

api ratings outliers 3

50% of calls from South America are outliers this week, which is a serious deterioration from the previous week, but over 36% outliers is still not good. These results suggest that Pager Duty need to take a good hard look at the configuration around this endpoint in particular and especially whether there are any issues around it for IBM Cloud and South America.

We don’t have any IBM Cloud locations in South America, so that can’t be the worst over cloud location.

api ratings outliers 4

There are always going to variations in performance between cloud locations because the world is large and the speed of light finite, but the goal of API engineering should be minimizing them as much as possible. Actively monitoring your APIs is a vital step in ensuring that all your users get the performance they expect.

API Ratings: Overall Quality

Week Ending 14 December 2020

Category

Organization

CASC score

 

Corporate
Infrastructure

 

Pivotal Tracker

9.55

COVID-19

nubentos.com

9.42

Search

 

Microsoft Bing

9.33

Social Networks

 

Google

9.62

UK Government

 

GOV.UK

9.50

US Government

 

Department of Justice

9.71

Messaging

 

Slack

9.52

Department of Justice in the US Government category takes the overall API ratings title again this week with a CASC score of 9.71, down from 9.81.

A CASC score of over 9.00 is very good and one of 9.50 or more exceptional. Three of the six categories are headed by organization with a CASC score of 9.50 or more this week. Sustaining a CASC score of >9.25 over a period of several weeks is a good showing and congratulations to those organizations that achieved it.

This week, all of the API ratings categories are topped by an API with a CASC score of more than 9.33.

API Ratings: Top Performers by Latency

Week Ending 14 December 2020

Category

Organization

Median Latency

 

Corporate
Infrastructure

 

Microsoft Office

213 ms

COVID-19

WHO

84 ms

Search

 

Google

399 ms

Social Networks

 

Google

61 ms

UK Government

 

Police.UK

84 ms

US Government

 

Department of Justice

46 ms

Messaging

 

Slack

260 ms

Department of Justice in the US Government takes the overall title again with a median latency of 46 ms, the same as last week.

An important caveat: medians can be misleading! An API might have a fast median latency but produce many slow outliers. So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score!

API Ratings: Worst Quality

Week Ending 14 December 2020

Category

Organization

CASC score

 

Corporate
Infrastructure

 

Nobody under 8.00!

 

 

COVID-19

thevirustracker.com

6.58

Search

 

Nobody under 8.00!

 

Social Networks

 

Flickr

7.67

UK Government

 

NHS

6.18

US Government

 

NASA

 

6.05

Messaging

 

Cisco Webex Teams

 

7.35

NASA in the US Government category takes the overall Wooden Spoon this week with a CASC score of 6.14.

We launched API.expert earlier in the year as a simple way to provide everybody with insights into the API economy and the leading APIs in different categories in particular. We are expanding our coverage of Open Finance APIs from around the world, so one of the major changes we made recently to these round ups has been to pull out the financial APIs.

See you again in a week as we look at API ratings when it will be almost Christmas! Also don’t forget to look out for our monthly report for November and the weekly and monthly financial API reports.

Do more with API monitoring!

If you are new to APImetrics, sign up for a free trial so see how they could work for you with our free API set.

If you have any questions, please don’t hesitate to reach out.