We are now in the final third of September(!) and that time of the week has already arrived again for us to look at some key API ratings over the past seven days.

We recently launched API.expert as a simple way to provide everybody with insights into the API economy and the leading API ratings in different categories in particular.

The basic service is free and will remain free. But other, related API ratings services will be built on top of it, including your very own API.expert dashboards if you want them!

Each week, month, and year we’ll look at API ratings in a variety of sectors for their quality CASC score using our patented system (US Patent 10,644,962, if you fancy a little light bedtime reading) that allows you to see the API ratings and compare them to other ones, as well as factors like uptime and availability.

If something is missing in our API ratings that you think should be here, don’t hesitate to drop us a line!

Now, onto what the past week tells us about the state of API ratings during this insane week of an unending year.

API Performance Headlines

To compile our API ratings, we look at more than 200 APIs. Then we pull all the metrics together to give you a general feeling for API quality in a particular category. We do see that certain providers consistently vie for top spot in their API ratings category, including GitHub and Google. Others do less well, such as Halifax and the NHS.

Top API Ratings – CASC Score

Sept 14-21, 2020

Category

Organization

CASC score

 

Corporate
Infrastructure

 

Github

9.68

COVID-19

COVID-10 Finnhub API

9.24

Cryptocurrency
Exchanges

 

Coinpaprika

9.35

Fintech

 

Stripe

9.50

Search

 

Google

9.43

Social Networks

 

Twitter

9.69

Open Banking – PSD2

 

Nordea Bank

9.89

Open Banking – UK Sandbox

 

Royal Bank of Scotland (Sandbox)

9.85

Open Banking – UK Production

 

Royal Bank of Scotland

8.76

Open Banking – UK Open Data

 

Allied Irish Bank (GB)

9.80

UK Government

 

GOV.UK

9.86

US Government

 

Department of Justice

9.72

Three changes in API ratings this week:

  • Coinpaprika swapping with OKEx in Cryptocurrency Exchanges
  • Royal Bank of Scotland topping the new category of Open Banking – UK Production
  • Allied Irish Bank (GB) replacing Bank of Ireland in Open Banking – UK Open Data

Nordea retains the overall title for the private sector with a CASC score of 9.89.

A CASC score of over 9.00 is very good and one of 9.50 or more exceptional for an API rating. Six of the eleven categories are headed by organization with a CASC score of 9.50 or more this week, the same as last week. Sustaining a CASC score of >9.25 over a period of several weeks is a good showing for API ratings. Congrats to those organizations that achieved it.

All API rating categories but one are headed by an API with CASC score of 9.20, which is an acceptable level of performance.

Top API Ratings – Latency

Sept 14-21, 2020

Category

Organization

Median Latency

 

Corporate
Infrastructure

 

Microsoft Office

234 ms

COVID-19

CDC Tools

163 ms

Cryptocurrency Exchanges

 

FTX

218 ms

Fintech

 

Stripe

302 ms

Search

 

Facebook

322 ms

Social Networks

 

Google

57 ms

Open Banking - PSD2

 

Railsbank

94 ms

Open Banking - UK Sandbox

 

Royal Bank of Scotland (Sandbox)

148 ms

Open Banking - UK Production

 

Cashplus

137 ms

Open Banking - UK Open Data

 

HSBC

81 ms

UK Government

 

Police.UK

90 ms

US Government

 

Department of Justice

51 ms

A stable week, with just one change in the API ratings: Cashplus topping the new category of Open Banking – UK Production.

Department of Justice takes the overall title with a median latency of 51 ms, up from 50 ms last week.

An important caveat for these API ratings: medians can be misleading! An API might have a fast median latency but produce many slow outliers.

These won’t affect the median, but they mean that users can experience many calls that were unacceptably slow.

So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score!

As so often, FTX tops the Cryptocurrency Exchange API rating category by median latency. This week they are down to 14th out of 18 overall with a CASC score of 8.29, placing them in the Green Zone, but not as solidly as we’d like to see it.

Worst API Rating Overall

Sept 14-21, 2020

Category

Organization

CASC Score

 

Corporate
Infrastructure

 

Cisco Spark

7.93

COVID-19

Coronavirus Data API (thevirustracker.com)

6.52

Cryptocurrency
Exchanges

 

RadarRelay

6.97

Fintech

 

OBIE Directory (Prod Auth)

 

 

7.36

Search

 

Nobody below 8.00!

 

 

Social Networks

 

Nobody below 8.00!

 

 

Open Banking - PSD2

 

BNP Parribas (Sandbox)

 

7.69

Open Banking - UK Sandbox

 

Barclays (Sandbox Auth)

1.00

Open Banking - UK Production

 

Creation Cards (Auth)

4.05

Open Banking - UK Open Data

 

HSBC

7.98

UK Government

 

NHS

6.99

US Government

 

Federal Communications Commission

7.52

Five changes this week in the API ratings!

  • Cisco Spark replacing DocuSign in Corporate Infrastructure
  • OBIE Directory (Production Auth) a new entry in Fintech
  • BNP Paribas (Sandbox) a new entrant in Open Banking – PSD2
  • Creation Cards (Auth) taking the bottom spot in the new category of Open Banking – UK Production
  • Halifax replacing HSBC in Open Banking – UK Open Data

Barclays (Sandbox Auth) again takes the overall API rating loser spot for the week with a CASC score of 1.00.

Something of interest

As you can see in our revised API ratings, APImetrics has recently been able to start monitoring not just the sandbox environments of some of the banks in the UK that are offering open banking, but also certain APIs in the production environments too.

Generally, we separate the functionality the bank provides into an auth piece and a operational piece for convenience. It is possible and useful to test the auth piece without having credentials to test the operational piece.

API ratings playing in Sandboxes

We can see that Royal Bank of Scotland tops both API rating categories. (We don’t currently monitor the NatWest test environment.) Royal Bank of Scotland and NatWest are brands of the same bank, so we would expect them to have similar performance.

Interestingly, the auth pieces have similar latency for both the sandbox and production environments. We see a lot of outliers for the auth endpoints in both the sandbox and production environments.

For the operational piece, we see that the number of outliers in the production environment is over 30 times as many as for the sandbox.

API ratings playing in Sandboxes

This might be because of heavier load on the production environment. Note that the median latency is higher for the production environment by 63 ms.

Using an active API performance and quality monitoring system like APImetrics allows you to determine exactly where the rate determining steps are with your APIs, and where the issues that might affect user experience could be.

It also lets you compare yourself with your peers. Both Lloyds Banking and Allied Irish Bank (GB) have pretty solid sandboxes, but they are much slower than Royal Bank of Scotland.

Now with APIs and API ratings, sometimes you might be comparing apples and oranges. Two corporate infrastructure or even banking APIs might be doing quite different things. But this isn’t the case with open banking where all the banks are implementing the same APIs.

So it is absolutely crucial to benchmark yourself to get true API ratings. If your competitor has a median latency of 148 ms for their operational piece, why is your latency 850 ms?

See you again in a week as we look once more at API ratings in the last week of September. Where did this month go?