API Ratings | Sept 14-21, 2020: Playing in Sandboxes

We are now in the final third of September(!) and that time of the week has already arrived again for us to look at some key API ratings over the past seven days.

We recently launched API.expert as a simple way to provide everybody with insights into the API economy and the leading API ratings in different categories in particular.

The basic service is free and will remain free. But other, related API ratings services will be built on top of it, including your very own API.expert dashboards if you want them!

Each week, month, and year we’ll look at API ratings in a variety of sectors for their quality CASC score using our patented system (US Patent 10,644,962, if you fancy a little light bedtime reading) that allows you to see the API ratings and compare them to other ones, as well as factors like uptime and availability.

If something is missing in our API ratings that you think should be here, don’t hesitate to drop us a line!

Now, onto what the past week tells us about the state of API ratings during this insane week of an unending year.

API Performance Headlines

To compile our API ratings, we look at more than 200 APIs. Then we pull all the metrics together to give you a general feeling for API quality in a particular category.
We do see that certain providers consistently vie for top spot in their API ratings category, including GitHub and Google. Others do less well, such as Halifax and the NHS.

Top API Ratings – CASC Score

Sept 14-21, 2020

Category

Organization

CASC score

Corporate
Infrastructure

Github

9.68

COVID-19

COVID-10 Finnhub API

9.24

Cryptocurrency
Exchanges

Coinpaprika

9.35

Fintech

Stripe

9.50

Search

Google

9.43

Social Networks

Twitter

9.69

Open Banking – PSD2

Nordea Bank

9.89

Open Banking – UK Sandbox

Royal Bank of Scotland (Sandbox)

9.85

Open Banking – UK Production

Royal Bank of Scotland

8.76

Open Banking – UK Open Data

Allied Irish Bank (GB)

9.80

UK Government

GOV.UK

9.86

US Government

Department of Justice

9.72

Three changes in API ratings this week:

  • Coinpaprika swapping with OKEx in Cryptocurrency Exchanges
  • Royal Bank of Scotland topping the new category of Open Banking – UK Production
  • Allied Irish Bank (GB) replacing Bank of Ireland in Open Banking – UK Open Data

Nordea retains the overall title for the private sector with a CASC score of 9.89.

A CASC score of over 9.00 is very good and one of 9.50 or more exceptional for an API rating. Six of the eleven categories are headed by organization with a CASC score of 9.50 or more this week, the same as last week. Sustaining a CASC score of >9.25 over a period of several weeks is a good showing for API ratings. Congrats to those organizations that achieved it.

All API rating categories but one are headed by an API with CASC score of 9.20, which is an acceptable level of performance.

Top API Ratings – Latency

Sept 14-21, 2020

Category

Organization

Median Latency

Corporate
Infrastructure

Microsoft Office

234 ms

COVID-19

CDC Tools

163 ms

Cryptocurrency Exchanges

FTX

218 ms

Fintech

Stripe

302 ms

Search

Facebook

322 ms

Social Networks

Google

57 ms

Open Banking - PSD2

Railsbank

94 ms

Open Banking - UK Sandbox

Royal Bank of Scotland (Sandbox)

148 ms

Open Banking - UK Production

Cashplus

137 ms

Open Banking - UK Open Data

HSBC

81 ms

UK Government

Police.UK

90 ms

US Government

Department of Justice

51 ms

A stable week, with just one change in the API ratings: Cashplus topping the new category of Open Banking – UK Production. Department of Justice takes the overall title with a median latency of 51 ms, up from 50 ms last week. An important caveat for these API ratings: medians can be misleading! An API might have a fast median latency but produce many slow outliers. These won’t affect the median, but they mean that users can experience many calls that were unacceptably slow. So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score! As so often, FTX tops the Cryptocurrency Exchange API rating category by median latency. This week they are down to 14th out of 18 overall with a CASC score of 8.29, placing them in the Green Zone, but not as solidly as we'd like to see it.

Worst API Rating Overall

Sept 14-21, 2020

Category

Organization

CASC Score

Corporate Infrastructure

Cisco Spark

7.93

COVID-19

Coronavirus Data API (thevirustracker.com)

6.52

Cryptocurrency Exchanges

RadarRelay

6.97

Fintech

OBIE Directory (Prod Auth)

7.36

Search

Nobody below 8.00!

Social Networks

Nobody below 8.00!

Open Banking - PSD2

BNP Parribas (Sandbox)

7.69

Open Banking - UK Sandbox

Barclays (Sandbox Auth)

1.00

Open Banking - UK Production

Creation Cards (Auth)

4.05

Open Banking - UK Open Data

HSBC

7.98

UK Government

NHS

6.99

US Government

Federal Communications Commission

7.52

Five changes this week in the API ratings!
  • Cisco Spark replacing DocuSign in Corporate Infrastructure
  • OBIE Directory (Production Auth) a new entry in Fintech
  • BNP Paribas (Sandbox) a new entrant in Open Banking – PSD2
  • Creation Cards (Auth) taking the bottom spot in the new category of Open Banking – UK Production
  • Halifax replacing HSBC in Open Banking – UK Open Data
Barclays (Sandbox Auth) again takes the overall API rating loser spot for the week with a CASC score of 1.00.

Something of interest

As you can see in our revised API ratings, APImetrics has recently been able to start monitoring not just the sandbox environments of some of the banks in the UK that are offering open banking, but also certain APIs in the production environments too. Generally, we separate the functionality the bank provides into an auth piece and a operational piece for convenience. It is possible and useful to test the auth piece without having credentials to test the operational piece. API ratings playing in Sandboxes We can see that Royal Bank of Scotland tops both API rating categories. (We don’t currently monitor the NatWest test environment.) Royal Bank of Scotland and NatWest are brands of the same bank, so we would expect them to have similar performance. Interestingly, the auth pieces have similar latency for both the sandbox and production environments. We see a lot of outliers for the auth endpoints in both the sandbox and production environments. For the operational piece, we see that the number of outliers in the production environment is over 30 times as many as for the sandbox. API ratings playing in Sandboxes This might be because of heavier load on the production environment. Note that the median latency is higher for the production environment by 63 ms. Using an active API performance and quality monitoring system like APImetrics allows you to determine exactly where the rate determining steps are with your APIs, and where the issues that might affect user experience could be. It also lets you compare yourself with your peers. Both Lloyds Banking and Allied Irish Bank (GB) have pretty solid sandboxes, but they are much slower than Royal Bank of Scotland. Now with APIs and API ratings, sometimes you might be comparing apples and oranges. Two corporate infrastructure or even banking APIs might be doing quite different things. But this isn’t the case with open banking where all the banks are implementing the same APIs. So it is absolutely crucial to benchmark yourself to get true API ratings. If your competitor has a median latency of 148 ms for their operational piece, why is your latency 850 ms? See you again in a week as we look once more at API ratings in the last week of September. Where did this month go?

Sign Up to Our Newsletter

Want to be updated on latest API Metrics Insights? Sign up to our newsletter and receive latest news and blog from us.

Add an API

Related Posts

Join Us Now!

Join the 100s of companies relying on APImetrics.
Share