API Metrics Fintech, Jan 25–Feb 1: Open Banking UK Production

Something of interest

In Open Banking – UK – Production API metrics category, Nationwide has a degree of middle table respectability coming in at 19 out of 40 with a Green Zone CASC score of 8.36 and a 100% pass rate. It could do better, but it’s doing OK.

But, as we discussed last week, in Open Banking – UK – Sandbox, Nationwide has been taking the overall wooden spoon amongst financial APIs. And here’s the thing: it’s the same four endpoints in both cases.

Everything looks fine for the production endpoints. OAuth Access Token client_credentials could do with being perked up a bit, but the pass rates (100%) and outlier percentages are fine as far as API metrics are concerned.

The sandbox endpoints don’t quite 100%, but it is the median latency and outlier percentage that is absolutely killing these API metrics endpoints. And this has been going on now for several weeks.

This does look as though not enough attention is being paid to the sandbox. Now, people might say “It’s only a sandbox” or “Nobody uses it” or “It’s just an off-the-shelf bank-in-a-box.” But none of that matters.

A sandbox should be easier than a production environment. If all your endpoints in the sandbox have had median latency of about 30 s for several weeks, something is seriously amiss. The sandbox’s performance should, ideally, be as close as possible to that of the production environment not just in what the endpoints return, but also in how quickly and reliably they do it.

It’s certainly about time that a good hard look was taken at the Nationwide sandbox and just what is going on with those latencies. Because if they can do it for production – and they can – Nationwide can do it for the sandbox.

Fintech API Metrics: Overall Performance

Week Ending 1 February 2021

Category	Organization	CASC score
Cryptocurrency Exchanges	RadarRelay	9.38
Fintech	Stripe	9.28
Open Banking: PSD2	Railsbank	9.61
Open Banking UK: API Sandbox	Starling Bank (Sandbox)	9.39
Open Banking UK: Production	Creation Cards	9.34
Open Banking UK: Open Data	Bank of Ireland	9.66

Bank of Ireland in the Open Banking – UK – Open Data API metrics category takes the overall title again this week with a CASC score of 9.66, down from 9.71 last week. A CASC score of over 9.00 is very good and one of 9.50 or more exceptional. Two out of the six categories are headed by organization with a CASC score of 9.50 or more this week. Sustaining a CASC score of >9.25 over a period of several weeks is a good showing and congratulations to those organizations that achieved it. All categories are headed by an API with CASC score of 9.28 or more, which is a very acceptable level of performance.

Fintech API Metrics: Latency

Week Ending 1 February 2021

Category	Organization	Median Latency
Cryptocurrency Exchanges	Kraken	224 ms
Fintech	Stripe	325 ms
Open Banking: PSD2	Railsbank	105 ms
Open Banking UK: API Sandbox	Starling Bank (Sandbox)	144 ms
Open Banking UK: Production	Creation Cards	157 ms
Open Banking UK: Open Data	HSBC	87 ms

HSBC in Open Banking – UK – Open Data API metrics again takes the overall title again with a median latency of 87 ms, the same as last week. An important caveat: medians can be misleading! An API might have a fast median latency but produce many slow outliers. These won’t affect the median, but they mean that users can experience many calls that were unacceptably slow. So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score!

Fintech API Metrics: Worst Overall Quality

Week Ending 1 February 2021

Category	Organization	CASC score
Cryptocurrency Exchanges	Cryptocompare	7.03
Fintech	Square (Sandbox)	6.70
Open Banking: PSD2	Deutsche Bank	7.63
Open Banking UK: API Sandbox	Nationwide (Sandbox)	2.88
Open Banking UK: Production	Creation Card (Auth)	5.32
Open Banking UK: Open Data	HSBC	7.71

Nationwide (Sandbox) in the Open Banking – UK – Sandbox API metrics category takes the overall wooden spoon again with a CASC score of 2.88, up from 2.36 last week. See you again in a week as we plunge further into February. Also don’t forget to look out for our monthly report for January and the weekly and monthly non-financial API reports. We will also be publishing special reports on the State of the APIs in the 2020, so that’s something else to look forward to!

Do more with Fintech API monitoring!

If you're new to API monitoring, you can learn about our comprehensive features for testing, monitoring, security and performance and sign up for a free trial account.