API.expert Weekly Banking API Analysis: Lloyds Group APIs

This Week’s Banking API Highlights

• Tide in the Open Banking API – UK – Production category takes the overall title this week with a CASC score of 9.82.
• HSBC in the Open Banking API – UK – Open Data category takes the overall title again with a median latency of 88 ms, the same as last week.
• Halifax in the Open Banking API – UK – Open Data category takes the overall wooden spoon this week with a CASC score of 4.59.
• We look at exactly what went wrong with the three Lloyds Group APIs in the Open Banking API – UK – Open Data category last week.

Something of interest

The three Lloyds Group APIs in the Open Banking – UK – Open Data category had a bad week last week. All three APIs had CASC scores of less than 5.00, placing them firmly in the Red Zone. Only two other APIs out of 17 are in the Amber Zone. So the Lloyds Group API really were struggling compared to their peers. What went wrong?

Remember that the Open Data APIs are directly comparable in terms of functionality and thus there is really no reason for significant differences in performance between them. Yet the Lloyds Group are three of the four slowest in the category and Halifax is by far the slowest. But it is in the pass rate and outliers that the three APIs are underperforming.

The Lloyds Get ATMs V2.2 endpoint is typical of the Lloyds Group endpoints this week. It has a low pass rate, a lot of outliers and is very slow.

The big problem is an outage on Saturday, 13 March 2021 from about 13:41 UTC, when we observed a HTTP 502 Bad Gateway followed by many HTTP 404 Not Found through to 19:52 UTC when the endpoint started passing again. Note that this was not a continuous outage.
We saw passes between 16:28 UTC and 16:59 UTC, which was followed by a 502 and then 404s. We also saw one 503 (Service Unavailable). Note that we also observed a cluster of 404s, 200 Slow Failures and incomplete (timed-out) calls later in the evening of 13 March between 22:24 UTC and 23:36 UTC. These can be on the chart above.
We have also this week seen a cluster 404s and 502s on 16 March, so this is an ongoing issue, suggesting that the original outage wasn’t a planned outage. The three Lloyds Group API has been having sustained issues over the last few days.
This is why you need to be actively monitoring your APIs, so that you know both if there are problems with your API right now, but also if there have been problems that spontaneously resolved themselves. Those kinds of problems can indicate undiagnosed bugs and they can come back to bite you when you least expect and, inevitably, when they will be most costly to you.

Banking API Analysis: Overall Performance

Week Ending 15 March 2021

Category	Organization	CASC score
Cryptocurrency Exchanges	RadarRelay	9.35
Fintech	Square (Sandbox)	9.44
Open Banking: PSD2	BBVA	9.51
Open Banking UK: API Sandbox	Royal Bank of Scotland (Sandbox)	9.58
Open Banking UK: Production	Tide	9.82
Open Banking UK: Open Data	Bank of Ireland	9.70

Tide in the Open Banking – UK – Production category takes the overall title this week with a CASC score of 9.82. When you look at CASC scores you are looking for a ranking of at least 9.00. Even better is if you can achieve a CASC score of 9.50 or better. This week we we are looking at organizations which achieve CASC scores of 9.50 or better for both this week and last week, that was the case in at least four of seven categories. That's impressive and these organizations should be proud. All six categories are headed by an API with CASC score of 9.30 or more, which is a very acceptable level of performance.

Banking API Analysis: Latency

Week Ending 15 March 2021

Category	Organization	Median Latency
Cryptocurrency Exchanges	Kraken	218 ms
Fintech	Stripe	328 ms
Open Banking: PSD2	Railsbank	104 ms
Open Banking UK: API Sandbox	Royal Bank of Scotland (Sandbox)	145 ms
Open Banking UK: Production	Tide (Auth)	151 ms
Open Banking UK: Open Data	HSBC	88 ms

HSBC in the Open Banking – UK – Open Data category takes the overall title again with a median latency of 88 ms, the same as last week. One thing that should be considered when looking at CASC scores. It's possible for an API to have periods of time when they are very fast but also periods when they are extremely slow. This means that users will very often have a subpar experience. You don't want to simply be fast from time to time. You want to be consistently running swiftly. It is not a good idea to rely on your fastest performance. Overall speed is the real issue.

Banking API Analysis: Worst Overall Quality

Week Ending 15 March 2021

Category	Organization	CASC score
Cryptocurrency Exchanges	Bancor Network	5.08
Fintech	Nobody under 8.00!
Open Banking: PSD2	Nobody under 8.00!
Open Banking UK: API Sandbox	Barclays (Sandbox)	6.24
Open Banking UK: Production	B (Auth)	5.09
Open Banking UK: Open Data	Halifax	4.59

HSBC in the Open Banking – UK – Open Data category takes the overall wooden spoon this week with a CASC score of 4.59. See you again in a week as we head further into March (though there are still a whole two weeks to go!). Also don’t forget to look out for our monthly report for February and the weekly and monthly non-financial API reports. We will also be publishing special reports on the State of the APIs in the 2020, so that’s something else to look forward to!

Do more with Fintech API monitoring!

If you're new to API monitoring, you can learn about our comprehensive features for testing, monitoring, security and performance and sign up for a free trial account.