THE CLOUD FOR APIS: 2022 IN REVIEW

4XX CLIENT-SIDE WARNING – 4xx HTTP status codes are generated when a request is made to an endpoint that does not exist or for which the user lacks the appropriate authorization. Because these types of issues indicate that the web server receiving the request is behaving as expected, 4xx client-side warnings generally should not be included when determining the performance and quality of an API endpoint. 

5XX SERVER ERROR – A 5xx server error is an actual reported error from the application server hosting the APIs. 

95TH PERCENTILE – The latency of the slowest 5% of calls expressed in milliseconds. 

99TH PERCENTILE – The latency of the slowest 1% of calls expressed in milliseconds. 

AGENT – The APImetrics software agent runs at various cloud locations around the world enabling synthetic calls to be generated as if they were being made by an end user or partner. 

API – Although an Application Programming Interface (API) is a general concept in computer systems, in the current context we are concerned only with web APIs. A user makes an HTTP request to a published API endpoint. The request causes the web server that receives the request to return a payload containing information in a specified format or cause the state of some remote resource to be changed. APIs can thus be used to exchange useful data and information between systems. 

API CALL – An API call is a single HTTP request made to a particular endpoint. Details of the request and the response are stored by APImetrics for further analysis to determine the performance and quality of the endpoint. 

AUTHENTICATION AND AUTHORIZATION – Access to a particular API endpoint may depend on validating the identity of the requesting party and whether it has been granted the appropriate authorization. This might involve encrypted passwords or tokens generated and managed through a protocol such as OAuth 2.0 supplemented by a specification such as FAPI (Financial-grade API). 

AVAILABILITY – Closely linked to pass rate. Strictly, the availability should always be higher than the pass rate. Calls to an endpoint may not pass because of authentication and authorization issues or because the request is malformed, but the endpoint is still available. APImetrics analyzes the results and calculates pass rate and estimates availability. 

AWS – Amazon Web Services – observer agents running on AWS servers in different regions. 

AZURE – Azure from Microsoft – observer agents running on Microsoft servers in different regions. 

CASC SCORE – Cloud API Service Consistency (CASC) is an APImetrics-proprietary patented technology that combines various measures of API performance such as availability, latency, reliability, and a number of outliers, benchmarked against our unrivalled collection of historical API call records, to give a single blended metric much like a credit rating. The CASC score lets you see at a glance the quality of an API endpoint, whether it is getting better or worse, and how it compares to other endpoints.

CDN – Content Delivery Network – used by IT systems to optimize the performance of their APIs and websites.

CLOUD PROVIDER – An organization that provides a commercial service, hosting applications at a server. Well-known cloud providers include Google Cloud Platform (Google), Amazon Web Services (AWS), and Microsoft Azure (Azure), IBM Cloud (IBM), all of which have many locations around the world.

CLOUD – Distributed computing service using server resources from providers such as Amazon, Microsoft, Google and IBM:

• AWS – Amazon Web Service
• Azure – Microsoft Cloud Services
• Google – Google Cloud Compute
• IBM – IBM Cloud Services

CMA9 BANKS – The CMA9 are nine large UK banks that are mandated by the Competition and Markets Authority (CMA) to expose certain Open Banking APIs and regularly return certain reports on the performance of the APIs. The banks are Allied Irish Bank, Bank of Ireland, Barclays, Danske, HSBC, Lloyds Group, Nationwide, NatWest Group, and Santander.

CONFIGURATION – Internal and external network configuration, such as load balancers at the API gateway that direct requests to specific IP addresses, can have a significant impact on API performance and quality. For instance, problems with external configuration such as routing tables can cause requests to be misdirected, and load balancers can direct requests to IP addresses that do not support a particular service.

DNS LATENCY – DNS (Domain Name Server) is a global service that identifies where a particular service is located on the internet. The lookup time is the time taken for the cloud service making the API call to identify where the target server is and route the request. The different techniques used for the lookup task will affect service quality and appear as latency.

DNS LOOKUP TIME – The time the cloud provider Domain Name Service (DNS) host takes to resolve the URI for the API.

DNS – Domain Name Service – the global system that allows the internet to identify the location of a specific internet service via a human readable name

DOWNLOAD TIME – The time taken for a request to be downloaded from the web server to the agent.

ENDPOINT – The Universal Resource Indicator (web address) that is called when you make an API call. For the API call to work you will have URI + parameters of the call + security. This task is different to simply looking up the URL of a website, where it is often just the URI that is needed.

FAILURE RATE – The proportion of calls made to an API endpoint that returns an unexpected response.

FAPI – Financial-grade API (FAPI) is a technical specification that the Financial-grade API Working Group of OpenID Foundation has developed. It uses OAuth 2.0 and OpenID Connect (OIDC) as its base and defines additional technical requirements for the financial industry and other industries that require higher API security.

GET – The simplest HTTP verb (others are HEAD, POST, PUT PATCH and DELETE) that sends a request to an API endpoint that gets a resource, such as a list of account transactions. Parameters and headers allow complex requests to be made with a GET.

GOOGLE CLOUD – Observer agents running on Google servers in different regions.

HANDSHAKE TIME – The time to complete the process that sets up an HTTP connection, which is called a handshake.

HTTP 200 CODES – Generally indicates a passing result (but not always!).

HTTP 400 CODES – Indicates that something has gone wrong somewhere but the API call has not necessarily failed – these often relate to security codes expiring – but can also indicate that another system has failed. These can be misleading and with API transactions can often indicate other problems.

HTTP 500 CODES – A failure has occurred – either an error code from the service itself, or an indication that something is not working in the back-end infrastructure or that a service is down for some reason.

HTTP STATUS CODE – A standard set of codes used by computer services to identify what is happening.

HTTP – Hyper Text Transfer Protocol – the basic communication system for the internet and APIs. IBM CLOUD – Observer agents running on IBM servers in different regions.

LATENCY – The time a specific action takes to complete – usually measured in milliseconds – 1 millisecond is 1/1000 of a second – to understand this in human terms, a normal camera flash lasts 1 millisecond and light itself can travel about 300 km (186 mi) in that time.

MEAN – The average value of data in a collection.

MEDIAN TOTAL TIME – The median latency of calls from our observer agents from the point at which the call is initiated to the point at which the last byte is received expressed in milliseconds (1/1000s of a second).

MEDIAN – The mid-point value in a data collection.

METRIC – A measure of some aspects of API endpoint performance such as the availability or median length of a latency component.

NETWORK INFRASTRUCTURE – The totality of the physical network elements that make up the systems that together comprise the internet. Includes switches, routers, and connectors such as fiber and microwave links.

NON-CONFORMANCE – An API endpoint that does not respond according to its published specification is non-conformant. Typically, this might mean that the return payload has missing fields, contains incorrect information, or the endpoint is generating errors and warnings despite the call being made according to specification.

OBIE – Open Banking Implementation Entity (OBIE) is the UK entity managing standards for Open Banking within the United Kingdom.

OPEN BANKING – A new global paradigm for banking, financial, and payment services that enables innovative new products and user experiences powered by data and information exchange through APIs.

OUTLIERS – The percentage of calls over time that were decided to be a performance outlier by our AI.

PASS RATE – The % of calls in the given time that passed with either just a HTTP200 code (link) or met the criteria set in APImetrics for a successful pass.

PERFORMANCE – The set of metrics such as availability, latency, reliability, and number of outliers that define how an API endpoint has behaved over time.

PROCESSING TIME – How long the target server takes to handle the API call.

PSD2 – Payment Services Directive 2 (PSD2) is a pan-European agreement to open payments and banking services that is applicable to all financial service providers doing business in the EU and United Kingdom. Responsibility for the implementation of regulations lies with each country.

QUALITY – How good an API endpoint is from the end-user perspective. Although this can be challenging to measure, blended metrics such as the APImetrics CASC score provide a quantitative benchmark, allowing organizations to compare the quality of an API endpoint over time, or compare two API endpoints at a glance.

REGION – The global region where the call originates.

RELIABILITY – A reliable API endpoint tends to respond within a narrow range of time. A reliable endpoint may not necessarily be fast, but the variance in its latency will be relatively small.

SERVICE – A software service delivered via cloud APIs.

SPEED – The rate at which data is passed along a connection such as an intercontinental undersea fiber link. The more traffic, the slower the speed of the connection.

SSL HANDSHAKE TIME – The time taken for the Secure Sockets Layer (SSL) handshake to take place.

TCP CONNECT TIME – The time it takes for Transmission Control Protocol (TCP), or for the observer agent to connect to the server hosting the API.

TOTAL API CALL TIME – Used in this document to relate to the entire sequence of making an API call from DNS name lookup, through connection, security handshakes, to the point at which the final result is returned.

TOTAL TIME – The time between a request being made to an API endpoint and the whole of the response being received, including the name lookup (DNS) time.

UPLOAD TIME – How long the observer agent takes to upload the parameters of the call to the target server.

VERSION – APIs are often updated to make changes to the way the endpoints are invoked, or the content of the payload returned. It is important to ensure that the endpoint for the correct version is invoked. Often the URI for the endpoint will contain the version.