In 2022, we monitored 15 APIs in the US Governments collection. Of these, 14 had a CASC score in the Green Zone, of which 4 had a CASC score of 9.00 or more. Only NASA had a CASC score for the year in the Yellow Zone with a score of 7.78.
For the month of January 2023, 9 APIs were in the Green Zone (5 with a CASC score of 9.00 or more), 5 APIs were in the Yellow Zone, and one (FEMA) was in the Red Zone with a CASC score of 4.40.
For the week beginning 13 February 2023 , 12 APIs are in the Green Zone (7 with a CASC score of 9.00 or more), 3 APIs are in the Yellow Zone. There are no APIs is in the Red Zone.
This indicates that January 2023 was an unusually bad month for US Government APIs, compared to either the year of 2022 or the week beginning 13 February 2023.
U.S. Government API Performance Issues: A Closer Look
The FDA always has a very high percentage of outliers (21.19% for 2022, 28.04% for January 2023, and 27.61% for the week beginning 13 February 2023).
The Bureau of Labor Statistics always has an unusually low pass (95.22% in 2022, placing it second from bottom behind General Services Administration, 96.23% in January 2023, placing it second from bottom behind FEMA, 96.23% for the week beginning 13 February 2023, placing it at the bottom). These failures are almost entirely owing to timeouts and being unable to contact the service with no HTTP status code being returned.
As mentioned above, NASA was the only API to be in the Yellow Zone for 2022, the month of January 2023, and the week beginning 13 February 2023. It had the lowest CASC score in 2022 with a score of 7.78. It has the highest p99 time in 2022, January 2023, and the week beginning 13 February 2023.
It also has quite a high percentage of outliers (for the three periods in the range of 7.30 to 10.33%, and never better than third-worst).
The pass rate is about average. The NASA endpoints might be under high load perhaps partly explaining their poor performance.
Conclusions & Recommendations
- Most APIs in the U.S. Government collection are in the Green Zone most of the time, indicating that these are typically performant APIs.
- A number of APIS (Bureau of Labor Statistics, FDA, FEMA, NASA) have had issues that are persistent over time, but there is no indication that these sets of problems can be considered generic to U.S. government APIs.
- It would be worth increasing the number of endpoints monitored were possible and adding additional APIs to the collection.
- There does not appear to be an active centralized point of competence for APIs across the U.S. Government. The most recent Government API Community of Practice – Quarterly Check-In (organized by 18F/data.gov through the General Services Administration) we can find was in March 2019, but it’s possible these are continuing (although we would need a .gov or .mil email address to attend). This document, for instance, discusses best practices for API design, but nothing about API monitoring, API performance, and quality or API lifecycle governance.
- There does not appear to be a U.S. Government API Czar (or API Product Owner/Manager). There will be senior managers at the 18F, the GSA, and the US Digital Service as well as several agencies that are interested in API governance, but most of the documents online are several years old. It is possible that someone like Kin Lane might have knowledge of specific stakeholders. Mina Hsiang is the administrator of the US Digital Service but is probably too senior/strategic to be involved with API governance.
