The variation in median total time over 24 months shows a roughly similar pattern between clouds, as expected. The variation is primarily driven by changes on the server-side for API providers. (The median is a better metric here as it excludes the effect of outliers.)

What is noteworthy is that the performance of AWS and Google show corresponding trends with a downward (increased speed) tendency until July 2020 and an upward (decreased speed) tendency since then. IBM Cloud was the fastest cloud in January 2019 but demonstrated a less markedly downward trend through summer 2020 and a somewhat sharper one since then. Azure has consistently been about 90 ms slower than AWS and Google from late 2019, even though it was the fastest cloud in July 2019.

The different latency components show that in 2020 Azure was slower for all latency components except Mean Time to Upload the API Request, which does not affect overall performance because it is only a few tenths of a millisecond at most. Azure thus needs to improve its performance across all latency components to be competitive against the other clouds.

Note these numbers are mean values of the latency component. The median values are smaller as they are not affected by outliers. This is particularly noticeable for Mean DNS Time as this component is mainly affected by outliers.

Only AWS shows a very slight decrease in Mean Total Time for 2020 compared to 2019. All other clouds are about 30 ms slower overall. An increase in Mean Time to First Byte is the primary component responsible. AWS also shows this increase but compensates with a faster Mean DNS Time.

Note these are mean times and thus affected by outliers. AWS has reduced outliers and (see below) ensured that the Median DNS Time is close to the optimum of 4 ms.

Of the six regions, Europe and North America have been largely stable for 24 months. These regions are noticeably faster than the other regions, typically by about 400 ms. The other four regions showed a general downward (decreased latency) trend in 2019. East Asia and Oceania show a marked decrease in the second half of 2020. This could be ascribed to the increased load on infrastructure because of COVID-19 in rolling out new network capacity in the geographically diffuse Asia-Pacific region.

The quantized nature of the DNS Time is seen by looking at the median value for each cloud over 24 months. Typically, the DNS lookup time should be about 4 ms or 12 ms.

The mean value of the DNS Time is often much higher because of the impact of frequent outliers and calls to endpoints for which DNS is not optimized. AWS has been able to consistently provide a median DNS Time of about 4 ms since March 2020. This suggests a well-optimized cloud. IBM Cloud has been able to manage this performance at times, but not Azure or Google.