Lead Image © Lucy Baldwin, 123RF.com
Filesystem Murder Mystery
Hunting down a performance problem
As I was generating results for one of last Fall's columns, James Saryerwinnie of the Amazon AWS team was chasing an interesting performance problem and relaying his progress live on Twitter (Figure 1) [1], providing me yet another unnecessary opportunity to procrastinate. What followed was one of the most interesting Twitter threads in performance-tuning in recent memory and a great practical demonstration of the investigative nature of performance problems.
Figure 1: The original description of the problem.
Saryerwinnie was observing occasional unexplained delays of a few seconds while renaming some gigabyte-sized files [2] on a fast cloud instance [3]. I was able to reproduce his result in an AWS EC2 m5d.large instance running Ubuntu Server 18.04 (Figure 2). I used an EBS class gp2 volume of 8GB showing remarkable performance in this particular test (including a throughput of 175MBps), but similar results are achievable at other scales with some lucky timing.
...
Buy this article as PDF
(incl. VAT)
Buy ADMIN Magazine
US / Canada
UK / Australia
Related content
-
Tuning I/O Patterns in Fortran 90
In the second article of this three-part series, we look at simple write examples in Fortran 90 and track the output with strace to see how it affects I/O patterns and performance.
-
Tuning I/O Patterns in C
The language you choose to use affects I/O patterns and performance. We track a simple write I/O pattern with C and look at how to improve performance.
-
Tuning I/O Patterns in Python
In the third article of this three-part series, we look at simple write examples in Python and track the output with strace to see how it affects I/O patterns and performance.
-
Microsoft PowerToys for Windows 10
This collection of free tools helps you customize Windows 10 for individual workflows. -
Appendix – I/O Report from MPI Strace Analyzer
Appendix – I/O Report from MPI Strace Analyzer