Metadata for Your Data
File Size Statistics
Thr dtatistics for files sizes of the files in the pickle are:
- Average file size: 909.000KB
- Largest file: 19,804,344.000KB
- Smallest file: 0.000KB
- Standard deviation file size: 49,832.8061KB
Table 9: File Size Intervals
| Interval (KB) | No. of Files | % of Total | Cumulative % |
|---|---|---|---|
| 0–1 | 156,455 | 40.31 | 40.31 |
| 1–2 | 38,278 | 9.86 | 50.18 |
| 2–4 | 30,822 | 7.94 | 58.12 |
| 4–8 | 43,118 | 11.11 | 69.23 |
| 8–16 | 24,296 | 6.26 | 75.49 |
| 16–32 | 33,455 | 8.62 | 84.11 |
| 32–64 | 13,502 | 3.48 | 87.59 |
| 64–128 | 12,083 | 3.11 | 90.70 |
| 128–256 | 8,623 | 2.22 | 92.93 |
| 256–512 | 13,437 | 3.46 | 96.39 |
| 512–1,024 | 5,456 | 1.41 | 97.79 |
| 1,024–2,048 | 2,687 | 0.69 | 98.49 |
| 2,048–4,096 | 2,497 | 0.64 | 99.13 |
| 4,096–8,192 | 1,361 | 0.35 | 99.48 |
| 8,192–16,384 | 949 | 0.24 | 99.73 |
| 16,384–32,768 | 373 | 0.10 | 99.82 |
| 32,768–65,536 | 246 | 0.06 | 99.89 |
| 65,536–131,072 | 179 | 0.05 | 99.93 |
| 131,072–262,144 | 76 | 0.02 | 99.95 |
| 262,144–524,288 | 36 | 0.01 | 99.96 |
| 524,288-1,048,576 | 154 | 0.04 | 100.00 |
Table 10: Top 10 Largest Files
| Rank | File | Size (KB) |
|---|---|---|
| 1 | /home/laytonjb/CLUSTERBUFFER2/STRACE_PY/EXAMPLES/cesm-strace/strace.janus017.tar | 19,804,344 |
| 2 | /home/laytonjb/src/CFD/CFD_2/cfdpp.tar.gz | 8,473,315 |
| 3 | /home/laytonjb/LAPTOP/CLUSTERBUFFER/STRACE_PY/EXAMPLES/23andme-jeff-runs/all2/strace.26702.call_variants_1.out | 6,155,298 |
| 4 | /home/laytonjb/CLUSTERBUFFER2/STRACE_PY/EXAMPLES/23andme-jeff-runs/all2/strace.26702.call_variants_1.out | 6,155,298 |
| 5 | /home/laytonjb/src/CFD/CFD_2/overflow.tar.gz | 5,863,543 |
| 6 | /home/laytonjb/LAPTOP/CLUSTERBUFFER/STRACE_PY/EXAMPLES/23andme-jeff-runs/all2/strace.24999.count_covariates_1.out | 5,748,509 |
| 7 | /home/laytonjb/CLUSTERBUFFER2/STRACE_PY/EXAMPLES/23andme-jeff-runs/all2/strace.24999.count_covariates_1.out | 5,748,509 |
| 8 | /home/laytonjb/src/CFD/CFD_2/boeing_app_tuned.tar.gz | 4,240,130 |
| 9 | /home/laytonjb/LAPTOP/CLUSTERBUFFER/STRACE_PY/EXAMPLES/23andme-jeff-runs/all2/strace.24084.realign_indels_1.out | 4,225,463 |
| 10 | /home/laytonjb/CLUSTERBUFFER2/STRACE_PY/EXAMPLES/23andme-jeff-runs/all2/strace.24084.realign_indels_1.out | 4,225,463 |
Figure 5: File size (KB) histogram.
Biggest Users
Table 11: Top 10 Largest Files
| Rank | User | Total Size (KB) | % of Total |
|---|---|---|---|
| 1 | laytonjb | 353,100,132 | 99.9998 |
| 2 | root | 645 | 0.0002 |
| 3 | susy | 0 | 0.0000 |
Related content
-
How Old is That Data?
The explosion of data is a storage burden that all system administrators bear. The agedu tool lets you discover what data is being used.
-
I/O Profiling at the Block Level
Understanding how applications perform I/O is important not only because of the volume of data being written and read, but because the performance of some applications is dependent on how I/O is conducted. In this article we profile I/O at the block layer to help you make the best storage decisions.
-
Sharing Data with SSHFS
Sharing data saves space, reduces data skew, and improves data management. We look at the SSHFS shared filesystem, put it through some performance tests, and show you how to tune it.
-
Filesystem Encryption
The revelation of wide-spread government snooping has sparked a renewed interest in data storage security via encryption. In this article, we review some options for encrypting files, directories, and filesystems on Linux.
-
What Is an Inode?
Understanding inodes is key to a better understanding of HPC filesystems.