Archive and compression formats compared: ZIP, TAR, GZ, BZ2, XZ, Zstandard, 7Z and RAR
Understand the architectural difference between random-access archives (ZIP), sequential tape archives (TAR), solid compressors (7Z, RAR), and stream compressors (Gzip, Bzip2, XZ, Zstandard).
1. Archiving vs. compression: why Unix separates TAR from GZ, BZ2, XZ and ZST
In the Unix philosophy, combining multiple files and directories into a single byte stream (archiving) is separate from shrinking that byte stream (compression):
- TAR (.tar) concatenates files sequentially in 512-byte blocks—writing a 512-byte POSIX
ustarheader (containing the filename, octal permissions, uid/gid, mtime, and byte size) before each file's raw, uncompressed bytes. A pure.tarfile is always a multiple of 512 bytes and performs zero compression. - Stream compressors (.gz, .bz2, .xz, .zst) compress a single stream of bytes. When wrapped around a
.tararchive, they create a compressed tarball (.tar.gz/.tgz,.tar.bz2,.tar.xz,.tar.zst) that achieves natural solid compression across all files in the archive. - ZIP (.zip) combines archiving and per-file DEFLATE compression in one format, placing a random-access Central Directory at the end of the file so you can extract a single 4 KB file from a 2 GB ZIP archive without decompressing the preceding 1.99 GB.
| Format | Magic bytes (hex) | Algorithm | Random single-file access | Sample download |
|---|---|---|---|---|
| ZIP (.zip) | 50 4B 03 04 ('PK\x03\x04') | DEFLATE (per file) | Yes (via End Central Directory) | ZIP samples |
| TAR (.tar) | 75 73 74 61 72 ('ustar' at offset 257) | None (uncompressed 512B blocks) | Sequential header hops | TAR samples |
| Gzip (.gz / .tgz) | 1F 8B 08 | DEFLATE (RFC 1952 + CRC32) | No (sequential stream) | GZ samples |
| Bzip2 (.bz2) | 42 5A 68 ('BZh') | Burrows-Wheeler block sort | By 100–900 KB block | sample.bz2 |
| XZ (.xz) | FD 37 7A 58 5A 00 | LZMA2 + CRC64 | By XZ block index | sample.xz |
| Zstandard (.zst) | 28 B5 2F FD (RFC 8878) | LZ77 + Finite State Entropy | With seekable frame table | sample.zst |
| 7-Zip (.7z) | 37 7A BC AF 27 1C | LZMA / LZMA2 solid blocks | By solid block | sample-archive-7z.7z |
| RAR (.rar) | 52 61 72 21 1A 07 ('Rar!') | Proprietary RARlabs LZ/PPM | By solid/non-solid block | Documented (proprietary encoder) |
2. Critical security checks when extracting uploaded archives
- Zip Slip (path traversal via
../): An archive entry name is just an arbitrary string inside the header. If an archive contains an entry named../../../../etc/cron.d/pwnor..\startup.batand your extractor joins that name directly to the output directory without verifyingresolvedPath.startsWith(targetDir), the extractor writes outside the target directory. - Zip Bombs and decompression ratio limits: Never decompress an untrusted archive to completion without tracking total uncompressed bytes written so far (not just the `uncompressed_size` field in the ZIP header, which can be spoofed).
- Symlink attacks in TAR and ZIP: POSIX tar archives (`typeflag = '2'`) and Unix-created ZIP files can contain symbolic links pointing to `/etc/passwd` or application secrets. Reject or strip symlink and device-node entries when unpacking user uploads.