QuickLZ is an open-source, byte-oriented data compression library written in C that is explicitly designed to be one of the fastest compression algorithms in the world. Developed by Lasse Mikkel Reinhold, it focuses strictly on maximizing streaming throughput and processing speed rather than achieving the tightest possible compression ratio.
At its core, QuickLZ is heavily optimized for modern CPU architectures. It processes data at speeds reaching hundreds of megabytes per second per CPU core, making it ideal for real-time applications where low latency is critical. How QuickLZ Works
QuickLZ is a byte-aligned implementation based on the classic LZ77 (Lempel-Ziv) algorithm.
Dictionary-Based Matches: It scans input data to find recurring sequences. Instead of re-writing those sequences, it replaces them with a compressed token consisting of a distance pointer and a match length.
No Entropy Coding: Unlike heavier tools (like ZIP or Gzip), QuickLZ drops the slow, mathematical entropy-coding phase (like Huffman coding). It simply copies literals or matches in their raw, byte-entirety, saving massive amounts of CPU cycles.
Three Selectable Levels: QuickLZ can be configured into three distinct operational modes:
Level 1: Delivers the fastest possible compression speed with a minimal compression footprint. Level 2: Acts as a balanced intermediate tier.
Level 3: Tailored to deliver the fastest decompression speed and the best compression ratio of the three. Key Advantages
Extreme Hardware Speed: Reaches benchmarking speeds around 308 MB/s per core, reducing CPU bottlenecks during bulk data operations.
Tiny Memory Footprint: The algorithm requires very little scratchpad memory—often needing only around 36 KB of state structure to compress data. This makes it highly portable for resource-constrained systems like the RT-Thread Real-Time Operating System.
Simple Integration: Consisting of a lightweight C implementation (quicklz.c and quicklz.h), it is notoriously trivial to drop into existing software architectures.
Streaming Support: It natively supports streaming modes, allowing applications to compress data on-the-fly as packets move through a network or pipe. QuickLZ vs. Modern Competitors
While QuickLZ pioneered high-speed streaming compression in the 2000s and 2010s, the data ecosystem has largely transitioned to newer, highly optimized alternatives. Primary Strength Typical Use Cases QuickLZ Historic speed, low memory (~36KB) Legacy embedded devices, historic database engines LZ4 Incredible decompression speed (>1 GB/s) Real-time logging, database engines (ClickHouse) Zstandard (zstd) Best balance of ratio and speed Modern file backups, network data transfers (Kafka)
Note: For many years, enterprise tools like Percona XtraBackup used QuickLZ (via the .qp qpress file format) by default. However, modern ecosystems have officially deprecated QuickLZ in favor of newer options like LZ4 and Zstandard due to better hardware scaling. Licensing Model QuickLZ operates under a dual-license model:
Open Source: Available for free under the GNU General Public License (GPL v1, v2, or v3). If you use it under this tier, your derived software must also be open-source.
Commercial License: A paid tier meant for proprietary, closed-source commercial software deployments.
Are you planning to implement QuickLZ in an embedded system, a database, or a custom application? Tell me about your environment, and I can provide specific integration examples or recommend the best modern library for your needs. RT-Thread-packages/quicklz – GitHub
Leave a Reply