Intel researchers have pushed a ternary language model below the theoretical 1.58-bit limit, compressing one checkpoint to 1.485 bits per weight while boosting decoding speed by up to 18% on CPUs and 27% on GPUs. The team developed a new storage format called BITCOS that exploits the abundance of zero-valued weights in real models, packing the same model into less space without retraining or changing its output. Published by Intel, the work challenges the widespread assumption that 1.58 bits represents a hard floor for ternary models.

The 1.58-bit figure assumes ternary models distribute their three possible weight values—negative one, zero, and positive one—equally, but Intel's analysis of 29 checkpoints from seven model families found zeros made up between 29.7% and 51.5% of all weights. The sparsest checkpoint, a ternary variant of Qwen3-1.7B created using CAT-Q post-training quantization, contained 51.48% zeros and compressed down to 1.485 bits per weight with BITCOS. Standard five-trit packing fits five ternary values into an eight-bit byte for 1.6 bits each, but storing weights in 128-element blocks leaves the final byte partially empty, raising the real rate to 1.625 bits per weight. BITCOS beat five-trit packing in 26 of the 29 checkpoints examined, crossing the breakeven threshold once zero weights exceeded 37.5% of the total.

The BITCOS format splits model weights into two separate streams: one bit per weight records whether it's zero or nonzero, and a second bit stores the sign only for nonzero values. According to the report, a positive or negative weight consumes two bits under this scheme, while a zero needs just the presence bit because it carries no sign. If z represents the fraction of zero weights, BITCOS uses 2 minus z bits per weight, falling from 1.6 bits at 40% zeros to 1.485 bits at 51.5%. The authors note that because the format changes only how weights are stored, unpacking restores the original values without affecting accuracy.

BITCOS works by reducing the volume of weight data traveling through memory during token-by-token decoding with small batch sizes, and Intel built custom unpacking kernels for AVX-512 and AVX2 CPUs as well as Xe2 GPUs to realize the speed gains. On AVX-512 hardware, the kernel uses the presence bitmap as a mask and the pdep instruction to scatter compacted sign bits across nonzero weight positions, while Xe2 GPUs rely on a 2KB lookup table because they lack an equivalent instruction. Benchmarks across five systems showed BITCOS ran 10% to 18% faster than 2-bit kernels on a 64-core Xeon server, 2% to 15% faster on a 24-core Core Ultra 9, 9% to 22% faster on the integrated Arc 140V, and 2% to 27% faster on the discrete Arc Pro B70. However, the format didn't win on an eight-core Lunar Lake CPU, where sufficient bandwidth made unpacking the bottleneck and Intel's fixed 2-bit kernel outperformed BITCOS on every model tested.

The paper has not been peer-reviewed, all five test systems used Intel hardware, and the end-to-end benchmarks covered seven models at batch size one, with no testing yet on Nvidia, AMD, or Arm platforms. The report notes that the right optimization depends on whether compute, memory, or bandwidth constrains the workload, and BITCOS delivers its largest advantages when memory movement—not unpacking overhead—is the limiting factor. The format joins other compression efforts aimed at fitting smaller models into faster inference pipelines for AI agents, particularly in scenarios where reducing weight data can directly speed up decoding. Organizations running ternary models with high zero density stand to gain immediate throughput improvements without retraining, though hardware and batch size will determine whether BITCOS or simpler fixed-bit formats deliver better real-world performance. The choice between maximum compression and fastest unpacking will increasingly shape how production systems balance model size against inference speed.