Adaptive Neural Compression

Adaptive Neural Compression

A research project exploring learned representations for data compression, built with PyTorch and designed for bandwidth-constrained aerial networks.

Project Overview

Traditional compression codecs are hand-crafted for specific media types. This project investigates an alternative: training neural networks to learn compact, task-aware representations of data end to end. The goal is a compression pipeline that adapts its bitrate and representation quality to the content being encoded, and ultimately to the constraints of the network carrying it, such as the MQTT-based drone swarm networks modeled in my OMNeT++ simulation workspace.

Technologies Used

• Language: Python
• ML Framework: PyTorch
• Numerical Computing: NumPy
• Data Handling: Custom PyTorch Dataset and DataLoader pipelines

Key Features

1. Learned Representations: Autoencoder-style models that map raw data into compact latent codes instead of relying on hand-crafted transforms.
2. Adaptive Bitrate Control: A quantization scheme that trades fidelity for size based on a target compression ratio, letting the same model serve different bandwidth budgets.
3. End-to-End Training: Loss functions that jointly optimize reconstruction quality and code entropy, so the network learns to discard truly redundant information.
4. Simulation Integration: Designed to feed into the MQTT drone network simulation for evaluating transmission overhead under real network constraints.

Project Details

The core of the project is a training loop that treats compression as an optimization problem: minimize the distortion introduced by the encoding while keeping the encoded size small. The encoder compresses inputs into latent codes, a learned quantizer discretizes them, and the decoder reconstructs the original signal. Reconstruction error and an entropy penalty are combined into a single loss, allowing fine-grained control over the rate-distortion trade-off.

Challenges

• Rate-Distortion Balance: Tuning the entropy penalty so the model produces small codes without destroying reconstruction quality.
• Quantization Training: Backpropagating through discrete quantization steps, which have no meaningful gradient.
• Generalization: Ensuring the learned codec performs well on inputs the model has never seen, not just the training distribution.

Solutions

• Straight-Through Estimation: Passing gradients through the quantizer unchanged while using its discrete output during the forward pass, a standard trick for training quantized networks.
• Tunable Entropy Weight: Exposing the entropy coefficient as a training hyperparameter, enabling a family of models spanning different compression ratios from a single architecture.
• Strict Data Split: Holding out entire data segments from training to measure true generalization of the learned representations.

Conclusion

Adaptive Neural Compression demonstrates that compression can be learned rather than engineered. By combining autoencoder architectures with entropy-aware training in PyTorch, the project lays the groundwork for adaptive, content-aware codecs, and for transmitting sensor and telemetry data efficiently over constrained networks like drone swarms.