OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inference
Attention mechanisms are central to the success of large language models (LLMs), enabling them to capture intricate token dependencies and implicitly assign importance to each token. Recent...
House of Cards: Massive Weights in LLMs
Massive activations, which manifest in specific feature dimensions of hidden states, introduce a significant bias in large language models (LLMs), leading to an overemphasis on the corresponding...
Efficient Neural Video Representation with Temporally Coherent Modulation
Implicit neural representations (INR) has found successful applications across diverse domains. To employ INR in real-life, it is important to speed up training. In the field of INR for video...
Diversify, Contextualize, and Adapt: Efficient Entropy Modeling for Neural Image Codec
Designing a fast and effective entropy model is challenging but essential for practical application of neural codecs. Beyond spatial autoregressive entropy models, more efficient backward...
Diversify, Contextualize, and Adapt: Efficient Entropy Modeling for Neural Image Codec
Unsupervised Representation Transfer for Small Networks: I Believe I Can Distill On-the-Fly
SeeThroughNet: Resurrection of Auxiliary Loss by Preserving Class Probability Information
D-3DLD: Depth-Aware Voxel Space Mapping for Monocular 3D Lane Detection with Uncertainty
Segmenting 2K-Videos at 36.5 FPS with 24.3 GFLOPs: Accurate and Lightweight Realtime Semantic Segmentation Network
Neural Image Compression with Text-guided Encoding for both Pixel-level and Perceptual Fidelity
Neural Weight Compression for Language Models
Efficient compression of language model weights is increasingly critical as model scale and deployment grow. Yet, most existing methods rely on handcrafted transforms and heuristics, reflecting...
Neural Image Compression with Text-guided Encoding for both Pixel-level and Perceptual Fidelity
Recent advances in text-guided image compression have shown great potential to enhance the perceptual quality of reconstructed images. These methods, however, tend to have significantly degraded...