Efficient Foundation-Model Adaptation & Post-Training
Modern post-training increasingly treats pretrained and specialized models themselves as reusable sources of knowledge, alongside data. My research explores efficient and reliable ways to adapt, compress, and integrate this knowledge for domain specialization and capability consolidation.
Research includes VB-LoRA (NeurIPS 2024) and Uni-LoRA (NeurIPS 2025 Spotlight), which explore reparameterization and parameter sharing for extreme parameter efficiency in fine-tuning and their effects on optimization; Beta-KD (CVPR 2026), an uncertainty-aware knowledge distillation approach for reliably transferring knowledge from imperfect supervision into compact models; and combining capabilities from multiple specialized models into a unified multitask model.
Related Publications
- VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks, NeurIPS 2024
- Uni-LoRA: One Vector is All You Need, NeurIPS 2025 Spotlight
- Uncertainty-Aware Knowledge Distillation for Multimodal Large Language Models, CVPR 2026