Multi-Objective Self-Supervised Representation Learning

Labeled data is the scarcest resource in computer vision. This line of work asks how far a vision transformer can get without it — and, more distinctively, how multiple self-supervised objectives should share one model.

Cross-Scale MAE (NeurIPS 2023) tackled the problem in remote sensing, where the same scene appears at wildly different ground resolutions across sensors. By enforcing consistency between representations of the same imagery at different scales during masked-autoencoder pre-training, it produced more robust features for downstream transfer. Pre-trained ViT-B and ViT-L weights are released in the lab’s GitHub repository.

MEDiC (arXiv 2026) moved to general imagery, exploring distillation from a frozen CLIP teacher: token-level distillation, CLS alignment, and MAE-style pixel reconstruction combined as objectives, with an “evolved part masking” scheme that progresses from spatial to semantic masks derived from CLIP attention.

ExPLoRe (ECCV 2026) resolves the tension MEDiC exposed: different objectives want different things from different image patches. ExPLoRe repurposes the dispatch weights of a Soft mixture-of-experts as learned per-patch loss coefficients, routing each patch’s training signal to the objectives it serves best — reaching 80.6% linear-probe and 85.3% fine-tune accuracy on ImageNet-1K with a ViT-B/16.

The through-line is a lab-wide conviction: representation learning works best when the model is allowed to decide, per location and per scale, which learning signal matters.