Poster
in
Workshop: DIG-BUGS: Data in Generative Models (The Bad, the Ugly, and the Greats)
Implementing Adaptations for Vision AutoRegressive Model
Kaif Shaikh · Antoni Kowalczuk · Franziska Boenisch · Adam Dziedzic
Keywords: [ image autoregressive models ] [ adaptations ] [ differential privacy ]
Vision AutoRegressive model (VAR) was recently introduced as an alternative to Diffusion Models (DMs) in image generation domain.In this work we focus on its adaptations, which aim to fine-tune pre-trained models to perform specific downstream tasks, like medical data generation. While for DMs there exist many techniques, adaptations for VAR remain underexplored. Similarly, differentially private (DP) adaptations---ones that aim to preserve privacy of the adaptation data---have been extensively studied for DMs, while VAR lacks such solutions. In our work, we implement and benchmark many strategies for VAR, and compare them to state-of-the-art DM adaptation strategies. We observe that VAR outperforms DMs for non-DP adaptations, however, the performance of DP suffers, which necessitates further research in private adaptations for VAR.Code is available at https://github.com/sprintml/finetuningvardp.