Poster
in
Workshop: Methods and Opportunities at Small Scale (MOSS)
Encoding Domain Insights into Multi-modal Fusion: Improved Performance at the Cost of Robustness
Jackson Michaels · Sidong Zhang · Madalina Fiterau
Keywords: [ Interpretability ] [ Robustness ] [ Small-Scale Experiments ] [ Synthetic Tasks ] [ Inductive Priors ] [ Multi-modal Fusion ]
Using small-scale experiments with real and synthetic tasks, we compare multi-modal fusion methods, including a proposed `Product Fusion', to demonstrate how encoding task-specific priors affects performance. Our results highlight a crucial trade-off: aligning fusion design with priors boosts clean-data accuracy with limited data but significantly diminishes robustness to noisy inputs.