Imputation is a critical preprocessing step for handling missing data, yet conventional imputation methods can introduce or amplify unfairness across protected groups. We theoretically show that imputation fairness directly impacts downstream fairness in terms of accuracy parity at inference time, when the predictive model is trained solely on fully observed data. Motivated by this insight, we introduce a novel adversarial framework called Fair Adversarial IMputation (FAIM) that can be integrated with any gradient-based imputation model. Our method incorporates a tunable fairness-accuracy trade-off parameter, allowing practitioners to balance imputation performance and imputation fairness. We empirically validate FAIM on two real-world datasets, showing significant improvements in group-wise imputation fairness. Furthermore, we assess the downstream fairness impact on a synthetic dataset derived from a real-world dataset, confirming that fairer imputations lead to fairer predictive outcomes at inference time.