Foundation models are transforming Earth observation, yet struggle to generalize across bands and sensors to handle the data for different applications. We introduce GeoCrossBench, a novel benchmark that extends the standard GeoBench, to evaluate this critical cross-band capability in remote sensing foundation models. We measure generalization by augmenting datasets with additional optical and radar data, training on RGB, then testing on other bands. We first evaluate existing models, as a reality check on current performance and for analysis of pretraining effects, then evaluate our own self-supervised extension of the ChannelViT model, ChiViT, to improve cross-band performance. While our ChiViT demonstrates strong results compared to currently available remote sensing specific models, none of them outperforms general-purpose vision models like DINOv2. These findings highlight the necessity of benchmarks like GeoCrossBench to advance robust foundation models for comprehensive Earth observation.