Automatic Modulation Classification (AMC) serves as a foundational pillar for cognitive radio systems, enabling critical functionalities including dynamic spectrum allocation, non-cooperative signal surveillance, and adaptive waveform optimization. However, practical deployment of AMC faces a fundamental challenge: prediction ambiguity arising from intrinsic similarity among modulation schemes and exacerbated under low signal-to-noise ratio (SNR) conditions. This phenomenon manifests as near-identical probability distributions across confusable modulation types, significantly degrading classification reliability. To address this, we propose Fuzzy Regularization-enhanced AMC (FR-AMC), a novel framework that integrates uncertainty quantification into the classification pipeline. The proposed FR has three features: (1) Explicitly model prediction ambiguity during backpropagation, (2) dynamic sample reweighting through adaptive loss scaling, (3) encourage margin maximization between confusable modulation clusters. Experimental results on benchmark datasets demonstrate that the FR achieves superior classification accuracy and robustness compared to compared methods, making it a promising solution for real-world spectrum management and communication applications.

In Automatic Modulation Recognition (AMC), the similarity between modulation methods, especially under low signal-to-noise ratio (SNR) conditions, leads to ambiguous model predictions and undermines recognition reliability. To address this challenge, we explore a new framework designed to alleviate the issue. Our experiments show that the similarity between modulation types is reflected in the model's predicted probability values. Specifically, more similar classes tend to produce closer prediction probabilities. We model this phenomenon by incorporating uncertainty. Building on this, we propose a novel fuzzy regularized AMC framework (FR-AMC). This framework effectively mitigates modulation confusion by quantifying the uncertainty in the predicted probability distribution during training and applying a penalty to high-uncertainty predictions. Experiments on benchmark datasets show that FR-AMC improves classification accuracy and robustness. It offers a practical path toward more reliable cognitive radio systems and shows potential for other fine-grained classification tasks.