This paper proposes a novel method to improve locomotion for a single quadruped robot using multi-agent deep reinforcement learning (MARL). Many existing methods employ single-agent reinforcement learning for individual robots or MARL for cooperative tasks in multi-robot systems. Unlike existing methods, this paper proposes using MARL for the locomotion of a single quadruped robot. We propose a learning structure called multi-agent reinforcement learning for single quadruped robot locomotion (MASQ), where each leg is considered an agent to explore the action space of the quadruped robot, shares a global critic, and learns cooperatively. Experimental results demonstrate that MASQ not only accelerates learning convergence but also enhances robustness in real-world settings, indicating that applying MASQ to single robots, such as quadrupeds, could surpass traditional single-robot reinforcement learning approaches. Our study provides insightful guidance on integrating MARL with single-robot locomotion.