Large Language Models (LLMs) face accuracy and complex reasoning challenges in specialized medical domains like orthopedics. We introduce OrthoGraphRAG, a multi-level Graph Retrieval-Augmented Generation (GraphRAG) framework, to address these issues. OrthoGraphRAG constructs a novel multi-level knowledge graph linking private clinical knowledge with public UMLS data, building on recent medical GraphRAG advancements. The framework retrieves query-entity-based subgraphs, augments them with clinical note text, allowing an LLM to synthesize informed responses from combined graph and textual evidence. Evaluated on real-world orthopedic clinic letters with diverse query complexities, OrthoGraphRAG demonstrated effectiveness, particularly in contextual reasoning integrating private patient data with broader medical knowledge. This multi-level GraphRAG approach offers a promising path to safer, more capable, and contextually aware LLMs for specialized clinical applications. Our code is released at: https://github.com/venkateshtata/OrthoGraphRAG.