In recent years, the advent of large language models (LLMs) has revolutionized the field of artificial intelligence, demonstrating exceptional language comprehension and reasoning abilities. These models offer promising solutions to complex challenges, particularly in the realm of Information Retrieval (IR), where accurately discerning a user's search intent from concise queries poses a persistent challenge. This study tries to tackle this issue by introducing an LLM Alignment via AI feedback pipeline. It begins with constructing an LLM-generated preference dataset. The dataset is then employed to align another Language Model dedicated to query reformulation, which involves expanding initial short queries into more detailed expressions that accurately reflect the user's search intent. This advanced query refinement approach would significantly enhances search systems' ability to satisfy users' information need effectively.