Skip to main content
Prompt

Prompt Feedback Optimization

Based on your provided input-output examples, Prompt Feedback Optimization learns effective patterns from those examples and automatically generates a refined prompt aligned with your expectations.

Effect Demonstration

Prompt Before Optimization The original prompt instructs the model to classify automotive-related articles into six categories — Product Analysis, Dealer Sales Promotion, Classic Nostalgia, Quality Complaints, Sales Performance, or Other — and output the result in JSON format: {"type":"<category>"}. Prompt After Optimization The optimized prompt consists of three parts:
  1. Original Prompt (the original classification instruction)
  2. Few-shot Examples (three input-output pairs), e.g., "ID3 sales article → Dealer Sales Promotion"
  3. Content Hints (additional clarifications on category boundaries)
Inference Result Comparison:
  • Before Optimization (Incorrect): Input a text discussing the relationship between quality complaints and sales performance for a certain automotive brand. Model output: {"type":"Quality Complaints"}. Status: Completed. Input Tokens: 265, Output Tokens: 6.
  • After Optimization (Correct): The model classifies the same text, outputting {"type": "Sales Performance"}. Status: Completed. Word Count: 16, Input Tokens: 1152, Output Tokens: 7.

Feature Overview

Prompt Feedback Optimization performs the following:
  • Injects example data directly into the prompt.
  • Conducts multi-round automated evaluation, reflection, and optimization of prompt behavior (based on outputs generated by the inference model), ultimately synthesizing improved content instructions.
Compared to generic prompt engineering, Prompt Feedback Optimization uses your provided examples as the ground-truth evaluation standard, significantly improving response quality in your actual use cases.
  • Inference Model: We recommend using Qwen-Max.
  • Training Examples: Alibaba Cloud Bailian automatically selects a subset of your example dataset to inject into the prompt. We recommend including 5-10 examples, ensuring at least one example per scenario.
  • Evaluation Dataset: Bailian evaluates prompt performance across both your evaluation dataset and the chosen inference model. We recommend at least 20 evaluation examples — more data generally yields better prompt optimization.

Practical Case Study

Background

To improve content management efficiency for an automotive forum, you plan to deploy a large language model (LLM) to classify forum posts into predefined categories. The classification schema is defined as follows: 
Which of the following categories does the following automotive-related article belong to?
"Product Analysis",
"Dealer Sales Promotion",
"Classic Nostalgia",
"Quality Complaints",
"Sales Performance",
"Other".

Output the result strictly in JSON format: {"type":"<category>"}
When you use the above instruction as your initial prompt, the model fails to classify accurately. For instance, the following article should be classified as "Sales Performance", but the model misclassifies it: After some experimentation, you determine that systematic prompt engineering is the best path forward. To improve classification accuracy, you manually annotate representative articles. Below are your labeled examples:

Manually Annotated Examples

Training Examples:
queryanswer
Title: A certain brand's vehicle chassis produces abnormal noise; owner claims there's a "duck" quacking inside.Body: A certain brand's vehicle chassis produces abnormal noise; owner claims there's a "duck" quacking inside. This is a video summary: A certain brand vehicle owner reports unusual "duck-like" noises during driving. Inspection reveals the issue stems from misaligned lower control arm bushings. Previously, the owner replaced the assembly at a third-party repair shop using non-OEM parts; mismatched hole diameters caused persistent issues. The repair team recommends replacing with OEM parts to ensure lasting resolution and avoid repeated repairs. They emphasize avoiding short-term cost savings that lead to wasted time and money — advocating against unnecessary rework.{"type": "Quality Complaints"}
Title: Certain Brand Vehicle Earns Full "Excellent" Rating in C-IASI Safety Index — Your Safety, Guaranteed!Body: Certain Brand Vehicle achieves full "excellent" ratings in the China Insurance Automotive Safety Institute (C-IASI) safety index assessment, validating its outstanding safety performance. With superior safety configurations and stable mechanical performance, Certain Brand Vehicle has become drivers' top choice for peace of mind. Choose Certain Brand Vehicle — let safety be your standard feature!{"type": "Product Analysis"}
Title: Spotting a Certain Model Car in Real Life Is Rare.Body: The Certain Model is currently the most expensive sedan offered by its brand, yet its visibility on city roads sometimes lags behind older million-RMB-tier models. Yesterday while washing my car, I spotted one — initially mistaking it for another model due to its unusually wide front end. Upon checking the rear, I noticed letters beneath the logo — confirming it was indeed the Certain Model. Yet no matter how closely I examined it, I couldn't see what justifies its price tag — it even looks less premium than the Starry Sky Edition of another model. No wonder only a handful sell annually.{"type": "Other"}
Title: Certain Model Car — No Beating Around the Bush! Surprises Await Below!!!!Body: I admit — I still can't let you go. At month-end, not a single Certain Model sold — my manager came knocking again. "Only one unit left. No joke — this affects my career."{"type": "Dealer Sales Promotion"}
Title: Is Certain Model Car Inventory Overflowing? Dealers: Clearance Sale — Drive Away for 15,000?!Body: In the late 1990s, Certain Brand pioneered joint-venture vehicles in China, building a solid foundation with early models. These models gained traction not only due to their early entry into the Chinese market but also because they represented luxury and scarcity — symbols of rising living standards. Today, classic models remain available at much lower prices — reflecting enduring consumer recognition and historical brand equity.{"type": "Classic Nostalgia"}
Evaluation Examples:
queryanswer
Title: Joint Venture — Certain Brand & Certain Group Launch ISO 9001 Project.Body: In [Month] [Year], Certain Consultant initiated the ISO 9001 Quality Management System consulting project for Certain Brand & Certain Group's joint venture — Certain Automobile (Beijing) Technology Co., Ltd.{"type": "Other"}
Title: Certain Model Car{"type": "Other"}
Title: #Which Is Better: Certain Model Car or Another Model Car?# Comfort vs. Features — It's Personal.{"type": "Other"}
Title: After One Week of Careful Research and Comparison, I Decided to Purchase the 2024 Certain Brand 5-Seat 2.0T 330 Model.Body: After one week of thorough research and comparison, I chose the 2024 Certain Brand 5-seat 2.0T 330 model. Its position in my mind was secured by outstanding performance and competitive pricing.{"type": "Product Analysis"}
Title: Certain Brand 2022 A7L 2.0TFSI 45TFSI S-line White Sorcerer Edition Full-Vehicle Wrap{"type": "Other"}
You decide to leverage Alibaba Cloud Bailian's Prompt Feedback Optimization feature, combining your manually annotated examples to generate a production-ready prompt.

Step-by-Step Workflow

On the Alibaba Cloud Bailian Prompt Engineering page, click Feedback Optimization.
  1. Step 1: Select the Inference Model. Bailian will conduct multi-round prompt evaluation and refinement on this model.
  2. Step 2: Enter the Prompt to optimize (simply describe your task objective).
  3. Step 3 (Optional): Select training examples. Supports Direct Upload and Select from Example Library.
    • Example data will be injected into the optimized prompt. Recommended: 5-10 examples, covering all categories.
    • Training examples used in this case: sample.xlsx.
  4. Step 4: Upload the Evaluation Dataset.
    • This dataset serves as the benchmark for selecting the optimal prompt.
    • Recommended: at least 20 examples — more data improves optimization quality.
    • Evaluation data used in this case: evaluation.xlsx.
  5. Step 5: Click Start Optimization.

Using the Optimized Prompt

Once complete, you may save the optimized prompt as a reusable Prompt Template, or directly create an AI Agent application based on it.
Overview
Getting Started
Applications
Prompt
Plugins
Task Center
Components
Invocation
Permissions
Changelog