This guide walks you through quickly building a private-knowledge Q&A application using Alibaba Cloud Bailian’s agent application development capabilities.
Large language models (LLMs) cannot directly answer questions about proprietary or domain-specific knowledge. However, with Alibaba Cloud Bailian’s agent application development capabilities and your own private knowledge documents, you can build an LLM-powered Q&A application that answers domain-specific questions—no coding required.
An LLM’s knowledge comes solely from its training data and does not include any private, domain-specific, or up-to-date information. As a result, asking it directly about specific products or internal knowledge often yields inaccurate or hallucinated responses.
Retrieval-Augmented Generation (RAG) is a technique that integrates external knowledge bases with large language models. It retrieves relevant information and feeds it to the model, enabling answers grounded in your proprietary knowledge. Its core workflow is as follows:
You’ve now mastered the fundamentals of building a basic RAG application. Next steps include:
Q: Why are answers inaccurate—or why isn’t the application using knowledge from my knowledge base?
A: This may stem from multiple points in the RAG pipeline:
A: Enable the Intervene Agent Responses feature and configure a fallback reply for cases where no knowledge base results are found.
Why Can’t LLMs Answer Private Knowledge Questions?
Core Concept
An LLM’s knowledge comes solely from its training data and does not include any private, domain-specific, or up-to-date information. As a result, asking it directly about specific products or internal knowledge often yields inaccurate or hallucinated responses.
Solution: RAG (Retrieval-Augmented Generation)
Retrieval-Augmented Generation (RAG) is a technique that integrates external knowledge bases with large language models. It retrieves relevant information and feeds it to the model, enabling answers grounded in your proprietary knowledge. Its core workflow is as follows:
- Data Processing & Vectorization: Uploaded documents are automatically split into chunks. An embedding model converts each chunk into a mathematical vector, which is stored in a dedicated vector database.
- Retrieval: When a question arrives, the system vectorizes it and retrieves the Top-k most semantically similar document chunks from the vector database.
- Generation: The original question and retrieved document snippets are combined into a prompt and submitted to the LLM, which then generates the final answer.
Effect Comparison
| Q&A App Without RAG | Q&A App With RAG |
|---|---|
| Without a dedicated knowledge base, the LLM cannot accurately answer questions like “What phones does Alibaba Cloud Bailian offer?” | With a proprietary knowledge base, the LLM can accurately answer questions about “Alibaba Cloud Bailian phones.” |
Building a Private-Knowledge Q&A Application
Creating an Agent
1
Create a Blank Application
Go to the Application Management page. Click Create Application, then select the Agent Application tab and click Create Now.
2
Select a Model
Choose a large language model for your application. We recommend Qwen-Max.
Using LLMs incurs charges. Alibaba Cloud Bailian offers limited-time free quotas. To check your remaining free quota, go to the Model Marketplace and click View Details on any model card.
3
Design the Prompt
Also known as the System Prompt, this defines the agent’s role and responsibilities. We recommend setting the prompt to:“You are an Alibaba Cloud Bailian phone shopping assistant. Your task is to help customers compare phone specifications, analyze their needs, and provide personalized recommendations.”
4
Configure Welcome Message and Sample Questions
You can set a welcome message and predefine common questions to jumpstart conversations.
- Welcome message: “Hi! I’m your Alibaba Cloud Bailian phone shopping assistant—I’m here to help you find the perfect Bailian phone.”
- Sample Question 1: “What phone models does Alibaba Cloud Bailian offer?”
- Sample Question 2: “Please recommend the best-value phone for me.”
- Sample Question 3: “Help me pick the best camera phone from Alibaba Cloud Bailian under ¥3,000.”
5
Test the Application
Click any sample question on the right-hand side to start a test conversation.
Because no “Alibaba Cloud Bailian phones” knowledge has been added yet, the Bailian application’s responses will be generic—or even fabricated. Next, we’ll walk you through configuring a knowledge base.
Creating a Knowledge Base
1
Access the Knowledge Base Page
Go to the Knowledge Base page and click Create Knowledge Base to build your custom knowledge base.
2
Enter a Knowledge Base Name
Enter a Knowledge Base Name, then click Next.
In this tutorial, we name the knowledge base “Alibaba Cloud Bailian Phones.” The Create Knowledge Base wizard uses default configurations, which you do not need to modify for this tutorial.
3
Select Category
Under Select Category, check Default Category, then click Next.
Here, you specify the source documents for your knowledge base. The wizard uses default settings—no changes needed for this tutorial.
4
Choose Chunking Strategy
Select Smart Chunking, then click Import Complete to successfully create the knowledge base.
“Smart Chunking” is a built-in strategy optimized for retrieval performance across most document types.
5
Wait for Parsing to Complete
Depending on your document size, Alibaba Cloud Bailian requires ~1–2 minutes to parse and index content. Please wait patiently.
Attaching the Knowledge Base to Your Agent
1
Enter Application Configuration
Go to the Application Management page. Locate your “Alibaba Cloud Bailian Phone Shopping Assistant” application and click Configure to open the configuration interface.
2
Add the Knowledge Base
Click the + button next to Knowledge > Documents, then select your newly created knowledge base to complete the attachment.
Testing and Publishing
1
Test Q&A Performance
Click any sample question on the right—for example: “Help me pick the best camera phone from Alibaba Cloud Bailian under ¥3,000.”With RAG enabled, the Bailian application now delivers accurate, knowledge-grounded answers to your shopping queries.
2
Publish the Application
Once satisfied with the behavior, click Publish to deploy your application.
Best Practices
You’ve now mastered the fundamentals of building a basic RAG application. Next steps include:
- Optimize RAG Performance: Learn how to fine-tune RAG at every stage—query rewriting, retrieval, recall, and re-ranking. See Knowledge Base Optimization.
- Add External Tools: Extend your agent’s capabilities by integrating plugins, MCP tools, and more.
- Build Advanced Applications: Use Workflow Applications for complex process orchestration, or Pro-Code Applications to build sophisticated AI services programmatically.
- External Integration: Integrate your application into business systems via the DashScope SDK or HTTP APIs.
Frequently Asked Questions
Q: Why are answers inaccurate—or why isn’t the application using knowledge from my knowledge base?A: This may stem from multiple points in the RAG pipeline:
- Document chunking breaks semantic coherence.
- The embedding model fails to capture core meaning accurately.
- Retrieval misses the most relevant document chunks.
- Use well-structured documents (e.g., Markdown, JSON).
- In knowledge base configuration, switch to Custom Mode, experiment with different embedding models and ranking settings to improve response quality.
A: Enable the Intervene Agent Responses feature and configure a fallback reply for cases where no knowledge base results are found.