Understand how RAG, chunking, embedding, indexing, and retrieval work
What Is RAG?
RAG (Retrieval-Augmented Generation) is a technique that integrates a retrieval system with a large language model (LLM). When a user submits a query, the system first retrieves relevant text snippets from a knowledge base, then feeds both the retrieved snippets and the original query into the LLM to generate a grounded, evidence-based response.
Processing Pipeline
| Stage | Purpose |
|---|---|
| Parsing | Identifies headings, paragraphs, tables, images, and formulas in content and stores them in structured format |
| Chunking | Splits content into small, searchable units—typically 256–4096 tokens |
| Embedding | Converts text into vector representations for semantic matching |
| Indexing | Stores chunks and their embeddings to enable fast search |
| Retrieval | Returns the top-K most relevant chunks given a query |
| Reranking | Reorders initial retrieval results to improve relevance |
| Generation | The LLM generates the final natural-language answer using retrieved chunks as context |
Knowledge Base Types
| Type | Suitable Data | Retrieval Method | Typical Use Cases |
|---|---|---|---|
| Document Knowledge Base | PDF / Word / Markdown / HTML | Hybrid vector + keyword search | Product manuals, contract templates, internal wikis |
| Data Knowledge Base | CSV / Excel / RDS tables | Natural language → SQL translation | Order data, inventory reports |
| Image Knowledge Base | Product photos / design mockups / posters | Reverse image search, multimodal Q&A | Visual search, image-based question answering |
| Audio/Video Knowledge Base | Meeting recordings / training courses | Speech-to-text followed by text retrieval | Video content search |
Key Terminology
| Term | Description |
|---|---|
| Knowledge Base | A logical collection of documents sharing the same chunking, embedding, and indexing configurations |
| Document | Original content uploaded or synced into the knowledge base—e.g., files, web pages, or database records |
| Chunk | The smallest retrievable unit after document splitting |
| Top-K | Number of chunks returned by retrieval; common values are 3–10 |
| Relevance Score | Semantic similarity between a retrieved chunk and the query, ranging from 0 to 1 |
| Data Connector | Integration channel for external data sources (e.g., OSS, MySQL, Yuque) |
| Retrieval Service | Configuration unit enabling federated search across multiple knowledge bases, with independent parameter tuning |
| Q&A Service | Adds LLM-powered generation on top of retrieval, returning natural-language answers with citations |
| Business Space | Resource isolation unit—multiple spaces can be created under one account |
Retrieval Modes
| Mode | How It Works | Use Cases |
|---|---|---|
| Ultra-Fast Mode | Single-round direct retrieval; lowest latency | Simple factual questions |
| Multi-Turn Intelligent Mode | LLM-driven iterative search planning with automatic query rewriting | Complex questions, multi-step reasoning |