Skip to main content

Dataset

RAG datasets (formerly Data Connections) provide a unified entry point for enterprise databases, document systems, and object storage

This page describes RAG datasets (formerly Data Connections). The new Connector for agent scenarios supports more systems and exposes them to agents over MCP. Legacy data connections must be migrated before September 30, 2026 — see Data Connection Migration.
A dataset is the unified entry point for managing external data sources in RAG. By creating a dataset, applications can securely access data stored in enterprise databases, document systems, and object storage—and query or reference that data in real time during conversations.

Dataset Types

Datasets are categorized into two types based on how data is stored and accessed: Platform-Hosted and Streaming.
Access PatternDatasetData Storage MethodUse Cases
Platform-HostedFilesStored on platformUpload and manage unstructured documents (PDF, Word, Markdown, etc.)
Platform-HostedTablesStored on platformImport and query structured tabular data (CSV, Excel, etc.)
StreamingMySQLData remains in original database; accessed in real timeConnect to a MySQL database and execute SQL queries
StreamingPostgreSQLData remains in original database; accessed in real timeConnect to a PostgreSQL database and execute SQL queries
StreamingPolarDB-X 2.0Data remains in original database; accessed in real timeConnect to Alibaba Cloud PolarDB-X 2.0 distributed database

Prerequisites

  • Account Permissions: Alibaba Cloud primary account or RAM user with dataset management permissions.
  • Data Source Preparation (by dataset type):
DatasetRequired Preparation
Files / TablesFiles ready for upload
MySQLA MySQL database instance (Alibaba Cloud RDS or self-managed), accessible over public or private network
PostgreSQLA PostgreSQL database instance where the wal_level parameter is set to logical
PolarDB-X 2.0An Alibaba Cloud PolarDB-X 2.0 instance located in a region that supports private network access

Creating a Dataset

Navigate to the console at Data Ingestion → Dataset, click Create Dataset, select the dataset type, and complete the configuration as described below.
  • Files
  • Tables
  • MySQL
  • PostgreSQL
  • PolarDB-X 2.0
The Files dataset manages unstructured documents (e.g., PDF, Word).
  1. Select Files as the dataset type.
  2. Enter a Dataset Name and Description. The description helps improve application invocation accuracy—include details about data content and intended usage.
  3. Under Storage Location, select Use Platform Storage: Data is stored in platform-provided storage space, with a limit of up to 100,000 files or 1 TB. Free tier available for limited time.
After completing all fields, click Confirm to finish creation.

Importing Data

After creating a dataset, import data into it.

Files Dataset

  1. From the Dataset list page, click Details for the target dataset to open its detail page, then navigate to the Files tab.
  2. In the left-hand Category tree, select an existing category or click + to create a new one.
  3. Click Import Data, choose Local Import, and upload files.
  4. Select a Parsing Method (default or custom); the system automatically parses uploaded files.

Database Datasets

For database datasets (MySQL / PostgreSQL / PolarDB-X 2.0), data remains in the original database—no manual import is required. Once the dataset is created, data can be queried in real time directly from applications.

Data Import API

In addition to console operations, you can manage data imports programmatically using APIs. See API Reference — Data Import.
The Data Import API (/api/v1/connector/dash/) uses the same workspace domain and authentication method as other APIs. For details, see API Overview.
After configuring your dataset, proceed to Create a Knowledge Base to build searchable indexes from the connected data.