Host unstructured documents such as PDF, Word, and Markdown, organize them with categories, and import them for agents to search and reference.
The File connector manages unstructured documents. After a file is uploaded, the platform parses it into a searchable format, and an agent uses the automatically generated tools to find files by keyword and obtain download links.
It requires no credentials from any external system, which makes it the best App to start with.
The description guides an agent when it decides whether to call this connector. We recommend that you state the data content and its purpose, for example, product manuals and release notes, to be referenced when answering questions about product features.
After the connector is created, 2 tools are generated automatically:
Expand a tool in the Available tools section of the App details page to view these parameters.
The platform supports common document formats, including PDF, Word, Markdown, and TXT.
A category is the unit that organizes files inside a connector, and every file belongs to one. Categories support multi-level nesting, so you can divide them by business line, document type, or time period. The Table connector organizes its data with categories as well.
Files with no category specified go into the default category.
On the file management page of the connector, click the create icon in the category area on the left and enter a category name. When you create one under an existing category, the new category becomes its child.
How you divide categories directly affects both the retrieval scope and the maintenance cost later. A few guidelines:
Perform the delete action on the category.
Tools can only find content after files are imported. Create or select a category before you import.
If the files are already in OSS, there is no need to download and re-upload them.
Prerequisites:
Tags apply a first layer of filtering during retrieval, narrowing the scope to improve efficiency.
Specify tags at retrieval time, and the application filters files by tag first, then searches within the filtered results.
For files that are already imported, you can add or replace tags, which supports two modes:
After import, a file moves through the following statuses:
The file list on the connector detail page shows the current status of every file.
During peak request hours, parsing may take several hours. Under high concurrency it can take even longer because of resource queuing, and occasional parsing timeouts occur.
Recommendations:
Create a connector
1
Open the connection dialog
On the Apps page, find the File connector card and click Connect.
2
Enter the basic information
| Field | Required | Description |
|---|---|---|
| Connector name | Yes | Use a name that is easy to recognize. Up to 64 characters |
| Connector description | No | Describes which documents are stored and in what scenarios they are used |
3
Confirm the storage location
Storage location currently offers only one option, Use platform storage. Select it and continue. Data is stored in the storage space provided by Alibaba Cloud Model Studio, with a quota of up to 200,000 files and 1 TB, free for a limited time. For details, see Quotas and Limits.
4
Confirm creation
Click Confirm. After creation, the connector type cannot be changed.
Automatically generated tools
After the connector is created, 2 tools are generated automatically:
| Tool | Function | Input parameters |
|---|---|---|
| Search files | Queries the file list by keywords in file titles and returns download links for the files | keyWord (string, required): a keyword from the file titlemaxCount (integer, optional): the maximum number of file download links to return. Default 5, max 10 |
| Get file | Retrieves a file by file ID and returns the download link for the file | fileId (string, required): the file ID |
Search files matches keywords in titles and does not perform semantic retrieval over document body text. When you need to derive an answer from document content, import the files into a knowledge base and use knowledge retrieval.
Supported file formats
The platform supports common document formats, including PDF, Word, Markdown, and TXT.
Categories
A category is the unit that organizes files inside a connector, and every file belongs to one. Categories support multi-level nesting, so you can divide them by business line, document type, or time period. The Table connector organizes its data with categories as well.
Files with no category specified go into the default category.
Create a category
On the file management page of the connector, click the create icon in the category area on the left and enter a category name. When you create one under an existing category, the new category becomes its child.
Each workspace can create at most 500 categories. If you need more, submit a ticket to request a quota increase.
How to divide categories
How you divide categories directly affects both the retrieval scope and the maintenance cost later. A few guidelines:
| Situation | Recommendation |
|---|---|
| Content types differ a lot | Divide by type, for example product manuals, internal policies, and meeting notes |
| Queries need to be isolated per team | Divide by team and limit the category scope during retrieval |
| Content keeps growing | Layer by time, for example using the year as the top-level category |
Categories are mainly for organization and scoping, and they do not carry access control. When you need to isolate data access, use separate workspaces.
Delete a category
Perform the delete action on the category.
Import data
Tools can only find content after files are imported. Create or select a category before you import.
Local upload
1
Open the file management page
Click the connector card to open its details, then open the file management page.
2
Choose a category
Select an existing category in the category area on the left, or create a new one.
3
Open the import dialog
Click Import data, choose local upload as the import method, and then select the files to upload.
4
Choose a parsing method
You can use the default settings, or customize them and configure parsing rules per format to improve parsing quality.
5
Configure tags
Optionally configure tags for the files. Tags are used later to limit the retrieval scope — see Tags below.
6
Confirm the import
Click Confirm. The system starts parsing and importing, and you can watch the task progress on the page.
Batch import from OSS
If the files are already in OSS, there is no need to download and re-upload them.
Prerequisites:
- Service-linked role authorization is complete.
- The target bucket has the tag
bailian-datahub-accesswith the valueread.
my-bucket/docs/ means all files under that directory, and my-bucket/docs/foo.md means a single file.
The OSS import here stores a copy of each file on the platform. When you want an agent to read the bucket in real time without creating copies, use the OSS connector instead.
Tags
Tags apply a first layer of filtering during retrieval, narrowing the scope to improve efficiency.
| Constraint | Value |
|---|---|
| Number of tags per file | At most 100 |
| Length of a single tag | No more than 32 characters |
| Total length of all tags | No more than 700 characters |
| Mode | Behavior |
|---|---|
OVERWRITE | Replaces all existing tags |
APPEND | Appends to the existing tags |
File status
After import, a file moves through the following statuses:
The file list on the connector detail page shows the current status of every file.
Parsing time
During peak request hours, parsing may take several hours. Under high concurrency it can take even longer because of resource queuing, and occasional parsing timeouts occur.
Recommendations:
- Wait patiently or retry later, and do not resubmit the same batch of files repeatedly.
- Schedule large imports outside peak hours whenever possible.
- Check the file status on the connector detail page. When parsing fails, the page shows the reason for the failure.
Considerations
- An imported file is stored as an independent copy and is not linked to the original data. Modifying the original file does not sync automatically.
- Only files imported within the last 90 days can be viewed. Beyond that time range, files are no longer displayed, but they are not deleted.
- Imported files are available only to users in the current workspace.