Skip to main content

Dataset Management

Bailian CLI command reference for dataset upload, validation, and management.

Manage datasets for model fine-tuning from the terminal: upload, validate locally, view, and delete.
Append --help to any command for full parameter details, e.g. bl dataset upload --help. Command details on this page are verified against bl 2.5.1.

Command reference

bl dataset upload

Upload a dataset file (.jsonl or .zip) to Bailian. Requires API Key authentication. Usage 
bl dataset upload --file <path> [--purpose <name>] [--schema <chatml|dpo|cpt|tts|image|video>] [--no-validate] [--full-validate]
ParameterTypeRequiredDescription
--file <path>stringyesLocal dataset file (.jsonl or .zip; ≤200MB SFT/DPO, ≤300MB CPT, ≤2GB media zip)
--purpose <name>stringnoDataset purpose tag (default: "fine-tune"; e.g. "evaluation")
--schema <s>stringnoRecord schema: "chatml" (SFT), "dpo" (chosen/rejected), "cpt" (raw text), "tts" (audio), "image" (image generation), or "video" (video generation). Default auto-detects per record.
--no-validateswitchnoSkip the local JSONL pre-flight check (not recommended)
--full-validateswitchnoJSON.parse every line instead of sampling (slower)
Example: Upload dataset file to Bailian 
bl dataset upload --file train.jsonl
  • Supports .jsonl (text) and .zip (audio/image/video archives with a data.jsonl manifest). Six record schemas are recognized: chatml = {messages:[...]} (SFT); dpo = {messages:[...], chosen, rejected}; cpt = {text:"..."} (continual pre-training, raw text); tts = {wav_fn:"train/xxx.wav", text:"..."} (audio fine-tuning); image = {img_path:"..."} (image generation); video = {first_frame_path:...} (video generation).
  • With no --schema, a record carrying wav_fn is validated as TTS, img_path as image, chosen/rejected as DPO, text (no messages) as CPT, otherwise ChatML.
  • Upload cap: 200MB SFT/DPO text, 300MB CPT, 2GB media zip. Upload uses the OpenAI-compatible /compatible-mode/v1/files endpoint so the purpose tag is persisted (the DashScope-native /api/v1/files drops it).

bl dataset validate

Locally validate a dataset file (.jsonl or .zip) without uploading. No authentication required. Usage 
bl dataset validate --file <path> [--full-validate] [--schema <chatml|dpo|cpt|tts|image|video>]
ParameterTypeRequiredDescription
--file <path>stringyesLocal dataset file (.jsonl or .zip)
--full-validateswitchnoJSON.parse every line instead of sampling (slower)
--schema <s>stringnoRecord schema: "chatml" (SFT), "dpo" (chosen/rejected), "cpt" (raw text), "tts" (audio), "image" (image generation), or "video" (video generation). Default auto-detects per record.
Example: Locally validate dataset file 
bl dataset validate --file train.jsonl
  • Default scan: every line gets a structural check, then ~160 lines (front 50, evenly spaced 100, last 10) are JSON.parsed against the active schema.
  • Schemas: chatml = {messages:[...]} (SFT); dpo = {messages:[...], chosen, rejected}; cpt = {text:"..."} (continual pre-training, raw text); tts = {wav_fn:"train/xxx.wav", text:"..."} (audio fine-tuning); image = {img_path:"..."} (image generation); video = {first_frame_path:"...", video_path:"..."} (video generation, i2v first-frame or kf2v first+last-frame with last_frame_path).
  • With no --schema, a record carrying wav_fn is validated as TTS, img_path as image, first_frame_path/video_path as video, chosen/rejected as DPO, text (no messages) as CPT, otherwise ChatML. Pass --schema to require a specific shape on every record.
  • ZIP archives (.zip) are validated structurally (data.jsonl present, media references resolve) in addition to per-record content checks. Use --full-validate to JSON.parse every line.

bl dataset list

List uploaded dataset files. Requires API Key authentication. Usage 
bl dataset list [--page <n>] [--page-size <n>] [--purpose <name>]
ParameterTypeRequiredDescription
--page <n>numbernoPage number (default: 1)
--page-size <n>numbernoResults per page (default: 10, max 100)
--purpose <name>stringnoFilter by purpose (e.g. "fine-tune", "evaluation"). Omit to list all.
Example: List datasets filtered by purpose 
bl dataset list --purpose fine-tune

bl dataset get

Get details of a single dataset file. Requires API Key authentication. Usage 
bl dataset get --file-id <id>
ParameterTypeRequiredDescription
--file-id <id>stringyesDataset file ID
Example: Retrieve dataset file details by ID 
bl dataset get --file-id file-xxx

bl dataset delete

Delete a dataset file by ID. Requires API Key authentication. Usage 
bl dataset delete --file-id <id>
ParameterTypeRequiredDescription
--file-id <id>stringyesDataset file ID
--yesswitchnoConfirm this high-risk operation
This permanently deletes the specified dataset file and cannot be undone.
Example: Delete a dataset file by ID 
bl dataset delete --file-id file-id-xxx