Skip to main content

Add Trajectory Evaluation Data

Trajectory data structure and how to add data to trajectory evaluation datasets

A trajectory is structured, time-series data generated by an AI Agent during task execution. It fully records the agent's thinking, actions, and observations across multi-turn interactions, starting from the user instruction. Trajectory evaluation assesses the reliability of the agent's execution process—for example, whether steps are logically coherent and whether irrelevant or redundant steps exist—complementing evaluation of the final output alone.

What is a trajectory

Trajectory data is in JSON format. Its core fields include:
FieldDescription
schema_versionTrajectory schema version
session_idSession ID
trajectory_idTrajectory ID
agentInformation about the agent that executed the trajectory (name, version, etc.)
stepsList of steps. Each step records the source (user or agent), message, timestamp, and execution details (Span ID, duration, token consumption, etc.)
extraTrajectory-level execution details: Span ID, Span kind, status, duration, semantic profile (semantic_profile), and input/output details. agent, steps, and final_metrics each have their own extra that carries additional information
final_metricsAggregated metrics, including total steps, token consumption, tool call count, and first-token latency
Trajectory data example (excerpt): 
{
  "schema_version": "ATIF-v1.8",
  "session_id": "0efe2dc279b09aaf9d51b378af01f520",
  "trajectory_id": "0efe2dc279b09aaf9d51b378af01f520:f4f2aa362afe7ae7",
  "agent": {
    "name": "invoke_agent_app",
    "version": "2"
  },
  "steps": [
    {
      "source": "user",
      "message": "Hello",
      "timestamp": "2026-10-13T02:11:36.660Z",
      "step_id": 1
    },
    {
      "source": "agent",
      "message": "Hello! How can I help you?",
      "timestamp": "2026-10-13T02:11:36.660Z",
      "step_id": 2
    }
  ],
  "extra": {
    "span_kind": "CHAIN",
    "operation_type": "invoke_agent_app",
    "duration_ms": 4499,
    "status": "1",
    "semantic_profile": "bailian.trace-atif/v1"
  },
  "final_metrics": {
    "total_steps": 2,
    "total_prompt_tokens": 4984,
    "total_completion_tokens": 52
  }
}
When adding a single trajectory record, click Trajectory Sample to insert a complete sample trajectory as a reference template for manual entry.

Creating a trajectory evaluation dataset

When creating an evaluation dataset, select Trajectory as the scenario. The default table schema is generated automatically:
ColumnData typeRequiredDescription
inputStringYesUser input
reference_outputStringNoReference output
trajectoryTrajectoryYesAgent execution trajectory
You can continue editing the column settings on top of the default schema. For detailed steps, see Create and Manage Datasets.

Adding trajectory data

Adding a single record

On the dataset details page, click Add Data > Add Single Record, then fill in the fields in the dialog:
  • input (String, required): user input, up to 10,000 characters.
  • reference_output (String, optional): reference output, up to 10,000 characters.
  • trajectory (Trajectory, required): agent execution trajectory.
The trajectory field is edited in a code editor that supports folding, expanding, and copying. The following actions are available above the trajectory field:
  • Trajectory Sample: Insert a sample trajectory JSON. If trajectory content already exists, inserting the sample overwrites the current content and requires confirmation in a dialog.
  • Import by Trace ID: Enter a Trace ID to import the trajectory data of that Trace. Make sure the Trace is currently valid.
  • Preview: Render the current trajectory data as a Trace view, showing the call chain, inputs and outputs, status, duration, and token consumption.
Trajectory data must be a valid JSON object, and a single trajectory cannot exceed 1 MB; otherwise it cannot be saved.

Batch import

On the dataset details page, click Add Data > Batch Import:
  1. Upload a local file. Supported formats: .xls, .xlsx, and .csv. Maximum file size: 20 MB.
  2. Select the import mode: Append Data (keeps existing data and appends new data to the end) or Full Overwrite (clears existing data before writing).
  3. Configure column mapping: map the data columns in the file to the dataset columns, and verify that the field types match.
If some records fail during import, download the log from the import result to check the failure reasons.

Backfilling evaluation data from Traces

Traces in Application Observability can be exported directly as trajectory evaluation data: select multiple Traces in the Trace list, configure the mapping between Trace fields and evaluation dataset fields, and import them into the dataset. See "Add to Evaluation Set" in Application Observability.