The tool-call ID exists. The training trace is still wrong.

A result points to call-17. You search the dataset and find call-17. Everything looks connected. The problem is that the call belongs to a different conversation.

That is a useful small check to make while agent post-training is getting more attention. Hugging Face's October 20 curriculum describes learning from agent traces and multi-turn tool interactions, not just isolated answers. The source is A curriculum for post-training agents. This walkthrough examines one illustrative record-linking rule; it does not implement that curriculum or claim to validate its trainers.

We built a nine-record mock example in JSONL, Parquet and Arrow IPC. Its lesson is independent of the reader: an identifier can be present and still refer to the wrong scope. You can run the reference without installing RowCove.

Disclosure: we are the RowCove team, developers of a commercial local data workspace. This article was prepared with AI assistance. All records are programmatically generated mock data. The reference below was executed with Python and PyArrow 23.0.1. There is no customer dataset or benchmark here.

Three traces, one misleading match

Each mock trace contains a question, a tool call and a tool result:

A and B deliberately reuse the same call ID. Our example allows that because they are different traces. C is different: its result points to an ID that was not declared earlier in C. A dataset-wide set of call IDs contains both call-17 and call-18, so it would accept every result in the table.

Under this contract, the lookup key is (trace_id, tool_call_id). The exact contract is important. A real system may require globally unique IDs, include nested runs or retries, or use another event format. Adapt the rule to the actual producer; do not silently turn this example into a universal standard.

Establish the scope before judging the record

For a useful investigation, retain three things together: the trace identity, the complete call ID and the record's source location. A shortened grid label or an ID copied without its trace can lead you to the wrong conversation.

In a visual workflow, inspect the full value, filter the intended source scope and keep the original source file and row. A first-page preview can help you understand the structure. It cannot establish that no matching call exists somewhere else. A collection page with a continuation cursor is likewise not proof that the collection has been exhausted.

The independent reference below scans all nine generated records for each format. The fixture order is explicit: calls precede their expected results. It is not inferred from arbitrary directory ordering. Real traces split across files need a defined ordering or a different reconciliation algorithm. Reporting an orphan before reading a later-arriving call can create a false alarm.

Run the mock check

Use a disposable Python environment with PyArrow installed. Save the following as check_agent_traces.py and run python check_agent_traces.py. It creates only temporary generated fixtures, opens them through each format's reader, checks results and confirms the source hashes remain unchanged.

"""Tiny illustrative trace contract, not a complete trainer validator or benchmark."""
import hashlib
import json
import tempfile
from pathlib import Path

import pyarrow as pa
import pyarrow.parquet as pq


def issues(events):
    calls = set()
    failures = []
    for source, row_number, event in events:
        key = (event["trace_id"], event["tool_call_id"])
        if event["event_kind"] == "call":
            calls.add(key)
        elif event["event_kind"] == "result" and key not in calls:
            failures.append({"source": source, "row": row_number,
                             "trace_id": key[0], "tool_call_id": key[1],
                             "issue": "no preceding call in this trace"})
    return failures


def run():
    schema = pa.schema([(k, pa.string()) for k in
                        ["trace_id", "event_kind", "tool_call_id", "content"]])
    rows_by_trace = []
    for name, call_id, result_id in [("mock-trace-A", "call-17", "call-17"),
                                     ("mock-trace-B", "call-17", "call-17"),
                                     ("mock-trace-C", "call-18", "call-17")]:
        rows_by_trace.append([
            {"trace_id": name, "event_kind": "user", "tool_call_id": None, "content": "Generated mock question."},
            {"trace_id": name, "event_kind": "call", "tool_call_id": call_id, "content": "Generated mock tool call."},
            {"trace_id": name, "event_kind": "result", "tool_call_id": result_id, "content": "Generated mock tool result."},
        ])
    report = {"mock_only": True, "pyarrow": pa.__version__, "formats": {}}
    with tempfile.TemporaryDirectory(prefix="rowcove-trace-lesson-") as tmp:
        root = Path(tmp)
        for fmt in ["jsonl", "parquet", "arrow"]:
            sources = []
            for i, rows in enumerate(rows_by_trace):
                path = root / f"mock-{i:02d}.{fmt}"
                table = pa.Table.from_pylist(rows, schema=schema)
                if fmt == "jsonl":
                    path.write_text("".join(json.dumps(r) + "\n" for r in rows), encoding="utf-8")
                elif fmt == "parquet":
                    pq.write_table(table, path)
                else:
                    with pa.OSFile(str(path), "wb") as sink:
                        with pa.ipc.new_file(sink, schema) as writer:
                            writer.write_table(table)
                sources.append(path)
            before = {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in sources}
            events = []
            for path in sources:  # Explicit mock chronology; not filesystem-order inference.
                if fmt == "jsonl":
                    rows = [json.loads(line) for line in path.read_text(encoding="utf-8").splitlines()]
                elif fmt == "parquet":
                    rows = pq.read_table(path).to_pylist()
                else:
                    with pa.memory_map(str(path), "r") as source:
                        rows = pa.ipc.open_file(source).read_all().to_pylist()
                events.extend((path.name, n, row) for n, row in enumerate(rows, 1))
            found = issues(events)
            assert len(events) == 9 and len(found) == 1
            assert found[0]["trace_id"] == "mock-trace-C" and found[0]["row"] == 3
            # A global-ID existence check incorrectly accepts trace C's result.
            all_ids = {r["tool_call_id"] for _, _, r in events if r["event_kind"] == "call"}
            assert all(r["tool_call_id"] in all_ids for _, _, r in events if r["event_kind"] == "result")
            assert before == {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in sources}
            report["formats"][fmt] = {"rows_checked": 9, "issues": found,
                                       "global_id_check_misses_error": True, "source_unchanged": True}
    assert issues([("mock", 1, {"trace_id": "A", "event_kind": "result", "tool_call_id": "x"})])
    return report


if __name__ == "__main__":
    print(json.dumps(run(), indent=2))

What should happen?

For each format, the result contains rows_checked: 9 and exactly one issue: mock-trace-C, row 3 of mock-02 in that format. Rows are one-based within the source file. The report also verifies that the simpler global-ID existence check misses the error. Both A and B pass; reusing an ID across them is not reported as an error under our stated contract.

The source hashes are checked after reading. This is a useful separation: inspection should not modify the evidence you are investigating. If you later correct a record, save a distinct artifact and reopen it to verify the intended value and source scope before giving it to another job.

Where the example stops

This deliberately small reference loads each mock file and retains preceding call identities in a Python set. Memory grows with the input and number of calls. It is not a bounded-memory validator for GB/TB corpora, and its tiny runtime says nothing about large-file performance.

It also does not check every schema variant, timestamp, retry, duplicate call, message role, tool argument, expected result count or trainer-specific rule. It does not execute tools, assess the truth of tool output, detect poisoned training data or guarantee model quality. A record-linking check and a model security evaluation answer different questions.

For a large source, budget the complete scan and any required ordering or external index. Include preparation, temporary storage and output costs when measuring a workflow. Do not compare a preview with a complete validation run.

Bring the finding back to the local record

RowCove can help with the visual part: open supported local JSONL, Parquet or Arrow sources, inspect complete values and retain source context while browsing supported results. Dataset-folder browsing is read-only. Supported individual source corrections can be staged and saved separately; native Parquet/Arrow edit-copy currently requires at most 128 MiB, 200,000 rows and 512 columns. The reference algorithm here is independent Python, not an automatic RowCove validator or a promised global cross-shard join.

Free Preview lets you inspect up to 100 read-only records without an account; supported full-file workflows require Pro. Start with the training-data workflow and format capabilities before choosing a larger task. Dataset processing is local. If you request results through an AI client using MCP, those results are intentionally shared with that client and may reach its model provider.

Count declarations before claiming a unique match

A technical response in the forum points out another failure mode: two declarations of the same scoped ID are collapsed by a set. The original snippet answers existence and scope; it does not retain multiplicity.

Under an explicit one-declaration-per-scoped-ID contract, distinguish no declaration, one declaration, and multiple declarations. A single declaration still has to precede the result. Repeated records can also represent valid retries or transport duplication, so use the producer's attempt/event identity or deduplication rule before judging them invalid.

The following extension checks the complete tiny mock scope in two passes. It reports orphan, unique preceding declaration, ambiguous declarations and out-of-order as separate outcomes. Its dictionaries and result list grow with the input; it is not a bounded-memory implementation for a large corpus.

"""Complete tiny mock scope, under an explicit one-declaration contract."""
from collections import defaultdict
import json


def classify(events):
    declarations = defaultdict(list)
    results = []
    for position, (trace, kind, call) in enumerate(events):
        key = (trace, call)
        if kind == "call":
            declarations[key].append(position)
        elif kind == "result":
            results.append((position, key))
    diagnostics = []
    for position, key in results:
        matches = declarations[key]
        if not matches:
            outcome = "orphan"
        elif len(matches) > 1:
            outcome = "ambiguous_declarations"
        elif matches[0] >= position:
            outcome = "out_of_order"
        else:
            outcome = "unique_preceding_declaration"
        diagnostics.append({"trace_id": key[0], "tool_call_id": key[1],
                            "declarations": len(matches), "outcome": outcome})
    return diagnostics


events = [
    ("mock-unique", "call", "call-17"),
    ("mock-unique", "result", "call-17"),
    ("mock-orphan", "result", "call-18"),
    ("mock-ambiguous", "call", "call-19"),
    ("mock-ambiguous", "call", "call-19"),
    ("mock-ambiguous", "result", "call-19"),
    ("mock-order", "result", "call-20"),
    ("mock-order", "call", "call-20"),
]
found = classify(events)
assert [row["outcome"] for row in found] == [
    "unique_preceding_declaration", "orphan", "ambiguous_declarations", "out_of_order"]
assert [row["declarations"] for row in found] == [1, 0, 2, 1]
# Same scoped ID twice is accepted by a set, which cannot retain multiplicity.
assert ("mock-ambiguous", "call-19") in {(t, c) for t, k, c in events if k == "call"}
if __name__ == "__main__":
    print(json.dumps({"mock_only": True, "complete_fixture_scanned": True,
                      "one_declaration_contract": True, "results": found}, indent=2))

Questions to ask of your own pipeline

Is the ID unique globally or only within a trace? Use the producer's contract. This example intentionally permits reuse between traces.

Does a clean schema prove the relationship is correct? No. All mock rows have the same field types; the error is in the relationship between values.

Can a clean preview validate the tail? No. Preserve the exact scope checked, including whether every relevant file and record was read.

Should you repair an orphan automatically? First establish the source and intended relationship. An ID that looks plausible is not evidence of the correct replacement.