Windows Preview available. Pro checkout is not open yet. Plans and capability details →
RowCoveAccount ↗

AI / Pretraining data

Inspect a corpus
one useful slice at a time.

A training corpus is usually more than one file. Keep split, shard and record provenance explicit so the next expensive operation uses the data you intended.

How should I browse a large pretraining dataset?

Open a supported shard folder, choose the intended split and recognized partitions, project a few useful fields and inspect bounded pages. Follow continuation cursors and trace suspicious records back to their source file.

1. Establish the dataset layout

Start with the dataset version and split: training, validation or test. Confirm whether the source is a JSONL stream, Parquet/Arrow shards or a compressed collection. Use the visible file/folder selection in Open data; inspect detected capabilities before reading.

Do not choose the parent of several unrelated dataset versions simply because it contains all the files. A narrower root reduces discovery work and makes the source scope easier to explain.

corpus-v3/
  split=train/
    language=en/part-0001.parquet
    language=en/part-0002.parquet
  split=validation/
    language=en/part-0001.parquet

Illustrative Hive-style layout. Partition pruning depends on the layout and reader actually recognized by RowCove.

2. Project the fields needed for the question

For a text-record investigation, a useful starting set might be an ID, text and source metadata. Confirm actual names and types in the schema. Avoid decoding every auxiliary field merely to answer one question.

Inspect complete text and nested metadata in the record inspector. A page describes observed records, not a complete schema reconciliation across all shards. Investigate missing or changed fields before treating them as uniform.

3. Continue a bounded scan correctly

Choose a supported partition scope first. Read one bounded page and retain its source-file context. If a continuation cursor is returned, continue with it—even if the current page has no matches. A scan may stop at a byte budget before reaching the next matching record.

Use visible progress and cancellation for longer operations. A bounded collection scan does not imply global sorting, a corpus-wide aggregate or an exact full row count.

4. Keep a reproducible finding

Record the dataset version, split, selected fields, predicates and source locator with the issue. Recheck the source record before handing a correction to the training pipeline. Supported single-file editing/export is a separate workflow; a shard-folder scan is not a corpus rewrite command.

For compressed sources, plan scratch disk space. For conversations and preference data, continue with the post-training guide. The format guide distinguishes table inspection from tensor and specialist decoding.

Try the first page on your own data.

Start with the free Windows Preview: source information and up to 100 records, read-only. Review reader capabilities before planning a full-file Pro workflow.

Pro checkout is being prepared. The initial Windows install enables signed in-app updates for later versions.