Windows Preview available. Pro checkout is not open yet. Plans and capability details →
RowCoveAccount ↗

Scale / Access planning

A terabyte is a scope.
Choose a useful slice.

Before scanning a very large collection, decide which files, fields and records can answer the question. Storage layout and access pattern matter as much as total size.

Has RowCove demonstrated a TB-scale performance benchmark?

Not yet. GB-scale workflows have documented acceptance, while TB-scale throughput and memory limits remain unverified. Use a representative subset to measure first-page latency, scan cost, cancellation and scratch requirements before a corpus-wide job.

1. Distinguish one stream from partitioned data

A terabyte of date/symbol-partitioned Parquet files has a different access pattern from a terabyte compressed into sequential text. Discover the actual source layout before deciding how to browse it.

For supported JSONL, Parquet and Arrow collections, select a narrow dataset root and recognized partition predicates. Begin with a training split, date interval, language or symbol scope that corresponds to the question. See pretraining shards and columnar dataset inspection.

2. Budget fields, records and query work

Project only the fields you need. Start with bounded pages and preserve continuation state. Nested text, very wide rows and unusually large cells can make a small row count expensive.

A bounded renderer does not prove a fixed total memory limit. Indexing, sorting and query preparation have separate costs, and a bounded cross-shard scan is not a global aggregate or sort. Measure the actual operation you intend to use.

3. Allow for decompression and output copies

Supported compressed-text workflows can require a full streaming decompression into a disk-backed cache. The output may be much larger than the compressed input. Choose a suitable destination and an output budget; monitor progress and cancel if the budget or source is wrong.

Allow separate space for the prepared source, query scratch and any exported copy. Preserve original data. Do not begin a corpus-wide rewrite just to answer a narrow inspection question.

4. Expand only after a representative measurement

  1. Record hardware, storage, file format, projection and source scope.
  2. Measure time to a usable page separately from a full query.
  3. Record cold and warm cases honestly, including slow cases.
  4. Observe native memory, renderer memory and temporary disk separately.
  5. Cancel a running operation and verify cleanup plus a successful retry.
  6. Check representative output values and types before expanding the job.

The performance page describes this model. Test your own workload in Preview before relying on a broader supported Pro workflow; specialist array, media and training-binary readers have separate capability boundaries.

Try the first page on your own data.

Start with the free Windows Preview: source information and up to 100 records, read-only. Review reader capabilities before planning a full-file Pro workflow.

Pro checkout is being prepared. The initial Windows install enables signed in-app updates for later versions.