Query Parquet and Arrow, then save a reviewed correction

Local workflow · Windows and macOS

Use projected fields for large columnar sources and keep bounded editing separate from whole-dataset rewriting.

Can I edit a large Parquet or Arrow file in RowCove?

Viewing and querying support larger columnar sources. Current edit-copy limits are 128 MiB per source, 200,000 rows and 512 columns; do not treat a GB-scale query as proof of GB-scale editing. Open an eligible individual file to stage corrections and save a separate copy.

1. Inspect the source and select useful fields

Select a Parquet, Arrow IPC or Feather file through Open data. Review detected columns and exact values. Use Columns to show the fields needed for the question; a projected read can avoid decoding unrelated columnar values. The same record population can occupy different physical sizes in compressed Parquet and Arrow.

For a partitioned folder, use the multi-shard guide. Collection browsing is read-only; an individual eligible source file has a separate edit workflow.

2. Query beyond the first page

Use full-file Filters to find the target category or interval, then apply a supported sort when ordering is relevant. Review the count and later result windows. A fast metadata/first-page operation is not a complete scan, and a filter-then-sort is not a sort of every source record.

The Parquet workflow and Arrow workflow explain format-specific access. Choose a useful scope before starting work that reads wide text or nested payloads.

3. Stage an eligible correction and verify the copy

For a source within the edit limits, switch to Edit, select the source cell, review the replacement and choose Stage edit. Save copy writes to a new destination. Reopen that artifact, check changed and unaffected values, and verify schema and units before using it downstream.

Match the source field type: a timestamp’s numeric unit, a decimal’s scale and a large unsigned identifier matter. Unsupported edits to complex values must be rejected rather than silently converted. File compression and batch layout can change when rewriting; save-copy is not byte-for-byte duplication.

The updated writer has regression coverage for exact unsigned integers, decimals, UTC timestamps, schema metadata and unchanged nested arrays. The native-type-preserving writer is included in RowCove 0.1.1.

Beyond the editing budget, retain query access and choose a deliberate downstream transformation workflow. Do not silently split a dataset or imply that editing one shard rewrites the entire collection.

4. Mock measurements by actual operation

All examples and measurements below use generated mock data. No customer files or trading records are used.

Same 1,048,576 logical records; three-trial medians
OperationParquet · 640.7 MB ZstdArrow IPC · 1,106.8 MB
Open and read first 100, all fields2.985 ms3.374 ms
Open and read first 100, projected id/group1.546 ms2.836 ms
Open and return all 10,810 matching records2.060 s1.159 s
Open, filter and numerically sort the matches2.092 s1.189 s
Open and read first/middle/final projected windows31.287 ms6.868 ms

Measured on Windows with a Release native engine build on 27 October 2026: three fresh-process trials per task, the same 1,048,576 logical mock records, and a 1,024-byte synthetic payload. Median elapsed times include opening, index/schema preparation and the requested work. UI rendering, IPC and the separate correctness pass are excluded. OS caches were not cleared; these are not cold-disk guarantees or a comparison with another application. These engineering measurements cover the updated engine included in 0.1.1; results depend on the device, source and requested operation.

All matching records, exact IDs and order were verified. Native query/sort peak working set was at most 31.4 MiB for Parquet and 28.5 MiB for Arrow; total desktop memory is not included.

A separate 100,000-record mock edit fixture used 256-byte payloads: complete one-label edit-copy medians were 148.649 ms for a 14.5 MB Parquet source and 35.551 ms for a 28.8 MB Arrow source. Every saved record and native schema was checked. These bounded edit results cannot be extrapolated to the larger query fixtures.

5. Put a price on repeated setup and waiting

Pro is useful when locating and correcting records is repeated work: fewer one-off conversion scripts, less manual file splitting and a reviewed independent output. Free Preview opens up to 100 records read-only. Try representative files first; occasional small-file inspection may already be covered by free tools. See introductory prices and renewal terms: first month US$5, then US$10/month; first year US$39, then US$100/year, for eligible new subscribers.

Time saved depends on the tasks you repeat and your existing tools. These timings do not establish purchase intent, paid conversion or a universal speed ranking.

Try the first page on your own data.

Start on Windows or macOS: source information and up to 100 records, read-only. Review reader capabilities before planning a full-file Pro workflow.

Version 0.1.1 · Windows x64 · macOS Apple Silicon · Install guide →

Reader capabilities →