A first page before a full scan.
Inspect source information and a bounded page. A complete row count is not a prerequisite for seeing records, and the displayed page keeps its scope explicit.
Native Rust data engine
Large datasets.
Less unnecessary work.
A high-performance data workspace should make the next useful operation fast, keep resource use visible and avoid loading the complete source just to show a page.
Read fewer fields and partitions.
Project required columns. Prune recognized Hive partitions in supported shard collections. Database predicates run in the native database engine, where suitable indexes can help.
Keep rendering out of the bottleneck.
Rust owns the local data work. The desktop view receives bounded results and virtualizes visible records. Progress and cancellation stay part of long operations.
Large files deserve a focused workflow.
Inspect a bounded first page, choose the fields you need and query a useful scope. Track progress, cancel longer work and verify a separate output copy.
Public demonstrations use synthetic mock records. They are workflow illustrations, not published speed or memory benchmarks.
Explore the mock research workflowTB-scale is a dataset planning problem.
Start with partition layout, projected fields, a bounded scan and the available disk budget. A terabyte spread across columnar shards has a different access cost from one compressed text stream.
A TB-scale throughput and memory benchmark has not been established. The guide explains how to scope the work and test a representative subset before committing to a full scan.
Plan the dataset accessMeasure useful work, not an animation.
| Measure | What to record | Why it matters |
|---|---|---|
| First useful page | Time from opening a source to usable records; source type and page size | Separates browsing from full indexing |
| Cold and warm query | Predicates, projection, cache state and rows scanned | Exposes preparation costs and repeat-work savings |
| Memory and scratch disk | Native process, desktop renderer and temporary storage separately | A bounded table does not imply a universal RAM cap |
| Cancel and recover | Time to stop, partial-file cleanup and a successful retry | A large job must remain controllable |
| Output integrity | Source unchanged; reopened output types, values and scope | Fast output is useful only when it is correct |
Compression changes the first-open cost.
Supported GZIP, Zstandard and BZIP2 sources can require streaming into a disk-backed cache. Set a destination and output budget, review progress and cancel when necessary. Preparation may need a complete decompression and substantial free disk space.
Pick a narrower question first.
One training split, a date partition or a small group of symbols is often the right starting scope. A bounded scan is different from a full-file aggregate, global sort or complete data-quality audit.
Choose a workflow guide