Skip to content

Rust API — Overview

The fastdpplot-core crate is a pure-Rust library that contains all data-intensive logic. It is compiled into a shared library and exposed to Python through the fastdpplot-py crate (PyO3).

Crate layout

crates/
├── fastdpplot-core/          ← pure library — no PyO3 dependency
│   └── src/
│       ├── lib.rs            ← re-exports public API
│       ├── types.rs          ← DotPoint, BinGrid, TileRequest
│       ├── error.rs          ← FastDpError (thiserror)
│       ├── io.rs             ← Parquet + TXT loaders
│       ├── fasta.rs          ← streaming FASTA parser
│       ├── binmat.rs         ← raw binary matrix loader + dtype inference
│       ├── dp_input.rs       ← high-level .bin + FASTA pipeline
│       ├── binning.rs        ← parallel 2-D histogram binning
│       ├── sparse.rs         ← COO / CSR conversion
│       └── tile.rs           ← tile-server entry point
└── fastdpplot-py/            ← PyO3 extension (cdylib)
    └── src/
        └── lib.rs            ← #[pyfunction] wrappers

Design principles

  • Zero unsafe in the public API. The single unsafe block in binning.rs is guarded by an index proof and documented.
  • Rayon everywhere. All hot paths (binning, matrix normalisation, TXT parsing) use rayon parallel iterators.
  • Streaming I/O. Parquet files are never fully resident in RAM; record batches are projected and processed one at a time.
  • Little-endian raw binary. The .bin format is always interpreted as little-endian, matching the output of most DP tools on x86/ARM hardware.

External dependencies

Crate Purpose
rayon 1.10 Data parallelism
thiserror 1.0 Ergonomic error enum
parquet 53 + arrow 53 Parquet & Arrow I/O
pyo3 0.21 Python bindings (fastdpplot-py only)

Module pages

Module Summary
types Core data structures (DotPoint, BinGrid, TileRequest)
binmat Binary matrix loading and dtype inference
fasta Streaming FASTA parser
binning Parallel 2-D histogram binning
sparse COO / CSR sparse format conversion
io Parquet and delimited-text loaders
dp_input High-level .bin + FASTA pipeline