Seven practices, one engineering team. We design, build, benchmark and operate the compute layer beneath your hardest problems — on-premise, hybrid or sovereign cloud.
A genomics pipeline and a trading engine are not the same machine. We start from your regulatory perimeter, your data gravity and your deadlines — then design backwards.
Solutions · Data & Storage
Ultra-low-latency architectures, high-IOPS tiers sized to accelerator demand, and multi-petabyte hybrid pipelines. We design the storage estate around data gravity first, because moving petabytes is slower and dearer than most architecture diagrams admit.
The First Constraint
A petabyte does not move because a diagram says it should. We start by measuring where your bytes already sit, how fast they grow, who reads them and what it costs in wall-clock hours to put them somewhere else.
Storage designs fail in a small number of predictable ways. A team buys accelerators, then finds the filesystem delivers a third of the read bandwidth the training loop needs. A pipeline is designed to read from an object store in another region, and the first full pass takes a fortnight. An archive tier is specified for cost per terabyte, then used as a working set.
All three are the same error: treating capacity as the design variable when the real variable is movement. The arithmetic below is not a benchmark. It is division, and it is the first slide in every storage review we run.
The Tiering Model
Tiering is not a licence to buy the cheapest medium. Each tier exists because a specific access pattern would be ruinous on the tier above it, or unbearably slow on the tier below.
01
Local or fabric-attached NVMe, sized to hold one job's working set with headroom for checkpoints. Microsecond-class latency, the highest cost per terabyte in the estate, and no resilience guarantee anyone should rely on. Scratch is purged by the scheduler epilogue. We write that into the Slurm job template and the user documentation. The first person to leave a thesis in scratch should learn the rule from a wiki page, not from a purge.
02
Lustre, BeeGFS, WEKA or VAST, chosen on your file size distribution and metadata rate rather than on a vendor's bandwidth slide. Lustre earns its place at rack scale with large sequential reads and a team that can operate it. BeeGFS trades the last slice of performance for markedly simpler operations. WEKA and VAST are where small files and metadata dominate, and where you accept a higher cost per terabyte to make that problem disappear. We shortlist two and run your data on both.
03
S3-compatible, erasure coded across nodes and racks, holding everything durably at a price that survives a budget review. Object is deliberately never in the hot path of a training loop: first-byte latency is tens of milliseconds, two to three orders of magnitude behind the NVMe tier. Versioning, object lock and the authoritative copy of every dataset live here.
04
LTO in a library, or a cloud archive class, at a small fraction of the cost per terabyte of object storage. Retrieval is measured in minutes to hours, which rules it out of any interactive workflow. It is also the only tier that is physically offline, and that single property is why it survives a compromised administrator credential when every online copy does not.
| Tier | Typical media | Read latency | Sustained read | Relative cost per TB | Designed for |
|---|---|---|---|---|---|
| NVMe scratch | PCIe Gen4 / Gen5 TLC NVMe, local or over NVMe-oF | 70–150 µs | 6–14 GB/s per node | Highest in the estate | One job's working set, checkpoint bursts, shuffle and spill |
| Parallel filesystem | NVMe metadata and hot pool, QLC or HDD capacity pool | 200 µs – 1 ms | 40–400 GB/s per cluster | 6–10× object | Shared hot dataset read concurrently by many nodes |
| Bulk object | QLC SSD or high-capacity HDD, erasure coded | 10–80 ms to first byte | 10–60 GB/s per cluster | Baseline | System of record, immutable copies, cross-site replication |
| Archive and tape | LTO-9 library, or cloud archive storage class | 30 s – 4 h to first byte | About 400 MB/s per drive | Roughly 0.1× object | Long retention, statutory records, offline last resort |
Ultra-Low-Latency Data Paths
Each of these technologies works. The question is whether the latency it removes is the latency you are actually paying. We measure that on your workload before quoting any of them.
Disaggregates NVMe from the host over RDMA or TCP, adding roughly ten to twenty microseconds over a local device. It earns its place where you want near-local latency with pooled capacity and independent failure domains. Re-pointing a shelf at a surviving node after a failure is the operational payoff. It does nothing at all if your bottleneck is a single-threaded reader issuing one request at a time.
Verdict — strong for pooled scratch, pointless behind a shallow queue depth
Takes the CPU out of the copy path and removes a context switch per transfer. On InfiniBand the fabric behaves itself by design. On RoCEv2 it does not. You need PFC, ECN and buffer tuning that agrees with your switch model. You also need a congestion plan for the incast a parallel filesystem creates on every collective read. An untuned RoCE fabric is slower and far less predictable than plain TCP.
Verdict — real gains, but the fabric configuration is the project
DMA straight from NVMe or the fabric into GPU memory, skipping the host bounce buffer and the page cache. Meaningful on large sequential reads into a training loop that is genuinely input-bound. Before we quote it we measure accelerator stall time per step. If your GPUs wait on I/O for four per cent of the step, the upper bound on what GPUDirect can return to you is four per cent.
Verdict — measure accelerator I/O stall time before you buy it
Removes syscall and page-cache overhead for applications written to use it. That rewrite is the real cost, and it lands on the application team rather than on the storage budget. For most workloads io_uring recovers the majority of the win without leaving the kernel or forking the codebase. Full SPDK belongs where a few microseconds of tail latency have a price attached — market data capture, order gateways, tick stores.
Verdict — decisive in trading systems, rarely justified elsewhere
High-IOPS Design
Bandwidth numbers sell storage. Metadata operations per second decide whether a genomics run or a VFX shot finishes inside its window.
Sizing to accelerator demand is arithmetic you can do on a whiteboard. A GPU node consuming 3 GB/s of training data needs 3 GB/s delivered to it. Add headroom for checkpoints, which arrive as one synchronised burst across every rank. Multiply by node count, then compare against the measured read bandwidth of the filesystem rather than the quoted one.
Then check the other axis, which is the one that usually breaks. A run that opens two million files an hour is issuing two million metadata requests an hour, and most filesystems saturate metadata long before they saturate bandwidth. The fix is rarely more capacity. It is more metadata servers, a filesystem with a distributed namespace, or a change to how the workflow lays out its intermediates.
If a filesystem benchmark is the only number in a storage proposal, the proposal has not been read against your workload.
Multi-Petabyte Hybrid Pipelines
Who may delete what, when, and on whose authority — that is a records management question. The storage system only enforces the answer. We ask for the answer in writing before configuring a single rule.
You cannot tier what you cannot see. We build an inventory first: size distribution, age, last-read time, owner, retention class and jurisdiction, held in a catalogue that outlives the migration. It is common to find a large share of an estate's capacity has not been read in over a year. It is also common that nobody currently employed knows who created it.
Every dataset maps to a retention class with a named owner. In a Commonwealth agency that means a records authority. In health it means the applicable state retention schedule. In publicly funded research it means the data management plan the funder accepted. We implement the schedule you give us, and we document what we implemented. We are infrastructure engineers, not your records authority or your legal advisers.
Jobs declare their input set. The scheduler stages it onto the hot tier before the job starts, and the epilogue releases it. That is Slurm prolog and epilog hooks, or a data-aware workflow engine such as Nextflow doing the same work one layer up. An accelerator waiting on a cold read is capacity you paid for and did not use. It is also invisible in any graph that counts allocated rather than busy.
A policy engine demotes cold data and promotes on access. Two rules survive every design review. Never demote an object a running or queued job has declared. Always log the demotion, so a researcher can find out why a read took forty seconds instead of four hundred microseconds. Silent tiering destroys trust in a platform faster than an outage does.
Egress is charged per gigabyte and it is the line item that turns a sensible hybrid design into a budget incident. Decide where the authoritative copy lives, and keep the read-heavy loop on the same side of the boundary as the data. Cache derived artefacts instead of re-reading source. Put a monthly egress ceiling in the runbook, with an alert set well below it.
Resilience & Recovery
Three copies, two media, one off-site. The rule is sound, and it is routinely satisfied on paper by three copies that share one credential, one control plane and one blast radius.
Ransomware changed what a backup has to survive. The threat model is no longer a failed drive or a flooded comms room. It is an authenticated administrator deleting snapshots and retention locks for a fortnight before encrypting the primary. A copy that can be removed with your own credentials is not a copy.
So we restate the rule in terms of what actually has to be true, and we test the restore rather than the backup job's exit code.
140PB
Aggregate capacity designed or operated by Cloud Natives across client estates, counting usable rather than raw.
1.2TB/s
Largest parallel filesystem delivered, measured with IOR and then re-measured on the client's own workload.
68%
Median improvement on read-heavy pipelines after a storage path and interconnect redesign.
4/ year
Full restore tests per year on managed storage platforms, with the measured restore time recorded against the RTO.
Questions We Get Asked
Short answers, including the ones that talk you out of buying something. If your question is not here, it is probably the interesting one — send it through.
It depends on two measurements: your file size distribution and your metadata operation rate. Lustre is excellent at rack scale for large sequential reads if you have, or will fund, the operational skill to run it. BeeGFS gives up a slice of peak performance for markedly simpler days. WEKA and VAST are built for the small-file and metadata-heavy end, and they price accordingly. We shortlist two, run your data on both, and give you the numbers with the method attached.
Sometimes, for large sequential reads with deep parallelism and a tolerant data loader. Usually not. Object stores answer in tens of milliseconds; a training step wants microseconds. So the answer is almost always a hot tier in front with explicit staging, rather than a faster object store. Where the dataset streams once per epoch and is already sharded into large files, reading object directly can be the cheaper design. We will say so when it is.
Only if your accelerators are demonstrably stalling on input. We measure I/O stall time per training step before recommending it, because the measurement sets the ceiling on the benefit. On small-file, metadata-bound datasets it changes very little, and the integration and driver-version constraints are real costs carried by your platform team for years.
Do the division before the meeting: bytes divided by link rate, then divided by about 0.7 for real-world efficiency. If the answer is more than a fortnight we discuss physical shipping, or a seed plus incremental sync. We also discuss the option nobody proposes often enough: leaving the data where it is and moving the compute to it. Migration windows that assume line rate are the most common reason a storage project slips a quarter.
Both are valuable on backup and archive tiers and often counterproductive on a hot tier, where inline processing adds latency to the exact path you paid to shorten. Virtual machine images and document stores deduplicate very well. Already-compressed media, encrypted blobs and most genomic formats do not, and a ratio quoted without naming the data type is a marketing number.
You do. We will design and operate the key management, and we will not be the single holder of the keys to your system of record. Key custody, escrow and the recovery procedure go into the handover document, and are rehearsed during the restore test. A tested backup you cannot decrypt is not a backup.
Jurisdiction is a design input on every tier, not a footnote. The architecture document names the physical location and the operating entity for each copy of your data, including backups and metadata. Some hybrid designs touch an offshore region — a management plane, a support tunnel, a telemetry endpoint. We name those in the document, rather than letting you discover them during an assessment.
Start With The Profile
Four things get us a long way: total capacity and growth rate, the file size distribution, the read pattern per job, and the retention obligation. From that we can usually tell you where the bottleneck is before anyone quotes hardware.
Bring a capacity report and a week of filesystem metrics if you have them. If you do not, we will help you collect them first.