Cloud Natives
Let's Talk

Solutions

Infrastructure for workloads that refuse to wait.

Seven practices, one engineering team. We design, build, benchmark and operate the compute layer beneath your hardest problems — on-premise, hybrid or sovereign cloud.

All solutions
AI & AI Factories Enterprise AI infrastructure, on-premise and hybrid LLM deployment, sovereign data governance, token pipeline optimisation. High Performance Computing Scalable compute clusters, R&D workload orchestration, advanced simulation and high-throughput parallel computing. Data Management & Storage Ultra-low-latency architectures, high-IOPS storage tiers and scalable multi-petabyte hybrid data pipelines. High-Frequency Trading Systems Ultra-low latency, mission-critical custom architectures and performance-tuned infrastructure down to the nanosecond. Quantum Computing Hybrid quantum-classical frameworks and exploratory development environments for teams starting the transition. Servers & Workstations Specialised compute, GPU-dense server racks and AI/ML data science workstations, built and burned in to spec. Managed Services 24/7 infrastructure monitoring, turnkey solution deployment, cloud optimisation and ongoing consulting. Start with a workload assessment Send us the job, the dataset shape and the deadline. You get a profiled bottleneck analysis and an honest answer about fit — before anything is quoted.
Not sure where your workload fits? Our engineers will benchmark it before you commit. Schedule a discovery call

Industries

Bespoke architecture, sector by sector.

A genomics pipeline and a trading engine are not the same machine. We start from your regulatory perimeter, your data gravity and your deadlines — then design backwards.

All industries
Government & Public Sector Data sovereignty, ISM controls evidenced through IRAP assessment, and resilient citizen service infrastructure. Defence & National Security Ruggedised mission-critical compute, strict sovereign compliance and secure air-gapped environments. Health Tech & Life Sciences Accelerating genomic sequencing, diagnostic AI workloads and medical imaging pipelines. eResearch & Academia Scalable HPC clusters enabling scientific discovery, climate modelling and multi-university collaboration. Media & Entertainment GPU rendering pipelines, VFX workflows and high-throughput media storage that keeps artists working. Startups & Scale-ups Right-sized, growth-oriented infrastructure that scales from proof-of-concept to production without a rebuild. The Arts & Culture Large-scale digital archiving, GPU-accelerated interactive media and creative technology platforms. Deployment stories from your sector How the constraint was identified, what the architecture looked like, and which numbers moved. Written by the engineers who built it.
Every engagement begins with a workload assessment, not a product catalogue. Read deployment stories

Solutions · AI & AI Factories

AI infrastructure that starts with the data boundary not the GPU count.

We design and operate enterprise AI estates where the weights, the prompts and the responses stay inside a perimeter you can point to on a diagram. On-premise, hybrid or sovereign cloud — sized against your own workload, governed to a standard an assessor will accept, and tuned until the cost per token is defensible.

The Constraint

Draw the boundary first. Everything else is a consequence of it.

Model choice, accelerator count and rack layout are downstream decisions. Until you can state which classes of data may cross which network edge, no honest sizing exercise is possible.

Most AI programmes stall at the same place. Not on model quality, and not on budget, but on a question nobody wrote down at the start: which of these artefacts is allowed to leave the building. There are more of them than people expect.

A prompt containing a patient record is a patient record. A vector index built from a case file is a lossy, partially reversible copy of that case file, and it deserves the classification of its source. Fine-tune deltas inherit the sensitivity of the corpus they were trained on, even though they look like anonymous tensors. Generated responses inherit the sensitivity of the retrieved context, not of the question that triggered them.

We run a boundary workshop before any architecture work. It produces one artefact: a table of data classes against permitted locations, signed by whoever owns the risk. That table decides the deployment posture, the logging design, the retention schedule and the network topology. Skip it and you will rebuild the estate the first time legal reads the logs.

Weights
Is the base model redistributable inside your perimeter under its licence, and do fine-tune deltas inherit the classification of the training corpus?
Prompts
Where may prompt text be written to disk, for how long, and who can read it during an incident?
Embeddings
A vector namespace is a derivative of the source documents. It is almost never less sensitive than they are.
Responses
Generated text carries the sensitivity of its retrieval context. Treat the output log as a copy of the corpus.
Telemetry
Token counts, queue depths and latency histograms usually can leave. Prompt bodies almost never can.
100%Inference retained in-country on sovereign builds, no third-party API egress
9Distinct data classes typically identified in a boundary workshop
3 wkMedian time from workshop to a costed sizing model
Architecture diagram — AI factory data boundary: corpus, weights, prompt path, vector namespace and audit sink, 1600×1200

Anatomy

An AI factory is five subsystems that have to agree.

The phrase gets used loosely. Concretely it means a GPU fabric, a storage tier that can keep that fabric fed, a scheduler, a model registry and an inference gateway. Under-build any one and the other four idle.

01

The GPU fabric

Distributed training runs at the speed of its collective operations. Every step ends in an all-reduce, and if that collective is slow the accelerators wait. What matters is per-GPU scale-out bandwidth, RDMA so gradients bypass the host CPU, and GPUDirect so they bypass host memory entirely.

We build on InfiniBand NDR or well-tuned RoCEv2, rail-optimised so each GPU has a dedicated path to its leaf. NVLink handles the intra-node case. The failure we see most often is oversubscription at the spine: a topology that benchmarks fine on one node and quietly loses a third of step time at thirty-two.

02

The storage tier that keeps it fed

Accelerator idle time is almost always a storage problem. Training I/O has three distinct shapes: small random reads across tokenised shards, large sequential reads for image and video corpora, and very bursty multi-terabyte checkpoint writes. A tier sized on capacity alone will starve on at least one of them.

We size on sustained GB/s per accelerator, then choose the technology — Lustre or BeeGFS for parallel throughput, an NVMe-oF flash namespace where latency dominates, object storage behind it for the cold corpus. Checkpoint write bandwidth deserves its own budget line: it sets how much of a long run you lose when a node fails at hour ninety.

03

The scheduler

An idle accelerator is the most expensive object in the building. Training wants gang scheduling, topology-aware placement and long exclusive reservations. Inference wants fast admission, fine-grained partitioning and preemption. Those are different products, and pretending otherwise is how estates end up at 40% utilisation.

We run Slurm for training and batch, Kubernetes with device plugins and MIG partitioning for serving, and a shared accounting layer so one report answers who consumed what. Fair-share weights and preemption policy are as much a governance document as a configuration file — we write them with the people who will argue about them.

04

The model registry

Twelve months after go-live somebody will ask which weights produced a particular response. Without a registry, answering that is archaeology. With one it is a lookup.

We store models as immutable, content-addressed artefacts with signed manifests. Each version records its base model, the training data snapshot identifier, the recipe, the evaluation results and the licence terms. Promotion from candidate to serving is a gate with named approvers, not a file copy. This is also what makes quantisation safe to adopt — see the evaluation gate below.

05

The inference gateway

The gateway exists because it is the only place you can enforce policy once rather than in every application. It terminates authentication, resolves which tenant is asking, routes by model and latency class, applies token budgets, performs redaction before anything is logged, and writes the audit record.

Behind it we run vLLM or TensorRT-LLM with an OpenAI-compatible surface, so the serving engine can be replaced without touching application code. Putting the gateway in late is the most common and most expensive sequencing error in an AI programme.

Deployment Postures

Three postures. Each one costs you something different.

We are not partisan about this. On-premise, hybrid and sovereign cloud all work. They fail differently, and the failure mode should be one your organisation can actually absorb.

On-premise

Weights, prompts, vector namespaces and logs never leave a facility whose access list you control. The boundary is physical, which is the only kind an assessor never argues with.

  • Buys you a hard perimeter, a flat unit cost at high utilisation and no egress meter
  • Costs you capital up front, accelerator lead times, and ownership of the power and cooling problem
  • Fails when utilisation stays low — idle hardware you already paid for is the worst outcome on this page

Suits — steady inference, classified data, long-lived fine-tunes

Hybrid, with burst training

Inference stays on owned hardware. Training bursts to elastic capacity for the few weeks a year you genuinely need several hundred accelerators, then gives them back.

  • Buys you elastic training without sizing the building for your annual peak
  • Costs you two control planes, two identity models and a data movement problem on every burst
  • Watch the egress meter: repatriating a large checkpoint set every month is a line item, not a rounding error

Suits — irregular training demand against steady serving load

Sovereign cloud

In-country regions with contractual assurances about data location and personnel. No capital outlay, and capacity available the week you ask for it rather than the quarter.

  • Buys you elasticity, a short start-up path and someone else's facilities problem
  • Costs you per-hour accelerator pricing that overtakes owned hardware above roughly half sustained utilisation
  • Ask three questions: where the control plane runs, which support staff can reach your tenancy, and which jurisdiction can compel disclosure

Suits — variable demand, shorter programmes, workloads the provider's assessment actually covers

Do the crossover arithmetic before the architecture review. The comparison that matters is not list price against hourly rate. It is the three-year total of hardware, power, cooling, floor space and staff against metered consumption at your real utilisation curve — plus egress, which only appears in the cloud column and only after the first repatriated checkpoint. In our experience the crossover sits somewhere between 45% and 60% sustained accelerator utilisation, and the honest recommendation for a team below that line is to rent. We will say so.
Ask us to model your crossover point

Sovereign AI Data Governance

Governance is a logging design problem.

A policy document does not constrain an AI system. The gateway, the retention job and the audit sink do. We build the controls first, then write the policy against what the system can actually prove.

Lineage, or you cannot investigate

Every served response should resolve to a model version, a weights digest, a retrieval set, a prompt template revision and a gateway decision record. Four of those five are cheap to capture on the way through and impossible to reconstruct afterwards.

Prompt logging is the real trade-off

This is the hardest decision on the page and we will not make it sound easy. Log full prompt and response text and you have created a new, highly sensitive corpus with its own classification, its own retention obligation and its own breach consequence. Log nothing and you cannot investigate an incident, evaluate quality or improve a retrieval pipeline.

Our default is structured metadata always, full text under a much shorter retention with field-level redaction applied before the write, and a break-glass path for investigations that is itself logged and reviewed. Every element of that default is negotiable, and each change gets written down with the person who accepted it.

Tenancy isolation, including the cache

Separate vector namespaces and separate encryption keys per tenant are table stakes. The one teams miss is the KV cache: shared prefix caching across tenants is a plausible cross-tenant inference channel. We disable it by default and enable it only within a single trust domain, with that decision recorded.

Who can see what

Platform engineers should be able to operate the estate without reading a single prompt. That is an access-control design — an operational telemetry plane separated from the content plane — not a sentence in a policy. Assessors test the design, not the sentence.

Lineage
Response → model version → weights digest → retrieval set → prompt template revision
Retention
Metadata on a long clock; prompt and response text on a short one, per data class
Redaction
Deterministic detection of identifiers at the gateway, applied before the log write, with a test suite
Tenancy
Per-tenant KV cache, vector namespace and encryption key; no cross-tenant prefix sharing
Access
Telemetry plane and content plane separated; break-glass access logged and reviewed monthly
Evidence
Signed model manifests, retention job logs, access review exports, redaction test results
The first question an assessor asks is not which model you used. It is who could have read the prompt, and how you would know.
Principal Engineer AI Infrastructure practice, Cloud Natives

Token Pipeline Optimisation

Throughput is won in the request path, not the purchase order.

Between a prompt arriving and its last token leaving there are six decisions that move throughput by multiples. Most estates we are asked to review are still running the defaults for all six.

Batching, then continuous batching

Static batching pads every request to the longest sequence present and holds the whole batch until the slowest finishes. Continuous batching admits and retires sequences at each decode step, so a two-hundred-token request is not stuck behind an eight-thousand-token one.

On mixed interactive traffic this is usually the single largest available throughput change. The cost is that tail latency becomes a function of admission policy, so you need a queue whose behaviour you can predict.

KV-cache management

The cache, not the weights, is what exhausts HBM. It grows with batch size multiplied by context length, and naive allocation fragments badly enough to waste a serious fraction of the memory you bought.

Paged attention removes most of the fragmentation. Prefix caching removes recomputation of shared system prompts. Both introduce policy questions — eviction behaviour under load, and whether a prefix may be shared across tenants. Usually it may not.

Quantisation, and what it actually costs

Weight-only FP8 or INT8 is close to free on most instruction-following work. INT4 schemes such as AWQ or GPTQ roughly halve memory again and lift throughput materially. The accuracy loss is real and it is not uniform.

Long-context reasoning, arithmetic and lower-resource languages degrade first, and general benchmarks are not sensitive enough to show it. We require a task-specific evaluation set before any quantised variant is promoted. A regression you cannot measure is a regression you cannot accept.

Speculative decoding

A small draft model proposes several tokens; the target model verifies them in one forward pass. The acceptance rate decides whether you win anything at all.

Acceptance is high on templated and predictable output, poor on open-ended creative text. Choose the wrong draft model and you have added latency and accelerator load for no benefit. This is a per-workload measurement, not an estate-wide default.

Request routing and SLO classes

Not every request deserves the largest model. Route on task, context length and latency class: a small model for classification, extraction and routing itself, the large model for the hard minority that needs it.

Separate interactive traffic from batch so an overnight extraction job cannot consume the queue a caseworker is waiting on. Routing is typically a larger cost lever than any kernel-level tuning, and it is cheaper to implement.

Admission control and back-pressure

Shedding load deliberately beats queueing it invisibly. Per-tenant token budgets, an explicit queue depth limit and a documented shed policy turn an outage into a degradation.

Without them, p99 time-to-first-token collapses during a spike while average throughput still looks healthy on the dashboard. That gap is where most AI service incidents actually begin.

Unit Economics

Cost per token is mostly decided by the building.

Accelerator price is the number everyone negotiates hardest and rarely the one that dominates. Utilisation, power, cooling and the amortisation window do.

The arithmetic is unsentimental. An accelerator drawing several hundred watts under sustained load, in a facility with a mediocre power usage effectiveness ratio, at commercial tariff, accumulates an energy and cooling bill over three years that is a meaningful fraction of its purchase price. Two of those at 45% utilisation cost far more per useful token than one at 85%.

Which is why we treat utilisation as the primary cost lever and the invoice as the secondary one. Routing, admission control and continuous batching raise useful work per watt. Direct-to-chip liquid cooling raises achievable rack density and lowers the cooling overhead, which is what actually moves the denominator. And a rack drawing well past a hundred kilowatts is a facilities conversation before it is a procurement one — floor loading, water loop, and what your switchboard will carry.

We publish a cost-per-million-tokens model with every design, with the inputs exposed so your finance team can disagree with our assumptions rather than with the conclusion.

41%

Lower cost per million tokens

Median reduction after continuous batching, paged attention and request routing were introduced to an existing serving estate.

2.6×

Throughput per accelerator

Improvement on mixed interactive traffic at equal p99 time-to-first-token, same hardware, same model weights.

1.18PUE

Facility efficiency

Achieved on a direct-to-chip liquid cooled deployment with a facility water loop, measured over a full summer.

86%

Sustained utilisation

Accelerator utilisation held across a trailing quarter after scheduler, routing and admission control tuning.

Placeholder data. These four figures are illustrative and must be replaced with audited measurements before launch. Each needs a stated measurement window, workload description and methodology footnote to survive a tender response.

Deployment Tiers

Indicative tiers, so you can sanity-check a budget.

Starting points for a conversation, not a price list. Every column changes once we have profiled your model, your context length and your concurrency curve.

Indicative AI factory tiers. Ranges are engineering estimates for planning only and are superseded by a sized design.
Dimension Single node — pilot Single rack — production Multi-rack — train and serve
Accelerators 8 GPUs in one chassis 32 to 64 GPUs across 4 to 8 nodes 256 to 1024 GPUs across 32 to 128 nodes
Scale-out interconnect NVLink within the node; 2 × 100 GbE uplink 400 Gb/s InfiniBand NDR or tuned RoCEv2, single leaf pair 400 to 800 Gb/s rail-optimised fat tree, non-blocking within the pod
Storage throughput, sustained 10 to 20 GB/s from local NVMe 60 to 150 GB/s parallel file system or NVMe-oF namespace 400 GB/s to 1.2 TB/s tiered flash over RDMA
Usable flash capacity 60 to 120 TB 0.5 to 2 PB 4 to 20 PB, with object storage behind it
Power envelope per rack 10 to 14 kW 40 to 80 kW 80 to 132 kW
Cooling Air, standard hot aisle containment Rear-door heat exchanger or direct-to-chip Direct-to-chip liquid; facility water loop required
Scheduling None, or single-team reservation Slurm or Kubernetes with MIG partitioning Slurm for training, Kubernetes for serving, one shared accounting layer
Typical use Evaluation, LoRA fine-tunes, a pilot inference endpoint Production inference for an enterprise, full fine-tunes of mid-size models Continued pre-training plus multi-tenant serving on one estate

Power figures are per rack at sustained load and exclude cooling overhead. Storage throughput is sustained aggregate read, not burst.

Procurement

The questions procurement actually asks.

Taken from real tender clarifications and security reviews. Where the honest answer is "that depends on your legal team", we say that instead of implying otherwise.

Send us your clarification questions

Let's Talk

Bring us the prompts you cannot send anywhere. We will benchmark them in your perimeter.

Send the model you are considering, a representative prompt distribution and the data classes involved. You get a boundary table, a profiled throughput and latency envelope, and a costed sizing model with three options — including the option of not building anything yet.

Schedule a discovery call All solutions
AI infrastructure enquiries
hello@cloudnatives.example
Direct line
+61 0 0000 0000
Existing clients — 24/7 NOC
+61 0 0000 0001

Sydney · Melbourne · Canberra. Australian owned, Australian staffed.