LLM porting and optimisation for AI hardware
We help AI chipmakers and accelerator developers bring open-weight LLMs to their hardware. From initial model bring-up to operator support, quantisation, kernel optimisation and production inference, we engineer the software stack that makes your silicon ready for real-world AI workloads.
Bring LLM support to new processors, NPUs and custom silicon.
Build model support, optimised kernels and production inference stacks.
Enable practical LLM workloads across edge and datacentre platforms.
Integrate models with proprietary runtimes, compilers and deployment environments.
A new processor is judged by whether the models customers expect actually run on it, how fast they run and how closely they match reference accuracy. Achieving this means addressing unsupported operators, quantisation errors, attention kernels that don't suit the memory architecture and inference runtimes that assume a conventional GPU. This engineering work can divert your internal team from its core hardware roadmap. We provide the specialised engineering capability to take it forward.
Most porting delays come from the seams between layers. Owning the whole path lets us trace a bad token back to its cause. The technologies listed are illustrative; we work with the ones your stack uses.
Llama, Qwen, Gemma, Mixtral, DeepSeek-R1, Mistral, Phi
Model selection, checkpoint compatibility, architecture and tokenizer validation. We select reference models covering dense and MoE architectures, different attention mechanisms and long-context workloads.
PyTorch, ONNX, MLIR, TVM, OpenXLA, ExecuTorch
Graph lowering, operator coverage, compilation and fusion, working with the compiler and deployment frameworks relevant to your hardware. We identify unsupported operators and dynamic-shape failures early.
Attention, GEMM, RMSNorm, RoPE, KV cache, sampling
We implement and optimise hardware-specific kernels and integrate or optimise the inference runtime, mapping attention and KV-cache access patterns to the target memory hierarchy, data-movement architecture and DMA capabilities.
BF16, FP16, FP8, INT8, INT4 | AWQ, GPTQ
We optimise numerical precision and quantisation for the formats your silicon accelerates, and validate against reference outputs using logit divergence, perplexity and workload-appropriate accuracy metrics.
GPU, NPU, custom ASIC, RISC-V vector extensions, FPGA, multi-chip
We profile hardware utilisation, memory bandwidth, compute efficiency and multi-chip scaling, analysing prefill and decode separately to identify compute, memory-bandwidth and interconnect bottlenecks.
A practical reference showing which models your hardware supports, the validated configurations, numerical accuracy, inference performance and known limitations. It becomes a shared engineering, product and sales reference for demonstrating hardware readiness to customers.
Identify supported operators, missing implementations and development priorities.
Integrate supported serving frameworks such as vLLM and llama.cpp, or adapt your own inference runtime to the target hardware and software backend.
Deliver validated checkpoints and reproducible accuracy comparisons.
Measure prefill and decode latency, throughput, tokens per watt, memory bandwidth and utilisation under documented workload and configuration settings.
Automated tests that track model correctness, inference performance and regressions across compiler, runtime and firmware updates.
Implement and validate tensor and pipeline parallelism, for workloads up to 70B-class models where your hardware supports it.
Each phase ends with a result you can show to your own leadership or customers.
First working reference model on target hardware
Validated outputs against reference implementations
Optimised kernels, inference and benchmark results
Production integration, automated regression tests and model-support matrix
You don't need a fully mature software ecosystem to begin. We can start with available simulators, emulators or early silicon, working alongside your compiler and firmware teams as the software stack evolves.
Share your architecture, toolchain status and the models your customers ask for first. We'll assess your requirements and outline a potential bring-up approach, key dependencies and initial engineering risks.