Kentino AI 128 Rome 5090 6704TOPS — 4× RTX 5090 Blackwell AI Server
Enterprise-grade platform, assembled and tested in the EU.
Kentino AI 128 Rome 5090 6704TOPS
128 GB VRAM Blackwell Inference Server
4x RTX 5090 | EPYC Milan | 6 704 TOPS INT8
Four Blackwell RTX 5090 with native fp8/fp4 tensor paths. Highest-throughput 4-GPU build on the Rome platform.
A 4U rack-mount inference server with four GeForce RTX 5090 pooled to 128 GB VRAM, one AMD EPYC 7643 Milan CPU (48C/96T), 512 GB DDR4 ECC (all 8 DIMM slots populated for max bandwidth), 2 TB NVMe boot, and dual synchronized 2 kW ATX PSU. Runs vLLM, SGLang, llama.cpp, ComfyUI with Blackwell-native fp8 and MXFP4 inference kernels.
Hardware
| Component | Detail |
|---|---|
| GPUs | 4x NVIDIA GeForce RTX 5090 32 GB GDDR7 (Blackwell, 575 W, PCIe 5.0 x16) |
| VRAM pool | 128 GB total across 4 cards (no NVLink on consumer 5090) |
| CPU | AMD EPYC 7643 Milan (48C/96T, 225 W, 128x PCIe 4.0 lanes) |
| Motherboard | ASRock Rack ROMED8-2T (SP3, 7x PCIe 4.0 x16, 8x DDR4 ECC, 2x 10 GbE, IPMI) |
| System RAM | 512 GB DDR4-2666 ECC RDIMM (8x 64 GB — all DIMM slots populated) |
| Boot / storage | 2 TB NVMe M.2 (PCIe 4.0 x4) |
| Power supply | Dual 2 kW ATX PSU with sync cable + 12VHPWR adapter set |
| Chassis | 4U rack-mount, 4x GPU, passive PCIe 4.0 x16 risers |
| Cooling | Arctic Freezer 4U-M SP3 tower + 3x 120 mm front intake + 1x 120 mm rear exhaust |
| Network | Onboard dual 10 GbE (Intel X550) |
Power envelope
- GPU draw: 4 x 575 W = 2 300 W
- System total at full load: ~2 650 W
- PSU total: 4 000 W (dual 2 kW synced) — 33.8 % headroom
- Dual PSU for split power delivery — each PSU feeds a portion of the system
Lane topology
ROMED8-2T fans out 128 PCIe Gen4 lanes from the EPYC directly to seven x16 slots; four populated by GPUs at Gen4 x16. No PCIe switch. No NVLink on consumer 5090 — inter-GPU peer-to-peer. Cards are Gen5 native; Rome caps at Gen4.
What you can run
With 128 GB pooled VRAM and Blackwell-native fp8 tensor paths, this server steps up to Qwen3-235B-A22B Q4 and gpt-oss-120b MXFP4 with real KV headroom — beyond what 4x RTX 4090 can reach.
LLMs — text / reasoning / coding
Chinese frontier
- Qwen3 / Qwen3.5 (Alibaba): Qwen3-235B-A22B Q3-Q4 (~112-132 GB) fits the 128 GB pool with 8-16k ctx — the hero config; Qwen3-32B dense bf16 (~65 GB) with massive KV; Qwen3-Coder-30B-A3B agentic at 1M ctx; Qwen3.5-122B-A10B Q6/fp8 (~75-80 GB); QwQ-32B bf16 reasoning
- DeepSeek: DeepSeek-V3/R1/V3.1/V3.2 fp8-native Q2 (~215 GB) with RAM spill across 512 GB host — feasible for batch; DeepSeek-R2 32B bf16 multi-stream (4 concurrent, one per card)
- GLM / Z.ai: GLM-4.5-Air 106B/12B fp8 (~106 GB) or Q6 comfortably; GLM-4.5/4.6/4.7 Q2_K_XL (~135 GB) tight with MoE offload
- Tencent Hunyuan: Hunyuan-A13B fp8 native (~80 GB) — Blackwell runs fp8 without upcast penalty; Hunyuan-Large Q2 with RAM spill
- ByteDance Seed-OSS-36B bf16 with 512k native; ERNIE-4.5-424B Q2 (~150 GB spill)
Western frontier
- Meta Llama: Llama 3.3 70B Q4 across 4x 5090 (~30-40 tok/s single-stream, ~270+ tok/s batch-32 vLLM); Llama 4 Scout 109B/17B MoE fp8/Q6 (~90 GB); Llama 4 Maverick 400B/17B Q3 (~188 GB spill)
- Mistral: Mistral Small 3 / Magistral / Devstral Small 2 (24B) bf16 multi-stream; Pixtral Large / Mistral Large 2 (123B) Q6 (~88 GB)
- OpenAI (open weights): gpt-oss-120b MXFP4 native (80 GB) with real KV and long context — Blackwell hero workload; gpt-oss-20b MXFP4
- Google Gemma 3: 27B multimodal bf16 (~54 GB) two concurrent streams; 12B / 4B
- Microsoft Phi-4 14B dense bf16; Phi-4-reasoning distilled
- NVIDIA Nemotron: Llama-3.1-Nemotron Ultra 253B Q3 (~119 GB) tight; Super 49B bf16 (~98 GB)
- Others: Cohere Command R+ 104B Q6 (~85 GB); Molmo 72B Q6-bf16 VLM; OLMo 2 32B; IBM Granite 4.0 H-Small
Vision-Language Models
Qwen3-VL-235B-A22B Q3-Q4; Qwen3-VL-32B bf16; InternVL3.5-241B-A28B Q4 (~135 GB tight); InternVL3 78B bf16; Llama 3.2 90B Vision Q6 (~74 GB); Pixtral Large 124B Q6 (~88 GB); Molmo 72B Q6/bf16; Gemma 3 27B multimodal bf16; GLM-4.6V 106B fp8.
Image generation
FLUX.1 [dev] bf16 and fp8 (~10-18 s/image at fp8); FLUX.1 Kontext [dev]; SD 3.5 Large bf16; HunyuanImage-2.1 bf16 and Q4; HunyuanImage-3.0 base (80B MoE, 13B active) bf16 (~80 GB, hero footprint); HunyuanDiT; Kolors / Kolors 2.0; AuraFlow v0.3; OmniGen v1; PixArt-Sigma.
Video generation
Wan 2.2 MoE two-expert bf16 (~54 GB, full ctx); Wan 2.2 TI2V-5B; HunyuanVideo 13B bf16 both experts (~60-80 GB); HunyuanVideo 1.5; CogVideoX-5B bf16; Open-Sora 2.0 11B bf16 (~24 GB); Genmo Mochi-1 bf16 (~42 GB); LTX-Video; Pyramid Flow; SVD / SV3D / SV4D; NVIDIA Cosmos.
Audio / Speech / TTS
- ASR: Whisper v3 large / turbo (~50x realtime); Parakeet-TDT; Canary 1B; Qwen3-ASR; SenseVoice
- TTS: CosyVoice 2 / 3; Kokoro 82M; Stable Audio Open; XTTS v2; StyleTTS 2; Step-Audio-EditX
- Realtime / S2S: Kyutai Moshi 7B; Step-Audio 2 mini/R1; Qwen2.5-Omni-7B
- Music / SFX: MusicGen / AudioGen / Bark; SeamlessM4T v2
Multi-model / multi-tenant serving
- 200B MoE at Q4 with batch inference (Qwen3-235B, GLM-4.5/4.6/4.7-Air) for 8-16 concurrent users
- fp8-native frontier — DeepSeek V3 family, Hunyuan-Large fp8 with Blackwell native paths
- Mixed resident stack: gpt-oss-120b MXFP4 + FLUX.1 + Whisper + Moshi on partitioned VRAM
- High-throughput 70B — tensor-parallel vLLM / SGLang with 200+ tok/s batch aggregate
Target workloads
- 200B+ MoE production serving at Q3-Q4 with real KV (Qwen3-235B, GLM-4.5-Air 106B)
- fp8-native frontier inference (DeepSeek V3/R1 fp8, Hunyuan fp8) — Blackwell runs without upcast
- High-throughput 70B serving — tensor-parallel batch via vLLM or SGLang
- Video generation studio at bf16 (Wan 2.2 dual-expert, HunyuanVideo 13B, Mochi-1)
- Multi-tenant mixed workload — 120B MoE + image gen + realtime voice all resident
Measured performance
Published references | NVIDIA RTX 5090 datasheet + community benchmarks
| Benchmark | Result |
|---|---|
| Per-card INT8 TOPS (NVIDIA datasheet) | 1 676 TOPS |
| Aggregate INT8 TOPS (4 cards) | 6 704 TOPS |
| Memory bandwidth per card | ~1 792 GB/s |
| Llama 3.3 70B Q6 via vLLM (community) | 60-90 tok/s single-stream, 300+ tok/s batch |
| Qwen3-235B-A22B Q3-Q4 | Fits 128 GB pool with 8-16k ctx |
| gpt-oss-120b MXFP4 native | 80 GB — comfortable with KV headroom |
Published external references, not measured on Kentino hardware. Kentino will publish first-party numbers after the first customer build.
Not ideal for
- Frontier 400B+ at Q4 (Kimi-K2, Mistral Large 3, Intern-S1-Pro — require 8-GPU or 6x RTX Pro 6000)
- PCIe Gen5-link-sensitive workloads — pick the Genoa SKU for native Gen5 x16
- Training from scratch (no NVLink on consumer 5090)
- ECC-sensitive 24/7 production — consumer 5090 has no ECC; prefer L40 or RTX Pro 6000 Server Edition
Warranty and lead time
Build includes assembly, BIOS configuration, driver install, burn-in testing, and functional verification. Lead time depends on component availability, confirmed at order.
Recommended add-ons
- Upgrade PSU to dual 2.5 kW (FSP) for sustained worst-case bf16 + video — recommended for 24/7
- 4 TB NVMe for model library + MoE weight staging
- 24U open cabinet for multi-server deployment
- Consider the Genoa-platform variant on request for Gen5 x16 link
The questions buyers ask us most often before ordering a server.
How long does it take?
Machines built from components we hold ship quickly; anything requiring a specific GPU generation depends on supply. We give you a date before you pay, and if it moves we tell you rather than letting you find out.
Can the configuration be changed before you build it?
Almost always. GPUs, memory, storage and cooling are chosen per order, and the listed configuration is a starting point rather than a fixed package. If you need more VRAM, faster storage or a different cooling approach, say so before you order and we will quote the change.
Can I collect the server in person?
You can. Our warehouse is in Prague, and collection in person is welcome — most people who come use the visit to go through the machine with our engineer and ask the questions that are awkward over email. For orders within the Czech Republic we also try to deliver personally and walk you through the setup on site.
Can I talk to someone who actually understands the workload?
Yes. We have an engineer who works on AI systems specifically, not a general sales desk. If your question is about batch sizes, quantisation, interconnect or where your bottleneck will be, ask it — that conversation usually changes the configuration for the better.
Which model can I run on this configuration?
Yes. Every machine is assembled, burn-in tested and benchmarked on real AI workloads before it ships, and it leaves us with an LLM already installed and running. You plug it in, connect it to your network and start work — the only decision left is which project it runs first.
How do you test a server before shipping?
We run it against actual AI workloads rather than synthetic scores: inference throughput, sustained load behaviour and thermals under continuous operation. You get the benchmark results with the machine, so the performance you were promised is the performance you can verify on day one.
Is the server ready to run when it arrives?
Yes. Every machine is assembled, burn-in tested and benchmarked on real AI workloads before it ships, and it leaves us with an LLM already installed and running. You plug it in, connect it to your network and start work — the only decision left is which project it runs first.
Ships from our EU warehouse. Heavy items may require freight arrangement — contact us for a shipping quote and lead time. 2-year limited warranty with advanced RMA support; extended warranty available.
Not exactly what you need?
Tell us your workload and we'll spec this platform around it — GPUs, memory, storage and cooling matched to what you actually run.