Kentino AI 96 Rome 4090 2644TOPS — 4× RTX 4090 AI Inference Server
Enterprise-grade platform, assembled and tested in the EU.
Kentino AI 96 Rome 4090 2644TOPS
96 GB VRAM Inference Server
4x RTX 4090 | EPYC Rome | 2 644 TOPS INT8
Measured on Kentino hardware. Llama 3.3 70B AWQ INT4 via vLLM 0.19.0.
A 4U rack-mount inference server with four GeForce RTX 4090 pooled to 96 GB VRAM, one AMD EPYC 7542 Rome CPU (32C/64T), 256 GB DDR4 ECC, 2 TB NVMe boot, and dual synchronized 2 kW ATX PSU. Runs vLLM, SGLang, llama.cpp, ComfyUI and every major open-weight inference stack out of the box.
Hardware
| Component | Detail |
|---|---|
| GPUs | 4x NVIDIA GeForce RTX 4090 24 GB GDDR6X (450 W, PCIe 4.0 x16) |
| VRAM pool | 96 GB total across 4 cards |
| CPU | AMD EPYC 7542 Rome (32C/64T, 225 W, 128x PCIe 4.0 lanes) |
| Motherboard | ASRock Rack ROMED8-2T (SP3, 7x PCIe 4.0 x16, 8x DDR4 ECC, 2x 10 GbE, IPMI) |
| System RAM | 256 GB DDR4-2666 ECC RDIMM (4x 64 GB) |
| Storage | 2 TB NVMe M.2 (PCIe 4.0 x4) |
| PSU | Dual 2 kW ATX with sync cable |
| Chassis | 4U rack-mount, front-to-back directed airflow |
| Cooling | SP3 tower cooler, 3x front + 1x rear 120 mm industrial fans |
| Network | Onboard dual 10 GbE (Intel X550) |
Power envelope
- GPU draw: 4 x 450 W = 1 800 W
- System total: ~2 125 W
- PSU total: 4 000 W (dual 2 kW) — 46.9% headroom
- Split power delivery — single PSU failure = loss of 2 GPUs or 2 GPUs + motherboard
Lane topology
128 PCIe Gen4 lanes from EPYC to seven x16 slots; four populated by GPUs at Gen4 x16. No PCIe switch. No NVLink — peer-to-peer at 19-22 GB/s (Kentino measured).
What you can run
With 96 GB of pooled VRAM across 4 cards, this server handles open-weight LLMs, vision models, image and video generation, speech AI, and multi-tenant serving.
LLMs — text / reasoning / coding
Chinese frontier
- Qwen3 / Qwen3.5: Qwen3-72B Q4 (~15-20 tok/s); Qwen3-32B Q6; Qwen3-30B-A3B MoE Q4-Q6; Qwen3-Coder-30B-A3B at 256k; Qwen3.5-122B-A10B Q4; QwQ-32B
- DeepSeek: DeepSeek-R2 32B Q4-Q6 (92.7% AIME 2025); DeepSeek-R1-Distill-Qwen-32B bf16; DeepSeek-V2-Lite 16B
- GLM / Z.ai: GLM-4.5-Air 106B/12B Q4-Q5; GLM-4.6V-Flash; GLM-Zero 9B
- Hunyuan: Hunyuan-A13B Q4-Q6 (~48 GB) 256k ctx dual-mode reasoning
- Others: Seed-OSS-36B Q4 512k ctx; ERNIE-4.5-47B-A3B Q4; Yi-34B Q6; Baichuan-M2-32B; Step-3.5-Flash
Western frontier
- Meta Llama: Llama 3.3 70B Q4_K_M (~20 tok/s llama.cpp, ~179 tok/s batch-32 vLLM — Kentino measured); Llama 3.1 8B bf16 (~80-120 tok/s); Llama 4 Scout Q4
- Mistral: Small 3 24B bf16; Magistral Small 24B reasoning; Devstral Small 2 24B 256k ctx; Mixtral 8x7B Q6
- OpenAI: gpt-oss-20b MXFP4 (16 GB); gpt-oss-120b MXFP4 (80 GB tight)
- Others: Gemma 3 27B Q6 128k; Phi-4 14B bf16; Nemotron-Super 49B Q4; Granite 4.0 H-Small; OLMo 2 32B; Reka Flash 3; Command R 35B
Vision-Language Models
Qwen3-VL-8B/32B, Qwen3-VL-30B-A3B, Qwen3-Omni-30B-A3B; InternVL3 up to 78B Q4; InternVL3.5-38B; DeepSeek-VL2; Llama 3.2 11B Vision; Pixtral 12B; Molmo 7B; Gemma 3 12B/27B; PaliGemma 2; MiniCPM-V 2.6 / MiniCPM-o 2.6.
Image generation
FLUX.1 [dev]/[schnell] fp8 (~15-25 s per 1024x1024); FLUX.1 Kontext; FLUX Tools; SD 3.5 Large; SDXL; HunyuanImage-2.1 bf16 (~34 GB) 2K native; Kolors 2.0; AuraFlow; OmniGen v1.
Video generation
Wan 2.2 T2V-A14B/I2V-A14B MoE (~54 GB bf16); Wan 2.2 TI2V-5B 720p@24fps; HunyuanVideo 13B Q4-Q5; HunyuanVideo 1.5; CogVideoX-5B; Open-Sora 2.0; Mochi-1; LTX-Video; SVD/SV3D/SV4D; NVIDIA Cosmos Predict 2.
Audio / Speech / TTS
- ASR: Whisper v3 turbo (~50x realtime); Parakeet-TDT 1.1B; Canary 1B; Qwen3-ASR; SenseVoice
- TTS: CosyVoice 3.0; Kokoro 82M; Stable Audio Open; Step-Audio-EditX
- Realtime: Kyutai Moshi (200 ms full-duplex); Step-Audio 2 mini; Qwen2.5-Omni-7B
- Music: MusicGen; AudioGen; Suno Bark; SeamlessM4T v2
Multi-model serving
- 4-8 concurrent users on 32-72B LLMs via vLLM / SGLang tensor-parallel
- Mixed: Qwen3-32B + FLUX.1 + Whisper-turbo + Moshi with partitioned VRAM
- LoRA/QLoRA fine-tuning 32-72B; full-param 7-14B
- RAG with Command R+ or Qwen3 + BGE-M3/E5/Jina
Target workloads
- Inference gateway for 50-200 seat org (70B Q4-Q6, 4-8 concurrent sessions)
- Batch diffusion/video pipeline (SDXL + FLUX.1 + Wan 2.2 overnight)
- LoRA/QLoRA fine-tuning lab for 7-34B domain adaptations
- RAG document assistant (Qwen3-VL + BGE-M3 + Command R, 32k ctx)
- Mixed single-box: chat + image + ASR + realtime voice on partitioned VRAM
Measured performance
Kentino bench | 2026-04-10 | 4x RTX 4090 + EPYC 7542 + ROMED8-2T
| Benchmark | Result |
|---|---|
| Sustained compute (fp16) | 647.7 TFLOPS |
| vLLM Llama 3.3 70B AWQ INT4 (single) | 8.0 tok/s |
| vLLM Llama 3.3 70B AWQ INT4 (batch-32) | 179.3 tok/s aggregate |
| llama.cpp Llama 3.3 70B Q4_K_M (single) | 20.3 tok/s |
| Prompt eval | 1 568 tok/s |
| GPU memory bandwidth | 920 GB/s per card |
| NVMe read/write | 4 589 / 4 213 MB/s |
| Peak thermal (GPU+CPU burn) | 73 C, 0.6% drop |
vLLM used awq kernel — 2-3x possible with awq_marlin.
Not ideal for
- Frontier 100B+ dense at bf16 (DeepSeek V3/R1, GLM-4.5+, Kimi-K2, Mistral Large 3 — require 256+ GB VRAM)
- Training from scratch (consumer RTX 4090 lack NVLink)
Warranty and lead time
Build includes assembly, BIOS configuration, driver install, burn-in testing, and functional verification. Lead time depends on component availability, confirmed at order.
Recommended add-ons
- Upgrade RAM to 512 GB (add 4x 64 GB DDR4 — four DIMM slots open)
- 4 TB NVMe secondary drive for dataset/model staging
- 24U open cabinet for multi-server deployments
The questions buyers ask us most often before ordering a server.
How long does it take?
Machines built from components we hold ship quickly; anything requiring a specific GPU generation depends on supply. We give you a date before you pay, and if it moves we tell you rather than letting you find out.
Can the configuration be changed before you build it?
Almost always. GPUs, memory, storage and cooling are chosen per order, and the listed configuration is a starting point rather than a fixed package. If you need more VRAM, faster storage or a different cooling approach, say so before you order and we will quote the change.
Can I collect the server in person?
You can. Our warehouse is in Prague, and collection in person is welcome — most people who come use the visit to go through the machine with our engineer and ask the questions that are awkward over email. For orders within the Czech Republic we also try to deliver personally and walk you through the setup on site.
Can I talk to someone who actually understands the workload?
Yes. We have an engineer who works on AI systems specifically, not a general sales desk. If your question is about batch sizes, quantisation, interconnect or where your bottleneck will be, ask it — that conversation usually changes the configuration for the better.
Which model can I run on this configuration?
Yes. Every machine is assembled, burn-in tested and benchmarked on real AI workloads before it ships, and it leaves us with an LLM already installed and running. You plug it in, connect it to your network and start work — the only decision left is which project it runs first.
How do you test a server before shipping?
We run it against actual AI workloads rather than synthetic scores: inference throughput, sustained load behaviour and thermals under continuous operation. You get the benchmark results with the machine, so the performance you were promised is the performance you can verify on day one.
Is the server ready to run when it arrives?
Yes. Every machine is assembled, burn-in tested and benchmarked on real AI workloads before it ships, and it leaves us with an LLM already installed and running. You plug it in, connect it to your network and start work — the only decision left is which project it runs first.
Ships from our EU warehouse. Heavy items may require freight arrangement — contact us for a shipping quote and lead time. 2-year limited warranty with advanced RMA support; extended warranty available.
Not exactly what you need?
Tell us your workload and we'll spec this platform around it — GPUs, memory, storage and cooling matched to what you actually run.