GPU Time Slicing

October 7, 2026

What it is

GPU time-slicing lets multiple Kubernetes pods share a single physical GPU by handing each one exclusive access in rotating time intervals. It’s configured through the NVIDIA Kubernetes device plugin — a ConfigMap change, no special hardware, no MIG required.

sharing:
  timeslicing:
    strategy: single
    resources:
    - name: nvidia.com/gpu
      replicas: 4

Each physical GPU is advertised as 4 virtual nvidia.com/gpu devices. A pod requesting one gets scheduled onto a replica. Under the hood, CUDA time-slices the GPU across the sharing processes.

Why it matters for AI infrastructure

Inference GPUs routinely sit at 10–30% utilization. Time-slicing packs several workloads onto one card — fewer GPUs bought, lower cost per workload. For a budget review, that’s the whole argument.

How it works — and where it breaks

The catch list matters more than the feature list:

No memory isolation. Every replica sees the full GPU memory. One greedy workload can OOM-kill its neighbors. There is no cgroup-style memory limit for GPU RAM here.
No performance isolation. A compute-heavy neighbor steals your time slices. Latency-sensitive inference next to a training job is a bad day.
Blurry observability. DCGM reports at the physical GPU level — per-replica utilization is not natively visible. You know the card is busy; attributing who is harder.

Time-slicing vs MIG

	Time-slicing	MIG
Isolation	None (software)	Hardware: memory + compute + fault
Hardware	Any NVIDIA GPU	A100 / H100 / H200 and up
Granularity	Any replica count	Fixed profiles (e.g. 1g.5gb)
Use for	Cost	Guarantees

Rule of thumb: time-slicing for cost, MIG for isolation. Dev/test, batch inference, and tolerant workloads — slice away. Multi-tenant production with SLOs — you want MIG’s hardware walls.

Enterprise angle

Time-slicing is the fastest way to lift GPU utilization without spending a dollar — which is exactly why it deserves scrutiny before production. The failure mode isn’t performance, it’s blame: when four teams share a card with no isolation, the 2 AM page has no clear owner. If you slice, pair it with per-workload memory requests as a convention, DCGM alerts on the physical card, and a runbook that names who gets evicted first.