Sharing the GPU: MIG vs Time-Slicing for Mixed Inference on Kubernetes

A team brings inference on-prem for all the right reasons — data residency, cost at volume, an exit from per-token pricing — and buys the iron to do it. Then the first nvidia-smi on a production node tells an uncomfortable story: an 80GB H100 running a single quantised model, sitting at low double-digit utilisation most of the day, with an embedding service next to it queued for a GPU that does not exist in the budget. The capacity was bought. It is simply not being used, because Kubernetes, by default, hands a whole GPU to one pod and calls it scheduled.

This is the problem the vLLM deployment guide deliberately left alone. Serving a model well on Kubernetes is about throughput, batching, and the request path; it assumes the GPU is yours to fill. The question underneath — how many workloads can safely share one physical GPU, and by what mechanism — is a scheduling and isolation problem, and getting it wrong wastes money (idle silicon) or reliability (a batch job that pages out your inference server mid-request). This article is the operator-level map: MIG, time-slicing, MPS, and now DRA — what each actually isolates, and how to run a mixed inference-plus-embedding estate on hardware you own without buying a GPU per model.

The utilisation you are already paying for

Start with the economics, because they make sharing a sovereignty question, not a tuning one. When you rent inference, idle capacity is the vendor's problem — priced into the per-token rate. When you own the GPU, idle capacity is your capex depreciating in real time. A node you sized with the VRAM-and-throughput math is a fixed cost the moment it racks; every hour a low-QPS model holds a whole card, you amortise a datacentre GPU across a workload that needed a fraction of one.

The scale is easy to overstate, so be precise about what is attributable. The Uptime Institute's analysis notes that even well-optimised training runs reach only 35–45% of the compute the silicon can deliver, and that inference is typically worse — without pinning a clean percentage to it. Treat "most single-tenant inference GPUs run well under half-busy" as a defensible rule of thumb, not a benchmarked constant, and then measure your own nodes. The point is not the number. It is that a whole class of models — embeddings, rerankers, classifiers, moderation — individually cannot fill a modern datacentre GPU, and collectively will if you let them share one.

Three ways to share one GPU

There are three mechanisms for putting more than one workload on a single NVIDIA GPU, and they differ on the axis that matters most for mixed inference: isolation. MIG isolates in hardware. MPS isolates spatially but softly. Time-slicing does not isolate at all. Everything else — how you configure them, how the scheduler sees them — follows from that one distinction.

Click to zoom

All three are provisioned by the same operator. The NVIDIA GPU Operator — calendar-versioned, currently in the v26.x line — deploys the whole node stack as DaemonSets: the driver, the container toolkit, the Kubernetes device plugin, GPU Feature Discovery, the DCGM exporter, and the MIG Manager, alongside Node Feature Discovery for labelling. You do not run nvidia-smi mig by hand across a fleet; you declare a configuration and the operator reconciles nodes to it — which makes the choice below a config change, not an afternoon of SSH.

MIG: hardware partitions with real walls

Multi-Instance GPU is the only option that isolates in silicon. On Ampere and later datacentre parts — A30, A100, H100, H200, and the Blackwell B200/GB200 generation — MIG carves one physical GPU into as many as seven instances, each with a dedicated slice of streaming multiprocessors, its own L2 cache partition, and its own framebuffer memory with its own bandwidth. A fault in one instance does not touch the others. To a pod it looks like a smaller, private GPU — because it is one.

The profiles are a fixed menu, named <compute-slices>g.<memory-GB>. An 80GB A100 or H100 exposes 1g.10gb (up to seven), 1g.20gb (up to four), 2g.20gb (up to three), 3g.40gb (up to two), 4g.40gb, and the whole card as 7g.80gb. An H200's larger memory shifts the numbers — 1g.18gb, 3g.71gb, 7g.141gb — but the shape is identical. The slices are additive against a budget of seven compute units and eight memory units, so a geometry of 3g.40gb + 2g.20gb + 2×1g.10gb sums to a full H100: forty gigabytes for a mid-size LLM, twenty for a reranker, and two ten-gigabyte instances for embeddings and a burst lane.

# GPU Operator ClusterPolicy: enable MIG, "mixed" strategy for
# heterogeneous profiles on the same card.
migManager:
  enabled: true
mig:
  strategy: mixed   # heterogeneous profiles -> nvidia.com/mig-<profile> names
                    # ("single" = one identical profile/node -> plain nvidia.com/gpu)

---
# Named geometry the MIG Manager applies (mig-parted format):
apiVersion: v1
kind: ConfigMap
metadata:
  name: custom-mig-config
data:
  config.yaml: |
    version: v1
    mig-configs:
      mixed-inference:
        - devices: [0]
          mig-enabled: true
          mig-devices:
            "3g.40gb": 1   # LLM       (vLLM)
            "2g.20gb": 1   # reranker
            "1g.10gb": 2   # embeddings + a burst/dev lane
# Then label the node:
#   kubectl label node gpu-01 nvidia.com/mig.config=mixed-inference

MIG's strength and its limitation are the same fact: the geometry is static. You cannot resize a 3g.40gb instance while a pod runs on it; changing the layout means draining the affected instances and having the MIG Manager reconfigure the card. On bare metal the operator can often apply a new geometry without a reboot, but the reconfiguration is an operation, not a scheduling decision. This is the right trade for a stable estate — a known set of models with known sizes — and the wrong one for a cluster whose demand changes shape hour to hour. That gap is what DRA closes.

Time-slicing: density without walls

Time-slicing is the opposite trade. It does not partition the GPU at all; it tells the Kubernetes device plugin to advertise one physical GPU as several schedulable replicas, and lets the driver context-switch between the pods that land on them. There is no hardware boundary. The pods share the full GPU, its full memory, and — critically — its full fault domain. It is configured in a single ConfigMap consumed by the GPU Operator.

# time-slicing-config ConfigMap, referenced by the GPU Operator
version: v1
flags:
  migStrategy: none
sharing:
  timeSlicing:
    renameByDefault: true          # advertise nvidia.com/gpu.shared, not gpu
    failRequestsGreaterThanOne: true   # reject pods asking for >1 shared replica
    resources:
      - name: nvidia.com/gpu
        replicas: 4                # 4 schedulable slots per physical GPU

That makes its best-fit obvious: bursty, low-duty-cycle traffic where two pods rarely contend hard at once and the cost of contention is a slower response, not a violated guarantee — internal developer endpoints, batch embedding jobs, CI inference, demo environments. A latency-SLO'd customer-facing model sharing a time-sliced GPU with a batch job is not a configuration; it is an incident with a scheduled start time.

MPS: the spatial middle path

Between the two sits the Multi-Process Service. Where time-slicing interleaves whole GPU turns, MPS lets multiple processes run kernels concurrently in one shared context, and can enforce soft ceilings on each client's active-thread percentage and memory — finer control, and less context-switch overhead. The catch is in the word "soft": MPS was built for cooperative, trusted workloads, and a fatal CUDA error in one client can take down the control daemon and every client sharing it. It is also mutually exclusive with MIG on the same resources — you layer MPS over a full GPU, not over a slice. Treat it as a throughput optimisation for workloads you already trust to co-exist, not as an isolation mechanism you would defend to an auditor.

DRA: fractional scheduling becomes a Kubernetes primitive

Until recently, all of the above was expressed through the device plugin's blunt vocabulary: a node advertises a count of a named resource, and a pod asks for an integer of it. That model cannot say "a GPU with at least 40GB and NVLink," cannot create a MIG partition on demand to satisfy a request, and cannot cleanly share one device across claims. Dynamic Resource Allocation replaces it. DRA reached general availability in Kubernetes 1.34, released on 1 September 2025, with resource.k8s.io/v1 as the stable API and the feature on by default — the point at which building on it stopped being a bet.

DRA reframes a GPU request as a claim. A workload describes what it needs through structured parameters — device class, attributes, selectors — and a driver publishes what each node has as ResourceSlice objects the scheduler reads at allocation time. For NVIDIA GPUs, the driver can configure MIG partitions and the NVLink fabric as part of fulfilling a claim: partition-on-demand, not partition-in-advance — the capability MIG's static geometry lacked, delivered as core Kubernetes scheduling rather than a vendor add-on.

# The shape of a claim (resource.k8s.io/v1, Kubernetes 1.34+).
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: mig-3g-40gb
spec:
  spec:
    devices:
      requests:
        - name: gpu
          deviceClassName: mig.nvidia.com
          selectors:
            - cel:
                expression: device.attributes["profile"].value == "3g.40gb"
---
apiVersion: apps/v1
kind: Deployment
# … pod template:
spec:
  resourceClaims:
    - name: llm-gpu
      resourceClaimTemplateName: mig-3g-40gb
  containers:
    - name: vllm
      resources:
        claims:
          - name: llm-gpu

The governance shift matters as much as the mechanism. At KubeCon + CloudNativeCon Europe 2026 in Amsterdam, NVIDIA moved its DRA Driver for GPUs from a vendor-owned repository into the Kubernetes project itself — the code now lives under kubernetes-sigs, under community governance rather than one vendor's roadmap. (Do not confuse this with NVIDIA's KAI Scheduler, which became a CNCF Sandbox project at the same event; separate announcements.) The practical effect is that the primitive your fractional-GPU strategy depends on is now standard Kubernetes plumbing with multi-vendor backing — AWS, Google Cloud, Microsoft, Red Hat, and SUSE among the co-signers — not a proprietary interface you would have to migrate off later.

Mixed inference in practice: a decision, not a default

Put the pieces together on one node. A single H100, partitioned into the geometry above, runs four workloads that would otherwise have demanded three or four cards: a mid-size chat model on the 40GB instance, a reranker on the 20GB, an embedding model on one 10GB instance, and a burst lane on the other. Each is a hardware-isolated tenant — the embedding service cannot page out the LLM, and a crash in the burst lane is contained. DRA binds each claim to its slice; the GPU Operator keeps the node reconciled.

Click to zoom

The rule of thumb that survives contact with production: isolate across trust or SLO boundaries, share within them. Two teams, or a customer-facing model beside an internal one, or anything with a latency SLO — MIG, because the boundary must be real. A pool of a team's own bursty, best-effort jobs — time-slicing, because the density is worth the shared fate and nobody is promised a millisecond. The mistake is picking one mechanism for the whole cluster; the estate wants both, chosen per workload class, which is precisely what a golden-path platform layer should encode so application teams request a capability, not a partitioning scheme.

Proving it worked: the DCGM feedback loop

A sharing strategy you cannot measure is a guess. Allocation is not utilisation — a MIG instance can be bound to a pod and idle, and a time-sliced GPU can be oversubscribed into thrashing. The GPU Operator already ships the dcgm-exporter, so the evidence is one scrape away in your own observability stack. The fields that answer the real questions are specific.

# Is a slice actually busy, or just allocated?
#   SM_ACTIVE = fraction of time ≥1 warp is resident on an SM.
avg by (GPU_I_ID, GPU_I_PROFILE) (DCGM_FI_PROF_SM_ACTIVE)

# Memory pressure per instance — the time-slicing OOM early-warning.
DCGM_FI_DEV_FB_USED / (DCGM_FI_DEV_FB_USED + DCGM_FI_DEV_FB_FREE)

# Are the tensor cores earning the GPU, or is it memory-bound?
avg by (GPU_I_ID) (DCGM_FI_PROF_PIPE_TENSOR_ACTIVE)

The loop is the point. Partition on your best sizing guess, run the real workload, read SM_ACTIVE and framebuffer pressure per instance, and re-cut the geometry when the data disagrees with the plan — which, with DRA, is a claim change, not a maintenance window. Own the hardware, own the telemetry, and let measured reality, not a vendor's default, decide how the silicon is carved.

The audit and multi-tenancy angle

For anyone running inference under a compliance regime, isolation is not a performance knob — it is a control, and an auditor will ask whether one workload on a shared GPU could observe or disrupt another. MIG gives a defensible answer: separate SMs, cache, and memory, plus hardware fault isolation — a boundary you can point to in a design document. Time-slicing and MPS give the opposite answer: a shared context and a shared fault domain, where the separation is scheduling policy, not silicon. That distinction decides which mechanism is admissible for which data.

The rule follows directly. Workloads processing different tenants' data, or data under different regulatory classifications, belong on MIG instances or separate GPUs — never time-sliced together, because "the scheduler kept them apart" is not a sentence that survives an incident review. Best-effort jobs handling the same class of data can share a time-sliced pool freely. Encoding that as policy — regulated namespaces may only request MIG-backed resource classes — turns intention into enforcement, the same way a golden-path platform governs the rest of the cluster. An isolation boundary is only as strong as the guardrail that stops someone scheduling around it at 2 a.m.

The exit ramp

Every capability we recommend has to answer how you leave it, and GPU sharing has a specific lock-in risk: proprietary schedulers. Adopt a vendor's scheduling stack that owns placement and partitioning behind its own CRDs, and your capacity model lives in a product you cannot easily replace. Build on the Kubernetes-native primitives instead: the device plugin and, increasingly, DRA. Because DRA is now core Kubernetes under community governance, a claim-based estate is portable across any conformant cluster and any DRA-capable GPU vendor.

1. Keep partitioning in Git, not in a console. The MIG geometry, the time-slicing ConfigMap, and the DRA resource classes are declarations — version them, review them, and a rebuilt node comes back identical. A geometry set by hand is a geometry you will not remember under pressure.
2. Request capabilities, not devices. Application teams should ask for gpu-small or gpu-isolated-40gb through a resource class, never a raw profile string. That indirection is what lets you move a workload from time-slicing to MIG, or from A100 to H100, without touching the workload.
3. Prefer DRA and the device plugin to a proprietary scheduler. A third-party GPU scheduler can be a reasonable optimisation, but only when its absence is survivable. If removing it strands your capacity model, it is lock-in wearing the costume of a feature.
4. Keep the telemetry in a sink you own. dcgm-exporter into your own Prometheus means your utilisation history — the evidence for every sizing and sharing decision — outlives any single tool and is not trapped in a vendor dashboard.

The long game

The specific silicon in your racks today will be two generations old before the depreciation schedule ends. Blackwell replaces Hopper; the profile tables shift; a model that needed 40GB this year needs less next. If your GPU strategy is a pile of hand-tuned per-node geometries, every hardware refresh is a migration project. If it is a set of declared resource classes and claims, the refresh is the operator reconciling new nodes to the same abstractions, and your workloads never learn the card changed underneath them.

That is the sovereign position, and it is the same one that runs through owning your models, your gateway, and your data path: the durable asset is not the hardware but the ability to change the hardware without renegotiating everything built on it. MIG, time-slicing, and DRA are not three products to choose between once. They are three tools whose right mix changes with your workloads and your silicon — and the teams that keep the choice cheap expressed it as reversible configuration over standard interfaces, not as a purchase they have to defend. Own the GPU, and then own the more valuable thing: the freedom to carve it however the next workload demands.