MIG vs Time-Slicing: Sharing GPUs on Kubernetes

A team brings inference on-prem for all the right reasons — data residency, cost at volume, an exit from per-token pricing — and buys the iron to do it. Then the first nvidia-smi on a production node tells an uncomfortable story: an 80GB H100 running a single quantised model, sitting at low double-digit utilisation most of the day, with an embedding service next to it queued for a GPU that does not exist in the budget. The capacity was bought. It is simply not being used, because Kubernetes, by default, hands a whole GPU to one pod and calls it scheduled.

This is the problem the vLLM deployment guide deliberately left alone. Serving a model well on Kubernetes covers throughput, batching, and the request path — assuming the GPU is yours to fill. What remains is how many workloads can safely share one physical GPU, and by what mechanism: get it wrong and you waste money (idle silicon) or reliability (a batch job paging out inference mid-request). This article is the operator-level map: MIG, time-slicing, MPS, and DRA — what each isolates, and how to run mixed inference-plus-embedding on hardware you own without a GPU per model.

The utilisation you are already paying for

Start with the economics: sharing is a sovereignty question, not a tuning one. When you rent inference, idle capacity is the vendor's problem — priced into the per-token rate. When you own the GPU, idle capacity is your capex depreciating in real time. A node sized with the VRAM-and-throughput math is fixed cost the moment it racks; every hour a low-QPS model holds a whole card, you amortise a datacentre GPU across a workload that needed a fraction of one.

Be precise about what is attributable. The Uptime Institute notes that even well-optimised training reaches only 35–45% of what the silicon can deliver, and inference is typically worse — without pinning a clean percentage. Treat "most single-tenant inference GPUs run well under half-busy" as a rule of thumb, not a benchmark, and measure your own nodes. The point is not the number: embeddings, rerankers, classifiers, and moderation each cannot fill a modern datacentre GPU, and collectively will if you let them share one.

Three ways to share one GPU

Three mechanisms put more than one workload on a single NVIDIA GPU; they differ on the axis that matters for mixed inference: isolation. MIG isolates in hardware. MPS isolates spatially but softly. Time-slicing does not isolate at all. Configuration and how the scheduler sees them follow from that distinction.

Click to zoom

All three are provisioned by the same operator. The NVIDIA GPU Operator — calendar-versioned, currently v26.x — deploys the node stack as DaemonSets: driver, container toolkit, device plugin, GPU Feature Discovery, DCGM exporter, and MIG Manager, plus Node Feature Discovery for labelling. You declare a configuration; the operator reconciles nodes to it — a config change, not an afternoon of SSH.

MIG: hardware partitions with real walls

Multi-Instance GPU is the only option that isolates in silicon. On Ampere and later datacentre parts — A30, A100, H100, H200, and Blackwell B200/GB200 — MIG carves one physical GPU into as many as seven instances, each with dedicated streaming multiprocessors, its own L2 cache partition, and its own framebuffer and bandwidth. A fault in one instance does not touch the others. To a pod it looks like a smaller, private GPU — because it is one.

The profiles are a fixed menu, named <compute-slices>g.<memory-GB>. An 80GB A100 or H100 exposes 1g.10gb (up to seven), 1g.20gb (up to four), 2g.20gb (up to three), 3g.40gb (up to two), 4g.40gb, and the whole card as 7g.80gb. An H200's larger memory shifts the numbers — 1g.18gb, 3g.71gb, 7g.141gb — but the shape is identical. The slices are additive against a budget of seven compute units and eight memory units, so a geometry of 3g.40gb + 2g.20gb + 2×1g.10gb sums to a full H100: forty gigabytes for a mid-size LLM, twenty for a reranker, and two ten-gigabyte instances for embeddings and a burst lane.

# GPU Operator ClusterPolicy: enable MIG, "mixed" strategy for
# heterogeneous profiles on the same card.
migManager:
  enabled: true
mig:
  strategy: mixed   # heterogeneous profiles -> nvidia.com/mig-<profile> names
                    # ("single" = one identical profile/node -> plain nvidia.com/gpu)

---
# Named geometry the MIG Manager applies (mig-parted format):
apiVersion: v1
kind: ConfigMap
metadata:
  name: custom-mig-config
data:
  config.yaml: |
    version: v1
    mig-configs:
      mixed-inference:
        - devices: [0]
          mig-enabled: true
          mig-devices:
            "3g.40gb": 1   # LLM       (vLLM)
            "2g.20gb": 1   # reranker
            "1g.10gb": 2   # embeddings + a burst/dev lane
# Then label the node:
#   kubectl label node gpu-01 nvidia.com/mig.config=mixed-inference

MIG's strength and its limitation are the same fact: the geometry is static. You cannot resize a 3g.40gb instance while a pod runs on it; changing the layout means draining the affected instances and having the MIG Manager reconfigure the card. On bare metal the operator can often apply a new geometry without a reboot, but the reconfiguration is an operation, not a scheduling decision. This is the right trade for a stable estate — a known set of models with known sizes — and the wrong one for a cluster whose demand changes shape hour to hour. That gap is what DRA's DynamicMIG path is built to close — alpha and off by default today; static MIG still needs geometry fixed in advance.

Time-slicing: density without walls

Time-slicing is the opposite trade. It does not partition the GPU; it tells the device plugin to advertise one physical GPU as several schedulable replicas, and the driver context-switches between pods that land on them. There is no hardware boundary. Pods share the full GPU, memory, and — critically — fault domain. Configured in a single ConfigMap consumed by the GPU Operator.

# time-slicing-config ConfigMap, referenced by the GPU Operator
version: v1
flags:
  migStrategy: none
sharing:
  timeSlicing:
    renameByDefault: true          # advertise nvidia.com/gpu.shared, not gpu
    failRequestsGreaterThanOne: true   # reject pods asking for >1 shared replica
    resources:
      - name: nvidia.com/gpu
        replicas: 4                # 4 schedulable slots per physical GPU

Best fit is obvious: bursty, low-duty-cycle traffic where pods rarely contend hard and contention costs a slower response, not a violated guarantee — internal endpoints, batch embeddings, CI inference, demos. A latency-SLO'd customer-facing model sharing a time-sliced GPU with a batch job is not a configuration; it is an incident with a scheduled start time.

MPS: the spatial middle path

Between the two sits the Multi-Process Service. Where time-slicing interleaves whole GPU turns, MPS lets multiple processes run kernels concurrently in one shared context, with soft ceilings on each client's active-thread percentage and memory — finer control, less context-switch overhead. The catch is "soft": MPS is for cooperative, trusted workloads, and a fatal CUDA error in one client can take down the control daemon and every client sharing it. It is mutually exclusive with MIG on the same resources — layer MPS over a full GPU, not a slice. Treat it as a throughput optimisation for trusted co-tenants, not isolation you would defend to an auditor.

DRA: fractional scheduling becomes a Kubernetes primitive

Until recently, all of the above used the device plugin's blunt vocabulary: a node advertises a count of a named resource, and a pod asks for an integer of it. That model cannot say "a GPU with at least 40GB and NVLink," and cannot cleanly share one device across claims. Dynamic Resource Allocation replaces those limits. DRA reached general availability in Kubernetes 1.34, released on 27 August 2025, with resource.k8s.io/v1 as the stable API and the feature on by default — the point at which building on it stopped being a bet. On-demand MIG partitioning is not part of that GA surface; default NVIDIA DRA still binds pre-cut instances that the MIG Manager already cut.

DRA reframes a GPU request as a claim. A workload describes what it needs through structured parameters — device class, attributes, selectors — and a driver publishes what each node has as ResourceSlice objects the scheduler reads at allocation time. For NVIDIA GPUs, DRA binds claims to MIG instances the driver discovers (static MIG — pre-cut with the GPU Operator MIG Manager, enabled by default) or creates partitions on demand only when DynamicMIG is fully enabled: the driver's alpha DynamicMIG feature gate, GPU allocation via gpuResourcesEnabledOverride=true, and — on Kubernetes 1.34–1.35 — the cluster feature gate DRAPartitionableDevices on the kube-apiserver and kube-scheduler only (Beta and default-on from Kubernetes 1.36). ComputeDomains for Multi-Node NVLink are the officially supported NVLink path. Dynamic MIG is alpha and off by default as of driver v0.4.x; pin the driver's own MIG guide and feature-gates reference for the full prerequisite set rather than a partial flag list.

# The shape of a claim (resource.k8s.io/v1, Kubernetes 1.34+).
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: mig-3g-40gb
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: mig.nvidia.com
            count: 1
            selectors:
              - cel:
                  expression: device.attributes["gpu.nvidia.com"].profile == "3g.40gb"
---
apiVersion: apps/v1
kind: Deployment
# … pod template:
spec:
  resourceClaims:
    - name: llm-gpu
      resourceClaimTemplateName: mig-3g-40gb
  containers:
    - name: vllm
      resources:
        claims:
          - name: llm-gpu

The governance shift matters as much as the mechanism. At KubeCon + CloudNativeCon Europe 2026 in Amsterdam, NVIDIA moved its DRA Driver for GPUs into the Kubernetes project — the code now lives under kubernetes-sigs, under community governance rather than one vendor's roadmap. (Do not confuse this with NVIDIA's KAI Scheduler, which was accepted as a CNCF Sandbox project in December 2025 and was referenced at the same event; separate announcements.) The practical effect: the primitive your fractional-GPU strategy depends on is standard Kubernetes plumbing with multi-vendor backing — AWS, Google Cloud, Microsoft, Red Hat, and SUSE among the co-signers — not a proprietary interface you would migrate off later.

Mixed inference in practice: a decision, not a default

Put the pieces together on one node. A single H100, partitioned into the geometry above, runs four workloads that would otherwise have needed three or four cards: a mid-size chat model on the 40GB instance, a reranker on the 20GB, embeddings on one 10GB instance, and a burst lane on the other. Each is hardware-isolated — embeddings cannot page out the LLM; a burst-lane crash is contained. DRA binds each claim to its (pre-cut) slice; the GPU Operator keeps the node reconciled.

Click to zoom

The production rule of thumb: isolate across trust or SLO boundaries, share within them. Two teams, a customer-facing model beside an internal one, or anything with a latency SLO — MIG, because the boundary must be real. A team's own bursty, best-effort jobs — time-slicing, because density is worth the shared fate. The mistake is one mechanism for the whole cluster; the estate wants both, per workload class — what a golden-path platform layer should encode so teams request a capability, not a partitioning scheme.

Proving it worked: the DCGM feedback loop

A sharing strategy you cannot measure is a guess. Allocation is not utilisation — a MIG instance can be bound and idle; a time-sliced GPU can thrash under oversubscription. The GPU Operator already ships the dcgm-exporter, so the evidence is one scrape away for stock counters (framebuffer used/free, tensor-pipe activity) in your own observability stack — and one custom metrics ConfigMap away for DCGM_FI_PROF_SM_ACTIVE, which is commented out in dcgm-exporter's default-counters.csv. Enable that field (and other DCP counters you need) before relying on SM-resident busy queries. The fields that answer the real operational questions are specific.

# Memory pressure per instance — stock counters; time-slicing OOM early-warning.
DCGM_FI_DEV_FB_USED / (DCGM_FI_DEV_FB_USED + DCGM_FI_DEV_FB_FREE)

# Are the tensor cores earning the GPU, or is it memory-bound?
#   PIPE_TENSOR_ACTIVE is available from the default exporter CSV.
avg by (GPU_I_ID) (DCGM_FI_PROF_PIPE_TENSOR_ACTIVE)

# Deeper busy signal — fraction of time ≥1 warp is resident on an SM.
#   Off by default in default-counters.csv — enable via ConfigMap first.
avg by (GPU_I_ID, GPU_I_PROFILE) (DCGM_FI_PROF_SM_ACTIVE)

The loop is the point. Partition on your best sizing guess, run the real workload, read framebuffer pressure and tensor-pipe activity per instance (add SM_ACTIVE once enabled), and re-cut the geometry when the data disagrees with the plan. Own the hardware, own the telemetry, and let measured reality, not a vendor's default, decide how the silicon is carved.

The audit and multi-tenancy angle

Under a compliance regime, isolation is not a performance knob — it is a control, and an auditor will ask whether one workload on a shared GPU could observe or disrupt another. MIG gives a defensible answer: separate SMs, cache, and memory, plus hardware fault isolation — a boundary you can cite in a design document. Time-slicing and MPS give the opposite: shared context and fault domain, where separation is scheduling policy, not silicon. That distinction decides which mechanism is admissible for which data.

The rule follows directly. Workloads processing different tenants' data, or data under different regulatory classifications, belong on MIG instances or separate GPUs — never time-sliced together, because "the scheduler kept them apart" does not survive an incident review. Best-effort jobs on the same class of data can share a time-sliced pool freely. Encoding that as policy — regulated namespaces may only request MIG-backed resource classes — turns intention into enforcement, the same way a golden-path platform governs the rest of the cluster. An isolation boundary is only as strong as the guardrail that stops someone scheduling around it at 2 a.m.

The exit ramp

Every capability we recommend has to answer how you leave it, and GPU sharing has a lock-in risk: proprietary schedulers. Adopt a vendor stack that owns placement and partitioning behind its own CRDs, and your capacity model lives in a product you cannot easily replace. Build on Kubernetes-native primitives instead: the device plugin and, increasingly, DRA. Because DRA is core Kubernetes under community governance, a claim-based estate is portable across any conformant cluster and any DRA-capable GPU vendor. The harder version of that question is whether the silicon itself is replaceable — AMD ROCm as a second source covers what changes in the cluster when part of the fleet is not NVIDIA, including the resource-name behaviour that has no MIG equivalent.

1. Keep partitioning in Git, not in a console. MIG geometry, time-slicing ConfigMap, and DRA resource classes are declarations — version them, review them, and a rebuilt node comes back identical. Geometry set by hand is geometry you will not remember under pressure.
2. Request capabilities, not devices. Application teams should ask for gpu-small or gpu-isolated-40gb through a resource class, never a raw profile string. That indirection lets you move a workload from time-slicing to MIG, or A100 to H100, without touching the workload.
3. Prefer DRA and the device plugin to a proprietary scheduler. A third-party GPU scheduler can be a reasonable optimisation, but only when its absence is survivable. If removing it strands your capacity model, it is lock-in in costume.
4. Keep the telemetry in a sink you own. dcgm-exporter into your own Prometheus means utilisation history — evidence for every sizing and sharing decision — outlives any single tool and is not trapped in a vendor dashboard.

The long game

The silicon in your racks today will be two generations old before the depreciation schedule ends. Blackwell replaces Hopper; profile tables shift; a model that needed 40GB this year needs less next. If your GPU strategy is hand-tuned per-node geometries, every hardware refresh is a migration. If it is declared resource classes and claims, the refresh is the operator reconciling new nodes to the same abstractions — workloads never learn the card changed underneath them.

That is the sovereign position — the same one that runs through owning your models, gateway, and data path: the durable asset is not the hardware but the ability to change it without renegotiating everything built on it. MIG, time-slicing, and DRA are not three products to choose between once. They are tools whose right mix changes with workloads and silicon — keep the choice cheap as reversible config over standard interfaces, not a purchase you have to defend. Own the GPU, then own the more valuable thing: freedom to carve it however the next workload demands. Partitioning is one gate in a longer sequence — the decision order for the whole on-premise AI stack sets out which layers have to be answered before this one.