Kubernetes 1.37 Brings Pod-Level Resource Managers to Beta

Daily Code Guide Kubernetes News

Kubernetes 1.37 moves Pod-Level Resource Managers to Beta, extending the platform’s Topology, CPU, and Memory Managers so they can make allocation decisions using resources declared for an entire Pod. The feature is disabled by default, so operators must enable it deliberately and test it on supported Linux nodes.

The change is most relevant to workloads that combine a performance-sensitive primary container with sidecars or other supporting containers. With the right topology scope and resource-manager policies, a Pod can reserve a shared resource budget while giving eligible containers exclusive CPU and memory slices. That can make it possible to preserve dedicated resources for critical work without requiring every container in the Pod to receive an exclusive allocation.

What changed in Kubernetes 1.37

The Pod-Level Resource Managers documentation identifies the capability as Beta beginning in Kubernetes 1.37. It builds on Pod-level resource specifications by allowing kubelet’s resource managers to use spec.resources at the Pod level for allocation and NUMA-alignment decisions.

Two feature gates are relevant:

  • PodLevelResources governs the use of Pod-level resource specifications.
  • PodLevelResourceManagers enables the resource-manager behavior that acts on those Pod-level resources.

Feature gates are component-specific key-value settings, so enabling the capability requires configuring the relevant Kubernetes components rather than assuming that one setting automatically applies everywhere. The feature-gate reference should be used alongside the Pod-level resource configuration documentation when planning enablement.

At Pod scope, the Topology Manager can align the Pod’s total resource budget to NUMA topology and divide that budget into exclusive container slices and a Pod-shared pool. At container scope, eligible containers can receive exclusive, NUMA-aligned resources from the node’s allocatable pool, while the Pod-level budget limits the group’s collective consumption.

Ad

How Pod-level resource declarations work

Pod-level CPU and memory budgets are declared with spec.resources.requests and spec.resources.limits. The configuration task for Pod-level resources documents CPU, memory, and hugepages as the supported Pod-level resource types in Kubernetes 1.37.

When both Pod-level and container-level values are present, the Pod-level requests and limits take precedence. That behavior makes the Pod declaration an aggregate budget rather than merely an additional hint alongside each container’s configuration. Teams should therefore review existing manifests carefully before adding Pod-level values, particularly where container-level resource settings were previously used to express the workload’s intended limits.

Pod-level budgets do not automatically make every container exclusive. A container is eligible for exclusive allocation when its CPU and memory requests equal its limits and its CPU request is a positive integer, subject to the configured topology scope and resource-manager policies. This allows a Pod to mix different allocation behaviors: a primary container can qualify for exclusive resources while sidecars use the remaining Pod-shared pool.

Standard init containers and restartable init containers are supported, with resource reservation and reuse following their lifecycle behavior. Operators should account for those lifecycle rules when estimating how much capacity a Pod will reserve over time.

Supported platforms and policies

The Beta feature has important boundaries. Pod-level resource-manager behavior is implemented for the static CPU Manager policy and Static Memory Manager policy. The Memory Manager BestEffort policy is not supported for this functionality.

Support is limited to Linux nodes. On Windows nodes, the resource managers act as a no-op for Pod-level allocations, so a manifest that relies on this behavior should not be treated as portable across operating systems.

The feature also introduces a distinction between Pod-level resource specifications and resource-manager behavior. Pod-level resource specifications are documented as Beta since Kubernetes 1.34 and enabled by default, while the configuration task still requires the relevant PodLevelResources setting for Pod-level specifications. The separate PodLevelResourceManagers gate is required for Topology, CPU, and Memory Managers to allocate based on the Pod-level values.

Why NUMA scope matters

NUMA alignment is one of the main operational reasons to evaluate this feature. With Topology Manager configured for Pod scope, kubelet considers the Pod’s total budget when selecting NUMA placement. It can then partition that placement between exclusive allocations and a shared pool.

With container scope, the allocation decision is made for eligible containers individually. Those containers can receive exclusive resources from the node’s allocatable pool, while the Pod-level budget caps the combined use of the Pod. The two scopes therefore represent different placement models, and administrators should test the selected scope against the workload rather than assuming that Pod and container scope produce identical results.

A Pod can fail admission when the node cannot meet its NUMA-alignment requirements. In that situation, the documented operational response is to reschedule the workload on a suitable node. NUMA alignment and exclusive slices describe resource-management and performance behavior; they should not be interpreted as a security isolation boundary.

Operational behavior to plan for

Pod-level reservations have lifecycle implications. If a container using the Pod-shared pool restarts, the Pod’s total reservation remains in place. The reserved capacity is released when the entire Pod terminates, not whenever an individual shared-pool container exits.

Exclusive CPU containers have CFS quota enforcement disabled according to the documented behavior, while containers in the Pod-shared pool retain quota enforcement against the remaining Pod budget. This difference can affect performance testing and monitoring, especially when a workload includes both exclusive and shared containers.

In Kubernetes 1.37, the node-local PodResources gRPC API can report Pod-level allocations when PodLevelResourceManagers is enabled. The supplied documentation confirms that reporting capability, but the precise schemas, field masks, and scope-by-scope reporting tables should be taken from the relevant API reference rather than inferred from this feature overview.

Downgrades also require planning. Kubernetes 1.37 uses a forward-compatible kubelet checkpoint format intended to avoid startup failures during downgrades. However, Kubernetes 1.36 kubelets do not restore active Pod-level resource assignments. Teams should therefore include active assignments and resource-manager state in version-transition testing.

What you should do

  1. Identify suitable workloads. Start with Pods that have a performance-sensitive primary container and sidecars or supporting containers that can use a shared pool.
  2. Confirm node support. Use Linux nodes with the static CPU Manager and Static Memory Manager policies required by the feature.
  3. Plan feature-gate configuration. Enable PodLevelResources for Pod-level specifications and PodLevelResourceManagers for resource-manager allocation behavior on the required components.
  4. Review manifest precedence. Check how Pod-level requests and limits will replace or override existing container-level values.
  5. Validate exclusive-allocation criteria. For containers that need exclusive resources, use equal CPU and memory requests and limits, with a positive integer CPU request.
  6. Test both placement and failure behavior. Exercise NUMA placement, admission failures, container restarts, and the distinction between Pod scope and container scope.
  7. Monitor the rollout. Use the documented resource-manager allocation and assignment metrics along with kubelet and Topology Manager errors.
  8. Document downgrade procedures. Pay particular attention to transitions involving Kubernetes 1.36 kubelets, which do not restore active Pod-level assignments.

In-place resizing of Pod-level resources is a separate capability. It requires the InPlacePodLevelResourcesVerticalScaling feature gate, documented as Beta in Kubernetes 1.37, and should not be assumed to be automatically included merely because Pod-Level Resource Managers are Beta.

Availability and caveats

Pod-Level Resource Managers are Beta in Kubernetes 1.37 and disabled by default. The documented resource-manager implementation does not provide a complete container-runtime compatibility matrix, so operators should avoid assuming support beyond the stated Linux and policy requirements.

The feature is intended to improve allocation and placement control, not to create a new security boundary. Its benefits and risks should be evaluated through workload performance, capacity planning, observability, and upgrade testing.

Sources

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top
Ad
Ad
Ad