Kubernetes 1.37 Adds Alpha Resize Preemption

Daily Code Guide Kubernetes News

Kubernetes 1.37 introduces an Alpha, disabled-by-default capability that allows the scheduler to preempt lower-priority Pods when a higher-priority Pod has a Deferred in-place CPU or memory resize. The change gives platform teams another way to satisfy resource growth without recreating the resizing Pod, but it also makes priority policy and workload disruption more consequential.

The feature is named InPlacePodVerticalScalingSchedulerPreemption. It builds on Kubernetes in-place Pod resizing, which has been stable since version 1.35. Because the new scheduler behavior is still Alpha, it should be evaluated as an opt-in operational change rather than treated as a general Kubernetes 1.37 default.

What changed in Kubernetes 1.37

In-place Pod resizing lets a Pod request changes to its CPU or memory resources without recreating the Pod. A resize can become Deferred when the requested allocation cannot currently be fulfilled but may become possible later—for example, after another Pod is removed or scaled down. The kubelet retries these Deferred resizes.

With the new scheduler-preemption feature enabled, kube-scheduler can respond to a Deferred resize by preempting lower-priority Pods on the affected node. The goal is to free enough CPU or memory capacity for a higher-priority Pod’s resize request. This addresses a limitation of Deferred resizing: previously, the request could remain pending until capacity became available through another event or through operator intervention.

The behavior is documented in the Pod resize documentation and described by KEP-5836. It does not make every resize succeed. Node capacity and kubelet feasibility still apply, and an Infeasible resize is not automatically fixed simply by enabling the gate.

Ad

Priority determines who gets capacity

Pod priority is central to both retry ordering and preemption. Deferred resize retries are ordered first by Pod priority. When priorities are equal, Guaranteed Pods are considered ahead of Burstable Pods, followed by how long the Pod has remained Deferred when other factors are equal.

The scheduler-preemption behavior follows Kubernetes’ broader priority model: lower-priority Pods may be selected as victims when resources are insufficient for a higher-priority workload. A PodDisruptionBudget can influence victim selection, but it does not turn preemption into a guarantee that a particular workload will be protected. Quality of Service is also not the scheduler’s primary preemption criterion.

These mechanisms should not be conflated with node-pressure eviction. Scheduler preemption is driven by a higher-priority scheduling or resize need, while kubelet eviction is associated with node resource pressure. They can have different operational causes and effects.

A higher-priority resize becoming pending also does not permanently block other pending resizes. The documented behavior allows other resizes to continue being retried, including if the higher-priority resize later becomes Deferred again.

Availability and enablement requirements

InPlacePodVerticalScalingSchedulerPreemption is Alpha in Kubernetes 1.37 and disabled by default. The supplied Kubernetes documentation lists these prerequisites:

  • The Kubernetes server must be version 1.33 or later.
  • InPlacePodVerticalScaling must be enabled on the control plane and all nodes according to the resize task documentation.
  • InPlacePodVerticalScalingSchedulerPreemption must be enabled on the control plane.
  • Pods must use resource requests and limits that can be changed through the documented Pod resize mechanism.
  • Pod priorities must be configured meaningfully for the intended retry and preemption behavior.

Feature gates are supplied as comma-separated key=value entries through the --feature-gates command-line option on the relevant Kubernetes components. The scheduler-preemption entry is:

InPlacePodVerticalScalingSchedulerPreemption=true

The resize documentation also identifies the base InPlacePodVerticalScaling gate as a prerequisite on the control plane and nodes, even though in-place Pod resizing is described as stable and locked since Kubernetes 1.35. Administrators should follow the version-specific documentation for their deployment rather than assuming that the Alpha scheduler behavior is enabled merely because the base resize feature is available.

Operational impact for platform teams

The main benefit is continuity for the Pod being resized. A higher-priority workload may be able to grow in place instead of being recreated or moved when the node is temporarily short of CPU or memory. That can reduce disruption for the resizing workload.

The cost is shifted to lower-priority workloads. Preempted Pods may be terminated gracefully according to Kubernetes’ preemption behavior, but they are still subject to disruption and rescheduling. Enabling the feature therefore changes the consequences of the priority classes already present in a cluster.

PriorityClass policy deserves particular attention in multi-tenant environments. Kubernetes documentation warns that untrusted users who can create Pods at the highest priorities could evict other workloads or cause them to remain unscheduled. ResourceQuota is identified as an administrative control for limiting consumption of priority classes. The new resize path does not remove that administrative concern.

KEP-5836 also documents design considerations involving races between Deferred resizes and newly arriving higher-priority Pods, scheduler restarts and state recovery, possible additional preemption, and coordination between the scheduler and kubelet. These points are important when designing tests and operational runbooks. They should not be read as a guarantee that every proposed KEP detail is implemented identically in every released environment.

What you should do before enabling it

  1. Review priority policy. Confirm which workloads may preempt others and whether users or teams can create high-priority Pods. Use ResourceQuota or equivalent documented controls where necessary.
  2. Verify the feature prerequisites. Check the server version and the required base and scheduler-preemption feature gates across the documented control-plane and node scope.
  3. Test Deferred resizing. Create representative scenarios in a non-production environment where a higher-priority Pod requests more CPU or memory than the node currently has available.
  4. Observe victim behavior. Test lower-priority workloads, graceful termination, PodDisruptionBudget effects, and rescheduling rather than assuming every possible victim outcome.
  5. Monitor resize status. Inspect PodResizePending and PodResizeInProgress conditions, including their reasons and messages. These fields help distinguish Deferred, Infeasible, in-progress, and error states from the resources actually allocated by the kubelet.
  6. Exercise recovery scenarios. Include competing higher-priority workloads and scheduler restart scenarios in testing before relying on the capability for important workloads.

When preemption may not be the right answer

This Alpha feature is one capacity-management option, not a universal replacement for autoscaling or reserved capacity. The supplied Kubernetes sources do not establish that either approach is preferable for a particular environment. Teams should evaluate those choices against workload behavior, node capacity, disruption tolerance, and platform policy rather than assuming that resize-triggered preemption is always the best solution.

Similarly, enabling the feature does not provide a security, isolation, or protection guarantee for critical workloads. It makes deliberate priority assignment and admission controls more important. The exact behavior of every mixed-version control-plane and worker-node rollout combination is not established by the supplied documentation, so this feature should be introduced through a tested, environment-specific rollout plan.

For Kubernetes 1.37 users, the practical message is straightforward: scheduler preemption can make Deferred in-place resizes more responsive, but it is an Alpha opt-in with real disruption trade-offs. Treat PodPriority, resize status, capacity planning, and tenant controls as part of the same change rather than enabling the gate in isolation.

Sources

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top
Ad
Ad
Ad