Kubernetes v1.35 Brings Kubelet Configuration Drop-in Directory to General Availability

Kubernetes v1.35 brings kubelet configuration drop-in directory support to general availability, marking a significant improvement in how administrators manage configuration across diverse cluster environments. This stable feature streamlines kubelet configuration management, particularly for large-scale deployments with heterogeneous node pools.

The --config-dir command line argument is now production-ready, letting you point to a directory of kubelet configuration drop-in files. The kubelet automatically merges all files in that directory with your main configuration, enabling administrators to maintain a consistent base configuration while applying targeted customizations for specific node groups—no complex tooling or manual file management required.

The challenge: kubelet configuration at scale

Large Kubernetes clusters rarely consist of uniform nodes. GPU-accelerated instances, edge compute nodes, and standard workhorses each demand different kubelet settings. Managing these variations creates several operational headaches:

• Configuration drift: Subtle differences between node configurations lead to unpredictable behavior across your cluster
• Node group customization: Different hardware profiles and workload patterns require distinct kubelet parameters
• Operational overhead: Maintaining separate complete configuration files for each node type is tedious and error-prone
• Change management: Coordinating configuration updates across multiple node pools becomes a logistical puzzle

Previously, administrators faced an unappealing choice: use one monolithic configuration for all nodes, manually juggle multiple complete configuration files, or build custom tooling. Each approach had significant drawbacks. The graduation of configuration drop-in directory support to stable provides a fully supported, elegant solution.

Practical applications

Heterogeneous node pool management

Consider a cluster with standard compute nodes, high-capacity nodes with GPUs or substantial memory, and edge nodes with specialized constraints. Configuration drop-ins let you layer settings cleanly.

Base configuration

File: 00-base.conf

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
clusterDNS:
 - "10.96.0.10"
clusterDomain: cluster.local

High-capacity node override

File: 50-high-capacity-nodes.conf

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
maxPods: 50
systemReserved:
 memory: "4Gi"
 cpu: "1000m"

Edge node override

File: 50-edge-nodes.conf (edge compute typically has lower capacity)

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
evictionHard:
 memory.available: "500Mi"
 nodefs.available: "5%"

High-capacity nodes inherit the base configuration plus capacity-specific overrides, while edge nodes combine base settings with edge-specific parameters. The layering is transparent and predictable.

Gradual configuration rollouts

Rolling out configuration changes becomes safer with drop-ins:

1. Add a new drop-in file with a high numeric prefix (e.g., 99-new-feature.conf)
2. Test changes on a canary node group
3. Expand the rollout incrementally
4. Once validated, consolidate changes into your base configuration

Inspecting the merged configuration

With configuration spread across multiple files, you'll want to verify the final result. The kubelet's /configz endpoint shows the merged configuration:

# Start kubectl proxy
kubectl proxy

# In another terminal, fetch the merged configuration
# Change the '<node-name>' placeholder before running the curl command
curl -X GET http://127.0.0.1:8001/api/v1/nodes/<node-name>/proxy/configz | jq .

This reveals the actual runtime configuration after all merging, including any settings specified via command-line arguments.

For comprehensive setup instructions, configuration examples, and merging behavior details, consult the official documentation:

• Set Kubelet Parameters Via A Configuration File
• Kubelet Configuration Directory Merging

Best practices

When implementing kubelet configuration drop-ins:

1. Test incrementally: Validate new drop-in configurations on a small node subset before cluster-wide deployment to contain potential issues

2. Version control everything: Track your drop-in files (or their generation source) in version control alongside infrastructure code for change history and rollback capability

3. Use numeric prefixes deliberately: Name files with numeric prefixes (00-, 50-, 90-) to control merge order explicitly and make configuration layering immediately clear

4. Watch for editor artifacts: Text editors often create backup files (.bak, .swp, ~ suffixes) in the working directory. Keep these out of your configuration directory to prevent the kubelet from processing them unintentionally

Acknowledgments

This feature emerged from collaborative work within SIG Node. Thanks to all contributors who designed, implemented, tested, and documented this capability through its evolution from alpha in v1.28, beta in v1.30, to general availability in v1.35.

Share feedback on this feature by joining the Kubernetes Node Special Interest Group, participating in the public Slack channel (#sig-node), or filing issues on GitHub.

Get involved

Have feedback, questions, or production experience with kubelet configuration management? Join the conversation:

• SIG Node community page
• Kubernetes Slack in the #sig-node channel
• SIG Node mailing list

SIG Node welcomes your real-world insights on this feature.