<!-- BEGIN MUNGE: GENERATED_TOC -->
- [v1.29.10](#v12910)
- [Downloads for v1.29.10](#downloads-for-v12910)
- [Source Code](#source-code)
- [Client Binaries](#client-binaries)
- [Server Binaries](#server-binaries)
- [Node Binaries](#node-binaries)
- [Container Images](#container-images)
- [Changelog since v1.29.9](#changelog-since-v1299)
- [Changes by Kind](#changes-by-kind)
- [Feature](#feature)
- [Bug or Regression](#bug-or-regression)
- [Other (Cleanup or Flake)](#other-cleanup-or-flake)
- [Dependencies](#dependencies)
- [Added](#added)
- [Changed](#changed)
- [Removed](#removed)
- [v1.29.9](#v1299)
- [Downloads for v1.29.9](#downloads-for-v1299)
- [Source Code](#source-code-1)
- [Client Binaries](#client-binaries-1)
- [Server Binaries](#server-binaries-1)
- [Node Binaries](#node-binaries-1)
- [Container Images](#container-images-1)
- [Changelog since v1.29.8](#changelog-since-v1298)
- [Changes by Kind](#changes-by-kind-1)
- [Feature](#feature-1)
- [Bug or Regression](#bug-or-regression-1)
- [Dependencies](#dependencies-1)
- [Added](#added-1)
- [Changed](#changed-1)
- [Removed](#removed-1)
- [v1.29.8](#v1298)
- [Downloads for v1.29.8](#downloads-for-v1298)
- [Source Code](#source-code-2)
- [Client Binaries](#client-binaries-2)
- [Server Binaries](#server-binaries-2)
- [Node Binaries](#node-binaries-2)
- [Container Images](#container-images-2)
- [Changelog since v1.29.7](#changelog-since-v1297)
- [Changes by Kind](#changes-by-kind-2)
- [API Change](#api-change)
- [Bug or Regression](#bug-or-regression-2)
- [Dependencies](#dependencies-2)
- [Added](#added-2)
- [Changed](#changed-2)
- [Removed](#removed-2)
- [v1.29.7](#v1297)
- [Downloads for v1.29.7](#downloads-for-v1297)
- [Source Code](#source-code-3)
- [Client Binaries](#client-binaries-3)
- [Server Binaries](#server-binaries-3)
- [Node Binaries](#node-binaries-3)
- [Container Images](#container-images-3)
- [Changelog since v1.29.6](#changelog-since-v1296)
- [Important Security Information](#important-security-information)
- [CVE-2024-5321: Incorrect permissions on Windows containers logs](#cve-2024-5321-incorrect-permissions-on-windows-containers-logs)
- [Changes by Kind](#changes-by-kind-3)
- [Feature](#feature-2)
- [Bug or Regression](#bug-or-regression-3)
- [Dependencies](#dependencies-3)
- [Added](#added-3)
- [Changed](#changed-3)
- [Removed](#removed-3)
- [v1.29.6](#v1296)
- [Downloads for v1.29.6](#downloads-for-v1296)
- [Source Code](#source-code-4)
- [Client Binaries](#client-binaries-4)
- [Server Binaries](#server-binaries-4)
- [Node Binaries](#node-binaries-4)
- [Container Images](#container-images-4)
- [Changelog since v1.29.5](#changelog-since-v1295)
- [Changes by Kind](#changes-by-kind-4)
- [API Change](#api-change-1)
- [Feature](#feature-3)
- [Bug or Regression](#bug-or-regression-4)
- [Dependencies](#dependencies-4)
- [Added](#added-4)
- [Changed](#changed-4)
- [Removed](#removed-4)
- [v1.29.5](#v1295)
- [Downloads for v1.29.5](#downloads-for-v1295)
- [Source Code](#source-code-5)
- [Client Binaries](#client-binaries-5)
- [Server Binaries](#server-binaries-5)
- [Node Binaries](#node-binaries-5)
- [Container Images](#container-images-5)
- [Changelog since v1.29.4](#changelog-since-v1294)
- [Changes by Kind](#changes-by-kind-5)
- [Bug or Regression](#bug-or-regression-5)
- [Other (Cleanup or Flake)](#other-cleanup-or-flake-1)
- [Dependencies](#dependencies-5)
- [Added](#added-5)
- [Changed](#changed-5)
- [Removed](#removed-5)
- [v1.29.4](#v1294)
- [Downloads for v1.29.4](#downloads-for-v1294)
- [Source Code](#source-code-6)
- [Client Binaries](#client-binaries-6)
- [Server Binaries](#server-binaries-6)
- [Node Binaries](#node-binaries-6)
- [Container Images](#container-images-6)
- [Changelog since v1.29.3](#changelog-since-v1293)
- [Important Security Information](#important-security-information-1)
- [CVE-2024-3177: Bypassing mountable secrets policy imposed by the ServiceAccount admission plugin](#cve-2024-3177-bypassing-mountable-secrets-policy-imposed-by-the-serviceaccount-admission-plugin)
- [Changes by Kind](#changes-by-kind-6)
- [Feature](#feature-4)
- [Bug or Regression](#bug-or-regression-6)
- [Dependencies](#dependencies-6)
- [Added](#added-6)
- [Changed](#changed-6)
- [Removed](#removed-6)
- [v1.29.3](#v1293)
- [Downloads for v1.29.3](#downloads-for-v1293)
- [Source Code](#source-code-7)
- [Client Binaries](#client-binaries-7)
- [Server Binaries](#server-binaries-7)
- [Node Binaries](#node-binaries-7)
- [Container Images](#container-images-7)
- [Changelog since v1.29.2](#changelog-since-v1292)
- [Changes by Kind](#changes-by-kind-7)
- [Feature](#feature-5)
- [Bug or Regression](#bug-or-regression-7)
- [Other (Cleanup or Flake)](#other-cleanup-or-flake-2)
- [Dependencies](#dependencies-7)
- [Added](#added-7)
- [Changed](#changed-7)
- [Removed](#removed-7)
- [v1.29.2](#v1292)
- [Downloads for v1.29.2](#downloads-for-v1292)
- [Source Code](#source-code-8)
- [Client Binaries](#client-binaries-8)
- [Server Binaries](#server-binaries-8)
- [Node Binaries](#node-binaries-8)
- [Container Images](#container-images-8)
- [Changelog since v1.29.1](#changelog-since-v1291)
- [Changes by Kind](#changes-by-kind-8)
- [Feature](#feature-6)
- [Bug or Regression](#bug-or-regression-8)
- [Dependencies](#dependencies-8)
- [Added](#added-8)
- [Changed](#changed-8)
- [Removed](#removed-8)
- [v1.29.1](#v1291)
- [Downloads for v1.29.1](#downloads-for-v1291)
- [Source Code](#source-code-9)
- [Client Binaries](#client-binaries-9)
- [Server Binaries](#server-binaries-9)
- [Node Binaries](#node-binaries-9)
- [Container Images](#container-images-9)
- [Changelog since v1.29.0](#changelog-since-v1290)
- [Changes by Kind](#changes-by-kind-9)
- [API Change](#api-change-2)
- [Feature](#feature-7)
- [Bug or Regression](#bug-or-regression-9)
- [Other (Cleanup or Flake)](#other-cleanup-or-flake-3)
- [Dependencies](#dependencies-9)
- [Added](#added-9)
- [Changed](#changed-9)
- [Removed](#removed-9)
- [v1.29.0](#v1290)
- [Downloads for v1.29.0](#downloads-for-v1290)
- [Source Code](#source-code-10)
- [Client Binaries](#client-binaries-10)
- [Server Binaries](#server-binaries-10)
- [Node Binaries](#node-binaries-10)
- [Container Images](#container-images-10)
- [Changelog since v1.28.0](#changelog-since-v1280)
- [Urgent Upgrade Notes](#urgent-upgrade-notes)
- [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade)
- [Changes by Kind](#changes-by-kind-10)
- [Deprecation](#deprecation)
- [API Change](#api-change-3)
- [Feature](#feature-8)
- [Documentation](#documentation)
- [Failing Test](#failing-test)
- [Bug or Regression](#bug-or-regression-10)
- [Other (Cleanup or Flake)](#other-cleanup-or-flake-4)
- [Dependencies](#dependencies-10)
- [Added](#added-10)
- [Changed](#changed-10)
- [Removed](#removed-10)
- [v1.29.0-rc.2](#v1290-rc2)
- [Downloads for v1.29.0-rc.2](#downloads-for-v1290-rc2)
- [Source Code](#source-code-11)
- [Client Binaries](#client-binaries-11)
- [Server Binaries](#server-binaries-11)
- [Node Binaries](#node-binaries-11)
- [Container Images](#container-images-11)
- [Changelog since v1.29.0-rc.1](#changelog-since-v1290-rc1)
- [Changes by Kind](#changes-by-kind-11)
- [Feature](#feature-9)
- [Dependencies](#dependencies-11)
- [Added](#added-11)
- [Changed](#changed-11)
- [Removed](#removed-11)
- [v1.29.0-rc.1](#v1290-rc1)
- [Downloads for v1.29.0-rc.1](#downloads-for-v1290-rc1)
- [Source Code](#source-code-12)
- [Client Binaries](#client-binaries-12)
- [Server Binaries](#server-binaries-12)
- [Node Binaries](#node-binaries-12)
- [Container Images](#container-images-12)
- [Changelog since v1.29.0-rc.0](#changelog-since-v1290-rc0)
- [Dependencies](#dependencies-12)
- [Added](#added-12)
- [Changed](#changed-12)
- [Removed](#removed-12)
- [v1.29.0-rc.0](#v1290-rc0)
- [Downloads for v1.29.0-rc.0](#downloads-for-v1290-rc0)
- [Source Code](#source-code-13)
- [Client Binaries](#client-binaries-13)
- [Server Binaries](#server-binaries-13)
- [Node Binaries](#node-binaries-13)
- [Container Images](#container-images-13)
- [Changelog since v1.29.0-alpha.3](#changelog-since-v1290-alpha3)
- [Changes by Kind](#changes-by-kind-12)
- [API Change](#api-change-4)
- [Feature](#feature-10)
- [Bug or Regression](#bug-or-regression-11)
- [Other (Cleanup or Flake)](#other-cleanup-or-flake-5)
- [Dependencies](#dependencies-13)
- [Added](#added-13)
- [Changed](#changed-13)
- [Removed](#removed-13)
- [v1.29.0-alpha.3](#v1290-alpha3)
- [Downloads for v1.29.0-alpha.3](#downloads-for-v1290-alpha3)
- [Source Code](#source-code-14)
- [Client Binaries](#client-binaries-14)
- [Server Binaries](#server-binaries-14)
- [Node Binaries](#node-binaries-14)
- [Container Images](#container-images-14)
- [Changelog since v1.29.0-alpha.2](#changelog-since-v1290-alpha2)
- [Urgent Upgrade Notes](#urgent-upgrade-notes-1)
- [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade-1)
- [Changes by Kind](#changes-by-kind-13)
- [Deprecation](#deprecation-1)
- [API Change](#api-change-5)
- [Feature](#feature-11)
- [Documentation](#documentation-1)
- [Failing Test](#failing-test-1)
- [Bug or Regression](#bug-or-regression-12)
- [Other (Cleanup or Flake)](#other-cleanup-or-flake-6)
- [Dependencies](#dependencies-14)
- [Added](#added-14)
- [Changed](#changed-14)
- [Removed](#removed-14)
- [v1.29.0-alpha.2](#v1290-alpha2)
- [Downloads for v1.29.0-alpha.2](#downloads-for-v1290-alpha2)
- [Source Code](#source-code-15)
- [Client Binaries](#client-binaries-15)
- [Server Binaries](#server-binaries-15)
- [Node Binaries](#node-binaries-15)
- [Container Images](#container-images-15)
- [Changelog since v1.29.0-alpha.1](#changelog-since-v1290-alpha1)
- [Changes by Kind](#changes-by-kind-14)
- [Feature](#feature-12)
- [Failing Test](#failing-test-2)
- [Bug or Regression](#bug-or-regression-13)
- [Other (Cleanup or Flake)](#other-cleanup-or-flake-7)
- [Dependencies](#dependencies-15)
- [Added](#added-15)
- [Changed](#changed-15)
- [Removed](#removed-15)
- [v1.29.0-alpha.1](#v1290-alpha1)
- [Downloads for v1.29.0-alpha.1](#downloads-for-v1290-alpha1)
- [Source Code](#source-code-16)
- [Client Binaries](#client-binaries-16)
- [Server Binaries](#server-binaries-16)
- [Node Binaries](#node-binaries-16)
- [Container Images](#container-images-16)
- [Changelog since v1.28.0](#changelog-since-v1280-1)
- [Changes by Kind](#changes-by-kind-15)
- [Deprecation](#deprecation-2)
- [API Change](#api-change-6)
- [Feature](#feature-13)
- [Documentation](#documentation-2)
- [Failing Test](#failing-test-3)
- [Bug or Regression](#bug-or-regression-14)
- [Other (Cleanup or Flake)](#other-cleanup-or-flake-8)
- [Dependencies](#dependencies-16)
- [Added](#added-16)
- [Changed](#changed-16)
- [Removed](#removed-16)
<!-- END MUNGE: GENERATED_TOC -->
# v1.29.10
## Downloads for v1.29.10
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes.tar.gz) | ce7bfce44a22d9788ef518c7a08e68be4ab8850447fe9035369d340adf87edd792b600a0bd7627ac7b0387a5a7684e7f19ceb7b75a6f4cd334ba7f436a682afc
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-src.tar.gz) | 783b848cd9ab5312dcca602dc5f9dd5da76f95891d35e02922e15466439758bbc141cc37c79690e18cdd648b8b9b3290b9569638d45eff6e62cd639286e8bec3
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-client-darwin-amd64.tar.gz) | 545396bfa63be41fdc9b2af83bffc4d3f285dcaf28e1a985a7e810c069fe17da9f5737c1ba635c821ed9792e645a071495065b4389c4c6e67fb95f0eb89bef9e
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-client-darwin-arm64.tar.gz) | 282e06ea245f0772fc08015d4e2cecfe447952f207aadc23c00557d4427be7889173bb3090d67069dc5e8a566d232df8fde477c9930a48cde75bb0649ba1411f
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-client-linux-386.tar.gz) | 38b3c8dcac8cebb5bc4bc111fbc27dce6784c2064147fc1a8d6b6f0cc96789fcaf4c0f3a9cefeaa3ae9e921ddddefd653672986efaf3764698ca135a4e622226
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-client-linux-amd64.tar.gz) | aa34097eda1272ea0d646de560829f7d698b2a97838889f20a0ac7fb88fe89cd7c5f088c67b373bd08ba864652c29f21954c99e7b1cc00393a5f6262fb31150a
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-client-linux-arm.tar.gz) | 6deb52392ba6664f0760b7db6455d339ac50506c824aefef082142323b4a1f11c376e33f041f1f64b202f7bfd5bc70ff51d315cd44dd5e3c0172d6114b72f9eb
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-client-linux-arm64.tar.gz) | e324347d6dd54d6201c098a75855905d08effde26d1b3c342c06705efe8860f799b02b3cf2911ecc44333e98145fce2fc2d78d0b30020bf13e99edd0267ce745
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-client-linux-ppc64le.tar.gz) | 28c62d9c4a27a31b1ada5ccb409d3c343be78045a1d81cc9b88bdf782375dd518ff5de89f60bde6292a338985538d4262aaee98f90d44802e403a489ddbe6f71
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-client-linux-s390x.tar.gz) | 5a851581bcdebca90557b6125eebe252100560fd7d3b2dc614b7facc252ecff541857021cdade38d63a812b444c30274ea057ef598a5c6b37cf11a0ffa787e30
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-client-windows-386.tar.gz) | 67946f344917c11dbc4294a69143a45579a38d89f9e78c75c5bc2b035d771235f09934bea6f0c5c4e6b279a99c3bb125298c968ddbc5a3d00e11e3b3074ff903
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-client-windows-amd64.tar.gz) | 182bfccd9e1587cfc5570554a4510af27c1ba069f302ded1a0b280700cf19778ad13312a7f7d1d23315e4ba80be98f00709e3090d61613253cbc177f9c5b3dcc
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-client-windows-arm64.tar.gz) | 37bcb12043a9ad699e5026a1c3b91551fe5db29ca8d24f999c07aaad334f9eb40e2498471cad3d68f105a90db1afe2d5ba3c18c795c7f0d97f2f746e533cbd42
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-server-linux-amd64.tar.gz) | 1b45ea775c4888587ef7edc91b9388e547cc535496d58603f22b4ef18cbe52cfd54595419743eac397b89ebcc3bcb87b1a712bc9d4de147201bbda92c64ece5d
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-server-linux-arm64.tar.gz) | f9460ef6f81cfe1c002ddba293876a33aa6be6dedc5437fc711dadb19ebd310a069b6b33e8e69ac855185238d998fdcf262b85278d5935ca4c2f57236241d93a
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-server-linux-ppc64le.tar.gz) | 0e2a4c8271eec4e5f3217b502b5c4b30bcaffad55226b5e371f1eca532cbe8e04bb796803e662345ff7de46c4c2c9d9fb5cde9b61df8565aa381c33b812e4891
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-server-linux-s390x.tar.gz) | c86087b4a327503bca118d2bfe6b2e2eef6734603c25e68cae3c4955948334d2b4d5f081d8893cc70c264247c0ec97396e32423db9e49035999dbf04f89f4c2d
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-node-linux-amd64.tar.gz) | e0df58dd88e0aac00186e25d3939e500703071c946d442f71d940aa4043636073d497ae6ad48e1108390035f7e1886d762b8d03cb6538b621b4e93878e1f1283
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-node-linux-arm64.tar.gz) | d429499c87a46fe1fa58c795678d8d11cac7eb7458669477a230d29cb32519496a5dc8ae1d8db104f047ab1ee66b18a159afb0c460f03aef0ad978714d82e550
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-node-linux-ppc64le.tar.gz) | 425650d1370e1b4aa3244e7f549a1a5b5ccfde84c4ac84f7303363957767cd24c748f7d4569b46fa9ee9a9ae676c94ad0fdd1fbf6f2535004af23fdd3456ac9f
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-node-linux-s390x.tar.gz) | 69edfa44446631f84578cceb11ce66c43d1d4547dc808146ff4bf639065b41415e7e210fa7e6e6a4389dfff74f879edb1079d9314f9ffc5d784a28f491449c47
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.10/kubernetes-node-windows-amd64.tar.gz) | bf0972dff63dbda8d94f0ad505c2d62dab0eb5eb6423e54dfad2b948e3585176b695e8adbf81340a08852e7ef2720bc0ea8378fa09bc65bbac0a8778c10d7434
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.10](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.10](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.10](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.10](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.10](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.10](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-s390x)
## Changelog since v1.29.9
## Changes by Kind
### Feature
- Kubernetes is now built with go 1.22.7 ([#127604](https://github.com/kubernetes/kubernetes/pull/127604), [@haitch](https://github.com/haitch)) [SIG Release and Testing]
- Kubernetes is now built with go 1.22.8 ([#128130](https://github.com/kubernetes/kubernetes/pull/128130), [@haitch](https://github.com/haitch)) [SIG Release and Testing]
### Bug or Regression
- Ensure daemonset controller to count old unhealthy pods towards max unavailable budget ([#127775](https://github.com/kubernetes/kubernetes/pull/127775), [@ncdc](https://github.com/ncdc)) [SIG Apps]
- Fix a bug on the endpoints controller that does not reconcile the Endpoint object after this is truncated (it gets more than 1000 endpoints addresses) ([#127417](https://github.com/kubernetes/kubernetes/pull/127417), [@aojea](https://github.com/aojea)) [SIG Apps, Network and Testing]
- Fixes a kubelet and kube-apiserver memory leak in default 1.29 configurations related to tracing. ([#126985](https://github.com/kubernetes/kubernetes/pull/126985), [@dashpole](https://github.com/dashpole)) [SIG API Machinery and Node]
- Fixes a regression introduced in 1.29 where conntrack entries for UDP connections
to deleted pods did not get cleaned up correctly, which could (among other things)
cause DNS problems when DNS pods were restarted. ([#127808](https://github.com/kubernetes/kubernetes/pull/127808), [@danwinship](https://github.com/danwinship)) [SIG Network]
- Kubeadm: fix wrong member list reported when removing an etcd member ([#127962](https://github.com/kubernetes/kubernetes/pull/127962), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle]
- Kubeadm: when adding new control plane nodes with "kubeamd join", ensure that the etcd member addition is performed only if a given member URL does not already exist in the list of members. Similarly, on "kubeadm reset" only remove an etcd member if its ID exists. ([#127621](https://github.com/kubernetes/kubernetes/pull/127621), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle]
- The scheduling queue didn't notice any extenders' failures, it could miss some cluster events,
and it could end up Pods rejected by Extenders stuck in unschedulable pod pool in 5min in the worst-case scenario.
Now, the scheduling queue notices extenders' failures and requeue Pods rejected by Extenders appropriately. ([#122022](https://github.com/kubernetes/kubernetes/pull/122022), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling]
### Other (Cleanup or Flake)
- Kubeadm: removed `socat` and `ebtables` from kubeadm preflight checks ([#127415](https://github.com/kubernetes/kubernetes/pull/127415), [@saschagrunert](https://github.com/saschagrunert)) [SIG Cluster Lifecycle]
## Dependencies
### Added
_Nothing has changed._
### Changed
_Nothing has changed._
### Removed
_Nothing has changed._
# v1.29.9
## Downloads for v1.29.9
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes.tar.gz) | c6f117b107d89ffdec55ab65852438a129f4058a5fabc74509a1c0767311eb1e50a493bbcfa8712843128fb4b48abc19a54f5f0db78e70bab36429b36b77146b
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-src.tar.gz) | 4299af0d2d7370661e7d83f622f355fcfa06116f37ea8965f72c9257121c7c021bfee7c97fc813b2048fb38701346c452db5669b17dcd2aff8c00f98b37588d8
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-client-darwin-amd64.tar.gz) | d92f2aa26442a4d239e393568c13a64952ef2bf134eab87a32ab2aec8e935ce7e7e003b68dd91ab25b172bb0ff3db8a5f3267fda7a94534252d8f3a94e3a1af5
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-client-darwin-arm64.tar.gz) | db2d7df3352fc5ab61d9b46435d766dd8af05ced99e2aa6f8952d38971e97208302296c8d02b572569824aab4c2cf781b72ad12fd551217d3b8f86188a749394
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-client-linux-386.tar.gz) | b590783c8ee02e3705be0b8acd0e8dfb28416ed5cf63b43c34da52f7530c849118379717bf90e452dbd3260febcdc2b6bf379624fa9316bfb01e4e11eb9923f5
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-client-linux-amd64.tar.gz) | be4204f070fe9f890b3fd34b2a784511b3f0393a4d6fca9c461963953da6a74c341920bbe84fa9068571da13243152f5b331809f0bd869c8044f960c913a8ef8
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-client-linux-arm.tar.gz) | b2a9ba3a0616ac8286f89054a0179985c330f122ae67f24c8f0f1da9e4f609ff2519c7a1fb47fe3a953297eb05a4cce31ab5986084efba7759bf45f0df092b84
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-client-linux-arm64.tar.gz) | 33dda6ec9de00bf54cce393a382ed364995d6e69f0ae7b9ffab85e058db2d5889a2b40a584b7d9aa9b4709f09ad7b9253e29ac0e659dffd690ff7c3290be4011
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-client-linux-ppc64le.tar.gz) | 84dddccac89d1a2db0174421579529f81c77f324fe177db18edac9d3ede351bb149f1be8696895dc8a57b277b1091bb4c178b5f88928caf06209d2e1cfa81ee0
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-client-linux-s390x.tar.gz) | 9397cbcd3fdbf8bd81db0ca226f0fc60620ac373a339c70a752dae68b62dd7f4bac235aff62a8053d7712be06d294bf87bce855c3f8c7b209fbc2f32f4ea1403
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-client-windows-386.tar.gz) | bf360ce4b7fe5e73d11a01b752627942df492fc8b651360de5f7de1f8db4a2fdea8d03c2418f2854f44c7704cea0c06b9861a01fc353cc14038c9452ae27af48
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-client-windows-amd64.tar.gz) | ccfd5cd57f56632b3b885a0efe77b8380705fe586a4f9bb1635bd6815b82820048e7f6d6b926e7ce1b6cd5678c252231dc6f6807834f1e181a6eb2c7cfcc79b9
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-client-windows-arm64.tar.gz) | ce005afa1ade3edb9588b1d72bcf44b0b89df522e8c44eae84d5545b2921e5ead6dd1f93a4c2c587f7011bdfff9fa36c6885d8ed39d3b51a46c53a95b21d1c89
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-server-linux-amd64.tar.gz) | e38727541b2bdd14200f7151e79ac049405fb019deaea4c7270fbe6fe449a1d7b1e7c7222ee6d794e4cb9ddfbb1f8e5d80a5bf8004f21af2729da146572aadf7
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-server-linux-arm64.tar.gz) | eaf41e251dd71f4c9a120553e6fa7d61b6fffea1de75a7646e01ada42a007ba5dbeb6dcbb392a5e64c39b7f1fccd236d2adec997cf36e0ed063a074d3dc19225
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-server-linux-ppc64le.tar.gz) | 40eae18be130082d7a98311327592a253dedf5ca49ea50c9d7396685143557f85cfd764d151c928fd41b3f100322e1549607aee33c6206981f999998eeab8196
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-server-linux-s390x.tar.gz) | 073a43837e477101b6fcc3df075b7e24e941961e9ef3d374e1b88c4223509460b932a341a40dc3b4a4530aca53974d370c90afe3c8b803f283aa18fa412c6f07
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-node-linux-amd64.tar.gz) | eaa139e26615df334213fdefce53ac1665c7034684672420ba50c454ee2b4b13e78e297f3aac6207813e02a55b27b83f1484493f846775c48b5d10cb2edae1d4
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-node-linux-arm64.tar.gz) | 54222a22b755fd1646f5afa8a56c87195cbe3a2f552f16b195254a1736a0047055a3ebea708392930ccc8c1dd4b4a7347ace0795bac3c1342633ed3151840c48
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-node-linux-ppc64le.tar.gz) | b3d3f9b4eb11775e63fa3f1618d9ac0f50dc2a0a1280820cc16b9d62efdee4ea9d50f3aad01436fa110c3943aa4af5bf2087560305f9301b0e4c9264481dd397
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-node-linux-s390x.tar.gz) | 19b6fa17b5c9c410dd833e4ebec15763eb5b2c2ff78a65d9766722ecd65ba7918509b622a5661002d0fc51b7e0b0b2db094061763196e3c56b2fe64a5dc0b37b
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.9/kubernetes-node-windows-amd64.tar.gz) | 1a707c4854bb64191db5170a52c57b7e833d7064b0c287bca58379636706e2d6ecb51e5298f0db2484aa191dc6c65dd314d632eb0863a155e85454c89a5ae4e9
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.9](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.9](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.9](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.9](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.9](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.9](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-s390x)
## Changelog since v1.29.8
## Changes by Kind
### Feature
- Kubernetes is now built with go 1.22.6 ([#126971](https://github.com/kubernetes/kubernetes/pull/126971), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing]
### Bug or Regression
- Fix a scheduler preemption issue where the victim pod was not deleted due to incorrect status patching. This issue occurred when the preemptor and victim pods had different QoS classes in their status, causing the preemption to fail entirely. ([#126694](https://github.com/kubernetes/kubernetes/pull/126694), [@Huang-Wei](https://github.com/Huang-Wei)) [SIG Scheduling]
- Fix race condition in kube-proxy initialization that could blackhole UDP traffic to service VIP. ([#126689](https://github.com/kubernetes/kubernetes/pull/126689), [@wedaly](https://github.com/wedaly)) [SIG Network]
- Fixed a bug where init containers may fail to start due to a temporary container runtime failure. ([#127214](https://github.com/kubernetes/kubernetes/pull/127214), [@SergeyKanzhelev](https://github.com/SergeyKanzhelev)) [SIG Node]
- Fixed a regression in 1.29+ default configurations, where regular init containers may fail to start due to a temporary container runtime failure. ([#127204](https://github.com/kubernetes/kubernetes/pull/127204), [@SergeyKanzhelev](https://github.com/SergeyKanzhelev)) [SIG Node]
- Terminated Pods on a node will not be re-admitted on kubelet restart. This fixes the problem of Completed Pods awaiting for the finalizer marked as Failed after the kubelet restart. ([#127209](https://github.com/kubernetes/kubernetes/pull/127209), [@SergeyKanzhelev](https://github.com/SergeyKanzhelev)) [SIG Node and Testing]
## Dependencies
### Added
_Nothing has changed._
### Changed
_Nothing has changed._
### Removed
_Nothing has changed._
# v1.29.8
## Downloads for v1.29.8
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes.tar.gz) | 2de71de7b3fd1683b895933fec27545a8f18bf9c416fe22fe93094ff9d7e1db565bf1c06228d454fa7e2003da440734aafc8b093165f11cc953dea7fc9d346a6
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-src.tar.gz) | 8c4da8b8bc42dc08a0b3d40d3cea8adbe5e1657c99543fbd257024b0b68a03fb50fea8deec48c9eedbc6790a896ec43b7ec93e7070a83fb45247b3de3cdf59a3
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-client-darwin-amd64.tar.gz) | 052855b12388ec052a0d91300b81b5bd9bc31eca27e416908f6ff8bfc63d5a89fe755b9f30704eca2a3947c79ba4132827bb22ec51b35351287d74d877b3723a
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-client-darwin-arm64.tar.gz) | 9b608bcaf11988e51a4a7e0ea9e9040836099a13f6583ab14bfd8838e0d66ac04548d52c480c07cb3bbe1f6b1294f39cb7333ce98c37d0320fb43f2886b91ff4
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-client-linux-386.tar.gz) | 45e25f9fa1c696d6e7231c67d8bb73565210a258a62c8b3036dbb199ec215db3f5db793f57031b7d484358935fdc391e89832493261770a2617bb851755b057e
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-client-linux-amd64.tar.gz) | 0d1459c40a6670635f0f324d0b04ebdf779c46d69d0713c124da4d23781d11e1c0ed9d44d8f8c5c92a71cfe433b914db102f7662660c87b540cf70fecd36981a
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-client-linux-arm.tar.gz) | 9d3ba7fbb303b91479883bcfc1026e6ff0ab83fbb22fc70f85206cc374972475244424704c0c8b6f320af64f871b9c4de8285f4228b8bf5efed89bc28b16ffc9
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-client-linux-arm64.tar.gz) | 41a165f1655aca5e72c9522cabeff9a7a81078a56c6ee1494bd50d10ed3581c5ea02ee9eaafc9f3e42c5e35a416e85fd244a5fed2b49ea7d29cd80fbb6bb68da
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-client-linux-ppc64le.tar.gz) | 98b5db0089eabdc349be36bf2308f044f621e2552db2461f4fa60727d9fcefb188eda2e83ab5e8e62fdd8e3a6f2f897f5c519c6ae6bad639c638c7a4fa7bec1e
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-client-linux-s390x.tar.gz) | c70bedc03846bceb3400d8eff26b473cd2292e25e4cb9d2ba0f149cebcc010c57e65a55e9055ae0a2471a708c56239d273da36047ec2b630c044e15c17674a9f
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-client-windows-386.tar.gz) | cc78a13d41515c098d50bdcfa58212acb2d0e17666feb421f7a88532cbcd45dd66cb92258c9e89e8a50d6c38eddc554023e0838fcdc5da97c50527229f942401
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-client-windows-amd64.tar.gz) | 60487f6641ae257c40e50f0f51c93ab2fefd2536ee9cec88dd276788f1a7372c94544d70789c826d0cca921cb349fa55fd5232ec28fd7ee556ae2090638a29d9
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-client-windows-arm64.tar.gz) | 8adb14dc3f8cbf5a1f4aee111b40080d61272c20bfa96b0a9c8b2ef7d61ff55593aa142bcc24a255c790a59fdbcc2c4deacbc259cef6b382bf08ce707749f9c5
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-server-linux-amd64.tar.gz) | 2fad1113b1befee57e01ae1342771183c834062e7366a4590f2ea0662c3087014bda55d27c64e5fae4d66a32ea8a866a302a98d9f422fff86fb7fa82817ed1d4
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-server-linux-arm64.tar.gz) | 9670fdb0d2f7f38a02e7cfe078a371715ae676cd402b1b6cebcec6c8e0b08bba98d8b08e037061a5af2d161c3a3ddf16a6ae293b79e06a791238174f7d646a3a
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-server-linux-ppc64le.tar.gz) | fc91e2f37b315c457ab7339cc372f806fd589fcf22c7697d18497f0ccb10e8c4bc3f5f204e1ab22427a5651c20e3d5756c60495322b9f758cd41d09c336fc1fb
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-server-linux-s390x.tar.gz) | c4da49e921a7e886dac59f72c324f7adec50eab339ddf09e9c29f05723ce20e3dfee76e31bce0f972cc26b02bca3c622ccd5c875fa82fa6b061329a98e3436c9
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-node-linux-amd64.tar.gz) | 9c3e50ff50ac37cdaa543fbcfc0bf92e55d0f8415b4c0d2eadcde2281a85f9df777c4d00062ffe2c72398712db02bee7d84ed374f11b0075056a39bf21abba5a
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-node-linux-arm64.tar.gz) | 19fa6d168c2326eb6b6ec329ca20b3c669db06af6141b9e5cd4f1be245f6b58af1f086cacfc7b8d663a5905773f639bb35bc0c84b8e25953ef352040e7dec980
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-node-linux-ppc64le.tar.gz) | 104be041c10379c5b1b1b371ee04fd0b9a83e08384ee98a20bdce3f865d0e0d2bd13640ee04bcc1ec46afb618b7c43b19ea99c58a5caf938807adecf86776d7a
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-node-linux-s390x.tar.gz) | 89aecc074b6cacda480f2e5add7584384fc884f2bb2e8058db977b3c96b3da23bd472c37ff18ff6f03996ac9e309f2b4a84333dffbccdb48104188455862ab42
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.8/kubernetes-node-windows-amd64.tar.gz) | d95c44ed0bbfb5ad8cc607350559c88baae5af1fc66b77ac0ee538d612d3a548060b9027663a58aa7c06f8269e435018ed5954752eb4a14c2a16223304454070
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.8](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.8](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.8](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.8](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.8](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.8](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-s390x)
## Changelog since v1.29.7
## Changes by Kind
### API Change
- Fixed a bug in the API server where empty collections of ValidatingAdmissionPolicies did not have an `items` field. ([#126157](https://github.com/kubernetes/kubernetes/pull/126157), [@xyz-li](https://github.com/xyz-li)) [SIG API Machinery]
- Use omitempty for optional Job Pod Failure Policy fields ([#126046](https://github.com/kubernetes/kubernetes/pull/126046), [@mimowo](https://github.com/mimowo)) [SIG Apps]
### Bug or Regression
- Fix the bug where PodIP field is temporarily removed for a terminal pod ([#125404](https://github.com/kubernetes/kubernetes/pull/125404), [@mimowo](https://github.com/mimowo)) [SIG Node and Testing]
- Fixed a bug that init containers with `Always` restartPolicy may not terminate gracefully if the pod hasn't initialized yet. ([#126332](https://github.com/kubernetes/kubernetes/pull/126332), [@gjkim42](https://github.com/gjkim42)) [SIG Node and Testing]
- Kube-apiserver: fixes a 1.27+ regression watching a single namespace via the deprecated /api/v1/watch/namespaces/$name endpoint where watch events were not delivered after the watch was established ([#126151](https://github.com/kubernetes/kubernetes/pull/126151), [@xyz-li](https://github.com/xyz-li)) [SIG API Machinery and Testing]
- Kube-apiserver: fixes a potential crash serving CustomResourceDefinitions that combine an invalid schema and CEL validation rules. ([#126167](https://github.com/kubernetes/kubernetes/pull/126167), [@cici37](https://github.com/cici37)) [SIG API Machinery and Testing]
- Mount-utils: treat syscall.ENODEV as corrupted mount ([#126174](https://github.com/kubernetes/kubernetes/pull/126174), [@dobsonj](https://github.com/dobsonj)) [SIG Storage]
- StatefulSet autodelete will respect controlling owners on PVC claims as described in https://github.com/kubernetes/enhancements/pull/4375 ([#126580](https://github.com/kubernetes/kubernetes/pull/126580), [@mattcary](https://github.com/mattcary)) [SIG Apps, Storage and Testing]
- Stop using wmic on Windows to get uuid in the kubelet ([#126012](https://github.com/kubernetes/kubernetes/pull/126012), [@marosset](https://github.com/marosset)) [SIG Node and Windows]
## Dependencies
### Added
_Nothing has changed._
### Changed
_Nothing has changed._
### Removed
_Nothing has changed._
# v1.29.7
## Downloads for v1.29.7
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes.tar.gz) | e8cd7e372f244dec2e1f8efb3fe8277dc5a246a819b9bb1c49961710ddc871d6fc6a6c1c209e25a82e5f3cc1c7be5bf1af3dd578a2d3394a718cf1a118c5b9f5
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-src.tar.gz) | 9b80bca98bfb889040fd110c630fc00714170548b61031b91802e7ec9ecf379d3c00fae116842bab5d119f5d26d4321b8cc2dcbd833b2220b033eb25dd6a1a34
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-client-darwin-amd64.tar.gz) | d73ba18bcbd070796e248f244c5ea80dcf166f4039e3e4cf2e04d8399e965088b077037ce0fac7d1677c5e3bcb8564da81cc3c54c70a7b98ac26d3200ed3fb90
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-client-darwin-arm64.tar.gz) | 282d6ed67198cecd84b8dd02483add2e1adf52e4eef0f4a1ec5dc29ae7be8e5a985d805468ba1435e1a4a43d86fecae1d7c9771bf245d3b2d96f725b846a471f
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-client-linux-386.tar.gz) | 50822a6a21bcef5ba579de7ca553f21cb30792f0c0cb2ad45fcce7b6b449ceb727f799fe27f321c5b53df520575aa3e25ba3afcf69b2a83e141fe4d46c10bced
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-client-linux-amd64.tar.gz) | dbb773987040f71deb0e45bf807369fd54980172d84635b28a9533c220e73e9f373c632f430a2d33c84e2e191e04837545de9259c7b91ff7e377eee563acbe8e
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-client-linux-arm.tar.gz) | 6f846858041b474be6725b9a2584989157961d27057f71de3218f0f89fb306549359b4a2dc4fba9cdc55fd8a13a578bfd6c3f102f20eac5a75b94378efeec633
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-client-linux-arm64.tar.gz) | 93ac7dafb6a6ddf653a0113289468cf954c80fb0b48a99f94d73912d13f40b6caeb567008e513611d4fdefa3696a7112d851a12b62bd3a1239db725bd4688655
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-client-linux-ppc64le.tar.gz) | 48e9a32bdba7f4149855530406a0a957556f5e7b5453c181dcc2e9a5ea193e9b9bc07a044c9c74de7ed7e30b67af00f62d263cdd45718afb0e84f7fda31b4e7b
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-client-linux-s390x.tar.gz) | 03e167d58c4a7aff42ade34697d5fece42c9f2dc6ea58a0ffc268bb2b03f893eb0b9ef34eca77998359851477620ba083adea3771d4765b2914ea47cc726762e
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-client-windows-386.tar.gz) | 7520b347039ca2965bc8de7cf73c7aeca57380651989e0499f9a346b09f40c124d12c87b89e39322c8fc12f819d769aacd5583c67eec230ee96e8575b068bea3
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-client-windows-amd64.tar.gz) | 06c09cc4e3ebe56dcb34840758b1f68fc9882da7127441ab617be96ae329fbed8e004563ee958c641443778ac0fe34074414ca575b0ec60dcdaa3c72d402ab2a
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-client-windows-arm64.tar.gz) | 3b51e77b8ca29d20f0f1433dae326885b490fdf0d771602cfb23b6d5cb0ab771d7eee5a28ee532158d8f29a47348ef925ce004911218286d99c977e9a07f30aa
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-server-linux-amd64.tar.gz) | 3c8163fb5eac5c15d25de4e36b6f43807bc0a0887c3eeff4e362a06f7244f9132128fd2f27c4ef569f2417c53e1c43c54e3d083a7f07050c47740368dd8612d2
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-server-linux-arm64.tar.gz) | e185cd258f3e81b947fa3ef9bcabcaf3252e1077aebe0acbd1aad63771b3da3989ad5123e7b3678daaba94c9eadfb438dc86e620c981584ba1d8b28b7bcaaf5d
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-server-linux-ppc64le.tar.gz) | ad53229c886cac117e097bc76a199d778dae6df451a24e57faae4e01f7331d68d852353dd0e62979c432664263821b6d922d3f1d7820f6e5aefb31b6ea26d1d2
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-server-linux-s390x.tar.gz) | 9fa62a0cbb85e2e14b2bc3563a1d6f101b208a27a0a1c86d8a6927e8756b1d7055dfc06ad1cd6da837c490ebe816c8d0812030c75d21ddfe821379bbe8593499
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-node-linux-amd64.tar.gz) | af4c39cd368f98cab39e6f4c7062b1a5f0ce0d90741f14b56da13d4bd8c1b63da15fe9e88ce3ac6521527d1819b579fcc96d55eb88c99b2d9454142c7b69c5fc
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-node-linux-arm64.tar.gz) | 57c1eb39a6b6f161e117a7c8c5d5ee800573db10bea106ffb2681eae84404f956761bf4d27034f79fec07045540d507d8cd7871b3b3ce68fd4d09a54ac96aafd
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-node-linux-ppc64le.tar.gz) | b82db79e9f74cea52405ddede53306f5ca5d283709c8dfe59c38ca3d5c9b0a0a0cc640c9de0c0b42015be2cd86cb2c98f4598e466096c6de96a4d2b39b95ddb5
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-node-linux-s390x.tar.gz) | e325787501190e4d3b62b2a20284184e0af96624911bb02816af008d4c1ab6841f12f4cd80fcd073abf02a124409b11017c74f51e6765da2aa006cd90e31e7f3
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.7/kubernetes-node-windows-amd64.tar.gz) | 800129df762412c07745b9b7a1e15402a1623f86151ae58769ff87ffbf7d578e9011e0a0e0cf40fb2bcbe058b5977a9c52c2d1bf221518ecde04f2399a0ad55f
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.7](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.7](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.7](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.7](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.7](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.7](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-s390x)
## Changelog since v1.29.6
## Important Security Information
This release contains changes that address the following vulnerabilities:
### CVE-2024-5321: Incorrect permissions on Windows containers logs
A security issue was discovered in Kubernetes clusters with Windows nodes
where BUILTIN\Users may be able to read container logs and NT
AUTHORITY\Authenticated Users may be able to modify container logs.
**Affected Versions**:
- kubelet <= 1.27.15
- kubelet <= 1.28.11
- kubelet <= 1.29.6
- kubelet <= 1.30.2
**Fixed Versions**:
- kubelet 1.27.16
- kubelet 1.28.12
- kubelet 1.29.7
- kubelet 1.30.3
This vulnerability was reported by Paulo Gomes @pjbgf from SUSE.
**CVSS Rating:** Medium (6.1) [CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:L/A:N](https://www.first.org/cvss/calculator/3.1#CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:L/A:N)
## Changes by Kind
### Feature
- Kubernetes is now built with go 1.22.4 ([#125668](https://github.com/kubernetes/kubernetes/pull/125668), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing]
- Kubernetes is now built with go 1.22.5 ([#125896](https://github.com/kubernetes/kubernetes/pull/125896), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing]
### Bug or Regression
- Add `/sys/devices/virtual/powercap` to default masked paths. It avoids the potential security risk that the ability to read these files may offer a power-based sidechannel attack against any workloads running on the same kernel. ([#125970](https://github.com/kubernetes/kubernetes/pull/125970), [@carlory](https://github.com/carlory)) [SIG Node]
- Drop additional rule requirement (cronjobs/finalizers) in the roles who use kubectl create cronjobs to be backwards compatible ([#124883](https://github.com/kubernetes/kubernetes/pull/124883), [@ardaguclu](https://github.com/ardaguclu)) [SIG CLI]
- Fix a bug that Pods could stuck in the unschedulable pod pool
if they're rejected by PreEnqueue plugins that could change its result by a change in resources apart from Pods.
DRA plugin is the only plugin that meets the criteria of the bug in in-tree,
and hence if you have `DynamicResourceAllocation` feature flag enabled,
your DRA Pods could be affected by this bug. ([#125644](https://github.com/kubernetes/kubernetes/pull/125644), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling and Testing]
- Fix endpoints status out-of-sync when the pod state changes rapidly ([#125675](https://github.com/kubernetes/kubernetes/pull/125675), [@tnqn](https://github.com/tnqn)) [SIG Apps, Network and Testing]
- For statically provisioned PVs, if its volume source is CSI type or it has migrated annotation, when it's deleted, the PersisentVolume controller won't changes its phase to the Failed state.
With this patch, the external provisioner can remove the finalizer in next reconcile loop. Unfortunately if the provious existing pv has the Failed state, this patch won't take effort. It requires users to remove finalizer. ([#126044](https://github.com/kubernetes/kubernetes/pull/126044), [@carlory](https://github.com/carlory)) [SIG Apps and Storage]
- Kubeadm: do not exit with an error if the "super-admin.conf" cannot create a ClusterRoleBinding for the "cluster-admin" user, due to the ClusterRoleBInding already existing. ([#125821](https://github.com/kubernetes/kubernetes/pull/125821), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle]
- NONE ([#126128](https://github.com/kubernetes/kubernetes/pull/126128), [@cji](https://github.com/cji)) [SIG Node and Windows]
## Dependencies
### Added
_Nothing has changed._
### Changed
_Nothing has changed._
### Removed
_Nothing has changed._
# v1.29.6
## Downloads for v1.29.6
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes.tar.gz) | f68274b7f82030c8f090b2bc6730484a771845632fb6f6d2bbd324ed36d5b02aea4fe9b7196c85a156a1979cb3b169ec802d919420eb73c356b2ebc914d72e2e
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-src.tar.gz) | 69747294aa4d4ec5be60769c8afcc995601f191b88dc9fefe8ab85cc5b09ffa7e8de8e3f3a7fe16cbd518b248ca4f335bd514b539627f606b9cb22889ec37e1e
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-client-darwin-amd64.tar.gz) | 50466e035d764f4d04720dd4cb5c0c5e39c2c4d8c6773a72ddff4b4d5f9045fd630e6dca43109553a3e6b3569f2792d7c66e5b286f6acf36add6d03344079c92
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-client-darwin-arm64.tar.gz) | 9d1f5c4008f40e4be0437457d2d872b58dea2bc52c7ed0145c7c452302f814be07f071f311844924cdebad890c80faaf191fcdb6198c8fd03fbebed0191e93b4
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-client-linux-386.tar.gz) | 3f0951f370c03d2e849bd38039ad83bdce1caefd069e532fc1af16489bc11480ae97f7f8ba620cbb5cc0dec1a9df7387b6b9577ea5d0077253e487e7c809f9d2
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-client-linux-amd64.tar.gz) | cd078f9b5bb8add3ee3a6a5bfe8bcd1e53d2c7e319d366a9136272c007380ec520c42251cfed73ad2dc37b5a2251f9e72c20c9fc2a85278e01937975f78a27c0
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-client-linux-arm.tar.gz) | a567700f0c2a092107f7c7768777eb8de953bce91529cf0d4b3c3b1808ce5008f71a95bc13d7f2551383cf4e225395eee1700a5001a6a33a3945e138486da943
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-client-linux-arm64.tar.gz) | e126cbe2502baa19f8e796df997bb2a08ecddb59d97f10c2fe74b6f77d5150ce9fdaa09765c31d3b4470e2db8a5f7f348602808d033632bfaf622e5fe093c930
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-client-linux-ppc64le.tar.gz) | c25d02402c58b6457f45c7e95b14a060fecad56504633ff33cf48374645c0ef2ec842063af7db9e3aac1fb305c4cc9e5bc92278898e0f4d1362834abdb7dfc79
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-client-linux-s390x.tar.gz) | 71ac395e28ea713151c54969f293a13c770fc72ab41a4f9fa8d81239f8e06e270ccc293b9ba5c66b33a2570ef1f882737237663aa16350bef8914d8d80da25c2
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-client-windows-386.tar.gz) | 3e96173fa38ee50e54a0278fc7706aabe084ca477d18b7eab73b1f063fb891f5bfa441c881b092a721229951a36e3f03c71936c3dbd66d6603ed6b6ea4c0d34b
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-client-windows-amd64.tar.gz) | 26890a01fa096d957db2eb179d8b84ace18b36a7cb408dcd23a0b631f26dd023cded5268154087e6a912e83141cd532b1d40b3af96d85cead7ed4f82ef183fcd
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-client-windows-arm64.tar.gz) | 0e0c4a8a75a5dbd84dbcbf4a99adfdd14c9f231d49be23d716a20fca5b78a85367932e973a020e8484c61b97c59660f1f1681e636b8e716957bc4ff3144ab457
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-server-linux-amd64.tar.gz) | e98ecf1787f1e1949a696f58a26df95c4e65675d9e83c36fd6a223dc38f3f9d11e82b13819076d332e262d59c3e292f42cd4918834c30687cbcec31d0b0c5e31
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-server-linux-arm64.tar.gz) | 0a1e2c8e1fbff2d134bf5a372ab89021352ca76ae9d464a2da65160b891da9ad380aaefe4cd7b1f3a194e133b55593e7f25feed3b99c0bd0170570baf0922c84
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-server-linux-ppc64le.tar.gz) | ae3211729b00cb5ce709508b815815f00c570ad6df5083ffd0ebc197b071cb1cfe26fbe779e1fda6aa1e62e8c32c3b9361abf2c85c5a757902080d21735c3cc3
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-server-linux-s390x.tar.gz) | e5ed1def383d447984c3d53e5f06b5301140934f89598951e9deccd76906b5d836e4ff1ed1c3376a015f0cc47ed283d55536999f49e744d3418af4e28d9b6752
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-node-linux-amd64.tar.gz) | 4271bb1624e83994c7687327a5e330cefbfc456f8bda2622c2450023de2fdf3afe463cf02035f5d3069efee02c44d3580c498d89f7e4cef14783f1fee19844ef
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-node-linux-arm64.tar.gz) | e56413199eb611973d71cd211e2385da8ceca459bb470f946dcc4463a93b275b5b4291c6a6145eb1c913d7a66f2f10d33d218dca929b4894f7f63d793db2ef47
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-node-linux-ppc64le.tar.gz) | d44b391039b928aeadb436fcd7a21a12d34c8279d235a69c219caad538a88d2441d33850449822214ecb990f2c62957883c01556ee88ec6306a76c5364600f68
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-node-linux-s390x.tar.gz) | 012da388d13853774be6a359ab6a3dac16564d35137f24b03705d5e88c381a31793ad670d193f64000699883378ae9d4a1db3c0ca14dced68894c1bc4c9ad5bf
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.6/kubernetes-node-windows-amd64.tar.gz) | e388cc29050e62c1217b9049863d102032112c1e4a3d61c9e019396363809ddcf35d7291f9c64327ce29060109acb65d72e2ba62293f9039635feb082d3c859d
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.6](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.6](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.6](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.6](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.6](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.6](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-s390x)
## Changelog since v1.29.5
## Changes by Kind
### API Change
- Improved scheduling performance when many nodes, and prefilter returns 1-2 nodes (e.g. daemonset)
For developers of out-of-tree PostFilter plugins, note that the semantics of NodeToStatusMap are changing: A node with an absent value in the NodeToStatusMap should be interpreted as having an UnschedulableAndUnresolvable status ([#125307](https://github.com/kubernetes/kubernetes/pull/125307), [@gabesaba](https://github.com/gabesaba)) [SIG Scheduling]
### Feature
- Kubernetes is now built with go 1.21.10 ([#124830](https://github.com/kubernetes/kubernetes/pull/124830), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing]
- Kubernetes is now built with go 1.21.11 ([#125367](https://github.com/kubernetes/kubernetes/pull/125367), [@cpanato](https://github.com/cpanato)) [SIG Architecture, Cloud Provider, Release, Storage and Testing]
### Bug or Regression
- Emition of RecreatingFailedPod and RecreatingTerminatedPod events has been removed from stateful set lifecycle. ([#123809](https://github.com/kubernetes/kubernetes/pull/123809), [@atiratree](https://github.com/atiratree)) [SIG Apps and Testing]
- Fixed PersistentolumeLabel providing wrong topology labels to Azure Disk PersistentVolumes when the external Azure cloud provider is used. ([#124528](https://github.com/kubernetes/kubernetes/pull/124528), [@jsafrane](https://github.com/jsafrane)) [SIG Cloud Provider]
- Improved scheduling latency when many gated pods ([#124849](https://github.com/kubernetes/kubernetes/pull/124849), [@gabesaba](https://github.com/gabesaba)) [SIG Scheduling and Testing]
- Kube-apiserver: fixes a 1.28 regression printing pods with invalid initContainer status ([#124909](https://github.com/kubernetes/kubernetes/pull/124909), [@liggitt](https://github.com/liggitt)) [SIG Node]
- Kube-scheduler: fixes a 1.29.5 regression that can lead to a scheduler crash when processing pods with affinity that doesn't match a real/valid node ([#125041](https://github.com/kubernetes/kubernetes/pull/125041), [@AxeZhan](https://github.com/AxeZhan)) [SIG Scheduling and Testing]
- Reduce critical section in watchcache to fix kube-apiserver scalability under heavy load of list requests ([#122027](https://github.com/kubernetes/kubernetes/pull/122027), [@wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery]
## Dependencies
### Added
_Nothing has changed._
### Changed
_Nothing has changed._
### Removed
_Nothing has changed._
# v1.29.5
## Downloads for v1.29.5
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes.tar.gz) | c08b09b3fb97968f9e6c59bccf20215cdbb61ebfee641bb8bae75262876eb855d7758017d7e5bf95987e6f728a28fa43da6a4025f532a9b2795349fba67dba6e
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-src.tar.gz) | ea8fd3c4b89ba2dcd0897124df9d9bbbe7d816a2d2e3d7a9a4dcb803f7f6aaaabe1dc9602cb37f698de3e846ad95ec0c4a8143b2a8214c1561f2753f165cc347
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-client-darwin-amd64.tar.gz) | 6bd763b2f5fa0e212551eb520a9e1f5f4e3ad8ed5b621b17d48a5f09b7f338ee98d8a6edaec182cd45de7279e56ddf1b93dd117512e3f44eb94624b34f819b70
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-client-darwin-arm64.tar.gz) | 89571830f9f957b941c87a6f7073fa7942a4e13cf5b3001bd6433811c022fc3b514d30bfc9d35ce0683ce1d9bc66a0aee8416556187e6fe5d58fab3ab98e061a
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-client-linux-386.tar.gz) | b0c225ae6e17a193a0532129e5b98f19a29dfc2d0075505738744bf265c53dc9b3219bbd0688ef75994f27fa3d0108182b9916de69a8f13c308ed91624dfc9c8
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-client-linux-amd64.tar.gz) | 8c03f8aa1cfe90b15f5ef590ef2224100a42aba7518049d4a65a10f2898b6c6354a66d2c3a2d24e06d5a34ff1dffaed8073fc3918768374a499e70ecc0fce90e
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-client-linux-arm.tar.gz) | 64e6ad92f21537e3191d8fe53d05a47074fd3b2a32d0bacdb001f74a65a8c55c92a83dc04b5151db3b62df9c81846a730a0f08b30e177a35fc92b8069cae94cf
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-client-linux-arm64.tar.gz) | 56c2a700ec996c1310747b7576ba155d018bf9496d131ec2dc69fe332fbd5c2e4b0ff7d6f3bd6dc621c5f6b067e5a540abc8a6450188fc50bd0152381d1d436e
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-client-linux-ppc64le.tar.gz) | d70cf260be09a1268c6c49b16ecc34ce875cdd9ddb544b9b4f445f08635b0bbb0039a0a287a38c757294cd2cb57a5b297e9eebd8192cd678656498b43bedd725
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-client-linux-s390x.tar.gz) | 7b373ac1336bda248b98bac9d92a44cb584436d3fb0b7092eda1510142c858f3ea91b73d3a64c556d60a0f2549d3a8da64f3263ae371b2b60fd73ecc917eb59e
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-client-windows-386.tar.gz) | fe58b678675a4b7de5e810cd7206de09d2f384fbad0abff8f74ee51ab0e3f8d0686dfb6ff1d556bfa4fbdc0a38f475d41447aea51041a5e6d99363444242d94c
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-client-windows-amd64.tar.gz) | b791403b1ae104afa945d1988bfde62330ff33661478b472a47410f4a5faca243ea790060be327a7946e7a45cf60d23204e6a7ce0809f0d29ce79ee5ffab5fe3
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-client-windows-arm64.tar.gz) | d29bd253bbd8a9c20924651cb9025de1266968ffd835c4af46d5ff0c867e454fd0f4995563add753c157fc4486b8b7a25d5fed439f7f13b8c0d14f98764cb269
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-server-linux-amd64.tar.gz) | 05a392b9882a33ee84bf3fc31ea54900e6f9ee48dd18a637fcf92bcd3c7e61052edce85da32a9da1cb530b471907ea16d903907de8ed85d7059acb89161d1f87
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-server-linux-arm64.tar.gz) | c8e2d743e4ed55ae6a8aded892b8786812bd9d682cc581e5b12d8211c974984b45d663026f06c0faee3aec97cf9aa84b1a19f09efe61b57168c3d7d2c5e1ba51
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-server-linux-ppc64le.tar.gz) | bb8f40ff65b879f9465ae76c9e235b963b39b2633af5307074c5008d6f6a27647cb645a27dfc5d970e69acd9c21c3d22d1e3fae8626e42c91edda1ba75b629a1
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-server-linux-s390x.tar.gz) | 6688da58a52bf0d6e9b20eceade272e69c3b4880f777da30ff6f6a1857b59bd90da7d5bdfbe4cd76d45768457152d3265375defabcd777a040c98a5f22f9097f
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-node-linux-amd64.tar.gz) | 44ca6907a7bd8b14db8ab15d6acd6da4e8c3717f94e83f57e4cb1efabef102e1085b25b118a5423420ee56c4e9445ded8b4b5465a151cadcf1b73a57b8aa3691
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-node-linux-arm64.tar.gz) | 75606598d442c09fc815da8704c6059c3128e93fb3d95f77884df05187ffe99da2ca3fde5c13cdd6b43eb2d046dcd320e8f7e786000058fbcb21c8fa8350eacd
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-node-linux-ppc64le.tar.gz) | 1c1319087a6d79bce42bc2ce9d179457378bc8a602fbd6acfb02daaec22da523bd0a6d5c8570da969ec518f5c3aebd846894a4cdcbd35b1800a95713b55c636d
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-node-linux-s390x.tar.gz) | 8a4e06779a0c89f9f9fd9a11d950865466414dbcdfdfc1d5309e6a8fa5dc190fdc5bedf8a04b2ac282c60939a70d1fa04485ed235e39386c6a7fba131ab70c8d
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.5/kubernetes-node-windows-amd64.tar.gz) | c193fa370cedbbf775db2e584461b3338bd5260b3822debd04ceac9b32936ebe11b8c46392299da05fc9b4d2bcb98980acb34c192b42aa452cf06369df14184f
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.5](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.5](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.5](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.5](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.5](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.5](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-s390x)
## Changelog since v1.29.4
## Changes by Kind
### Bug or Regression
- Fixed a bug that a pod may remain unscheduled if any PreFilter plugin returns nodes that do not exist. ([#124559](https://github.com/kubernetes/kubernetes/pull/124559), [@chengjoey](https://github.com/chengjoey)) [SIG Scheduling and Testing]
- Fixes a 1.29.0 regression that introduced a possible data race that could cause panics in kube-controller-manager and kube-scheduler ([#124518](https://github.com/kubernetes/kubernetes/pull/124518), [@wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery and Scheduling]
- Kubeadm: fix a bug when using external CA mode and trying to upgrade to 1.29 using "kubeadm upgrade apply". Show a warning that kubeadm cannot sign the new "super-admin.conf" as the host does not have a CA and show some instructions on how to manually migrate to the separate "admin.conf" and "super-admin.conf" kubeconfig files. ([#124682](https://github.com/kubernetes/kubernetes/pull/124682), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle]
### Other (Cleanup or Flake)
- For apiserver_storage_size_bytes metric, we are renaming the label for etcd to be "storage_cluster_id" instead of "cluster" to to reduce conflict and be very specific. ([#124294](https://github.com/kubernetes/kubernetes/pull/124294), [@dims](https://github.com/dims)) [SIG API Machinery, Instrumentation and Testing]
## Dependencies
### Added
_Nothing has changed._
### Changed
_Nothing has changed._
### Removed
_Nothing has changed._
# v1.29.4
## Downloads for v1.29.4
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes.tar.gz) | 837cc6ab833228e387e787bdb1508d74bbf79c380ac71fed7acaf9e239f3f2fcbe3fcfb9a9e41711620ec21e6cc3a5984148dc80515f37a6fabb02e50a82a29c
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-src.tar.gz) | 716c6fc59d8dfed72ed45dfa5535dff3bae3bd3bd9f8641c2068d76c06c21c7ebc8ba0626374312b4a20285277cb8ea4df446199caae9cc0d992346c9dc09479
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-client-darwin-amd64.tar.gz) | 01506990cf76344fb12207e3e88a7c38a926ad8ccffc00b0ddcfeff9a5312b01438ef8c813e877e4b856cf1cc3f52dada7cd687a487797168a3436b66c64fc9b
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-client-darwin-arm64.tar.gz) | f77fd94e97f1ecda0715f930d34ed85789f0eb0db6e3aacb35e7ebfbf101efe365169deff98c6b6d3d92b8a18d311032504adfe43e8c7f9fd4a42de2352de389
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-client-linux-386.tar.gz) | c665a345c445878120a05d2c9755a80109423e333bace0423b7208a3fea91c019e3b33c4bcb0761d52a0b5d17258249881b4dd1a5a9a584ae04e4887d3e34b96
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-client-linux-amd64.tar.gz) | c13235bd929eaaf4d0eaaa9ba883e95ce27a402ca7256c634e20a027fbf72db8834de8ea2ca7238e1fe92859e0edc7384a1cec7fbe2b7a5adf07b2e5cf99b04f
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-client-linux-arm.tar.gz) | 9c348edc150340219f4b9b8bfd17e1747df9884f9407126c1585c3a817fec5561d5d02ddaed0317ac515bf6142cc7530fcac9e735f60f92390d05e5517c7d166
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-client-linux-arm64.tar.gz) | 614cd5b5881c583505d089c09c221e4a06da0dc8b5ac70b3d93d7e2a58c8b439446a646d0bb53396c2a48535808503daa6aa1a37f43affe22176c2211fdc2cc4
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-client-linux-ppc64le.tar.gz) | 3c17be398175d0f882a0c1894c05d04fb564a4bd01ac95cbc5dab4902f7827425ff00d3fcee1fda22a31f81888effefff5a9705e266e44aa1a6c8be9ca42f0ca
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-client-linux-s390x.tar.gz) | 0a202fee4e78fffc1a25538529a9751dd7d421f75244cf6739332f606bfbf5ae455519f1f5b4378e7f22b4d8e1104f3eeb1acd37739e213b437db78f429dbc49
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-client-windows-386.tar.gz) | afa38bee4b8d09347a5d4c1b4fde74d337d42efc411d62b336c841b2bc7bf39d6355557a169132ff69ce5d239a01cd59298a061437580c168400399b0a6c71d9
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-client-windows-amd64.tar.gz) | 653c737e582a43dcb7f8475c61bbffbd892b363dbb015d30c7be9413839ffa2564a97ceeea3e273fd0c46025e06e8828a4eedd0ff7983ad848e5647bb03f2249
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-client-windows-arm64.tar.gz) | b779f64dac14f3d01b2565f093c27e71c793e0b0b2ff491419730d96ec5782152d7266c68f9b892264752022bb6ef600079649df9ab30e7a329bae2835a43803
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-server-linux-amd64.tar.gz) | 8554f4e2c828df8e2a3aafceea941712ca079f0710ac1390a1bde32e09fe9a588c217462a2666e0d542c42f1e42881fc975751ce7d5b81c9f88aed0c8302f6dc
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-server-linux-arm64.tar.gz) | 850c1233ac2b267964ed840b6f4c4d51a961d8e86b1bf78ae4bc50e31e8255f04fdbd50c91d0937b9ce197dcac30fcc3df5be1b537a6a962b9e984a4754e74bf
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-server-linux-ppc64le.tar.gz) | 45e1abd9ee2efd0acac599ff3fe360835f1d02cf1075ba6499d41a5f41bd98e96868fc9d3555c41a535a88a3450f64fb7552d6b9f6a281837451b059c8c3695c
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-server-linux-s390x.tar.gz) | 2c924db2f1a6ba83d8364d373d79714d6f7f2697fcefb3f9ca3b89ff46f194ae87b81d3567f2e77bbe9f490e68468c64cf86f0e952a7c49efc87600b92bc9a36
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-node-linux-amd64.tar.gz) | bf5eb2f4ef8e215941d98ca62bc4901fcad7f68dda153e1f077f9688b1e3f273d3b25bc48167cefd0a1f1ce1e0a3525d71bf7aa37b7fa8b8f639af35df0233bd
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-node-linux-arm64.tar.gz) | d983706b1975d3b8c3cb1dae833efc178337df2c2abdd835588a0be6dbaef55c27e1df993e8623b87f56f7f7a8f6de34390e8a085911644fa8af2d49f47512e0
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-node-linux-ppc64le.tar.gz) | 2d0358f1c7b6bc8146a7ec386b44e037dc4dd46b17584a276dc473ee57f74d0f5895b001217cef086785fa12482236fcfa96e005bc46c43fd42edd6e332f6b7f
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-node-linux-s390x.tar.gz) | be7a9cf871c0255df63e16f21005ecf19ad8702a3b4be6483ed158aa8a85ad5ed8719ddabaf85110a2e05ec777fb7426967926be8a7d3a3ee51b557c181b78b9
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.4/kubernetes-node-windows-amd64.tar.gz) | b4a632a37f76b2486e4ded68928edaa159b41d9f428c396710ca67582040439df0f11dfeeb39d4ba3eef02186ffe937eae08cd9d5fdad84bcf43a682d1226d13
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.4](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.4](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.4](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.4](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.4](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.4](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-s390x)
## Changelog since v1.29.3
## Important Security Information
This release contains changes that address the following vulnerabilities:
### CVE-2024-3177: Bypassing mountable secrets policy imposed by the ServiceAccount admission plugin
A security issue was discovered in Kubernetes where users may be able to launch containers that bypass the mountable secrets policy enforced by the ServiceAccount admission plugin when using containers, init containers, and ephemeral containers with the envFrom field populated.
**Affected Versions**:
- kube-apiserver v1.29.0 - v1.29.3
- kube-apiserver v1.28.0 - v1.28.8
- kube-apiserver <= v1.27.12
**Fixed Versions**:
- kube-apiserver v1.29.4
- kube-apiserver v1.28.9
- kube-apiserver v1.27.13
This vulnerability was reported by tha3e1vl.
**CVSS Rating:** Low (2.7) [CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:L/I:N/A:N](https://www.first.org/cvss/calculator/3.1#CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:L/I:N/A:N)
## Changes by Kind
### Feature
- Kubernetes is now built with go 1.21.9
- update debian-base to bookworm-v1.0.2 ([#124197](https://github.com/kubernetes/kubernetes/pull/124197), [@cpanato](https://github.com/cpanato)) [SIG API Machinery, Architecture, Cloud Provider, Release, Storage and Testing]
### Bug or Regression
- Fix pod restart after node reboot when NewVolumeManagerReconstruction feature gate is enabled and SELinuxMountReadWriteOncePod disabled ([#124140](https://github.com/kubernetes/kubernetes/pull/124140), [@bertinatto](https://github.com/bertinatto)) [SIG Node]
- Golang.org/x/net is bumped to v0.23.0 to address CVE-2023-45288 ([#124180](https://github.com/kubernetes/kubernetes/pull/124180), [@MadhavJivrajani](https://github.com/MadhavJivrajani)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node and Storage]
- Kube-apiserver: fixes a 1.27+ regression in watch stability by serving watch requests without a resourceVersion from the watch cache by default, as in <1.27 (disabling the change in #115096 by default). This mitigates the impact of an etcd watch bug (https://github.com/etcd-io/etcd/pull/17555). If the 1.27 change in #115096 to serve these requests from underlying storage is still desired despite the impact on watch stability, it can be re-enabled with a `WatchFromStorageWithoutResourceVersion` feature gate. ([#123973](https://github.com/kubernetes/kubernetes/pull/123973), [@serathius](https://github.com/serathius)) [SIG API Machinery]
- Kubeadm: fix panic in the command "kubeadm certs check-expiration" when "/etc/kubernetes/pki" exists but cannot be read. ([#124124](https://github.com/kubernetes/kubernetes/pull/124124), [@carlory](https://github.com/carlory)) [SIG Cluster Lifecycle]
- NONE ([#124327](https://github.com/kubernetes/kubernetes/pull/124327), [@ritazh](https://github.com/ritazh)) [SIG Auth]
- OpenAPI V2 will no longer publish aggregated apiserver OpenAPI for group-versions not matching the APIService specified group version ([#123624](https://github.com/kubernetes/kubernetes/pull/123624), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery and Testing]
## Dependencies
### Added
_Nothing has changed._
### Changed
- golang.org/x/crypto: v0.16.0 → v0.21.0
- golang.org/x/net: v0.19.0 → v0.23.0
- golang.org/x/sys: v0.15.0 → v0.18.0
- golang.org/x/term: v0.15.0 → v0.18.0
### Removed
_Nothing has changed._
# v1.29.3
## Downloads for v1.29.3
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes.tar.gz) | 4b168808633fe4333fa36d749f385df0a35b79b3246cf06c6a9216a8892b26139b2519b57bfaa6b075cf9099fe527526e7918e34fc0d70d3383db456283ca149
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-src.tar.gz) | fa2994aaf691d34c60745925686efd5ce85b180a591f0892087af3c831fb0703ac01ae80f0dde3b0b414d57419aa9a3956d26500755674a1a6d6e68d31b4a84f
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-client-darwin-amd64.tar.gz) | dbe82475b73cedefeb6cd296c38a49321c311a2b6dd893c2b4f4ffd608ff757629d20280dfa41e87dc8afdef0a733c734a4f583d2e883ced26767ce74e62032c
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-client-darwin-arm64.tar.gz) | 2dfbc1cd16547d8b58c5fdf12c46df6c3b90547ee27ed8d78de7668d4ad4cd6d75d6439a898358701fa41d87c0b21ba2fcb9363196edf327b2ea2956c1fef499
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-client-linux-386.tar.gz) | 8b86ed080cc9008f1dd479b7d69ba7c20ec89f0a4fc85c07dabc6a15102f877d5d4fc0aa50da0f5c177d1660adac3d24771725ac4f0c88788085f0e340927cfe
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-client-linux-amd64.tar.gz) | c9cc7ab9e3aa776f2daab3a9e10ee78d57d0c081ef43f8032de36a61c6425ba527d5df92611b058672be0975a6b97ad3f3a169e282c26275d2c0e59e1f9b1173
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-client-linux-arm.tar.gz) | 2c9c9da97b615f51aef33366a1be5ab3ccfadae8c023f81227d42d7b60df9d7735e403becd1f929178f72f7e568336317c7b76700faeced59fb5266a6982ca62
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-client-linux-arm64.tar.gz) | f66d3734ce43f54d2536f1e5ae20aa85e8602b86aad995573c852d6f03bc50907bd84bdd42d7cb14f246ab27ed0dc840502a540a0bf312b7faf9712053b0f775
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-client-linux-ppc64le.tar.gz) | 080a42f50891ebfcf4ec4588e8af6f5ccbed318825a31840a0219f511f84170eecd73b03460bf8284bdb7796015be2ba793dfd8017c96d41dc1cf41cfd9947ce
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-client-linux-s390x.tar.gz) | 3458ba376bad3bf098b4553007c9438c7bd47dc9b7be9d5e6fc05a81be3c3861781f385829be3e6072ded56f14d509fa6e11862802c7113f48fa26bda532622d
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-client-windows-386.tar.gz) | 225a6b79b442bcbe421317823c98ec3a7fd772ecd2e8758e37eee39e87da318585718dfe62583dfae0c2fcfeccf2290bef29b03029e14735a26fdded3b1febc9
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-client-windows-amd64.tar.gz) | fcfc410f59ff6dc053f7f7786cd486bcd534a419c6db022c0069c22b58d6bbf8ca5c4f58c3823b7bdc4d501dd886e86efeec04fb3469f938b0c8fe0fa545aece
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-client-windows-arm64.tar.gz) | 70100f18c229ecb675dfb459ca64b0545ff03d83c25a04d46e3a445d990649d80e4fb273bd04ef76f81ad06c412d8f2ade53d920b11d64fdfc7d0ab616b53649
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-server-linux-amd64.tar.gz) | 91349b3d1e2d769b4d727531a4c33a4a596dcadeaee617e339f543ca24892acf3930a8e45a810c012ece4fcd4d376700fd3810aef6cb40955a8c7b996789f514
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-server-linux-arm64.tar.gz) | 659e03e65bec126fac64ddc277d906afeefce6024a80fb6642990aaf588e54e6cd83edfa0f6e30a4570ded1cbc2c58be8f854ef8cf5a40a6e2d28e93288caf6f
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-server-linux-ppc64le.tar.gz) | f735fcdf243599ae8a70069cf3285577683f7a3f1890681da53c88cf50f973107695213401910c9849d6065a02934b8dc20704566edb2d85f019db099c6d0d04
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-server-linux-s390x.tar.gz) | 4888ccc80f171512c4ffb1cb3e61d5255c63aa331a5418830a28d7ad9360960e880666b9281a19e54c767636bc5c4eb66c253edab82fe8a3259df54fe885a1e2
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-node-linux-amd64.tar.gz) | 704a3039a9b203dee4f74b219162d8295dd85e16ed385fb8106a14e75f768623adf2da3083991625709ac403bd5785ea683f1451dbb58637b8b53be85627b12f
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-node-linux-arm64.tar.gz) | 794fcc502fe98b0196e06db1cd66f0700fe650d7ef423b85b4ecd60bf0397ab754ec3aebab79eb1b8dd42dc068a8522a9cc0a851bea977bd04801451ea653a25
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-node-linux-ppc64le.tar.gz) | 01f8bde765edec9bbaba7602570fbf1c2cb401185e2f61e855b1a4cfa1cdb3a91c6bf91111fccb9c3c3c425fa5ccd3d6edbf5ef9e62b643f7abca7b9be7d4539
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-node-linux-s390x.tar.gz) | 48573b3da001cb96113e8cd1fbfdaf017f8d00485848dfc7ff09b123ba26012ba0ac7ae27494619c1815dee6e621d1d55a86331ddd011baca6612df42f2d1fab
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.3/kubernetes-node-windows-amd64.tar.gz) | a1f66a9eac3b6af9138559a1630559ffedec93b2777d83886bcda36679fb453a26bd6e0fcac0ce9db89ff5dfb5cc4841fc927ed14dbd6d00b1fb12041a1b1d8c
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.3](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.3](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.3](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.3](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.3](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.3](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kubectl-s390x)
## Changelog since v1.29.2
## Changes by Kind
### Feature
- Kubernetes is now built with go 1.21.8
- update distroless-iptables to v0.4.6 ([#123773](https://github.com/kubernetes/kubernetes/pull/123773), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing]
### Bug or Regression
- Fix error when trying to expand a volume that does not require node expansion ([#123055](https://github.com/kubernetes/kubernetes/pull/123055), [@gnufied](https://github.com/gnufied)) [SIG Node and Storage]
- Fixed a bug that an init container with containerRestartPolicy with `Always` cannot update its state from terminated to non-terminated for the pod with restartPolicy with `Never` or `OnFailure`. ([#123709](https://github.com/kubernetes/kubernetes/pull/123709), [@gjkim42](https://github.com/gjkim42)) [SIG Apps]
- Fixed cleanup of Pod volume mounts when a file was used as a subpath. ([#123052](https://github.com/kubernetes/kubernetes/pull/123052), [@jsafrane](https://github.com/jsafrane)) [SIG Node]
- Fixed the disruption controller's PDB status synchronization to maintain all PDB conditions during an update. ([#122056](https://github.com/kubernetes/kubernetes/pull/122056), [@dhenkel92](https://github.com/dhenkel92)) [SIG Apps]
- Fixes an issue calculating total CPU usage reported for Windows nodes ([#122999](https://github.com/kubernetes/kubernetes/pull/122999), [@marosset](https://github.com/marosset)) [SIG Node and Windows]
- Prevent watch cache starvation by moving its watch to separate RPC and add a SeparateCacheWatchRPC feature flag to disable this behavior ([#123693](https://github.com/kubernetes/kubernetes/pull/123693), [@mengqiy](https://github.com/mengqiy)) [SIG API Machinery]
- Restore --verify-only function in code generation wrappers. ([#123261](https://github.com/kubernetes/kubernetes/pull/123261), [@skitt](https://github.com/skitt)) [SIG API Machinery]
- Updates google.golang.org/protobuf to v1.33.0 to resolve CVE-2024-24786 ([#123763](https://github.com/kubernetes/kubernetes/pull/123763), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node and Storage]
### Other (Cleanup or Flake)
- Etcd: Update to version 3.5.12 ([#123188](https://github.com/kubernetes/kubernetes/pull/123188), [@bzsuni](https://github.com/bzsuni)) [SIG API Machinery, Cloud Provider, Cluster Lifecycle and Testing]
## Dependencies
### Added
_Nothing has changed._
### Changed
- github.com/golang/protobuf: [v1.5.3 → v1.5.4](https://github.com/golang/protobuf/compare/v1.5.3...v1.5.4)
- google.golang.org/protobuf: v1.31.0 → v1.33.0
### Removed
_Nothing has changed._
# v1.29.2
## Downloads for v1.29.2
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes.tar.gz) | 5eec915a8064dc3323cec80db18d213bf6e8c7e805b01a071681e0d4c0af4785d6b8b63c8cd77077dac14347ea7410384ac4bd866db6f83e26808482fe357583
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-src.tar.gz) | 3ced73b388123240a2228506ffb678aea0983b2d152978255ce7e0488c5776749319170187365e3998fb9d92619abe46256036d02824fc75e59e246e27ba85de
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-client-darwin-amd64.tar.gz) | 3d7e9b86c93d4bdd1a32c30967fca3fa6fb932aab7582d8a5c7389b9f7a4646e4b39a7c478ae360d2ccdd3b52932b3c7f7a0094557693b046b141c556599e154
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-client-darwin-arm64.tar.gz) | a64501130b2521003f130c11d2022996795157969b234b0558bc014fb6416ec1e78e777c7c3cbb05eb65481aeeec1ed35646ed8245513af0ed3a2c88b901aa3c
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-client-linux-386.tar.gz) | bbb8c6649600866bbfbce27d9068bbed3542639ee403d86d3ecd54a69e3925ed56a0bb37c1a772a30f3057f817d89eb46768a5c72c64ef3a5e101f2465a2109f
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-client-linux-amd64.tar.gz) | 1526534f31ece4247a61eb794ed31fe2289e14387424c08cfe63770e7479df477e69836b068154f67beadbd0e039c540c73912becb03394d43302f839cad4dad
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-client-linux-arm.tar.gz) | f9fd808955cd98c29f357c439c536e980bd4a1cf04da33e0d305343fbbba2ce905b4de3e0359a0f4e01202c439e8ce16a44b49594cb5c92fda6caf506a1a1948
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-client-linux-arm64.tar.gz) | c5fe23c425e151307461bc807a64b539cb3b61b9688122bbc79a428d099a907ea11a4313b2f5eae4b70a190c6f296daffdcf199c81793a747d200d67d182c9ab
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-client-linux-ppc64le.tar.gz) | c923fd2e50f7eca933580d572d709b7330fd1d8d8ab325c45f0054ece390416682d6dd7c335a991d5f0b2a01b084e521dbab94cdb20893b29561314d0be91778
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-client-linux-s390x.tar.gz) | be407bd63812a2fef7d49f8a92a93d1ece357a455c12a96bc5ec4880a5210fe54e726d52e004395c240967cc25a8ce5a1b91071bb437296793b2db2c38eb4753
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-client-windows-386.tar.gz) | 32be8d5d05ca1ca789e71afb0671d33e232f5fc328f8dce507bde365395d589182e18bccc3b58c81c2538104e890c1c411a7b53762fb8547e2a125c9af7ad3cc
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-client-windows-amd64.tar.gz) | 002c0a681e3f766504b3479706978e028929287df7f3ea9af8a2294efecea61366f28c0524aee07261ba6d462b8c71635d4a3ec9dbb9c3a02e7fa64128dd5a3c
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-client-windows-arm64.tar.gz) | e74b551a1b94845bfe0c4f83463d97d9c0a523f18577d6be29a54d888df3618ea446e9801fafad07cb0f4c07fdc79559e95c4f0beb8863aae267986a7c23f143
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-server-linux-amd64.tar.gz) | d5575da7f28a5284d4ffb40ca1b597213e03c381e161c1ec2bdadd7fe0532d62f41c758443ecefed70f484fb770e0bac53218f0a429587ac983469a39e56979b
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-server-linux-arm64.tar.gz) | ed2bb79cc50679f2238d6247b2119b18764274160fa6f555ba7a112dc3f13c9870f1aea7eca3a572e81ca562f14d001f03307db785debe0b96ea026bc37ad456
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-server-linux-ppc64le.tar.gz) | 406687513b2d1b15cfbc87812918bc9f72e4899497d4f03d95f8cb9c510822591fd9c19f333e5eb14b2063be6a93f1e1ca99b2d371bd835f178dcf25f999e881
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-server-linux-s390x.tar.gz) | 2b7002771db07dfe220d4f9fec089e2a43a9216f9c608fea67a1f1fafa8ea828ebc019d8e89a82f8cae370d821d762252f02547e521451d2271885b70a542f77
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-node-linux-amd64.tar.gz) | 7b4ae81efc8a8bf0ece659c83dc6de0d4730b97d917e6b4c9f15fba9155422ee847bbd4464e241b9689afcda80ff2753d73e9663de167947f81084fac15b6e66
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-node-linux-arm64.tar.gz) | 29cde35274d9514eb5a8a252a9af57ac2f8a99f90341f707a061afea7478b83d541e1d1e2edf42efdeea9d0fe529d10b49129895c1c4fd201cf59aa1e90538d1
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-node-linux-ppc64le.tar.gz) | 5bdf77b98e171807c3ae05041a15b6574e09a6734e571d67c8c3b2e131b6722eb29879b82f2c95be7f6246d154bb6019d4f60f522f5b2f2e255127e000fc8aab
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-node-linux-s390x.tar.gz) | c45393ec1bc1853bd37374b67dde574b889ca849be662e2820f3c7d0b0ee962f317cc5d616827ae5b7215bdd38b08f2761cdc00f9c15182d33df085c726a66e7
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.2/kubernetes-node-windows-amd64.tar.gz) | 42b380ba122fc525146fa55032edeb6e6be74dd58f48ab4a8715bb66c1e031924f599f2e47cd30c36901ae2df591d736e46ff5fded43e533c71ced3f4bdd2230
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-s390x)
## Changelog since v1.29.1
## Changes by Kind
### Feature
- Add process_start_time_seconds to /metrics/slis endpoint of all components ([#122750](https://github.com/kubernetes/kubernetes/pull/122750), [@richabanker](https://github.com/richabanker)) [SIG Architecture, Instrumentation and Testing]
- Kubernetes is now built with go 1.21.7
- update setcap/debian-base to bookworm-v1.0.1
- update distroless-iptables to v0.4.5 ([#123218](https://github.com/kubernetes/kubernetes/pull/123218), [@cpanato](https://github.com/cpanato)) [SIG API Machinery, Architecture, Cloud Provider, Release, Storage and Testing]
### Bug or Regression
- Fix deprecated version for pod_scheduling_duration_seconds that caused the metric to be hidden by default in 1.29. ([#123042](https://github.com/kubernetes/kubernetes/pull/123042), [@alculquicondor](https://github.com/alculquicondor)) [SIG Instrumentation and Scheduling]
- Fixed a bug in ValidatingAdmissionPolicy that caused policies which were using CRD parameters to fail to synchronize ([#123080](https://github.com/kubernetes/kubernetes/pull/123080), [@alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery and Testing]
- Fixes a 1.29 regression in "kubeadm init" that caused a user-specified --kubeconfig file to be ignored. ([#122792](https://github.com/kubernetes/kubernetes/pull/122792), [@avorima](https://github.com/avorima)) [SIG Cluster Lifecycle]
- Fixes a race condition in the iptables mode of kube-proxy in 1.27 and later
that could result in some updates getting lost (e.g., when a service gets a
new endpoint, the rules for the new endpoint might not be added until
much later). ([#122756](https://github.com/kubernetes/kubernetes/pull/122756), [@hakman](https://github.com/hakman)) [SIG Network]
- If a pvc has an empty storageClassName, persistentvolume controller won't try to assign a default StorageClass ([#122704](https://github.com/kubernetes/kubernetes/pull/122704), [@carlory](https://github.com/carlory)) [SIG Apps and Storage]
- Kubeadm: do not upload kubelet patch configuration into `kube-system/kubelet-config` ConfigMap ([#123108](https://github.com/kubernetes/kubernetes/pull/123108), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle]
- Kubeadm: fix a bug where the --rootfs global flag does not work with "kubeadm upgrade node" for control plane nodes. ([#123096](https://github.com/kubernetes/kubernetes/pull/123096), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle]
## Dependencies
### Added
_Nothing has changed._
### Changed
_Nothing has changed._
### Removed
_Nothing has changed._
# v1.29.1
## Downloads for v1.29.1
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes.tar.gz) | e622256168b70f1c5968ba39673e8b403c898fb13a1710f4a674f666fccc1cf9414a31518810b4e7fbfad20a12264a37afafe8534ab6eb452cce588d2b92ceb0
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-src.tar.gz) | f229ea55c8afa9a165b245081738adfbcfa5ed41be2be9b2ed76ed0a789378614417e02109eb5717f7973a4b21a649bf18a99fbad16e05f9496e0b03ac785576
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-client-darwin-amd64.tar.gz) | 0e2b49ed96d24511a315453ee7325787217f8f4f403a8dd18a4d0642e5941ca9865f48b7293b39dade110de2e05f595f026e00cd1dfa639545590a60381c8980
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-client-darwin-arm64.tar.gz) | cabc9436ac37537c2ec1adf3e28df6461eaa98933cc20cb9862e8c7c72a1ea2201e5a7373d6f9242b381eadfbdf335455466b1c68a013c511486fc7c88e0782b
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-client-linux-386.tar.gz) | c25b56a20cdf64a9f250c8419d393f5808b78b5a5f3f9979cabc098f2874538f48231852aaa4bc1f88bf0abae9ef9b383c2c1a86c745da6ce173fe148a392bcf
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-client-linux-amd64.tar.gz) | 9142b014be0c14bd9965be6b55aa24db24c4bb82a4ae01ea39752998cca7b0e0c29403853bb461c2accf57177943803a314d278bb64e4e6896840742f3d347f1
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-client-linux-arm.tar.gz) | e1e43e744c263dc4cc89dc2347a3d9a8bffae3727a7d4d7386a4800164fc0b6acaed460fcd820832a4d37ad9051ec185498bd1c37ca1e440253853b62dcc85c3
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-client-linux-arm64.tar.gz) | 29a7770797f749941cd7147cd6ed5b6f836afbaea2d9f63351fb86b992af13867a08a4cf3ba8ff539b6db5c069ea4f0274a2bc85fd5301fca821817e404496dc
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-client-linux-ppc64le.tar.gz) | 96fa7e1b88b5053600ee885f3b7afab64ac72bc73332c4162bc770549b66b05ae26235c98ed11aeb6ff6e72b64d995b8aeeae725d3916322dc19c6612277bdb7
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-client-linux-s390x.tar.gz) | 5f942fd006d26cfc5350f51f97300e81f63e8c058230c91ec23479f57356d05cd51fc4466fa9a10d9b79b7fa12549a7aa7a87f6e025ecd0fc62b7744e5574303
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-client-windows-386.tar.gz) | c099b667667f50ab7b2639e8d19d655ced0caea77c4ad95b952fb604d453033218e2a2e44ee3c0e60845f3ebabd38dc3c0616761cf3382e6d9d8906614469e0c
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-client-windows-amd64.tar.gz) | e24da5de0c6c7d0865ad434270d019e5c5490078d86f490481b6f34069ddb7892bff2df1e13f6333569d3529859ac0e88553a652397b235b25ff426bfdbe4626
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-client-windows-arm64.tar.gz) | 4566eb1946642ded9e2ce3a79085947e8b712f534354ef1f24d80b6c48b9202c331e8d9260895f2f332ff1970bb634cbc30428df1089ff4f3dea52b4e8d211a8
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-server-linux-amd64.tar.gz) | b01f8ef7160045363b017e79f1b3c2c5b54a67aa3426c0e5c45562be88bd72a3cb5e72043764684e10e255ecbb19d697765755031aa3c1b49c4e4c12122b3077
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-server-linux-arm64.tar.gz) | 48b78293820ab61f5723d590787c888df51c134539cc224f1bcc2d63e78e9fe2f39e9f23192f47aa31456802d5981cd86fa56522ebfb7d63a5914d54c7682684
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-server-linux-ppc64le.tar.gz) | 93e8608acef77428dfecb38546b07bb2a9e9da998e674fc106e687f10387263135d081da10bbafc5dbd91f467620f999ab154975d20dc26088b1d9f2559a9582
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-server-linux-s390x.tar.gz) | 20458da5a5dd67e89a768037f408a440a26173afd2844696c658b768fd93a7fa4e39167d93bd8d804ccb562f13ef43b4cba65d0edbe55479d3335c5f68e1f12a
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-node-linux-amd64.tar.gz) | 89adcf2782d42b43bea9841366051c88267b30f32c32d4570fd6f2a0fe4d2fae7c79fa704be2135c60129d9f1fa7d7cf17fc1e0401b7a489175a872d6bdacc35
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-node-linux-arm64.tar.gz) | 15c542d2588981b13c98be42030b64e4e430e3879dd440184d5b50c3764272c8337f2416effee2697a1af2ab1617545b7375665db6cf05e6cb3a7f6e0a12f354
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-node-linux-ppc64le.tar.gz) | e57203fb80f7c433242b0000c7610daff2583d5c3e7b787aac2c2c15e7118cb77482ecf2b4586bd0105a7a8b875d84397e229848e41196be546b162c7332fb8d
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-node-linux-s390x.tar.gz) | 6eaa7087be6a0258e9bf03196667c11cccf447433b92f8239e035d4ee66d9cffbca1dc6ca9e35888d892b936148aa6972afea158eac29900fca3a1aecdcbabc1
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.1/kubernetes-node-windows-amd64.tar.gz) | ee54d0fc79b70cdb31d3a557cc170c61fa621956efa52c32c5d915dfa271691b4db60dce7e6efc84156dafab218a75feb76f0e8fd3aede2e80606ef6986d0348
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-s390x)
## Changelog since v1.29.0
## Changes by Kind
### API Change
- Fixes accidental enablement of the new alpha `optionalOldSelf` API field in CustomResourceDefinition validation rules, which should only be allowed to be set when the CRDValidationRatcheting feature gate is enabled. Existing CustomResourceDefinition objects which have the field set will retain it on update, but new CustomResourceDefinition objects will not be permitted to set the field while the CRDValidationRatcheting feature gate is disabled. ([#122343](https://github.com/kubernetes/kubernetes/pull/122343), [@jpbetz](https://github.com/jpbetz)) [SIG API Machinery]
### Feature
- Kubernetes is now built with Go 1.21.6 ([#122711](https://github.com/kubernetes/kubernetes/pull/122711), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing]
### Bug or Regression
- Allow deletion of pods that use raw block volumes on node reboot ([#122211](https://github.com/kubernetes/kubernetes/pull/122211), [@gnufied](https://github.com/gnufied)) [SIG Node and Storage]
- Fix an issue where kubectl apply could panic when imported as a library ([#122559](https://github.com/kubernetes/kubernetes/pull/122559), [@Jefftree](https://github.com/Jefftree)) [SIG CLI]
- Fix: Mount point may become local without calling NodePublishVolume after node rebooting. ([#119923](https://github.com/kubernetes/kubernetes/pull/119923), [@cvvz](https://github.com/cvvz)) [SIG Node and Storage]
- Fixed a regression since 1.24 in the scheduling framework when overriding MultiPoint plugins (e.g. default plugins).
The incorrect loop logic might lead to a plugin being loaded multiple times, consequently preventing any Pod from being scheduled, which is unexpected. ([#122366](https://github.com/kubernetes/kubernetes/pull/122366), [@caohe](https://github.com/caohe)) [SIG Scheduling]
- Fixed migration of in-tree vSphere volumes to the CSI driver. ([#122341](https://github.com/kubernetes/kubernetes/pull/122341), [@jsafrane](https://github.com/jsafrane)) [SIG Storage]
- QueueingHint implementation for NodeAffinity is reverted because we found potential scenarios where events that make Pods schedulable could be missed. ([#122327](https://github.com/kubernetes/kubernetes/pull/122327), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling]
- QueueingHint implementation for NodeUnschedulable is reverted because we found potential scenarios where events that make Pods schedulable could be missed. ([#122326](https://github.com/kubernetes/kubernetes/pull/122326), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling]
### Other (Cleanup or Flake)
- Reverts the EventedPLEG feature (beta, but disabled by default) back to alpha for a known issue ([#122718](https://github.com/kubernetes/kubernetes/pull/122718), [@pacoxu](https://github.com/pacoxu)) [SIG Node]
## Dependencies
### Added
_Nothing has changed._
### Changed
- golang.org/x/crypto: v0.14.0 → v0.16.0
- golang.org/x/mod: v0.12.0 → v0.14.0
- golang.org/x/net: v0.17.0 → v0.19.0
- golang.org/x/sync: v0.3.0 → v0.5.0
- golang.org/x/sys: v0.13.0 → v0.15.0
- golang.org/x/term: v0.13.0 → v0.15.0
- golang.org/x/text: v0.13.0 → v0.14.0
- golang.org/x/tools: v0.12.0 → v0.16.1
### Removed
_Nothing has changed._
# v1.29.0
[Documentation](https://docs.k8s.io)
## Downloads for v1.29.0
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes.tar.gz) | `f07879916d7c4c7f8059ff9fd3c0006ce9bceb540874e183268a2bf2936df2632c4a3878a613cf2d695a80796e6c3eb52de5e3d83a73c91cb9a0bb5627091bae`
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-src.tar.gz) | `a37a7927224785625e9863c1e2dcbc88943593d003b8d126fee63770e6b8eff122004d0f80e1301de34e8a2d6ce208ec6fa55cad3bbe8631b92e5469f45bd00d`
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-client-darwin-amd64.tar.gz) | `22da1d2a217a8de91c1a8c393d17eb5ca81e243a1a3e509f3a40fb91d623670ace4ee87a09218a184aaa2eec4ca9c5478b992b8c6f136c568767d6e9dea493bf`
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-client-darwin-arm64.tar.gz) | `cbc0cafecae18a50f98aaa8b508b1808a50b7a477638dc8699830a9dae7ffa83641f9fdb9f53616b32ebc8df84835fc847ea252c5ebe647c7d3462029a63b7a0`
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-client-linux-386.tar.gz) | `f7ace756a3b6c56f2620d0ea6236fb94328c0a928094e4be7fbb78990a5771e8628bd93eac34017f3c33505c0248e8a64f933724a5fec6b322cf54dc30901985`
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-client-linux-amd64.tar.gz) | `6ff15bed6030c47e2ce90723500f08fa9968413f5b858456d4395bc67ab529b0b523ad0521e03be37664965e2fa588680aa0a5180054bc5cb3bafeef1497029b`
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-client-linux-arm.tar.gz) | `bafe1ca945c41ae671029d5398e564bac0753400ee3a50dc0b4979284c0a905e8c77575d8b64b303e9c776d09c919d27f1f99847390d4e2e1c43be826a8dc1a4`
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-client-linux-arm64.tar.gz) | `f3bca520625eaf6e6dd9af4cc709ff20bfce4da298a03e0be8835013a95fe0d6a25693d7702a4739c9477f9d49d2492d739718245ff91716fff90f60279ff376`
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-client-linux-ppc64le.tar.gz) | `e6ea574272cefe9fd6e8eea2bddd89e1d67d0cb560089813e7429f3fb6d98be0c6601f33c8a0b2364d3becfb93c0904c171096ed6cafc4071e08851566d70d82`
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-client-linux-s390x.tar.gz) | `b67dd572d84382e3f713d56bfb371de379807dca52cc4a1e082d6f4720a12770354ef2c9eac93bfc73bc0ea5f4be293db3b6c03328b94a797c2da17b9c40d9f3`
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-client-windows-386.tar.gz) | `0cf4b665f46e36616452916d744367b0ae2238098705b32de79559d06ea551173ab95190a26e87bebc03e67a75dc6a65699be3ef3db12aef82f32b66fd5afb0e`
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-client-windows-amd64.tar.gz) | `69cbe2b3942ba7d9c66e99f819adca94a9c7b420ad72cfd74407954c23ad70a4e7e76296824c4899f88232cabffe08d364c96af83bdaa538f29fa1303bcda2fa`
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-client-windows-arm64.tar.gz) | `44b0d1a7904bc2bf754abecb9b43a9efdc7cf700ab18f2564d95d98b4e38fe6d91f066943db7105baea964f86d77ade3b1acd57c7aaf1cdf689660f0d4422960`
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-server-linux-amd64.tar.gz) | `651a8bf34acb6d61c39cc67ae23d9ef18204f95b309561d31f49da26c0c6a1b7585e7d7c2ac2f1522b2c326470a4e1ec9aa0dcf3bb1f66e1a41e6a2286e0aa5f`
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-server-linux-arm64.tar.gz) | `7f1f58b05c923d860f2daa6d31906faf834584b1560f4eda01ba5499338d07a7f183030ab625557b1f5df50a5f0ea30d97d487e2571c85260e5b88fc3519cd43`
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-server-linux-ppc64le.tar.gz) | `3ca2af4a7d68c0d84ef65e69190daeb2392946c87c6b8e84ff8d5cf917c979f0778fc00040d4b471e71b8474ca57ac8fdf786f006260d4403b53f59a203a48f1`
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-server-linux-s390x.tar.gz) | `dfa172456f98210e614a9a538b9027ba211cc19f6eec22a42d5e89ce12d7f5e7e58dfd3229bb974ecba31ffafdf1a5361aef18b9610a45614a181918d87500db`
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-node-linux-amd64.tar.gz) | `8057197e9354e2e0f48aab18c0ce87e4ea39c1682cfd4c491c2bc83f8881787b09cb0c9b9f4d7bef8fbe53cc4056f5381745dbfde7f7474bb76a2358b8b3953e`
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-node-linux-arm64.tar.gz) | `70d086c71f6258b1667bcb1efe60c15810b5b76848fdf26781c5a90efb8a78030e9ffb230bb0fd52d994f02b13c0b558c8e8ad3a42b601a0f9440a71cf91be2d`
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-node-linux-ppc64le.tar.gz) | `2740f6ac0dfeebbe4ba8804b43ec5968997d9137de9a9432861c3e71e614cb84b309da31bde3554f896f829a570c21b833f0af241659ad326fa753a80f185ec4`
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-node-linux-s390x.tar.gz) | `9877d5a6cc84569efe30256ba5e8095f38bfa0b11c28892499a12b577b467b516880a33022d88f65263c7ffa2a9a3687ef52cb85fa611e95b14ae0c5b7a79c5c`
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0/kubernetes-node-windows-amd64.tar.gz) | `66b264de5e810bff31c4cf7cc575c3c57fed491fa4e21de7035dad76127e17d5fc88aff9f65277adf0826b255bf9b983f61c91bff2f8386d950f87509db6ec6b`
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-s390x)
## Changelog since v1.28.0
## Urgent Upgrade Notes
### (No, really, you MUST read this before you upgrade)
- Stopped accepting component configuration for `kube-proxy` and `kubelet` during `kubeadm upgrade plan --config`. This was a legacy behavior that was not well supported for upgrades and could be used only at the plan stage to determine if the configuration for these components stored in the cluster needs manual version migration. In the future, `kubeadm` will attempt alternative component config migration approaches. ([#120788](https://github.com/kubernetes/kubernetes/pull/120788), [@chendave](https://github.com/chendave))
- `kubeadm`: a separate "super-admin.conf" file is now deployed. The User in `admin.conf` is now bound to a new RBAC Group `kubeadm:cluster-admins` that has `cluster-admin` `ClusterRole` access. The User in `super-admin.conf` is now bound to the `system:masters` built-in super-powers / break-glass Group that can bypass RBAC. Before this change, the default `admin.conf` was bound to `system:masters` Group, which was undesired. Executing `kubeadm init phase kubeconfig all` or just `kubeadm init` will now generate the new `super-admin.conf` file. The cluster admin can then decide to keep the file present on a node host or move it to a safe location. `kubadm certs renew` will renew the certificate in `super-admin.conf` to one year if the file exists; if it does not exist a "MISSING" note will be printed. `kubeadm upgrade apply` for this release will migrate this particular node to the two file setup. Subsequent kubeadm releases will continue to optionally renew the certificate in `super-admin.conf` if the file exists on disk and if renew on upgrade is not disabled. `kubeadm join --control-plane` will now generate only an `admin.conf` file that has the less privileged User. ([#121305](https://github.com/kubernetes/kubernetes/pull/121305), [@neolit123](https://github.com/neolit123))
## Changes by Kind
### Deprecation
- #### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:
<!--
This section can be blank if this pull request does not require a release note.
When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.
See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files
Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
--> ([#119495](https://github.com/kubernetes/kubernetes/pull/119495), [@bzsuni](https://github.com/bzsuni)) [SIG API Machinery]
- Creation of new `CronJob` objects containing `TZ` or `CRON_TZ` in `.spec.schedule`, accidentally enabled in `v1.22`, is now disallowed. Use the `.spec.timeZone` field instead, supported in `v1.25+` clusters in default configurations. See https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#unsupported-timezone-specification for more information. ([#116252](https://github.com/kubernetes/kubernetes/pull/116252), [@soltysh](https://github.com/soltysh))
- Removed the networking `alpha` API `ClusterCIDR`. ([#121229](https://github.com/kubernetes/kubernetes/pull/121229), [@aojea](https://github.com/aojea))
### API Change
- '`kube-apiserver`: adds `--authentication-config` flag for reading `AuthenticationConfiguration`
files. `--authentication-config` flag is mutually exclusive with the existing `--oidc-*`
flags.' ([#119142](https://github.com/kubernetes/kubernetes/pull/119142), [@aramase](https://github.com/aramase))
- '`kube-scheduler` component config (`KubeSchedulerConfiguration`) `kubescheduler.config.k8s.io/v1beta3`
is removed in `v1.29`. Migrated `kube-scheduler` configuration files to `kubescheduler.config.k8s.io/v1`.' ([#119994](https://github.com/kubernetes/kubernetes/pull/119994), [@SataQiu](https://github.com/SataQiu))
- A new sleep action for the `PreStop` lifecycle hook was added, allowing containers to pause for a specified duration before termination. ([#119026](https://github.com/kubernetes/kubernetes/pull/119026), [@AxeZhan](https://github.com/AxeZhan))
- Added CEL expressions to `v1alpha1 AuthenticationConfiguration`. ([#121078](https://github.com/kubernetes/kubernetes/pull/121078), [@aramase](https://github.com/aramase))
- Added Windows support for InPlace Pod Vertical Scaling feature. ([#112599](https://github.com/kubernetes/kubernetes/pull/112599), [@fabi200123](https://github.com/fabi200123)) [SIG Autoscaling, Node, Scalability, Scheduling and Windows]
- Added `ImageMaximumGCAge` field to Kubelet configuration, which allows a user to set the maximum age an image is unused before it's garbage collected. ([#121275](https://github.com/kubernetes/kubernetes/pull/121275), [@haircommander](https://github.com/haircommander))
- Added `UserNamespacesPodSecurityStandards` feature gate to enable user namespace support for Pod Security Standards.
Enabling this feature will modify all Pod Security Standard rules to allow setting: `spec[.*].securityContext.[runAsNonRoot,runAsUser]`.
This feature gate should only be enabled if all nodes in the cluster support the user namespace feature and have it enabled.
The feature gate will not graduate or be enabled by default in future Kubernetes releases. ([#118760](https://github.com/kubernetes/kubernetes/pull/118760), [@saschagrunert](https://github.com/saschagrunert)) [SIG API Machinery, Auth, Node and Release]
- Added `optionalOldSelf` to `x-kubernetes-validations` to support ratcheting CRD schema constraints. ([#121034](https://github.com/kubernetes/kubernetes/pull/121034), [@alexzielenski](https://github.com/alexzielenski))
- Added a new `ServiceCIDR` type that allows to dynamically configure the cluster range used to allocate `Service ClusterIPs` addresses. ([#116516](https://github.com/kubernetes/kubernetes/pull/116516), [@aojea](https://github.com/aojea))
- Added a new `ipMode` field to the `.status` of Services where `type` is set to `LoadBalancer`.
The new field is behind the `LoadBalancerIPMode` feature gate. ([#119937](https://github.com/kubernetes/kubernetes/pull/119937), [@RyanAoh](https://github.com/RyanAoh)) [SIG API Machinery, Apps, Cloud Provider, Network and Testing]
- Added options for configuring `nf_conntrack_udp_timeout`, and `nf_conntrack_udp_timeout_stream` variables of netfilter conntrack subsystem. ([#120808](https://github.com/kubernetes/kubernetes/pull/120808), [@aroradaman](https://github.com/aroradaman))
- Added support for CEL expressions to `v1alpha1 AuthorizationConfiguration` webhook `matchConditions`. ([#121223](https://github.com/kubernetes/kubernetes/pull/121223), [@ritazh](https://github.com/ritazh))
- Added support for projecting `certificates.k8s.io/v1alpha1` ClusterTrustBundle objects into pods. ([#113374](https://github.com/kubernetes/kubernetes/pull/113374), [@ahmedtd](https://github.com/ahmedtd))
- Added the `DisableNodeKubeProxyVersion` feature gate. If `DisableNodeKubeProxyVersion` is enabled, the `kubeProxyVersion` field is not set. ([#120954](https://github.com/kubernetes/kubernetes/pull/120954), [@HirazawaUi](https://github.com/HirazawaUi))
- Fixed a bug where CEL expressions in CRD validation rules would incorrectly compute a high estimated cost for functions that return strings, lists or maps.
The incorrect cost was evident when the result of a function was used in subsequent operations. ([#119800](https://github.com/kubernetes/kubernetes/pull/119800), [@jpbetz](https://github.com/jpbetz)) [SIG API Machinery, Auth and Cloud Provider]
- Fixed the API comments for the Job `Ready` field in status. ([#121765](https://github.com/kubernetes/kubernetes/pull/121765), [@mimowo](https://github.com/mimowo))
- Fixed the API comments for the `FailIndex` Job pod failure policy action. ([#121764](https://github.com/kubernetes/kubernetes/pull/121764), [@mimowo](https://github.com/mimowo))
- Go API: the `ResourceRequirements` struct was replaced with `VolumeResourceRequirements` for use with volumes. ([#118653](https://github.com/kubernetes/kubernetes/pull/118653), [@pohly](https://github.com/pohly))
- Graduated `Job BackoffLimitPerIndex` feature to `beta`. ([#121356](https://github.com/kubernetes/kubernetes/pull/121356), [@mimowo](https://github.com/mimowo))
- Marked the `onPodConditions` field as optional in `Job`'s pod failure policy. ([#120204](https://github.com/kubernetes/kubernetes/pull/120204), [@mimowo](https://github.com/mimowo))
- Promoted `PodReadyToStartContainers` condition to `beta`. ([#119659](https://github.com/kubernetes/kubernetes/pull/119659), [@kannon92](https://github.com/kannon92))
- The `flowcontrol.apiserver.k8s.io/v1beta3` `FlowSchema` and `PriorityLevelConfiguration` APIs has been promoted to `flowcontrol.apiserver.k8s.io/v1`, with the following changes:
- `PriorityLevelConfiguration`: the `.spec.limited.nominalConcurrencyShares` field defaults to `30` only if the field is omitted (v1beta3 also defaulted an explicit `0` value to `30`). Specifying an explicit `0` value is not allowed in the `v1` version in v1.29 to ensure compatibility with `v1.28` API servers. In `v1.30`, explicit `0` values will be allowed in this field in the `v1` API.
The `flowcontrol.apiserver.k8s.io/v1beta3` APIs are deprecated and will no longer be served in v1.32. All existing objects are available via the `v1` APIs. Transition clients and manifests to use the `v1` APIs before upgrading to `v1.32`. ([#121089](https://github.com/kubernetes/kubernetes/pull/121089), [@tkashem](https://github.com/tkashem))
- The `kube-proxy` command-line documentation was updated to clarify that
`--bind-address` does not actually have anything to do with binding to an
address, and you probably don't actually want to be using it. ([#120274](https://github.com/kubernetes/kubernetes/pull/120274), [@danwinship](https://github.com/danwinship))
- The `kube-scheduler` `selectorSpread` plugin has been removed, please use the `podTopologySpread` plugin instead. ([#117720](https://github.com/kubernetes/kubernetes/pull/117720), [@kerthcet](https://github.com/kerthcet))
- The `matchLabelKeys/mismatchLabelKeys` feature is introduced to the hard/soft `PodAffinity/PodAntiAffinity`. ([#116065](https://github.com/kubernetes/kubernetes/pull/116065), [@sanposhiho](https://github.com/sanposhiho))
- When updating a CRD, per-expression cost limit check are now skipped for `x-kubernetes-validations` rules of versions that are not mutated. ([#121460](https://github.com/kubernetes/kubernetes/pull/121460), [@jiahuif](https://github.com/jiahuif))
- `CSINodeExpandSecret` feature has been promoted to `GA` in this release and is enabled
by default. The CSI drivers can make use of the `secretRef` values passed in `NodeExpansion`
request optionally sent by the CSI Client from this release onwards. ([#121303](https://github.com/kubernetes/kubernetes/pull/121303), [@humblec](https://github.com/humblec))
- `NodeStageVolume` calls will now be retried if the CSI node driver is not running. ([#120330](https://github.com/kubernetes/kubernetes/pull/120330), [@rohitssingh](https://github.com/rohitssingh))
- `PersistentVolumeLastPhaseTransitionTime` is now beta and enabled by default. ([#120627](https://github.com/kubernetes/kubernetes/pull/120627), [@RomanBednar](https://github.com/RomanBednar))
- `ValidatingAdmissionPolicy` type checking now supports CRDs and API extensions types. ([#119109](https://github.com/kubernetes/kubernetes/pull/119109), [@jiahuif](https://github.com/jiahuif))
- `kube-apiserver`: added `--authorization-config` flag for reading a configuration file containing an `apiserver.config.k8s.io/v1alpha1 AuthorizationConfiguration` object. The `--authorization-config` flag is mutually exclusive with `--authorization-modes` and `--authorization-webhook-*` flags. The `alpha` `StructuredAuthorizationConfiguration` feature flag must be enabled for `--authorization-config` to be specified. ([#120154](https://github.com/kubernetes/kubernetes/pull/120154), [@palnabarun](https://github.com/palnabarun))
- `kube-proxy` now has a new nftables-based mode, available by running
`kube-proxy --feature-gates NFTablesProxyMode=true --proxy-mode nftables`
This is currently an alpha-level feature and while it probably will not
eat your data, it may nibble at it a bit. (It passes e2e testing but has
not yet seen real-world use.)
At this point it should be functionally mostly identical to the iptables
mode, except that it does not (and will not) support Service NodePorts on
127.0.0.1. (Also note that there are currently no command-line arguments
for the nftables-specific config; you will need to use a config file if
you want to set the equivalent of any of the `--iptables-xxx` options.)
As this code is still very new, it has not been heavily optimized yet;
while it is expected to _eventually_ have better performance than the
iptables backend, very little performance testing has been done so far. ([#121046](https://github.com/kubernetes/kubernetes/pull/121046), [@danwinship](https://github.com/danwinship))
- `kube-proxy`: Added an option/flag for configuring the `nf_conntrack_tcp_be_liberal` sysctl (in the kernel's netfilter conntrack subsystem). When enabled, `kube-proxy` will not install the `DROP` rule for invalid conntrack states, which currently breaks users of asymmetric routing. ([#120354](https://github.com/kubernetes/kubernetes/pull/120354), [@aroradaman](https://github.com/aroradaman))
### Feature
- #### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:
<!--
This section can be blank if this pull request does not require a release note.
When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.
See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files
Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
--> ([#119517](https://github.com/kubernetes/kubernetes/pull/119517), [@sanposhiho](https://github.com/sanposhiho)) [SIG Node, Scheduling and Testing]
- '`kubeadm`: added validation to verify that the `CertificateKey` is a valid hex
encoded AES key.' ([#120064](https://github.com/kubernetes/kubernetes/pull/120064), [@SataQiu](https://github.com/SataQiu))
- A customizable `OrderedScoreFuncs()` function was introduced. Out-of-tree plugins
that used the scheduler's preemption interface could implement this function
for custom preemption preferences or return nil to keep the current behavior. ([#121867](https://github.com/kubernetes/kubernetes/pull/121867), [@lianghao208](https://github.com/lianghao208))
- Added `apiextensions_apiserver_update_ratcheting_time` metric for tracking time taken during requests by feature `CRDValidationRatcheting`. ([#121462](https://github.com/kubernetes/kubernetes/pull/121462), [@alexzielenski](https://github.com/alexzielenski))
- Added `apiserver_envelope_encryption_dek_cache_filled` to measure number of records in data encryption key (DEK) cache. ([#119878](https://github.com/kubernetes/kubernetes/pull/119878), [@ritazh](https://github.com/ritazh))
- Added `apiserver_watch_list_duration_seconds` metrics which will measure response latency distribution in seconds for watchlist requests broken by group, version, resource and scope. ([#120490](https://github.com/kubernetes/kubernetes/pull/120490), [@p0lyn0mial](https://github.com/p0lyn0mial))
- Added `job_pods_creation_total` metrics for tracking Pods created by the Job controller labeled by events which triggered the Pod creation. ([#121481](https://github.com/kubernetes/kubernetes/pull/121481), [@dejanzele](https://github.com/dejanzele))
- Added `kubectl node drain` helper callbacks `OnPodDeletionOrEvictionStarted`
and `OnPodDeletionOrEvictionFailed`; people extending `kubectl` can use these
new callbacks for more granularity. Deprecated the `OnPodDeletedOrEvicted`
node drain helper callback. ([#117502](https://github.com/kubernetes/kubernetes/pull/117502), [@adilGhaffarDev](https://github.com/adilGhaffarDev))
- Added a new `--init-only` command line flag to `kube-proxy`. Setting the flag makes `kube-proxy` perform its initial configuration that requires privileged mode, and then exit. The `--init-only` mode is intended to be executed in a privileged init container, so that the main container may run with a stricter `securityContext`. ([#120864](https://github.com/kubernetes/kubernetes/pull/120864), [@uablrek](https://github.com/uablrek)) [SIG Network and Scalability]
- Added a new scheduler metric, `pod_scheduling_sli_duration_seconds`, and started the deprecation for `pod_scheduling_duration_seconds`. ([#119049](https://github.com/kubernetes/kubernetes/pull/119049), [@helayoty](https://github.com/helayoty))
- Added a return value to `QueueingHint` to indicate an error. If `QueueingHint` returns an error,
the scheduler logs it and treats the event as a `QueueAfterBackoff` so that
the Pod won't be stuck in the unschedulable pod pool. ([#119290](https://github.com/kubernetes/kubernetes/pull/119290), [@carlory](https://github.com/carlory))
- Added apiserver identity to the following metrics:
`apiserver_envelope_encryption_key_id_hash_total`, `apiserver_envelope_encryption_key_id_hash_last_timestamp_seconds`, `apiserver_envelope_encryption_key_id_hash_status_last_timestamp_seconds`, `apiserver_encryption_config_controller_automatic_reload_failures_total`, `apiserver_encryption_config_controller_automatic_reload_success_total`, `apiserver_encryption_config_controller_automatic_reload_last_timestamp_seconds`.
Fixed bug to surface events for the following metrics: `apiserver_encryption_config_controller_automatic_reload_failures_total`, `apiserver_encryption_config_controller_automatic_reload_last_timestamp_seconds`, `apiserver_encryption_config_controller_automatic_reload_success_total`. ([#120438](https://github.com/kubernetes/kubernetes/pull/120438), [@ritazh](https://github.com/ritazh))
- Added container filesystem to the `ImageFsInfoResponse`. ([#120914](https://github.com/kubernetes/kubernetes/pull/120914), [@kannon92](https://github.com/kannon92))
- Added multiplication functionality to `Quantity`. ([#117411](https://github.com/kubernetes/kubernetes/pull/117411), [@tenzen-y](https://github.com/tenzen-y))
- Added new feature gate called `RuntimeClassInImageCriApi` to address `kubelet` changes needed for KEP 4216.
Noteable changes:
1. Populate new `RuntimeHandler` field in CRI's `ImageSpec` struct during image pulls from container runtimes.
2. Pass `runtimeHandler` field in `RemoveImage()` call to container runtime in `kubelet`'s image garbage collection. ([#121456](https://github.com/kubernetes/kubernetes/pull/121456), [@kiashok](https://github.com/kiashok))
- Added support for split image filesystem in kubelet. ([#120616](https://github.com/kubernetes/kubernetes/pull/120616), [@kannon92](https://github.com/kannon92))
- Bumped `cel-go` to `v0.17.7` and introduced set `ext` library with new options. ([#121577](https://github.com/kubernetes/kubernetes/pull/121577), [@cici37](https://github.com/cici37))
- Bumped `distroless-iptables` to `0.3.2` based on Go `1.21.1`. ([#120527](https://github.com/kubernetes/kubernetes/pull/120527), [@cpanato](https://github.com/cpanato))
- Bumped `distroless-iptables` to `0.3.3` based on Go `1.21.2`. ([#121073](https://github.com/kubernetes/kubernetes/pull/121073), [@cpanato](https://github.com/cpanato))
- Bumped `distroless-iptables` to `0.4.1` based on Go `1.21.3`. ([#121216](https://github.com/kubernetes/kubernetes/pull/121216), [@cpanato](https://github.com/cpanato))
- Bumped distroless-iptables to 0.4.1 based on Go `1.21.3`. ([#121871](https://github.com/kubernetes/kubernetes/pull/121871), [@cpanato](https://github.com/cpanato))
- CEL can now correctly handle a CRD `openAPIV3Schema` that has neither `Properties` nor `AdditionalProperties`. ([#121459](https://github.com/kubernetes/kubernetes/pull/121459), [@jiahuif](https://github.com/jiahuif))
- CEL cost estimator no longer treats enums as unbounded strings when determining its length. Instead, the length is set to the longest possible enum value. ([#121085](https://github.com/kubernetes/kubernetes/pull/121085), [@jiahuif](https://github.com/jiahuif)) [SIG API Machinery]
- CRI: image pull per runtime class is now supported. ([#121121](https://github.com/kubernetes/kubernetes/pull/121121), [@kiashok](https://github.com/kiashok))
- Certain `requestBody` parameters in the OpenAPI `v3` are now correctly marked as required. ([#120735](https://github.com/kubernetes/kubernetes/pull/120735), [@Jefftree](https://github.com/Jefftree))
- Changed `kubectl help` to display basic details for subcommands from plugins. ([#116752](https://github.com/kubernetes/kubernetes/pull/116752), [@xvzf](https://github.com/xvzf))
- Changed the `KMSv2KDF` feature gate to be enabled by default. ([#120433](https://github.com/kubernetes/kubernetes/pull/120433), [@enj](https://github.com/enj)) [SIG API Machinery, Auth and Testing]
- Client-side apply will now use OpenAPI `v3` by default. ([#120707](https://github.com/kubernetes/kubernetes/pull/120707), [@Jefftree](https://github.com/Jefftree))
- Decoding etcd's response now respects the timeout context. ([#121614](https://github.com/kubernetes/kubernetes/pull/121614), [@HirazawaUi](https://github.com/HirazawaUi))
- Decoupled `TaintManager` from `NodeLifeCycleController` (KEP-3902). ([#119208](https://github.com/kubernetes/kubernetes/pull/119208), [@atosatto](https://github.com/atosatto))
- Enabled traces for KMSv2 encrypt/decrypt operations. ([#121095](https://github.com/kubernetes/kubernetes/pull/121095), [@aramase](https://github.com/aramase))
- Fixed `kube-proxy` panicking on exit when the `Node` object changed its `PodCIDR`. ([#120375](https://github.com/kubernetes/kubernetes/pull/120375), [@pegasas](https://github.com/pegasas))
- Fixed bugs in handling of server-side apply, create, and update API requests for objects containing duplicate items in keyed lists.
- A `create` or `update` API request with duplicate items in a keyed list no longer wipes out managedFields. Examples include env var entries with the same name, or port entries with the same containerPort in a pod spec.
- A server-side apply request that makes unrelated changes to an object which has duplicate items in a keyed list no longer fails, and leaves the existing duplicate items as-is.
- A server-side apply request that changes an object which has duplicate items in a keyed list, and modifies the duplicated item removes the duplicates and replaces them with the single item contained in the server-side apply request. ([#121575](https://github.com/kubernetes/kubernetes/pull/121575), [@apelisse](https://github.com/apelisse))
- Fixed overriding default `KubeletConfig` fields in drop-in configs if not set. ([#121193](https://github.com/kubernetes/kubernetes/pull/121193), [@sohankunkerkar](https://github.com/sohankunkerkar))
- Graduated API List chunking (aka pagination) feature to `stable`. ([#119503](https://github.com/kubernetes/kubernetes/pull/119503), [@wojtek-t](https://github.com/wojtek-t))
- Graduated the `ReadWriteOncePod` feature gate to `GA`. ([#121077](https://github.com/kubernetes/kubernetes/pull/121077), [@chrishenzie](https://github.com/chrishenzie))
- Graduated the following kubelet resource metrics to **general availability**:
- `container_cpu_usage_seconds_total`
- `container_memory_working_set_bytes`
- `container_start_time_seconds`
- `node_cpu_usage_seconds_total`
- `node_memory_working_set_bytes`
- `pod_cpu_usage_seconds_total`
- `pod_memory_working_set_bytes`
- `resource_scrape_error`
Deprecated (renamed) `scrape_error` in favor of `resource_scrape_error` ([#116897](https://github.com/kubernetes/kubernetes/pull/116897), [@Richabanker](https://github.com/Richabanker)) [SIG Architecture, Instrumentation, Node and Testing]
- Implemented API for streaming for the `etcd` store implementation.
When `sendInitialEvents ListOption` is set together with `watch=true`, it begins the watch stream with synthetic init events followed by a synthetic `Bookmark`, after which the server continues streaming events. ([#119557](https://github.com/kubernetes/kubernetes/pull/119557), [@p0lyn0mial](https://github.com/p0lyn0mial))
- Improved memory usage of `kube-scheduler` by dropping the `.metadata.managedFields` field that `kube-scheduler` doesn't require. ([#119556](https://github.com/kubernetes/kubernetes/pull/119556), [@linxiulei](https://github.com/linxiulei))
- In a scheduler with `Permit` plugins, when a Pod is rejected during `WaitOnPermit`, the scheduler records the plugin.
The scheduler will use the record to honor cluster events and queueing `hints registered` for the plugin, to inform whether to retry the pod. ([#119785](https://github.com/kubernetes/kubernetes/pull/119785), [@sanposhiho](https://github.com/sanposhiho))
- In-tree cloud providers are now switched off by default. Please use `DisableCloudProviders` and `DisableKubeletCloudCredentialProvider` feature flags if you still need this functionality. ([#117503](https://github.com/kubernetes/kubernetes/pull/117503), [@dims](https://github.com/dims))
- Introduced new apiserver metric `apiserver_flowcontrol_current_inqueue_seats`. This metric is analogous to `apiserver_flowcontrol_current_inqueue_requests`, but tracks the total number of seats, as each request can take more than one seat. ([#119385](https://github.com/kubernetes/kubernetes/pull/119385), [@andrewsykim](https://github.com/andrewsykim))
- Introduced the `job_finished_indexes_total` metric for the `BackoffLimitPerIndex` feature. ([#121292](https://github.com/kubernetes/kubernetes/pull/121292), [@mimowo](https://github.com/mimowo))
- Kubeadm: supported updating certificate organization during `kubeadm certs renew` operation. ([#121841](https://github.com/kubernetes/kubernetes/pull/121841), [@SataQiu](https://github.com/SataQiu))
- Kubernetes is now built with Go `1.21.0`. ([#118996](https://github.com/kubernetes/kubernetes/pull/118996), [@cpanato](https://github.com/cpanato))
- Kubernetes is now built with Go `1.21.1`. ([#120493](https://github.com/kubernetes/kubernetes/pull/120493), [@cpanato](https://github.com/cpanato))
- Kubernetes is now built with Go `1.21.2`. ([#121021](https://github.com/kubernetes/kubernetes/pull/121021), [@cpanato](https://github.com/cpanato))
- Kubernetes is now built with Go `1.21.4`. ([#121808](https://github.com/kubernetes/kubernetes/pull/121808), [@cpanato](https://github.com/cpanato))
- Kubernetes is now built with Go `v1.21.3`. ([#121149](https://github.com/kubernetes/kubernetes/pull/121149), [@cpanato](https://github.com/cpanato))
- List of metric labels can now be configured by supplying a manifest using the `--allow-metric-labels-manifest` flag. ([#118299](https://github.com/kubernetes/kubernetes/pull/118299), [@rexagod](https://github.com/rexagod))
- Listed the pods using `<PVC>` as an ephemeral storage volume in "Used by:" part of the output of `kubectl describe pvc <PVC>` command. ([#120427](https://github.com/kubernetes/kubernetes/pull/120427), [@MaGaroo](https://github.com/MaGaroo))
- Migrated the `nodevolumelimits` scheduler plugin to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#116884](https://github.com/kubernetes/kubernetes/pull/116884), [@mengjiao-liu](https://github.com/mengjiao-liu))
- Migrated the `volumebinding scheduler plugins` to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#116803](https://github.com/kubernetes/kubernetes/pull/116803), [@mengjiao-liu](https://github.com/mengjiao-liu))
- Priority and Fairness feature is now `stable`, the feature gate will be removed in `v1.31`. ([#121638](https://github.com/kubernetes/kubernetes/pull/121638), [@tkashem](https://github.com/tkashem))
- Promoted `PodHostIPs` condition to `beta`. ([#120257](https://github.com/kubernetes/kubernetes/pull/120257), [@wzshiming](https://github.com/wzshiming))
- Promoted `PodHostIPs` condition to `beta`. ([#121477](https://github.com/kubernetes/kubernetes/pull/121477), [@wzshiming](https://github.com/wzshiming))
- Promoted `PodReplacementPolicy` to `beta`. ([#121491](https://github.com/kubernetes/kubernetes/pull/121491), [@dejanzele](https://github.com/dejanzele))
- Promoted `ServiceNodePortStaticSubrange` to stable and lock to default. ([#120233](https://github.com/kubernetes/kubernetes/pull/120233), [@xuzhenglun](https://github.com/xuzhenglun))
- Promoted plugin subcommand resolution feature to `beta`. ([#120663](https://github.com/kubernetes/kubernetes/pull/120663), [@ardaguclu](https://github.com/ardaguclu))
- Removed `/livez` livezchecks for KMS v1 and v2 to ensure KMS health does not cause `kube-apiserver` restart. KMS health checks are still in place as a healthz and readiness checks. ([#120583](https://github.com/kubernetes/kubernetes/pull/120583), [@ritazh](https://github.com/ritazh))
- Restartable init containers resource in pod autoscaler are now calculated. ([#120001](https://github.com/kubernetes/kubernetes/pull/120001), [@qingwave](https://github.com/qingwave))
- Sidecar termination is now serialized and each sidecar container will receive a `SIGTERM` after all main containers and later starting sidecar containers have terminated. ([#120620](https://github.com/kubernetes/kubernetes/pull/120620), [@tzneal](https://github.com/tzneal))
- The CRD validation rule with feature gate `CustomResourceValidationExpressions` was promoted to `GA`. ([#121373](https://github.com/kubernetes/kubernetes/pull/121373), [@cici37](https://github.com/cici37))
- The KMSv2 features with feature gates `KMSv2` and `KMSv2KDF` are promoted to `GA`. The `KMSv1` feature gate is now disabled by default. ([#121485](https://github.com/kubernetes/kubernetes/pull/121485), [@ritazh](https://github.com/ritazh))
- The `--interactive` flag in `kubectl delete` is now visible to all users by default. ([#120416](https://github.com/kubernetes/kubernetes/pull/120416), [@ardaguclu](https://github.com/ardaguclu))
- The `CloudDualStackNodeIPs` feature is now `beta`, meaning that when using
an external cloud provider that has been updated to support the feature,
you can pass comma-separated dual-stack `--node-ips` to `kubelet` and have
the cloud provider take both IPs into account. ([#120275](https://github.com/kubernetes/kubernetes/pull/120275), [@danwinship](https://github.com/danwinship))
- The `Dockerfile` for the kubectl image has been updated with the addition of a specific base image and essential utilities (bash and jq). ([#119592](https://github.com/kubernetes/kubernetes/pull/119592), [@rayandas](https://github.com/rayandas))
- The `SidecarContainers` feature has graduated to `beta` and is enabled by default. ([#121579](https://github.com/kubernetes/kubernetes/pull/121579), [@gjkim42](https://github.com/gjkim42))
- The `kube-apiserver` will now expose four new metrics to inform about errors on the clusterIP and nodePort allocation logic. ([#120843](https://github.com/kubernetes/kubernetes/pull/120843), [@aojea](https://github.com/aojea))
- The `volume_zone` plugin will consider `beta` labels as `GA` labels during the scheduling
process. Therefore, if the values of the labels are the same, PVs with `beta` labels
can also be scheduled to nodes with `GA` labels. ([#118923](https://github.com/kubernetes/kubernetes/pull/118923), [@AxeZhan](https://github.com/AxeZhan))
- Updated the generic apiserver library to produce an error if a new API server is configured with support for a data format other than JSON, YAML, or Protobuf. ([#121325](https://github.com/kubernetes/kubernetes/pull/121325), [@benluddy](https://github.com/benluddy)) [SIG API Machinery]
- Use of secret-based service account tokens now adds an `authentication.k8s.io/legacy-token-autogenerated-secret` or `authentication.k8s.io/legacy-token-manual-secret` audit annotation containing the name of the secret used. ([#118598](https://github.com/kubernetes/kubernetes/pull/118598), [@yuanchen8911](https://github.com/yuanchen8911)) [SIG Auth, Instrumentation and Testing]
- `--sync-frequency` will not affect the update interval of volumes that use `ConfigMaps`
or `Secrets` when the `configMapAndSecretChangeDetectionStrategy` is set to `Cache`.
The update interval is only affected by `node.alpha.kubernetes.io/ttl` node annotation." ([#120255](https://github.com/kubernetes/kubernetes/pull/120255), [@likakuli](https://github.com/likakuli))
- `CRDValidationRatcheting`: added support for ratcheting `x-kubernetes-validations` in schema. ([#121016](https://github.com/kubernetes/kubernetes/pull/121016), [@alexzielenski](https://github.com/alexzielenski))
- `DevicePluginCDIDevices` feature has been graduated to `beta` and enabled by default in the kubelet. ([#121254](https://github.com/kubernetes/kubernetes/pull/121254), [@bart0sh](https://github.com/bart0sh))
- `ValidatingAdmissionPolicy` now preserves types of composition variables, and raises type-related errors early. ([#121001](https://github.com/kubernetes/kubernetes/pull/121001), [@jiahuif](https://github.com/jiahuif))
- `cluster/gce`: added webhook to replace `PersistentVolumeLabel` admission controller. ([#121628](https://github.com/kubernetes/kubernetes/pull/121628), [@andrewsykim](https://github.com/andrewsykim))
- `dra`: the scheduler plugin now avoids additional scheduling attempts in some cases by falling back to SSA after a conflict. ([#120534](https://github.com/kubernetes/kubernetes/pull/120534), [@pohly](https://github.com/pohly))
- `etcd`: image is now based on `v3.5.9`. ([#121567](https://github.com/kubernetes/kubernetes/pull/121567), [@mzaian](https://github.com/mzaian))
- `kube-apiserver` added:
- `alpha` support (guarded by the `ServiceAccountTokenJTI` feature gate) for adding a `jti` (JWT ID) claim to service account tokens it issues, adding an `authentication.kubernetes.io/credential-id` audit annotation in audit logs when the tokens are issued, and `authentication.kubernetes.io/credential-id` entry in the extra user info when the token is used to authenticate.
- `alpha` support (guarded by the `ServiceAccountTokenPodNodeInfo` feature gate) for including the node name (and uid, if the node exists) as additional claims in service account tokens it issues which are bound to pods, and `authentication.kubernetes.io/node-name` and `authentication.kubernetes.io/node-uid` extra user info when the token is used to authenticate.
- `alpha` support (guarded by the `ServiceAccountTokenNodeBinding` feature gate) for allowing `TokenRequests` that bind tokens directly to nodes, and (guarded by the ServiceAccountTokenNodeBindingValidation feature gate) for validating the node name and uid still exist when the token is used. ([#120780](https://github.com/kubernetes/kubernetes/pull/120780), [@munnerz](https://github.com/munnerz))
- `kube-apiserver` updated:
- Updated encryption configuration file watch logic from using inotify watch to polling at an interval of every minute. Moved the logic to polling because there are variations on file changes (like symlink swapping of directories that contain the encryption config) that the file watch logic would fail to detect. Polling at a set interval prevents any such issues. Note that there is no guarantee on how quickly the API server will process the encryption config. The `apiserver_encryption_config_controller_automatic_reload_last_timestamp_seconds` metric must be used to determine when the new config becomes effective.([#121310](https://github.com/kubernetes/kubernetes/pull/121310), [@nilekhc](https://github.com/nilekhc))
- `kube-controller-manager`: The `LegacyServiceAccountTokenCleanUp` feature gate is now `beta` and enabled by default. When enabled, legacy auto-generated service account token secrets are auto-labeled with a `kubernetes.io/legacy-token-invalid-since` label if the credentials have not been used in the time specified by `--legacy-service-account-token-clean-up-period` (defaulting to one year), **and** are referenced from the `.secrets` list of a ServiceAccount object, **and** are not referenced from pods. This label causes the authentication layer to reject use of the credentials. After being labeled as invalid, if the time specified by `--legacy-service-account-token-clean-up-period` (defaulting to one year) passes without the credential being used, the secret is automatically deleted. Secrets labeled as invalid which have not been auto-deleted yet can be re-activated by removing the `kubernetes.io/legacy-token-invalid-since` label. ([#120682](https://github.com/kubernetes/kubernetes/pull/120682), [@yt2985](https://github.com/yt2985))
- `kube-proxy` will only install the `DROP` rules for invalid `conntrack` states if
the `nf_conntrack_tcp_be_liberal` is not set. ([#120412](https://github.com/kubernetes/kubernetes/pull/120412), [@aojea](https://github.com/aojea))
- `kube-scheduler` implemented scheduling hints for the `NodeUnschedulable` plugin.
The scheduling hints allow the scheduler to only retry scheduling a `Pod`
that was previously rejected by the `NodeSchedulable` plugin if a new `Node` or a `Node` update sets `.spec.unschedulable` to false. ([#119396](https://github.com/kubernetes/kubernetes/pull/119396), [@wackxu](https://github.com/wackxu))
- `kube-scheduler` implements scheduling hints for the `NodeAffinity` plugin.
The scheduling hints allow the scheduler to only retry scheduling a `Pod`
that was previously rejected by the `NodeAffinity` plugin if a new `Node` or a `Node` update matches the `Pod`'s node affinity. ([#119155](https://github.com/kubernetes/kubernetes/pull/119155), [@carlory](https://github.com/carlory))
- `kubeadm`: promoted feature gate `EtcdLearnerMode` to `beta`. Learner mode for
joining `etcd` members is now enabled by default. ([#120228](https://github.com/kubernetes/kubernetes/pull/120228), [@pacoxu](https://github.com/pacoxu))
- `kubeadm`: turned on feature gate `MergeCLIArgumentsWithConfig` to merge the config from flag and config file, otherwise, if the flag `--ignore-preflight-errors` is set from the CLI, then the value from config file will be ignored. ([#119946](https://github.com/kubernetes/kubernetes/pull/119946), [@chendave](https://github.com/chendave))
- `kubeadm`: will now allow deploying a kubelet that is 3 versions older than the version of `kubeadm` (N-3). This aligns with the recent change made by SIG Architecture that extends the support skew between the control plane and kubelets. Tolerate this new kubelet skew for the commands `init`, `join` and `upgrade`. Note that if the `kubeadm` user applies a control plane version that is older than the `kubeadm` version (N-1 maximum) then the skew between the kubelet and control plane would become a maximum of N-2. ([#120825](https://github.com/kubernetes/kubernetes/pull/120825), [@pacoxu](https://github.com/pacoxu))
- `kubelet` , when using `--cloud-provider=external`, will now initialize the node addresses with the value of `--node-ip` , if it exists, or waits for the cloud provider to assign the addresses. ([#121028](https://github.com/kubernetes/kubernetes/pull/121028), [@aojea](https://github.com/aojea))
- `kubelet` allows pods to use the `net.ipv4.tcp_fin_timeout`, “net.ipv4.tcp_keepalive_intvl”
and “net.ipv4.tcp_keepalive_probes“ sysctl by default; Pod Security Admission
allows this sysctl in `v1.29+` versions of the baseline and restricted policies. ([#121240](https://github.com/kubernetes/kubernetes/pull/121240), [@HirazawaUi](https://github.com/HirazawaUi))
- `kubelet` now allows pods to use the `net.ipv4.tcp_keepalive_time` sysctl by default
and the minimal kernel version is 4.5; Pod Security Admission allows this sysctl
in `v1.29+` versions of the baseline and restricted policies. ([#118846](https://github.com/kubernetes/kubernetes/pull/118846), [@cyclinder](https://github.com/cyclinder))
- `kubelet` now emits a metric for end-to-end pod startup latency, including image pull. ([#121041](https://github.com/kubernetes/kubernetes/pull/121041), [@ruiwen-zhao](https://github.com/ruiwen-zhao))
- `kubelet` now exposes latency metrics of different stages of the node startup. ([#118568](https://github.com/kubernetes/kubernetes/pull/118568), [@qiutongs](https://github.com/qiutongs))
### Documentation
- Added descriptions and examples for the situation of using `kubectl rollout restart` without specifying a particular deployment. ([#120118](https://github.com/kubernetes/kubernetes/pull/120118), [@Ithrael](https://github.com/Ithrael))
- When the kubelet fails to assign CPUs to a Pod because there less available CPUs than the Pod requests, the error message changed from
`not enough cpus available to satisfy request` to `not enough cpus available to satisfy request: <num_requested> requested, only <num_available> available`. ([#121059](https://github.com/kubernetes/kubernetes/pull/121059), [@matte21](https://github.com/matte21))
### Failing Test
- Added mock framework support for unit tests for Windows in `kubeproxy`. ([#120105](https://github.com/kubernetes/kubernetes/pull/120105), [@princepereira](https://github.com/princepereira))
- DRA: when the scheduler had to deallocate a claim after a node became unsuitable for a pod, it might have needed more attempts than really necessary. This was fixed by first disabling allocations. ([#120428](https://github.com/kubernetes/kubernetes/pull/120428), [@pohly](https://github.com/pohly))
- E2e framework: retrying after intermittent `apiserver` failures was fixed in `WaitForPodsResponding` ([#120559](https://github.com/kubernetes/kubernetes/pull/120559), [@pohly](https://github.com/pohly))
- KCM specific args can be passed with `/cluster` script, without affecting CCM. New variable name: `KUBE_CONTROLLER_MANAGER_TEST_ARGS`. ([#120524](https://github.com/kubernetes/kubernetes/pull/120524), [@jprzychodzen](https://github.com/jprzychodzen)) [SIG Cloud Provider]
- `k8s.io/dynamic-resource-allocation`: DRA drivers updating to this release are compatible with Kubernetes `v1.27` and `v1.28`. ([#120868](https://github.com/kubernetes/kubernetes/pull/120868), [@pohly](https://github.com/pohly))
### Bug or Regression
- '`kubeadm`: printing the default component configs for `reset` and `join` is now
unsupported.' ([#119346](https://github.com/kubernetes/kubernetes/pull/119346), [@chendave](https://github.com/chendave))
- '`kubeadm`: removed `system:masters` organization from `etcd/healthcheck-client`
certificate.' ([#119859](https://github.com/kubernetes/kubernetes/pull/119859), [@SataQiu](https://github.com/SataQiu))
- Added `CAP_NET_RAW` to netadmin debug profile and removed privileges when debugging nodes. ([#118647](https://github.com/kubernetes/kubernetes/pull/118647), [@mochizuki875](https://github.com/mochizuki875))
- Added a check on a user attempting to create a static pod via the `kubelet` without specifying a name. They will now get a visible validation error. ([#119522](https://github.com/kubernetes/kubernetes/pull/119522), [@YTGhost](https://github.com/YTGhost))
- Added a redundant process to remove tracking finalizers from Pods that belong to Jobs. The process kicks in after the control plane marks a Job as finished. ([#119944](https://github.com/kubernetes/kubernetes/pull/119944), [@Sharpz7](https://github.com/Sharpz7))
- Added more accurate requeueing in scheduling queue for Pods rejected by the temporal failure (e.g., temporal failure on `kube-apiserver`). ([#119105](https://github.com/kubernetes/kubernetes/pull/119105), [@sanposhiho](https://github.com/sanposhiho))
- Allowed specifying `ExternalTrafficPolicy` for `Services` with `ExternalIPs`. ([#119150](https://github.com/kubernetes/kubernetes/pull/119150), [@tnqn](https://github.com/tnqn))
- Changed kubelet logs from `error` to `info` for uncached partitions when using CRI stats provider. ([#100448](https://github.com/kubernetes/kubernetes/pull/100448), [@saschagrunert](https://github.com/saschagrunert))
- Empty values are no longer assigned to undefined resources (CPU or memory) when storing the resources allocated to the pod in checkpoint. ([#117615](https://github.com/kubernetes/kubernetes/pull/117615), [@aheng-ch](https://github.com/aheng-ch))
- Fixed CEL estimated cost of `replace()` to handle a zero length replacement string correctly.
Previously this would cause the estimated cost to be higher than it should be. ([#120097](https://github.com/kubernetes/kubernetes/pull/120097), [@jpbetz](https://github.com/jpbetz)) [SIG API Machinery]
- Fixed OpenAPI v3 not being cleaned up after deleting `APIServices`. ([#120108](https://github.com/kubernetes/kubernetes/pull/120108), [@tnqn](https://github.com/tnqn))
- Fixed [121094](https://github.com/kubernetes/kubernetes/issues/121094) by re-introducing the readiness predicate for `externalTrafficPolicy: Local` services. ([#121116](https://github.com/kubernetes/kubernetes/pull/121116), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu))
- Fixed `kubectl events` not filtering events by `GroupVersion` for resources with a full name. ([#120119](https://github.com/kubernetes/kubernetes/pull/120119), [@Ithrael](https://github.com/Ithrael))
- Fixed `systemLogQuery` service name matching. ([#120678](https://github.com/kubernetes/kubernetes/pull/120678), [@rothgar](https://github.com/rothgar))
- Fixed a `1.27` scheduling regression that `PostFilter` plugin may not function if previous `PreFilter` plugins return `Skip`. ([#119769](https://github.com/kubernetes/kubernetes/pull/119769), [@Huang-Wei](https://github.com/Huang-Wei))
- Fixed a `v1.26` regression scheduling bug by ensuring that preemption is skipped when a `PreFilter` plugin returns `UnschedulableAndUnresolvable`. ([#119778](https://github.com/kubernetes/kubernetes/pull/119778), [@sanposhiho](https://github.com/sanposhiho))
- Fixed a `v1.28.0` regression where `kube-controller-manager` can crash when `StatefulSet` with `Parallel` policy and PVC labels are scaled up. ([#121142](https://github.com/kubernetes/kubernetes/pull/121142), [@aleksandra-malinowska](https://github.com/aleksandra-malinowska))
- Fixed a `v1.28` regression around restarting init containers in the right order relative to normal containers. ([#120281](https://github.com/kubernetes/kubernetes/pull/120281), [@gjkim42](https://github.com/gjkim42))
- Fixed a `v1.28` regression handling negative index json patches. ([#120327](https://github.com/kubernetes/kubernetes/pull/120327), [@liggitt](https://github.com/liggitt))
- Fixed a `v1.28` regression in scheduler: a pod with concurrent events could incorrectly get moved to the unschedulable queue where it could get stuck until the next periodic purging after 5 minutes, if there was no other event for it. ([#120413](https://github.com/kubernetes/kubernetes/pull/120413), [@pohly](https://github.com/pohly))
- Fixed a bug around restarting init containers in the right order relative to normal containers with `SidecarContainers` feature enabled. ([#120269](https://github.com/kubernetes/kubernetes/pull/120269), [@gjkim42](https://github.com/gjkim42))
- Fixed a bug in the cronjob controller where already created jobs might be missing from the status. ([#120649](https://github.com/kubernetes/kubernetes/pull/120649), [@andrewsykim](https://github.com/andrewsykim))
- Fixed a bug where `Services` using finalizers may hold onto `ClusterIP` and/or `NodePort` allocated resources for longer than expected if the finalizer is removed using the status subresource. ([#120623](https://github.com/kubernetes/kubernetes/pull/120623), [@aojea](https://github.com/aojea))
- Fixed a bug where an API group's path was not unregistered from the API server's root paths when the group was deleted. ([#121283](https://github.com/kubernetes/kubernetes/pull/121283), [@tnqn](https://github.com/tnqn)) [SIG API Machinery and Testing]
- Fixed a bug where containers would not start on `cgroupv2` systems where `swap` is disabled. ([#120784](https://github.com/kubernetes/kubernetes/pull/120784), [@elezar](https://github.com/elezar))
- Fixed a bug where the CPU set allocated to an init container, with containerRestartPolicy of `Always`, were erroneously reused by a regular container. ([#119447](https://github.com/kubernetes/kubernetes/pull/119447), [@gjkim42](https://github.com/gjkim42)) [SIG Node and Testing]
- Fixed a bug where the device resources allocated to an init container, with `containerRestartPolicy` of `Always`, were erroneously reused by a regular container. ([#120461](https://github.com/kubernetes/kubernetes/pull/120461), [@gjkim42](https://github.com/gjkim42))
- Fixed a bug where the memory resources allocated to an init container, with containerRestartPolicy of `Always`, were erroneously reused by a regular container. ([#120715](https://github.com/kubernetes/kubernetes/pull/120715), [@gjkim42](https://github.com/gjkim42)) [SIG Node]
- Fixed a concurrent map access in `TopologyCache`'s `HasPopulatedHints` method. ([#118189](https://github.com/kubernetes/kubernetes/pull/118189), [@Miciah](https://github.com/Miciah))
- Fixed a regression (`CLIENTSET_PKG: unbound variable`) when invoking deprecated `generate-groups.sh` script. ([#120877](https://github.com/kubernetes/kubernetes/pull/120877), [@soltysh](https://github.com/soltysh))
- Fixed a regression in `kube-proxy` where it might refuse to start if given
single-stack `IPv6` configuration options on a node that has both `IPv4` and
`IPv6` IPs. ([#121008](https://github.com/kubernetes/kubernetes/pull/121008), [@danwinship](https://github.com/danwinship))
- Fixed a regression in default configurations, which enabled `PodDisruptionConditions`
by default, that prevented the control plane's pod garbage collector from deleting
pods that contained duplicated field keys (environmental variables with repeated keys or
container ports). ([#121103](https://github.com/kubernetes/kubernetes/pull/121103), [@mimowo](https://github.com/mimowo))
- Fixed a regression in the default `v1.27` configurations in `kube-apiserver`: fixed the `AggregatedDiscoveryEndpoint` feature (`beta` in `v1.27+`) to successfully fetch discovery information from aggregated API servers that do not check `Accept` headers when serving the `/apis` endpoint. ([#119870](https://github.com/kubernetes/kubernetes/pull/119870), [@Jefftree](https://github.com/Jefftree))
- Fixed a regression in the kubelet's behavior while creating a container when the `EventedPLEG` feature gate is enabled. ([#120942](https://github.com/kubernetes/kubernetes/pull/120942), [@sairameshv](https://github.com/sairameshv))
- Fixed a regression since `v1.27.0` in the scheduler framework when running score plugins.
The `skippedScorePlugins` number might be greater than `enabledScorePlugins`,
so when initializing a slice the `cap(len(skippedScorePlugins) - len(enabledScorePlugins))` is negative,
which is not allowed. ([#121632](https://github.com/kubernetes/kubernetes/pull/121632), [@kerthcet](https://github.com/kerthcet))
- Fixed a situation when, sometimes, the scheduler incorrectly placed a pod in the `unschedulable` queue instead of the `backoff` queue. This happened when some plugin previously declared the pod as `unschedulable` and then in a later attempt encounters some other error. Scheduling of that pod then got delayed by up to five minutes, after which periodic flushing moved the pod back into the `active` queue. ([#120334](https://github.com/kubernetes/kubernetes/pull/120334), [@pohly](https://github.com/pohly))
- Fixed an issue related to not draining all the pods in a namespace when an empty selector, i.e., "{}," is specified in a Pod Disruption Budget (PDB). ([#119732](https://github.com/kubernetes/kubernetes/pull/119732), [@sairameshv](https://github.com/sairameshv))
- Fixed an issue where `StatefulSet` might not restart a pod after eviction or node failure. ([#120398](https://github.com/kubernetes/kubernetes/pull/120398), [@aleksandra-malinowska](https://github.com/aleksandra-malinowska))
- Fixed an issue where a `CronJob` could fail to clean up Jobs when the `ResourceQuota` for `Jobs` had been reached. ([#119776](https://github.com/kubernetes/kubernetes/pull/119776), [@ASverdlov](https://github.com/ASverdlov))
- Fixed an issue where a `StatefulSet` might not restart a pod after eviction or node failure. ([#121389](https://github.com/kubernetes/kubernetes/pull/121389), [@aleksandra-malinowska](https://github.com/aleksandra-malinowska))
- Fixed an issue with the `garbagecollection` controller registering duplicate event handlers if discovery requests failed. ([#117992](https://github.com/kubernetes/kubernetes/pull/117992), [@liggitt](https://github.com/liggitt))
- Fixed attaching volumes after detach errors. Now volumes that failed to detach are not treated as attached. Kubernetes will make sure they are fully attached before they can be used by pods. ([#120595](https://github.com/kubernetes/kubernetes/pull/120595), [@jsafrane](https://github.com/jsafrane))
- Fixed bug that kubelet resource metric `container_start_time_seconds` had timestamp equal to container start time. ([#120518](https://github.com/kubernetes/kubernetes/pull/120518), [@saschagrunert](https://github.com/saschagrunert)) [SIG Instrumentation, Node and Testing]
- Fixed inconsistency in the calculation of number of nodes that have an image, which affect the scoring in the `ImageLocality` plugin. ([#116938](https://github.com/kubernetes/kubernetes/pull/116938), [@olderTaoist](https://github.com/olderTaoist))
- Fixed issue with incremental id generation for `loadbalancer` and `endpoint` in `kubeproxy` mock test framework. ([#120723](https://github.com/kubernetes/kubernetes/pull/120723), [@princepereira](https://github.com/princepereira))
- Fixed panic in Job controller when `podRecreationPolicy: Failed` is used, and the number of terminating pods exceeds parallelism. ([#121147](https://github.com/kubernetes/kubernetes/pull/121147), [@kannon92](https://github.com/kannon92))
- Fixed regression with adding aggregated `APIservices` panicking and affected health check introduced in release `v1.28.0`. ([#120814](https://github.com/kubernetes/kubernetes/pull/120814), [@Jefftree](https://github.com/Jefftree))
- Fixed some invalid and unimportant log calls. ([#121249](https://github.com/kubernetes/kubernetes/pull/121249), [@pohly](https://github.com/pohly)) [SIG Cloud Provider, Cluster Lifecycle and Testing]
- Fixed stale SMB mount issue when SMB file share is deleted and then unmounted. ([#121851](https://github.com/kubernetes/kubernetes/pull/121851), [@andyzhangx](https://github.com/andyzhangx))
- Fixed the bug where images that were pinned by the container runtime could be garbage collected by `kubelet`. ([#119986](https://github.com/kubernetes/kubernetes/pull/119986), [@ruiwen-zhao](https://github.com/ruiwen-zhao))
- Fixed the bug where kubelet couldn't output logs after log file rotated when `kubectl logs POD_NAME -f` is running. ([#115702](https://github.com/kubernetes/kubernetes/pull/115702), [@xyz-li](https://github.com/xyz-li))
- Fixed the calculation of the requeue time in the cronjob controller, resulting in proper handling of failed/stuck jobs. ([#121327](https://github.com/kubernetes/kubernetes/pull/121327), [@soltysh](https://github.com/soltysh))
- Fixed the issue where pod with ordinal number lower than the rolling partitioning number was being deleted. It was coming up with updated image. ([#120731](https://github.com/kubernetes/kubernetes/pull/120731), [@adilGhaffarDev](https://github.com/adilGhaffarDev))
- Fixed tracking of terminating Pods in the Job status. The field was not updated unless there were other changes to apply. ([#121342](https://github.com/kubernetes/kubernetes/pull/121342), [@dejanzele](https://github.com/dejanzele))
- Forbidden sysctls for pod sharing the respective namespaces with the host are now checked when creating or updating pods without such sysctls. ([#118705](https://github.com/kubernetes/kubernetes/pull/118705), [@pacoxu](https://github.com/pacoxu))
- If a watch with the `progressNotify` option set is to be created, and the registry hasn't provided a `newFunc`, return an error. ([#120212](https://github.com/kubernetes/kubernetes/pull/120212), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery]
- Improved handling of jsonpath expressions for `kubectl wait --for`. It is now possible to use simple filter expressions which match on a field's content. ([#118748](https://github.com/kubernetes/kubernetes/pull/118748), [@andreaskaris](https://github.com/andreaskaris))
- In the `wait.PollUntilContextTimeout` function, if `immediate` is true, the condition will now be invoked before waiting, guaranteeing that the condition is invoked at least once and then wait a interval before executing again. ([#119762](https://github.com/kubernetes/kubernetes/pull/119762), [@AxeZhan](https://github.com/AxeZhan))
- Incorporating feedback on PR #119341 ([#120087](https://github.com/kubernetes/kubernetes/pull/120087), [@divyasri537](https://github.com/divyasri537)) [SIG API Machinery]
- KCCM: fixed transient node addition and removal caused by #121090 while syncing load balancers on large clusters with a lot of churn. ([#121091](https://github.com/kubernetes/kubernetes/pull/121091), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu))
- Kubeadm: changed the "system:masters" Group in the apiserver-kubelet-client.crt certificate Subject to be "kubeadm:cluster-admins" which is a less privileged Group. ([#121837](https://github.com/kubernetes/kubernetes/pull/121837), [@neolit123](https://github.com/neolit123))
- Metric buckets for `pod_start_duration_seconds` were changed to `{0.5, 1, 2, 3, 4, 5, 6, 8, 10, 20, 30, 45, 60, 120, 180, 240, 300, 360, 480, 600, 900, 1200, 1800, 2700, 3600}`. ([#120680](https://github.com/kubernetes/kubernetes/pull/120680), [@ruiwen-zhao](https://github.com/ruiwen-zhao))
- Mitigated http/2 DOS vulnerabilities for `CVE-2023-44487` and `CVE-2023-39325` for the API server when the client is unauthenticated. The mitigation may be disabled by setting the `UnauthenticatedHTTP2DOSMitigation` feature gate to `false` (it is enabled by default). An API server fronted by an L7 load balancer that already mitigates these http/2 attacks may choose to disable the kube-apiserver mitigation to avoid disrupting load balancer -> kube-apiserver connections if http/2 requests from multiple clients share the same backend connection. An API server on a private network may opt to disable the kube-apiserver mitigation to prevent performance regressions for unauthenticated clients. Authenticated requests rely on the fix in golang.org/x/net `v0.17.0` alone. https://issue.k8s.io/121197 tracks further mitigation of http/2 attacks by authenticated clients. ([#121120](https://github.com/kubernetes/kubernetes/pull/121120), [@enj](https://github.com/enj))
- No-op and GC related updates to cluster trust bundles no longer require attest authorization when the `ClusterTrustBundleAttest` plugin is enabled. ([#120779](https://github.com/kubernetes/kubernetes/pull/120779), [@enj](https://github.com/enj))
- Registered metric `apiserver_request_body_size_bytes` to track the size distribution of requests by `resource` and `verb`. ([#120474](https://github.com/kubernetes/kubernetes/pull/120474), [@YaoC](https://github.com/YaoC)) [SIG API Machinery and Instrumentation]
- Revised the logic for `DaemonSet` rolling update to exclude nodes if scheduling constraints are not met. This eliminates the problem of rolling updates to a `DaemonSet` getting stuck around tolerations. ([#119317](https://github.com/kubernetes/kubernetes/pull/119317), [@mochizuki875](https://github.com/mochizuki875))
- Scheduler: in 1.29 pre-releases, enabling contextual logging slowed down pod scheduling. ([#121715](https://github.com/kubernetes/kubernetes/pull/121715), [@pohly](https://github.com/pohly)) [SIG Instrumentation and Scheduling]
- Service Controller: will now update load balancer hosts after node's `ProviderID` is
updated. ([#120492](https://github.com/kubernetes/kubernetes/pull/120492), [@cezarygerard](https://github.com/cezarygerard))
- Setting the `status.loadBalancer` of a Service whose `spec.type` is not `LoadBalancer` was previously allowed, but any update to the `metadata` or `spec` would wipe that field. Setting this field is no longer permitted unless `spec.type` is `LoadBalancer`. In the very unlikely event that this has unexpected impact, you can enable the `AllowServiceLBStatusOnNonLB` feature gate, which will restore the previous behavior. If you do need to set this, please file an issue with the Kubernetes project to help contributors understand why you need it. ([#119789](https://github.com/kubernetes/kubernetes/pull/119789), [@thockin](https://github.com/thockin))
- The `--bind-address` parameter in kube-proxy is misleading, no port is opened with this address. Instead it is translated internally to "nodeIP". The nodeIPs for both families are now taken from the Node object if `--bind-address` is unspecified or set to the "any" address (0.0.0.0 or ::). It is recommended to leave `--bind-address` unspecified, and in particular avoid to set it to localhost (127.0.0.1 or ::1) ([#119525](https://github.com/kubernetes/kubernetes/pull/119525), [@uablrek](https://github.com/uablrek)) [SIG Network and Scalability]
- Updated `kube-openapi` to remove invalid defaults: OpenAPI spec no longer includes default of `{}` for certain fields where it did not make sense. ([#120757](https://github.com/kubernetes/kubernetes/pull/120757), [@alexzielenski](https://github.com/alexzielenski))
- Updated the CRI-O socket path, so users who configure kubelet to use a location like `/run/crio/crio.sock` don't see strange behaviour from CRI stats provider. ([#118704](https://github.com/kubernetes/kubernetes/pull/118704), [@dgl](https://github.com/dgl))
- Volume attach or publish operation will not fail at `kubelet` if target path directory already exists on the node. ([#119735](https://github.com/kubernetes/kubernetes/pull/119735), [@akankshapanse](https://github.com/akankshapanse))
- `cluster-bootstrap`: improved the security of the functions responsible for generation and validation of bootstrap tokens. ([#120400](https://github.com/kubernetes/kubernetes/pull/120400), [@neolit123](https://github.com/neolit123))
- `etcd`: updated to `v3.5.10`. ([#121566](https://github.com/kubernetes/kubernetes/pull/121566), [@mzaian](https://github.com/mzaian))
- `k8s.io/dynamic-resource-allocation/controller:` `UnsuitableNodes` can now handle a mix of allocated and unallocated claims correctly. ([#120338](https://github.com/kubernetes/kubernetes/pull/120338), [@pohly](https://github.com/pohly))
- `k8s.io/dynamic-resource-allocation/controller`: `ResourceClaimParameters` and `ResourceClassParameters` validation errors are now visible on `ResourceClaim`, `ResourceClass` and `Pod`. ([#121065](https://github.com/kubernetes/kubernetes/pull/121065), [@byako](https://github.com/byako))
- `k8s.io/dynamic-resource-allocation`: can now handle a `selected` node which isn't listed
as `potential` node. ([#120871](https://github.com/kubernetes/kubernetes/pull/120871), [@pohly](https://github.com/pohly))
- `kube-proxy` now reports its health more accurately in dual-stack clusters when there are problems with only one IP family. ([#118146](https://github.com/kubernetes/kubernetes/pull/118146), [@aroradaman](https://github.com/aroradaman))
- `kubeadm`: Fixed the bug where it always did CRI detection when `--config` was passed, even if it is not required by the subcommand. ([#120828](https://github.com/kubernetes/kubernetes/pull/120828), [@SataQiu](https://github.com/SataQiu))
- `kubeadm`: fixed `nil` pointer when `etcd` member is already removed. ([#119753](https://github.com/kubernetes/kubernetes/pull/119753), [@pacoxu](https://github.com/pacoxu))
- `kubeadm`: fixed the bug where `--image-repository` flag is missing for some init
phase sub-commands. ([#120072](https://github.com/kubernetes/kubernetes/pull/120072), [@SataQiu](https://github.com/SataQiu))
- `kubeadm`: improved the logic that checks whether a `systemd` service exists. ([#120514](https://github.com/kubernetes/kubernetes/pull/120514), [@fengxsong](https://github.com/fengxsong))
- `kubeadm`: will now use universal deserializer to decode static pod. ([#120549](https://github.com/kubernetes/kubernetes/pull/120549), [@pacoxu](https://github.com/pacoxu))
- `kubectl prune v2`: Switched annotation from `contains-group-resources` to `contains-group-kinds`,
because this is what we defined in the KEP and is clearer to end-users. Although the functionality is
in `alpha`, we will recognize the prior annotation. This migration support will be removed in `beta`/`GA`. ([#118942](https://github.com/kubernetes/kubernetes/pull/118942), [@justinsb](https://github.com/justinsb))
- `kubectl` will not print events if `--show-events=false` argument is passed to
describe PVC subcommand. ([#120380](https://github.com/kubernetes/kubernetes/pull/120380), [@MaGaroo](https://github.com/MaGaroo))
- `scheduler`: Fixed missing field `apiVersion` from events reported by the taint
manager. ([#114095](https://github.com/kubernetes/kubernetes/pull/114095), [@aimuz](https://github.com/aimuz))
### Other (Cleanup or Flake)
- Added automatic download of the CNI binary in `local-up-cluster.sh`, facilitating local debugging. ([#120312](https://github.com/kubernetes/kubernetes/pull/120312), [@HirazawaUi](https://github.com/HirazawaUi))
- Added context to `caches populated` log messages. ([#119796](https://github.com/kubernetes/kubernetes/pull/119796), [@sttts](https://github.com/sttts))
- Changed behavior of `kube-proxy` by allowing to set `sysctl` values lower than the existing one. ([#120448](https://github.com/kubernetes/kubernetes/pull/120448), [@aroradaman](https://github.com/aroradaman))
- Cleaned up `kube-apiserver` HTTP logs for impersonated requests. ([#119795](https://github.com/kubernetes/kubernetes/pull/119795), [@sttts](https://github.com/sttts))
- Deprecated the `--cloud-provider` and `--cloud-config` CLI parameters in kube-apiserver.
These parameters will be removed in a future release. ([#120903](https://github.com/kubernetes/kubernetes/pull/120903), [@dims](https://github.com/dims)) [SIG API Machinery]
- Dynamic resource allocation: will now avoid creating a new gRPC connection for every call of prepare/unprepare resource(s). ([#118619](https://github.com/kubernetes/kubernetes/pull/118619), [@TommyStarK](https://github.com/TommyStarK))
- E2E storage tests: setting test tags like `[Slow]` via the `DriverInfo.FeatureTag` field is no longer supported. ([#121391](https://github.com/kubernetes/kubernetes/pull/121391), [@pohly](https://github.com/pohly))
- Fixed an issue where the `vsphere` cloud provider would not trust a certificate if:
- The issuer of the certificate was unknown (`x509.UnknownAuthorityError`)
- The requested name did not match the set of authorized names (`x509.HostnameError`)
- The error surfaced after attempting a connection contained one of the substrings: "certificate is not trusted" or "certificate signed by unknown authority". ([#120736](https://github.com/kubernetes/kubernetes/pull/120736), [@MadhavJivrajani](https://github.com/MadhavJivrajani))
- Fixed bug where `Adding GroupVersion` log line was constantly repeated without any group version changes. ([#119825](https://github.com/kubernetes/kubernetes/pull/119825), [@Jefftree](https://github.com/Jefftree))
- Generated `ResourceClaim` names are now more readable because of an additional hyphen before the random suffix (`<pod name>-<claim name>-<random suffix>`). ([#120336](https://github.com/kubernetes/kubernetes/pull/120336), [@pohly](https://github.com/pohly))
- Graduated `JobReadyPods` to `stable`. The feature gate can no longer be disabled. ([#121302](https://github.com/kubernetes/kubernetes/pull/121302), [@stuton](https://github.com/stuton))
- Improved memory usage of `kube-controller-manager` by dropping the `.metadata.managedFields` field that `kube-controller-manager` doesn't require. ([#118455](https://github.com/kubernetes/kubernetes/pull/118455), [@linxiulei](https://github.com/linxiulei))
- Lower and upper case feature flag values are now allowed, but the name still has to match. ([#121441](https://github.com/kubernetes/kubernetes/pull/121441), [@soltysh](https://github.com/soltysh))
- Makefile and scripts now respect `GOTOOLCHAIN` and otherwise ensure `./.go-version` is used. ([#120279](https://github.com/kubernetes/kubernetes/pull/120279), [@BenTheElder](https://github.com/BenTheElder))
- Migrated the remainder of the scheduler to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#120933](https://github.com/kubernetes/kubernetes/pull/120933), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG Instrumentation, Scheduling and Testing]
- Optimized `NodeUnschedulable` Filter to avoid unnecessary calculations. ([#119399](https://github.com/kubernetes/kubernetes/pull/119399), [@wackxu](https://github.com/wackxu))
- Previous versions of Kubernetes on Google Cloud required that workloads (e.g. Deployments, DaemonSets, etc.) which used `PersistentDisk` volumes were using them in read-only mode. This validation provided very little value at relatively host implementation cost, and will no longer be validated. If this is a problem for a specific use-case, please set the `SkipReadOnlyValidationGCE` gate to false to re-enable the validation, and file a Kubernetes bug with details. ([#121083](https://github.com/kubernetes/kubernetes/pull/121083), [@thockin](https://github.com/thockin))
- Previously, the pod name and namespace were eliminated in the event log message. This PR attempts to add the preemptor pod UID in the preemption event message logs for easier debugging and safer transparency. ([#119971](https://github.com/kubernetes/kubernetes/pull/119971), [@kwakubiney](https://github.com/kwakubiney)) [SIG Scheduling]
- Promoted to conformance a test that verified that `Services` only forward traffic on the port and protocol specified. ([#120069](https://github.com/kubernetes/kubernetes/pull/120069), [@aojea](https://github.com/aojea))
- Removed `GA` feature gate about `CSIMigrationvSphere`. ([#121291](https://github.com/kubernetes/kubernetes/pull/121291), [@bzsuni](https://github.com/bzsuni))
- Removed `GA` feature gate about `ProbeTerminationGracePeriod`. ([#121257](https://github.com/kubernetes/kubernetes/pull/121257), [@bzsuni](https://github.com/bzsuni))
- Removed `GA` feature gate for `JobTrackingWithFinalizers` in `v1.28`. ([#119100](https://github.com/kubernetes/kubernetes/pull/119100), [@bzsuni](https://github.com/bzsuni))
- Removed `GA`ed feature gate `TopologyManager`. ([#121252](https://github.com/kubernetes/kubernetes/pull/121252), [@tukwila](https://github.com/tukwila))
- Removed `GA`ed feature gates `OpenAPIV3`. ([#121255](https://github.com/kubernetes/kubernetes/pull/121255), [@tukwila](https://github.com/tukwila))
- Removed `GA`ed feature gates `SeccompDefault`. ([#121246](https://github.com/kubernetes/kubernetes/pull/121246), [@tukwila](https://github.com/tukwila))
- Removed ephemeral container legacy server support for the server versions prior to `1.22`. ([#119537](https://github.com/kubernetes/kubernetes/pull/119537), [@ardaguclu](https://github.com/ardaguclu))
- Removed the `CronJobTimeZone` feature gate (the feature is stable and always enabled)
- Removed the `JobMutableNodeSchedulingDirectives` feature gate (the feature is stable and always enabled)
- Removed the `LegacyServiceAccountTokenNoAutoGeneration` feature gate (the feature is stable and always enabled) ([#120192](https://github.com/kubernetes/kubernetes/pull/120192), [@SataQiu](https://github.com/SataQiu)) [SIG Apps, Auth and Scheduling]
- Removed the `DownwardAPIHugePages` feature gate (the feature is stable and always enabled) ([#120249](https://github.com/kubernetes/kubernetes/pull/120249), [@pacoxu](https://github.com/pacoxu)) [SIG Apps and Node]
- Removed the `GRPCContainerProbe` feature gate (the feature is stable and always enabled). ([#120248](https://github.com/kubernetes/kubernetes/pull/120248), [@pacoxu](https://github.com/pacoxu))
- Renamed `apiserver_request_body_sizes` metric to `apiserver_request_body_size_bytes`. ([#120503](https://github.com/kubernetes/kubernetes/pull/120503), [@dgrisonnet](https://github.com/dgrisonnet))
- Set the resolution for the `job_controller_job_sync_duration_seconds` metric from `4ms` to `1min`. ([#120577](https://github.com/kubernetes/kubernetes/pull/120577), [@alculquicondor](https://github.com/alculquicondor))
- The `horizontalpodautoscaling` and `clusterrole-aggregation` controllers now assume the `autoscaling/v1` and `rbac.authorization.k8s.io/v1` APIs are available. If you disable those APIs and do not want to run those controllers, exclude them by passing `--controllers=-horizontalpodautoscaling` or `--controllers=-clusterrole-aggregation` to `kube-controller-manager`. ([#117977](https://github.com/kubernetes/kubernetes/pull/117977), [@liggitt](https://github.com/liggitt)) [SIG API Machinery and Cloud Provider]
- The metrics controlled by the `ComponentSLIs` feature-gate and served at `/metrics/slis` are now GA and unconditionally enabled. The feature-gate will be removed in `v1.31`. ([#120574](https://github.com/kubernetes/kubernetes/pull/120574), [@logicalhan](https://github.com/logicalhan))
- Updated CNI plugins to `v1.3.0`. ([#119969](https://github.com/kubernetes/kubernetes/pull/119969), [@saschagrunert](https://github.com/saschagrunert))
- Updated `cri-tools` to `v1.28.0`. ([#119933](https://github.com/kubernetes/kubernetes/pull/119933), [@saschagrunert](https://github.com/saschagrunert))
- Updated `distroless-iptables` to use `registry.k8s.io/build-image/distroless-iptables:v0.3.1`. ([#120352](https://github.com/kubernetes/kubernetes/pull/120352), [@saschagrunert](https://github.com/saschagrunert))
- Updated runc to `1.1.10`. ([#121739](https://github.com/kubernetes/kubernetes/pull/121739), [@ty-dc](https://github.com/ty-dc))
- Upgraded `coredns` to `v1.11.1`. ([#120116](https://github.com/kubernetes/kubernetes/pull/120116), [@tukwila](https://github.com/tukwila))
- `EnqueueExtensions` from plugins other than `PreEnqueue`, `PreFilter`, `Filter`, `Reserve` and `Permit` are now ignored.
It reduces the number of kinds of cluster events the scheduler needs to subscribe/handle. ([#121571](https://github.com/kubernetes/kubernetes/pull/121571), [@sanposhiho](https://github.com/sanposhiho))
- `GetPodQOS(pod *core.Pod)` function now returns the stored value from `PodStatus.QOSClass`, if set. To compute/evaluate the value of `QOSClass` from scratch, `ComputePodQOS(pod*core.Pod)` must be used. ([#119665](https://github.com/kubernetes/kubernetes/pull/119665), [@vinaykul](https://github.com/vinaykul))
- `RetroactiveDefaultStorageClass` feature gate that graduated to GA in `v1.28` and was unconditionally enabled has been removed in `v1.29`. ([#120861](https://github.com/kubernetes/kubernetes/pull/120861), [@RomanBednar](https://github.com/RomanBednar))
- `Statefulset` now waits for new replicas in tests when removing `.start.ordinal`. ([#119761](https://github.com/kubernetes/kubernetes/pull/119761), [@soltysh](https://github.com/soltysh))
- `ValidatingAdmissionPolicy` and `ValidatingAdmissionPolicyBinding` objects are
persisted in `etcd` using the `v1beta1` version. Either remove alpha objects, or disable the
alpha `ValidatingAdmissionPolicy` feature in a `v1.27` server before upgrading to a
`v1.28` server with the beta feature and API enabled. ([#120018](https://github.com/kubernetes/kubernetes/pull/120018), [@liggitt](https://github.com/liggitt))
- `client-go`: `k8s.io/client-go/tools` events and record packages now have new APIs for specifying a context and logger. ([#120729](https://github.com/kubernetes/kubernetes/pull/120729), [@pohly](https://github.com/pohly))
- `kube-controller-manager` help now includes controllers behind a feature gate in `--controllers` flag. ([#120371](https://github.com/kubernetes/kubernetes/pull/120371), [@atiratree](https://github.com/atiratree))
- `kubeadm`: removed `system:masters` organization from `apiserver-etcd-client`
certificate. ([#120521](https://github.com/kubernetes/kubernetes/pull/120521), [@SataQiu](https://github.com/SataQiu))
- `kubeadm`: removed leftover disclaimer that could be seen in the `kubeadm init phase certs` command help screen, since the "certs" phase of "init" is no longer alpha. ([#121172](https://github.com/kubernetes/kubernetes/pull/121172), [@SataQiu](https://github.com/SataQiu))
- `kubeadm`: updated warning message when swap space is detected. When swap is
active on Linux, `kubeadm` explains that swap is supported for cgroup v2 only and
is beta but disabled by default. ([#120198](https://github.com/kubernetes/kubernetes/pull/120198), [@pacoxu](https://github.com/pacoxu))
- `kubectl` will not support the `/swagger-2.0.0.pb-v1` endpoint that has been long deprecated. ([#119410](https://github.com/kubernetes/kubernetes/pull/119410), [@Jefftree](https://github.com/Jefftree))
- `scheduler`: handling of unschedulable pods because a `ResourceClass` is missing
is a bit more efficient and no longer relies on periodic retries. ([#120213](https://github.com/kubernetes/kubernetes/pull/120213), [@pohly](https://github.com/pohly))
## Dependencies
### Added
- cloud.google.com/go/dataproc/v2: v2.0.1
- github.com/danwinship/knftables: [v0.0.13](https://github.com/danwinship/knftables/tree/v0.0.13)
- github.com/distribution/reference: [v0.5.0](https://github.com/distribution/reference/tree/v0.5.0)
- github.com/google/s2a-go: [v0.1.7](https://github.com/google/s2a-go/tree/v0.1.7)
- google.golang.org/genproto/googleapis/bytestream: e85fd2c
### Changed
- cloud.google.com/go/accessapproval: v1.6.0 → v1.7.1
- cloud.google.com/go/accesscontextmanager: v1.7.0 → v1.8.1
- cloud.google.com/go/aiplatform: v1.37.0 → v1.48.0
- cloud.google.com/go/analytics: v0.19.0 → v0.21.3
- cloud.google.com/go/apigateway: v1.5.0 → v1.6.1
- cloud.google.com/go/apigeeconnect: v1.5.0 → v1.6.1
- cloud.google.com/go/apigeeregistry: v0.6.0 → v0.7.1
- cloud.google.com/go/appengine: v1.7.1 → v1.8.1
- cloud.google.com/go/area120: v0.7.1 → v0.8.1
- cloud.google.com/go/artifactregistry: v1.13.0 → v1.14.1
- cloud.google.com/go/asset: v1.13.0 → v1.14.1
- cloud.google.com/go/assuredworkloads: v1.10.0 → v1.11.1
- cloud.google.com/go/automl: v1.12.0 → v1.13.1
- cloud.google.com/go/baremetalsolution: v0.5.0 → v1.1.1
- cloud.google.com/go/batch: v0.7.0 → v1.3.1
- cloud.google.com/go/beyondcorp: v0.5.0 → v1.0.0
- cloud.google.com/go/bigquery: v1.50.0 → v1.53.0
- cloud.google.com/go/billing: v1.13.0 → v1.16.0
- cloud.google.com/go/binaryauthorization: v1.5.0 → v1.6.1
- cloud.google.com/go/certificatemanager: v1.6.0 → v1.7.1
- cloud.google.com/go/channel: v1.12.0 → v1.16.0
- cloud.google.com/go/cloudbuild: v1.9.0 → v1.13.0
- cloud.google.com/go/clouddms: v1.5.0 → v1.6.1
- cloud.google.com/go/cloudtasks: v1.10.0 → v1.12.1
- cloud.google.com/go/compute: v1.19.0 → v1.23.0
- cloud.google.com/go/contactcenterinsights: v1.6.0 → v1.10.0
- cloud.google.com/go/container: v1.15.0 → v1.24.0
- cloud.google.com/go/containeranalysis: v0.9.0 → v0.10.1
- cloud.google.com/go/datacatalog: v1.13.0 → v1.16.0
- cloud.google.com/go/dataflow: v0.8.0 → v0.9.1
- cloud.google.com/go/dataform: v0.7.0 → v0.8.1
- cloud.google.com/go/datafusion: v1.6.0 → v1.7.1
- cloud.google.com/go/datalabeling: v0.7.0 → v0.8.1
- cloud.google.com/go/dataplex: v1.6.0 → v1.9.0
- cloud.google.com/go/dataqna: v0.7.0 → v0.8.1
- cloud.google.com/go/datastore: v1.11.0 → v1.13.0
- cloud.google.com/go/datastream: v1.7.0 → v1.10.0
- cloud.google.com/go/deploy: v1.8.0 → v1.13.0
- cloud.google.com/go/dialogflow: v1.32.0 → v1.40.0
- cloud.google.com/go/dlp: v1.9.0 → v1.10.1
- cloud.google.com/go/documentai: v1.18.0 → v1.22.0
- cloud.google.com/go/domains: v0.8.0 → v0.9.1
- cloud.google.com/go/edgecontainer: v1.0.0 → v1.1.1
- cloud.google.com/go/essentialcontacts: v1.5.0 → v1.6.2
- cloud.google.com/go/eventarc: v1.11.0 → v1.13.0
- cloud.google.com/go/filestore: v1.6.0 → v1.7.1
- cloud.google.com/go/firestore: v1.9.0 → v1.11.0
- cloud.google.com/go/functions: v1.13.0 → v1.15.1
- cloud.google.com/go/gkebackup: v0.4.0 → v1.3.0
- cloud.google.com/go/gkeconnect: v0.7.0 → v0.8.1
- cloud.google.com/go/gkehub: v0.12.0 → v0.14.1
- cloud.google.com/go/gkemulticloud: v0.5.0 → v1.0.0
- cloud.google.com/go/gsuiteaddons: v1.5.0 → v1.6.1
- cloud.google.com/go/iam: v0.13.0 → v1.1.1
- cloud.google.com/go/iap: v1.7.1 → v1.8.1
- cloud.google.com/go/ids: v1.3.0 → v1.4.1
- cloud.google.com/go/iot: v1.6.0 → v1.7.1
- cloud.google.com/go/kms: v1.10.1 → v1.15.0
- cloud.google.com/go/language: v1.9.0 → v1.10.1
- cloud.google.com/go/lifesciences: v0.8.0 → v0.9.1
- cloud.google.com/go/longrunning: v0.4.1 → v0.5.1
- cloud.google.com/go/managedidentities: v1.5.0 → v1.6.1
- cloud.google.com/go/maps: v0.7.0 → v1.4.0
- cloud.google.com/go/mediatranslation: v0.7.0 → v0.8.1
- cloud.google.com/go/memcache: v1.9.0 → v1.10.1
- cloud.google.com/go/metastore: v1.10.0 → v1.12.0
- cloud.google.com/go/monitoring: v1.13.0 → v1.15.1
- cloud.google.com/go/networkconnectivity: v1.11.0 → v1.12.1
- cloud.google.com/go/networkmanagement: v1.6.0 → v1.8.0
- cloud.google.com/go/networksecurity: v0.8.0 → v0.9.1
- cloud.google.com/go/notebooks: v1.8.0 → v1.9.1
- cloud.google.com/go/optimization: v1.3.1 → v1.4.1
- cloud.google.com/go/orchestration: v1.6.0 → v1.8.1
- cloud.google.com/go/orgpolicy: v1.10.0 → v1.11.1
- cloud.google.com/go/osconfig: v1.11.0 → v1.12.1
- cloud.google.com/go/oslogin: v1.9.0 → v1.10.1
- cloud.google.com/go/phishingprotection: v0.7.0 → v0.8.1
- cloud.google.com/go/policytroubleshooter: v1.6.0 → v1.8.0
- cloud.google.com/go/privatecatalog: v0.8.0 → v0.9.1
- cloud.google.com/go/pubsub: v1.30.0 → v1.33.0
- cloud.google.com/go/pubsublite: v1.7.0 → v1.8.1
- cloud.google.com/go/recaptchaenterprise/v2: v2.7.0 → v2.7.2
- cloud.google.com/go/recommendationengine: v0.7.0 → v0.8.1
- cloud.google.com/go/recommender: v1.9.0 → v1.10.1
- cloud.google.com/go/redis: v1.11.0 → v1.13.1
- cloud.google.com/go/resourcemanager: v1.7.0 → v1.9.1
- cloud.google.com/go/resourcesettings: v1.5.0 → v1.6.1
- cloud.google.com/go/retail: v1.12.0 → v1.14.1
- cloud.google.com/go/run: v0.9.0 → v1.2.0
- cloud.google.com/go/scheduler: v1.9.0 → v1.10.1
- cloud.google.com/go/secretmanager: v1.10.0 → v1.11.1
- cloud.google.com/go/security: v1.13.0 → v1.15.1
- cloud.google.com/go/securitycenter: v1.19.0 → v1.23.0
- cloud.google.com/go/servicedirectory: v1.9.0 → v1.11.0
- cloud.google.com/go/shell: v1.6.0 → v1.7.1
- cloud.google.com/go/spanner: v1.45.0 → v1.47.0
- cloud.google.com/go/speech: v1.15.0 → v1.19.0
- cloud.google.com/go/storagetransfer: v1.8.0 → v1.10.0
- cloud.google.com/go/talent: v1.5.0 → v1.6.2
- cloud.google.com/go/texttospeech: v1.6.0 → v1.7.1
- cloud.google.com/go/tpu: v1.5.0 → v1.6.1
- cloud.google.com/go/trace: v1.9.0 → v1.10.1
- cloud.google.com/go/translate: v1.7.0 → v1.8.2
- cloud.google.com/go/video: v1.15.0 → v1.19.0
- cloud.google.com/go/videointelligence: v1.10.0 → v1.11.1
- cloud.google.com/go/vision/v2: v2.7.0 → v2.7.2
- cloud.google.com/go/vmmigration: v1.6.0 → v1.7.1
- cloud.google.com/go/vmwareengine: v0.3.0 → v1.0.0
- cloud.google.com/go/vpcaccess: v1.6.0 → v1.7.1
- cloud.google.com/go/webrisk: v1.8.0 → v1.9.1
- cloud.google.com/go/websecurityscanner: v1.5.0 → v1.6.1
- cloud.google.com/go/workflows: v1.10.0 → v1.11.1
- cloud.google.com/go: v0.110.0 → v0.110.6
- github.com/alecthomas/template: [fb15b89 → a0175ee](https://github.com/alecthomas/template/compare/fb15b89...a0175ee)
- github.com/cncf/xds/go: [06c439d → e9ce688](https://github.com/cncf/xds/go/compare/06c439d...e9ce688)
- github.com/coredns/corefile-migration: [v1.0.20 → v1.0.21](https://github.com/coredns/corefile-migration/compare/v1.0.20...v1.0.21)
- github.com/cyphar/filepath-securejoin: [v0.2.3 → v0.2.4](https://github.com/cyphar/filepath-securejoin/compare/v0.2.3...v0.2.4)
- github.com/docker/docker: [v20.10.21+incompatible → v20.10.24+incompatible](https://github.com/docker/docker/compare/v20.10.21...v20.10.24)
- github.com/emicklei/go-restful/v3: [v3.9.0 → v3.11.0](https://github.com/emicklei/go-restful/v3/compare/v3.9.0...v3.11.0)
- github.com/envoyproxy/go-control-plane: [v0.10.3 → v0.11.1](https://github.com/envoyproxy/go-control-plane/compare/v0.10.3...v0.11.1)
- github.com/envoyproxy/protoc-gen-validate: [v0.9.1 → v1.0.2](https://github.com/envoyproxy/protoc-gen-validate/compare/v0.9.1...v1.0.2)
- github.com/evanphx/json-patch: [v5.6.0+incompatible → v4.12.0+incompatible](https://github.com/evanphx/json-patch/compare/v5.6.0...v4.12.0)
- github.com/fsnotify/fsnotify: [v1.6.0 → v1.7.0](https://github.com/fsnotify/fsnotify/compare/v1.6.0...v1.7.0)
- github.com/go-logr/logr: [v1.2.4 → v1.3.0](https://github.com/go-logr/logr/compare/v1.2.4...v1.3.0)
- github.com/godbus/dbus/v5: [v5.0.6 → v5.1.0](https://github.com/godbus/dbus/v5/compare/v5.0.6...v5.1.0)
- github.com/golang/glog: [v1.0.0 → v1.1.0](https://github.com/golang/glog/compare/v1.0.0...v1.1.0)
- github.com/google/cadvisor: [v0.47.3 → v0.48.1](https://github.com/google/cadvisor/compare/v0.47.3...v0.48.1)
- github.com/google/cel-go: [v0.16.0 → v0.17.7](https://github.com/google/cel-go/compare/v0.16.0...v0.17.7)
- github.com/google/go-cmp: [v0.5.9 → v0.6.0](https://github.com/google/go-cmp/compare/v0.5.9...v0.6.0)
- github.com/googleapis/gax-go/v2: [v2.7.1 → v2.11.0](https://github.com/googleapis/gax-go/v2/compare/v2.7.1...v2.11.0)
- github.com/gorilla/websocket: [v1.4.2 → v1.5.0](https://github.com/gorilla/websocket/compare/v1.4.2...v1.5.0)
- github.com/grpc-ecosystem/grpc-gateway/v2: [v2.7.0 → v2.16.0](https://github.com/grpc-ecosystem/grpc-gateway/v2/compare/v2.7.0...v2.16.0)
- github.com/ishidawataru/sctp: [7c296d4 → 7ff4192](https://github.com/ishidawataru/sctp/compare/7c296d4...7ff4192)
- github.com/konsorten/go-windows-terminal-sequences: [v1.0.3 → v1.0.1](https://github.com/konsorten/go-windows-terminal-sequences/compare/v1.0.3...v1.0.1)
- github.com/mrunalp/fileutils: [v0.5.0 → v0.5.1](https://github.com/mrunalp/fileutils/compare/v0.5.0...v0.5.1)
- github.com/onsi/ginkgo/v2: [v2.9.4 → v2.13.0](https://github.com/onsi/ginkgo/v2/compare/v2.9.4...v2.13.0)
- github.com/onsi/gomega: [v1.27.6 → v1.29.0](https://github.com/onsi/gomega/compare/v1.27.6...v1.29.0)
- github.com/opencontainers/runc: [v1.1.7 → v1.1.10](https://github.com/opencontainers/runc/compare/v1.1.7...v1.1.10)
- github.com/opencontainers/selinux: [v1.10.0 → v1.11.0](https://github.com/opencontainers/selinux/compare/v1.10.0...v1.11.0)
- github.com/spf13/afero: [v1.2.2 → v1.1.2](https://github.com/spf13/afero/compare/v1.2.2...v1.1.2)
- github.com/stretchr/testify: [v1.8.2 → v1.8.4](https://github.com/stretchr/testify/compare/v1.8.2...v1.8.4)
- github.com/vmware/govmomi: [v0.30.0 → v0.30.6](https://github.com/vmware/govmomi/compare/v0.30.0...v0.30.6)
- go.etcd.io/bbolt: v1.3.7 → v1.3.8
- go.etcd.io/etcd/api/v3: v3.5.9 → v3.5.10
- go.etcd.io/etcd/client/pkg/v3: v3.5.9 → v3.5.10
- go.etcd.io/etcd/client/v2: v2.305.9 → v2.305.10
- go.etcd.io/etcd/client/v3: v3.5.9 → v3.5.10
- go.etcd.io/etcd/pkg/v3: v3.5.9 → v3.5.10
- go.etcd.io/etcd/raft/v3: v3.5.9 → v3.5.10
- go.etcd.io/etcd/server/v3: v3.5.9 → v3.5.10
- go.opentelemetry.io/contrib/instrumentation/github.com/emicklei/go-restful/otelrestful: v0.35.0 → v0.42.0
- go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc: v0.35.0 → v0.42.0
- go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp: v0.35.1 → v0.44.0
- go.opentelemetry.io/contrib/propagators/b3: v1.10.0 → v1.17.0
- go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc: v1.10.0 → v1.19.0
- go.opentelemetry.io/otel/exporters/otlp/otlptrace: v1.10.0 → v1.19.0
- go.opentelemetry.io/otel/metric: v0.31.0 → v1.19.0
- go.opentelemetry.io/otel/sdk: v1.10.0 → v1.19.0
- go.opentelemetry.io/otel/trace: v1.10.0 → v1.19.0
- go.opentelemetry.io/otel: v1.10.0 → v1.19.0
- go.opentelemetry.io/proto/otlp: v0.19.0 → v1.0.0
- golang.org/x/crypto: v0.11.0 → v0.14.0
- golang.org/x/mod: v0.10.0 → v0.12.0
- golang.org/x/net: v0.13.0 → v0.17.0
- golang.org/x/oauth2: v0.8.0 → v0.10.0
- golang.org/x/sync: v0.2.0 → v0.3.0
- golang.org/x/sys: v0.10.0 → v0.13.0
- golang.org/x/term: v0.10.0 → v0.13.0
- golang.org/x/text: v0.11.0 → v0.13.0
- golang.org/x/tools: v0.8.0 → v0.12.0
- google.golang.org/api: v0.114.0 → v0.126.0
- google.golang.org/genproto/googleapis/api: dd9d682 → 23370e0
- google.golang.org/genproto/googleapis/rpc: 28d5490 → b8732ec
- google.golang.org/genproto: 0005af6 → f966b18
- google.golang.org/grpc: v1.54.0 → v1.58.3
- google.golang.org/protobuf: v1.30.0 → v1.31.0
- k8s.io/gengo: c0856e2 → 9cce18d
- k8s.io/klog/v2: v2.100.1 → v2.110.1
- k8s.io/kube-openapi: 2695361 → 2dd684a
- k8s.io/utils: d93618c → 3b25d92
- sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.1.2 → v0.28.0
- sigs.k8s.io/structured-merge-diff/v4: v4.2.3 → v4.4.1
### Removed
- cloud.google.com/go/dataproc: v1.12.0
- cloud.google.com/go/gaming: v1.9.0
- github.com/blang/semver: [v3.5.1+incompatible](https://github.com/blang/semver/tree/v3.5.1)
- github.com/jmespath/go-jmespath/internal/testify: [v1.5.1](https://github.com/jmespath/go-jmespath/internal/testify/tree/v1.5.1)
- go.opentelemetry.io/otel/exporters/otlp/internal/retry: v1.10.0
# v1.29.0-rc.2
## Downloads for v1.29.0-rc.2
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes.tar.gz) | daaabe57e2da16a076380072bb0e4a178400f045bb82d44d338efa90641bb4e35b590764d9ab4f365219634149588526da57d1aaabdb1ed805ee0ccd9aed63b6
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-src.tar.gz) | c4a3ea15db8a7d0696f2ef4a2f3d1e65b89a931074043957fa59be2bb0fac04b9967e8eff1037b4c649fcdd34a3ad2b717d129b2ce2f45691675bbef95710833
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-client-darwin-amd64.tar.gz) | 3b22ba3aa4f778f0086e739999f81f8ca040f5a5b9b88a8e71c9cd94dc728ee9090f0c388a0020f458a6c13716c614b37e1be11b7e5402cc00380595ce3be21f
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-client-darwin-arm64.tar.gz) | ed2d2f866f28b5f157b1ca39ae2cb832cc934c2383f8cb90b5e79781a4835579dd99ebe2f34245c4a764a49ebcb343499fded8f180ac1a084acbfa2bfc38ef37
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-client-linux-386.tar.gz) | cdf10ab26e223742a882d19a36a932a3230a061e026514fd3cbe3d763f7de50372903e04ea9b85c2554980310d8fa8b8c140bb623e491a6c646af18499e14354
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-client-linux-amd64.tar.gz) | 7aa3da03d393b9da31ec3fd0c5c9694ab3a3e1bc7340108375238a5b132da8955db318559f2d6f7968f141802373fa3462a980932e5bb9d59b553816fc1bc2d4
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-client-linux-arm.tar.gz) | c24722f8f20f86a842f04e047041c631975ee4a8800da9cee1ff5415055842eb381b7c918b7da2c7421369d718b13e182008e08c2f5f2a5c5a1782c481851ddf
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-client-linux-arm64.tar.gz) | 1d4df5d1bb6fd5fdb8b9dc3b0bb7c8b7f3c155dab019222d9611706d5140206ac9a7eda6673a2a912e1738f02f8707184e4e711d4f82f5680a8098ce59ad9f74
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-client-linux-ppc64le.tar.gz) | c7187e7834f690958ea2b3f7aeac987f6efcd76b859487b391972a25daf347c87fc60e9e9b9a440a2c4df529c8f467b94790e6f8992cdbbac87f73d1373f03c0
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-client-linux-s390x.tar.gz) | e758559d18ef1510d50fa57177e2023ffc4bc19d3a4e302e984bef35701e9a47894e7c7d80589742bd24a337d75b35e83faf1c097202401deee01e0f5fe31829
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-client-windows-386.tar.gz) | 0ef788e96ea786b0d62de7dcc1315800a4106b367381e02731ca384bc89397aa7a1de5b678a0532df590c1f8d448206a236623ed729bc5c30b0e63317aec6a6c
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-client-windows-amd64.tar.gz) | 719c3c1f9b7beb199bd9d0a0b5c85d99b76bd462397a3d4b37fe8b5970e30e69e26f30815f97bb9a4049b28c7aaf0b384d1378a14e8273774d65f3549cbd3083
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-client-windows-arm64.tar.gz) | 24916863e604c14939ccd1574f754306215d603ff2efdd4dd00fa667923932aff88f8046fc8a4b7dea47f97ea5bfc53d4193fb293fa48fdefb4cd65c301e32ab
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-server-linux-amd64.tar.gz) | 88e7a746f69b3070908fc32213d6046a10c5df139be566f42b135fb344f9681f7d527e5d8b65214f3a5abc4564962d4d1f8391d6ce62ca3620e538bb0e836055
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-server-linux-arm64.tar.gz) | 69b9bf487dabf7ae466cb9cf0e569d04908c359dea2772ed2a0daa0c5be2d1389d09cac3eebc7c350af0fd0e0f30e2cfc86c4ede423a0cd8d00f62909e13fac4
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-server-linux-ppc64le.tar.gz) | 5925a756d4ab2be13141391c7c18888f2b7337aa3db05ac0cb0ed25ad66a6d290b38731e389b829571bb5fcb95b6b9f5b4d054058818adad66d1f39e45fa9356
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-server-linux-s390x.tar.gz) | 82e4aa613bffc8658e8a10c53269a7977219fd38b985fb8a4a4df78a8fa876521c1925ab927a74737e3a57c415963c13717720d106b6f086294c717bdc5f02c9
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-node-linux-amd64.tar.gz) | fd7be4f743a641f22c04d32749825d509b6c9ff095207997db40dc39cb1647bd03f47b71287339a02f303b2903ffe6bd0910bd051f20f44c03e7d853c2e794b5
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-node-linux-arm64.tar.gz) | 12454bf94d86282cbf04db188f504aed6d8c6eeda8f32b0001152aa7ed52c65c1ec0db90c99077b30987d40a4ccd6c707dcda1ce6be84c746f4c972af0b277f4
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-node-linux-ppc64le.tar.gz) | 1eb0d47ca00231df825daf2c39b203c438a7c1b1d9ec1e7bcefd63c16e89771316d9a2cf66c89ad6653e70fed3ccc38a650e83be106f2738c5ae97bbc055de7f
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-node-linux-s390x.tar.gz) | ff7f4da7e71859ebea3a70410575f7ffa52c476f4f6d76958366ca5e4ec7b315fdd4b5b8cbd48faaa800ffcfb64f9af85b6605582480242ae336c3fea84f8733
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.2/kubernetes-node-windows-amd64.tar.gz) | 8fc83a3735d163866e370df3c6bccbe15d2ce478b95bc1fe71b199952db68c95f31d8967514b0930c2ad7bc2734c2a4ab5d8cdc738611365215f7d88a19bc2d6
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.0-rc.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.0-rc.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.0-rc.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.0-rc.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.0-rc.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.0-rc.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-s390x)
## Changelog since v1.29.0-rc.1
## Changes by Kind
### Feature
- Bump distroless-iptables to v0.4.3 ([#122206](https://github.com/kubernetes/kubernetes/pull/122206), [@xmudrii](https://github.com/xmudrii)) [SIG Release and Testing]
- Kubernetes is now built with Go 1.21.5 ([#122201](https://github.com/kubernetes/kubernetes/pull/122201), [@xmudrii](https://github.com/xmudrii)) [SIG Release and Testing]
## Dependencies
### Added
_Nothing has changed._
### Changed
_Nothing has changed._
### Removed
_Nothing has changed._
# v1.29.0-rc.1
## Downloads for v1.29.0-rc.1
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes.tar.gz) | e44677de7af6634c31b86672dc6755d97ae145fd0497229c08b156d11dcdbc922f15c715fd878b585d21a6c7dd10fde0b43135f0b6f7e77a9f957f2280a32018
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-src.tar.gz) | 63e197478e315a64dae6282c1e4ce2b672f0a3941bea9920094b703d44a09aaed74228a7c29fbec4051a4cb832f6e791d54f5e76e006aada1216a502c6d2e744
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-client-darwin-amd64.tar.gz) | 167d931f6b9540b9fbd8501e3dd9d2ea032ead96f57cf4a929020fff3c4efb0456e3bc5eee2a8436670589ef18b48018dabf57081ff13393542007e9d0c8cb72
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-client-darwin-arm64.tar.gz) | 6623d8b08b69beea8832a1130c50624f248464a01fec4fce720700ccebdd3ed440af664f5beb49a294557db3c4ff7a8fecfd3cf48f9dabcc48b9bda2d791c08e
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-client-linux-386.tar.gz) | 11f6c0d6f0954938c4217436536a67713c403b1f3c2b988d26944374fab6f70c385cb9356e6aa51b480fdcd07539de4667be2e72fa8128ba792a607bb388254f
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-client-linux-amd64.tar.gz) | ee7605531629a2e320f299ef49bfb87566b73245f16dee79a40b824d637bd97ca8bd7f81a177320c2e4bbb82acd7f5d840359dd79c3062c2136e0b5f04eeb90e
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-client-linux-arm.tar.gz) | baa1af4d932c3d36ff084f2fc4c7676e76a2f0e4c4c746495f932c56a583b4390376b0b631b13f6dc3b03bd874d84c20f82e71d2483940b37ea439bc7d21dadf
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-client-linux-arm64.tar.gz) | 26a20dfbeca7abb73a73f1cfb5337b4af71bb3d2810d053f9d94e4a3709d282e515a542f02502a7e86adf3b32ac52e7b434d1604fa9664729682d083a55b314d
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-client-linux-ppc64le.tar.gz) | eca5ca3028b64ea44138b08831e998ac85bd054785099366e295b88b8b568c455a54f4fd110b568208169b9a3aef918c7f6caf8e05f9e73f85c26f973a589e2d
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-client-linux-s390x.tar.gz) | 5bcde8b36b8dc1d3ed83337322fc260311238e9f067301838c5002bb0dc63153f82c2602d8c9a28c1ae5dd85b0eecc4d3e8c31dcf75f4653c6b9bebd3a564321
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-client-windows-386.tar.gz) | d183c3183bfac0878377eaa8adf00e6ecdb3f252ed47180b8f9231d757b44c30931620b04da52c3470a1a278fa5a76e99f0b7c587b696f517caec5ff16103480
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-client-windows-amd64.tar.gz) | 7a51ed89ad5f850bfc94e4175294d944eae9628c281fea2c18939417f84c438b82246d262e645bea9fd257deb60b51c05b1c1ebd321b35aafeb87d1b4f83ebe6
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-client-windows-arm64.tar.gz) | 2efc1fd75461ed5e0bceba78681804567e731a545a801b28e8291d2f3cc8e2c8c22d3e418888a666dc1e40754681acbcbfc64fc81772088b8524fde7c55e7e3a
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-server-linux-amd64.tar.gz) | 28eeda8ab821891ca445b4a884ae70028146f6d4264e364a7ca88c819ea80bd95653c7f538af6c5f660921f140026f3d61c469afb0109c2f5111245592439fd7
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-server-linux-arm64.tar.gz) | d786685de63f060910181f0de66511fe8f8f0fa66f00ee0c76957246a62e3de3208ec009908c6700ff83dc7c5e5c24f3c1c3118d06b07568fcb004190d1e5fe6
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-server-linux-ppc64le.tar.gz) | b9d14fdcf282b4f9e523d81f71cd82d64bfb9bf9ba4affb7d6dbcfa8191f9b0d238f7091c65c1f1d516c6bcaa13f68affdd3bfda1ec759f35d505284a87494e1
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-server-linux-s390x.tar.gz) | f84c315e3d3ab6e3124106783e43b18d407d5c4ef09910641ae51c034b550fba0581515181ae4d355eea5b75eb0688460a891141c47d13be902e06156908d26d
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-node-linux-amd64.tar.gz) | b2a974c505878c757bc643c64cf24208ed92bfe7943f6e9e50d215f027e4fc2f4a578d3975721f53ff6a06e6560dec362887a236052213784222695c916a59fb
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-node-linux-arm64.tar.gz) | 7f7b6a3bcdef051ae0126a811d02b327f6f45b13050934dd4d90219a23efd5aeabb945e040a9aa57a24a4151222b558a4e2a996bad5e5f7ef508c26e6cca85ee
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-node-linux-ppc64le.tar.gz) | 6245a8645a6b52c2bd40e1104d4fbba015bf583f50478b2686ecf0c3c8fdc05c6729da76dc6301ca0f2f0d63f1a5df84a0871fb4793fc3aa9e8e2b6cdae37d34
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-node-linux-s390x.tar.gz) | fe530554696b84db43372ca48e95a01c46cf3a09dd51fb569b1792a341827a428bfbb04c3bab59d6aea095687eafe144534a0d98df8852d469e8bdc69a8d2d1a
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.1/kubernetes-node-windows-amd64.tar.gz) | 0a80378b4037f8d325fdbdc065ce5fd841b0b66242d64ec5c6b8ad6af0e430fc020d4fb0d624909b0725761d2c249a8f75c91eae22af03cefe4a4338b57d3d29
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.0-rc.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.0-rc.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.0-rc.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.0-rc.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.0-rc.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.0-rc.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-s390x)
## Changelog since v1.29.0-rc.0
## Dependencies
### Added
_Nothing has changed._
### Changed
_Nothing has changed._
### Removed
_Nothing has changed._
# v1.29.0-rc.0
## Downloads for v1.29.0-rc.0
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes.tar.gz) | 6a5b027a35b96d1cf8495efce0f9f518499b94e63e1d11058876d1b364d0bba42ccedac4612082771eb38cd54be0d8868a808de05c7e9077b8644f15a5c6f413
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-src.tar.gz) | d92897e5e28a14f0fbd3f03e9016e9c86f30bf097c4e709e6dba74b1a9897ce016e3c3a44aed9d5f851af1f5d5bd0ea2240efe8d8d12d7893b7f9cff66caff55
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-client-darwin-amd64.tar.gz) | a4e8dd4e65158024a46843701ed24082eefde5d407c6d6a191b7b7f690413ea65c5422ba578e2813cd6624ac7327174554d879dbbbb324b56fbfe99892eb8d80
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-client-darwin-arm64.tar.gz) | ef14378eaa3a35a34ab5e9b06c9856ff46165bbee2a4efc1b8512de47e8f584449d94155665978eca6264e23f131e31d072f9333117c11a3e92aadeea367b8e5
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-client-linux-386.tar.gz) | 686a8b69525e8e1494cdc890e8023ba60f86e41ceb28cb5df7e33f152ecc3ac8c62b0b1d24fa6c8198278a9d585bfd8962d058daf7f27dfa658580598b45cafe
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-client-linux-amd64.tar.gz) | 7ebe8d866f8fd1dccd6761be0ad5096cb861e5fb20bdea0ac65a3d63230d9a7d47df16a6933fcf4069cf819ab90d12eaf87ec53873eacd88c3feab009e85e430
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-client-linux-arm.tar.gz) | f8a336b48c27819f979336fff3ffa7eeb5512330f3eafe7a3b85ea65a4d94b213ed20d4d35d6fe3f92cb557037a051023eb47fd6e6dcaf3e0a6fe88a5c6cd632
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-client-linux-arm64.tar.gz) | a72602ac48b13c6a97883c34170fd64095539d4f9a3900367ff628a195aa931c27d7c9582f864c669332bbc58b4883d5e41bc65d5ac83337bdf7066e538deceb
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-client-linux-ppc64le.tar.gz) | 002b2e685758ad6fa2a18d7706a335249f55a786a4315d3f2cab8e34d38a01302af91063742729de664c7ab06bd656b388166f82010af36e43e931e8ddd93752
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-client-linux-s390x.tar.gz) | 21da21e1f7ba24b6967b5e22abb62e1c1691cd7cc15eb5ecd9777fa51d788a7a132f31c04306d3a59e5cee96bb58b9d1838630de1bcbf168cabe8f4afb514501
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-client-windows-386.tar.gz) | 21494c5fe65e6a9aaf2f7f11996219155ed85a4f54d048b64df05de1adbd925af40ee51d4119801333143364902b9805cefceafac8d407f62eef1e7f07b686ee
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-client-windows-amd64.tar.gz) | eaaddeac2e0a69a618f606574044eec8b41f4c3d4f6cf0045e4456ad57d44c865d1f183b6e0929f6913e28febe67b178662dfce3e40395c6d97180985b4fb48d
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-client-windows-arm64.tar.gz) | dad6a73bf2530c0c2f58b8e77956ec444b6795c9882b0f2b960998fbd9e22720fc6fff114af3b0ad10655e9e1d627f70bc6f67fdd388d0e995aeb9bd4bf9bea2
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-server-linux-amd64.tar.gz) | c64651213144ef4696fa11da0ec93c6fd7540798bfc28df8e69ee8bdb35dbc7114ee043cd38dc86c75a3dbff5e45ed4474be22ce74b8c4b3206030a10cd20f8d
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-server-linux-arm64.tar.gz) | 0724ce02d551d72f39c7ac6b29c78dcaeae7878126b33cde7a949d0b9be0b35b3977f5494ab48f02a382bf83a70a8ad035f4962b0644a6fedc084068b525ddf9
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-server-linux-ppc64le.tar.gz) | 7f527bb02e046308b2720a99d8f6ac13e1daee23b44e77603b75aa5569a9b4baf29a7b19f3076219ff94f296ce8316fdaadb9cabaf1c58173a7e3719e94f3917
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-server-linux-s390x.tar.gz) | 0285e04f2834bdbb66b46193f54724e6f9264ff992b10dbaa3694abbab297f5e1f4e95ade14f7dcb41f856d9e3a292f1af16f3ed59e2b02961451973d4972f1d
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-node-linux-amd64.tar.gz) | b4c5e4a0e818eb9f88128e2a051591b4955a858e400489d04b75cdfb68eb3a7d004ced839c2916bf5ca885d7ae496fb68c0620b2c3352cb5435c435756b0a70a
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-node-linux-arm64.tar.gz) | e525decd637860b9621ec7ec8c42913c419bb81577a1a359e752e7628507b6e9b1a82889ef0ba17ca975aa8630edba12a87d38b75ea3f9e213493873036b92c8
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-node-linux-ppc64le.tar.gz) | 4282286b775a5bdaab753c911fa0f351476d89070a34569bedb104cb2c56a408d125d44c4895cc28fcb8cd5c12585f3cbecccfe045880b546766202113a703b1
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-node-linux-s390x.tar.gz) | af88dac8622e10e336e5d79f9d4511de3eceed384da210dda83223c7b6582133acaa7d8f6b361cf213a4ca3ea51379bd828816c991ed7b4c62cf6fd9830f0c30
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0-rc.0/kubernetes-node-windows-amd64.tar.gz) | 7b322df6a7e9e0b0b881f99d7ef76b3eda0f856345eb888efa62daf1f3638f88a630fc40626800075db90215c68a956fec7ce274381e06a173d494e4b03b4f49
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.0-rc.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.0-rc.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.0-rc.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.0-rc.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.0-rc.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.0-rc.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-s390x)
## Changelog since v1.29.0-alpha.3
## Changes by Kind
### API Change
- Added support for projecting certificates.k8s.io/v1alpha1 ClusterTrustBundle objects into pods. ([#113374](https://github.com/kubernetes/kubernetes/pull/113374), [@ahmedtd](https://github.com/ahmedtd)) [SIG API Machinery, Apps, Auth, Node, Storage and Testing]
- Adds `optionalOldSelf` to `x-kubernetes-validations` to support ratcheting CRD schema constraints ([#121034](https://github.com/kubernetes/kubernetes/pull/121034), [@alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery]
- Fix API comment for the Job Ready field in status ([#121765](https://github.com/kubernetes/kubernetes/pull/121765), [@mimowo](https://github.com/mimowo)) [SIG API Machinery and Apps]
- Fix API comments for the FailIndex Job pod failure policy action. ([#121764](https://github.com/kubernetes/kubernetes/pull/121764), [@mimowo](https://github.com/mimowo)) [SIG API Machinery and Apps]
### Feature
- A customizable OrderedScoreFuncs() function is introduced. Out-of-tree plugins that use scheduler's preemption interface can implement this function for custom preemption preferences, or return nil to keep current behavior. ([#121867](https://github.com/kubernetes/kubernetes/pull/121867), [@lianghao208](https://github.com/lianghao208)) [SIG Scheduling]
- Bump distroless-iptables to 0.4.1 based on Go 1.21.3 ([#121871](https://github.com/kubernetes/kubernetes/pull/121871), [@cpanato](https://github.com/cpanato)) [SIG Testing]
- Fix overriding default KubeletConfig fields in drop-in configs if not set ([#121193](https://github.com/kubernetes/kubernetes/pull/121193), [@sohankunkerkar](https://github.com/sohankunkerkar)) [SIG Node and Testing]
- KEP-4191- add support for split image filesystem in kubelet ([#120616](https://github.com/kubernetes/kubernetes/pull/120616), [@kannon92](https://github.com/kannon92)) [SIG Node and Testing]
- Kubeadm: support updating certificate organization during 'kubeadm certs renew' ([#121841](https://github.com/kubernetes/kubernetes/pull/121841), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle]
- Kubernetes is now built with Go 1.21.4 ([#121808](https://github.com/kubernetes/kubernetes/pull/121808), [@cpanato](https://github.com/cpanato)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Storage and Testing]
### Bug or Regression
- Fix: statle smb mount issue when smb file share is deleted and then unmount ([#121851](https://github.com/kubernetes/kubernetes/pull/121851), [@andyzhangx](https://github.com/andyzhangx)) [SIG Storage]
- KCCM: fix transient node addition + removal caused by #121090 while syncing load balancers on large clusters with a lot of churn ([#121091](https://github.com/kubernetes/kubernetes/pull/121091), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) [SIG Cloud Provider, Network and Testing]
- Kubeadm: change the "system:masters" Group in the apiserver-kubelet-client.crt certificate Subject to be "kubeadm:cluster-admins" which is a less privileged Group. ([#121837](https://github.com/kubernetes/kubernetes/pull/121837), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle]
- Scheduler: in 1.29 pre-releases, enabling contextual logging slowed down pod scheduling. ([#121715](https://github.com/kubernetes/kubernetes/pull/121715), [@pohly](https://github.com/pohly)) [SIG Instrumentation and Scheduling]
### Other (Cleanup or Flake)
- Update runc to 1.1.10 ([#121739](https://github.com/kubernetes/kubernetes/pull/121739), [@ty-dc](https://github.com/ty-dc)) [SIG Architecture and Node]
## Dependencies
### Added
_Nothing has changed._
### Changed
- github.com/mrunalp/fileutils: [v0.5.0 → v0.5.1](https://github.com/mrunalp/fileutils/compare/v0.5.0...v0.5.1)
- github.com/opencontainers/runc: [v1.1.9 → v1.1.10](https://github.com/opencontainers/runc/compare/v1.1.9...v1.1.10)
### Removed
_Nothing has changed._
# v1.29.0-alpha.3
## Downloads for v1.29.0-alpha.3
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes.tar.gz) | 998a680aee880601d65c14cf43a8ace13aacb3d693ac2f32c40ddc5c0a567fd4cc5627f397bf5612ed83d6b37ef568260f2700d46592bbc74174e155bf8f0606
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-src.tar.gz) | ca46836dabd989a8dc6ee61032ab7f73747a5e2ef3bc11437e4036d95cbfbb9574f647b1672a098625729b62f1ff663726fcaf2dc3ea472e7b27d6b373d8afa9
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-client-darwin-amd64.tar.gz) | b82008b54b2a90e3640e786782cc20cf3a7d6a5011974f6710d418770541b53edb7d9d4ccd9489d4d81fcf7df7db38a3766db19898c86381b6fcfd7b261bc06a
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-client-darwin-arm64.tar.gz) | b389eece6ea7ba07fdff76a6acdf36e77ed81e474277b62ef40b91ccc0d00c37f6f7c1194cacb14df844b1ea4dc66895b1c73bfedba570c71b72c6ab9a697861
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-client-linux-386.tar.gz) | 5fa044082a1d2fb9d0b428fd2ba913196b4891f4c0571a7061ef1b6fee19ff820ff2b67506edad27fe0ddc735630d7398f66160836620188a527a8f3dbeb6b09
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-client-linux-amd64.tar.gz) | 782d262f696e9b706de195870e5589fe3a0c4c11698574709668d4f60fcfe3cfb0137e86fb2d43a27a297bf88c29552216d30ac72255b4a757525f0b7e2385a1
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-client-linux-arm.tar.gz) | cd9038cd3fa938aac9a0b462f7c6822d031f4e05e2529df378b4069f2d69d362236e5fc6e464d20cc42549f84f283f302b6cb33eb4a128ab91dcaa1cf04552e8
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-client-linux-arm64.tar.gz) | d0fdf61def1be6c3b9e5259c13e8dbd764af44ed3dcdeb83c6a7d6cfe87b2293cbd88ceaad0aac87b448fe4766635b6b9eca40bc1a302f717d1bc0e26dac60ed
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-client-linux-ppc64le.tar.gz) | c8b148404eecdff20939f0bec92024be58cb9629802c3768085834998e97b82e87f37443a867dfbb73e9922aada038d308ef02ec52a078aefb1f76360220c77b
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-client-linux-s390x.tar.gz) | fb7070ef9d610fae614eadf9ec7fcfe68958143010b709586f0339309336e87f06d03ff8df5108b340ea063082e7b1d393b519ee6a4b4ed302427fd66e896295
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-client-windows-386.tar.gz) | c0021a7668504a0a2be408cb2d1754bd20fc9afeeeb31dfb11f11787eb7047540c544888058300a83662dab845c726c7001562ee712b4c1d485ff0c3a88827f9
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-client-windows-amd64.tar.gz) | 5678ab6523345ec38ba49a81c945223112228e75b82d0b459f0c9d6c37d3a0c93af4b82f05226e2d1110c1315ae5ed7ed3ad0bb085afad7f376b9715fe8fee75
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-client-windows-arm64.tar.gz) | 2d7ac1add995683b29396fbbc06b15bf81ca62338ba0ed6d4738283752560fa167d6b9ffc44aeb4ed9b1bae6bc2ed8fe1d6346436c34a2acd3a5467ac68041bb
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-server-linux-amd64.tar.gz) | a948e26c77fb7cef3c50543c0b92c1ec1085516c4f16f9cd6695a02e1d88e44d5cd1f6fcc13bda48a580a26d024d1665ca2a960b37b31f0a7f0186e941a21e7e
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-server-linux-arm64.tar.gz) | 14538ca02dc149a57c1cb30206b281248f9a84b024dad777e0326a9c6dc6c74228211e1a49f0480e2b7f825b12891d176489bcafcfab0fa05b18acc33c5044f8
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-server-linux-ppc64le.tar.gz) | d347d6072f5a4c6c14ddb9418eadcc075824fa2dd15e49bfb79ee3fab7b2cc0efdd18021d7baed9024a4b83b4f9b800cedaeb8fa3917bfd47c4e5935146fc9fc
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-server-linux-s390x.tar.gz) | 210ed1f933ba611cc3a828382ad15b02d2e35e74e99baed7077c21708249c5a561963e25f2582562773f1ea8f3eccb89b9fa25ca58da6eec9a516652efd432a5
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-node-linux-amd64.tar.gz) | 6d9b9e382a3137a2622a4631162b5c6a0c0c709fe95b76a7d5af610aec2a292b2f5a0b3378ddf8243450d774f6c1cd2ca16cbf240aed7109d819cd366b7abda9
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-node-linux-arm64.tar.gz) | 0951c701155c914a0578dab9c8d584d32e8260ca923e1efccd0739db2065bf3d37d5d1b6584bc38f7873e20d164e57603e79abfecfbbe89e5386b6d7738d521b
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-node-linux-ppc64le.tar.gz) | afbbedb58bd8344608e1fe047666914874419aef7f31c057a992e0dc24acae6151a7b0c53c2cfc8144ab8e0e914ee8b3a2f11adbb3791fb3b412172ade67439d
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-node-linux-s390x.tar.gz) | 10c73d669dde0841078e5cee9158fa1a551c8bfe668d07beadf316386f815979f8729b89a0bed7e9e76350e82f6fd94d204187b9c3fe6e2bc1aabb2e580fee87
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.3/kubernetes-node-windows-amd64.tar.gz) | c94d9f4979aeebfae9e66e029ea99ef6e349209f641d853032f87bb9cb646e885b995a512be3eef8e8cf2def76418e70086a03c00121e22c082b67b45562a6c5
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.0-alpha.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.0-alpha.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.0-alpha.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.0-alpha.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.0-alpha.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.0-alpha.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-s390x)
## Changelog since v1.29.0-alpha.2
## Urgent Upgrade Notes
### (No, really, you MUST read this before you upgrade)
- Kubeadm: deploy a separate "super-admin.conf" file. The User in "admin.conf" is now bound to a new RBAC Group "kubeadm:cluster-admins" that have "cluster-admin" ClusterRole access. The User in "super-admin.conf" is bound to the "system:masters" built-in super-powers / break-glass Group that can bypass RBAC. Before this change the default "admin.conf" was bound to "system:masters" Group which was undesired. Executing "kubeadm init phase kubeconfig all" or just "kubeadm init" will now generate the new "super-admin.conf" file. The cluster admin can then decide to keep the file present on a node host or move it to a safe location. "kubadm certs renew" will renew the certificate in "super-admin.conf" to one year if the file exists. If it does not exist a "MISSING" note will be printed. "kubeadm upgrade apply" for this release will migrate this particular node to the two file setup. Subsequent kubeadm releases will continue to optionally renew the certificate in "super-admin.conf" if the file exists on disk and if renew on upgrade is not disabled. "kubeadm join --control-plane" will now generate only an "admin.conf" file that has the less privileged User. ([#121305](https://github.com/kubernetes/kubernetes/pull/121305), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle]
- Stop accepting component configuration for kube-proxy and kubelet during `kubeadm upgrade plan --config`. This is a legacy behavior that is not well supported for upgrades and can be used only at the plan stage to determine if the configuration for these components stored in the cluster needs manual version migration. In the future, kubeadm will attempt alternative component config migration approaches. ([#120788](https://github.com/kubernetes/kubernetes/pull/120788), [@chendave](https://github.com/chendave)) [SIG Cluster Lifecycle]
## Changes by Kind
### Deprecation
- Creation of new CronJob objects containing `TZ` or `CRON_TZ` in `.spec.schedule`, accidentally enabled in 1.22, is now disallowed. Use the `.spec.timeZone` field instead, supported in 1.25+ clusters in default configurations. See https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#unsupported-timezone-specification for more information. ([#116252](https://github.com/kubernetes/kubernetes/pull/116252), [@soltysh](https://github.com/soltysh)) [SIG Apps]
- Remove the networking alpha API ClusterCIDR ([#121229](https://github.com/kubernetes/kubernetes/pull/121229), [@aojea](https://github.com/aojea)) [SIG Apps, CLI, Cloud Provider, Network and Testing]
### API Change
- A new sleep action for the PreStop lifecycle hook is added, allowing containers to pause for a specified duration before termination. ([#119026](https://github.com/kubernetes/kubernetes/pull/119026), [@AxeZhan](https://github.com/AxeZhan)) [SIG API Machinery, Apps, Node and Testing]
- Add ImageMaximumGCAge field to Kubelet configuration, which allows a user to set the maximum age an image is unused before it's garbage collected. ([#121275](https://github.com/kubernetes/kubernetes/pull/121275), [@haircommander](https://github.com/haircommander)) [SIG API Machinery and Node]
- Add a new ServiceCIDR type that allows to dynamically configure the cluster range used to allocate Service ClusterIPs addresses ([#116516](https://github.com/kubernetes/kubernetes/pull/116516), [@aojea](https://github.com/aojea)) [SIG API Machinery, Apps, Auth, CLI, Network and Testing]
- Add the DisableNodeKubeProxyVersion feature gate. If DisableNodeKubeProxyVersion is enabled, the kubeProxyVersion field is not set. ([#120954](https://github.com/kubernetes/kubernetes/pull/120954), [@HirazawaUi](https://github.com/HirazawaUi)) [SIG API Machinery, Apps and Node]
- Added Windows support for InPlace Pod Vertical Scaling feature. ([#112599](https://github.com/kubernetes/kubernetes/pull/112599), [@fabi200123](https://github.com/fabi200123)) [SIG Autoscaling, Node, Scalability, Scheduling and Windows]
- Added `UserNamespacesPodSecurityStandards` feature gate to enable user namespace support for Pod Security Standards.
Enabling this feature will modify all Pod Security Standard rules to allow setting: `spec[.*].securityContext.[runAsNonRoot,runAsUser]`.
This feature gate should only be enabled if all nodes in the cluster support the user namespace feature and have it enabled.
The feature gate will not graduate or be enabled by default in future Kubernetes releases. ([#118760](https://github.com/kubernetes/kubernetes/pull/118760), [@saschagrunert](https://github.com/saschagrunert)) [SIG API Machinery, Auth, Node and Release]
- Added options for configuring nf_conntrack_udp_timeout, and nf_conntrack_udp_timeout_stream variables of netfilter conntrack subsystem. ([#120808](https://github.com/kubernetes/kubernetes/pull/120808), [@aroradaman](https://github.com/aroradaman)) [SIG API Machinery and Network]
- Adds CEL expressions to v1alpha1 AuthenticationConfiguration. ([#121078](https://github.com/kubernetes/kubernetes/pull/121078), [@aramase](https://github.com/aramase)) [SIG API Machinery, Auth and Testing]
- Adds support for CEL expressions to v1alpha1 AuthorizationConfiguration webhook matchConditions. ([#121223](https://github.com/kubernetes/kubernetes/pull/121223), [@ritazh](https://github.com/ritazh)) [SIG API Machinery and Auth]
- CSINodeExpandSecret feature has been promoted to GA in this release and enabled by default. The CSI drivers can make use of the `secretRef` values passed in NodeExpansion request optionally sent by the CSI Client from this release onwards. ([#121303](https://github.com/kubernetes/kubernetes/pull/121303), [@humblec](https://github.com/humblec)) [SIG API Machinery, Apps and Storage]
- Graduate Job BackoffLimitPerIndex feature to Beta ([#121356](https://github.com/kubernetes/kubernetes/pull/121356), [@mimowo](https://github.com/mimowo)) [SIG Apps]
- Kube-apiserver: adds --authorization-config flag for reading a configuration file containing an apiserver.config.k8s.io/v1alpha1 AuthorizationConfiguration object. --authorization-config flag is mutually exclusive with --authorization-modes and --authorization-webhook-* flags. The alpha StructuredAuthorizationConfiguration feature flag must be enabled for --authorization-config to be specified. ([#120154](https://github.com/kubernetes/kubernetes/pull/120154), [@palnabarun](https://github.com/palnabarun)) [SIG API Machinery, Auth and Testing]
- Kube-proxy now has a new nftables-based mode, available by running
kube-proxy --feature-gates NFTablesProxyMode=true --proxy-mode nftables
This is currently an alpha-level feature and while it probably will not
eat your data, it may nibble at it a bit. (It passes e2e testing but has
not yet seen real-world use.)
At this point it should be functionally mostly identical to the iptables
mode, except that it does not (and will not) support Service NodePorts on
127.0.0.1. (Also note that there are currently no command-line arguments
for the nftables-specific config; you will need to use a config file if
you want to set the equivalent of any of the `--iptables-xxx` options.)
As this code is still very new, it has not been heavily optimized yet;
while it is expected to _eventually_ have better performance than the
iptables backend, very little performance testing has been done so far. ([#121046](https://github.com/kubernetes/kubernetes/pull/121046), [@danwinship](https://github.com/danwinship)) [SIG API Machinery and Network]
- Kube-proxy: Added an option/flag for configuring the `nf_conntrack_tcp_be_liberal` sysctl (in the kernel's netfilter conntrack subsystem). When enabled, kube-proxy will not install the DROP rule for invalid conntrack states, which currently breaks users of asymmetric routing. ([#120354](https://github.com/kubernetes/kubernetes/pull/120354), [@aroradaman](https://github.com/aroradaman)) [SIG API Machinery and Network]
- PersistentVolumeLastPhaseTransitionTime is now beta, enabled by default. ([#120627](https://github.com/kubernetes/kubernetes/pull/120627), [@RomanBednar](https://github.com/RomanBednar)) [SIG Storage]
- Promote PodReadyToStartContainers condition to beta. ([#119659](https://github.com/kubernetes/kubernetes/pull/119659), [@kannon92](https://github.com/kannon92)) [SIG Node and Testing]
- The flowcontrol.apiserver.k8s.io/v1beta3 FlowSchema and PriorityLevelConfiguration APIs has been promoted to flowcontrol.apiserver.k8s.io/v1, with the following changes:
- PriorityLevelConfiguration: the `.spec.limited.nominalConcurrencyShares` field defaults to `30` only if the field is omitted (v1beta3 also defaulted an explicit `0` value to `30`). Specifying an explicit `0` value is not allowed in the `v1` version in v1.29 to ensure compatibility with 1.28 API servers. In v1.30, explicit `0` values will be allowed in this field in the `v1` API.
The flowcontrol.apiserver.k8s.io/v1beta3 APIs are deprecated and will no longer be served in v1.32. All existing objects are available via the `v1` APIs. Transition clients and manifests to use the `v1` APIs before upgrading to v1.32. ([#121089](https://github.com/kubernetes/kubernetes/pull/121089), [@tkashem](https://github.com/tkashem)) [SIG API Machinery and Testing]
- The kube-proxy command-line documentation was updated to clarify that
`--bind-address` does not actually have anything to do with binding to an
address, and you probably don't actually want to be using it. ([#120274](https://github.com/kubernetes/kubernetes/pull/120274), [@danwinship](https://github.com/danwinship)) [SIG Network]
- The matchLabelKeys/mismatchLabelKeys feature is introduced to the hard/soft PodAffinity/PodAntiAffinity. ([#116065](https://github.com/kubernetes/kubernetes/pull/116065), [@sanposhiho](https://github.com/sanposhiho)) [SIG API Machinery, Apps, Cloud Provider, Scheduling and Testing]
- ValidatingAdmissionPolicy Type Checking now supports CRDs and API extensions types. ([#119109](https://github.com/kubernetes/kubernetes/pull/119109), [@jiahuif](https://github.com/jiahuif)) [SIG API Machinery, Apps, Auth and Testing]
- When updating a CRD, per-expression cost limit check is skipped for x-kubernetes-validations rules of versions that are not mutated. ([#121460](https://github.com/kubernetes/kubernetes/pull/121460), [@jiahuif](https://github.com/jiahuif)) [SIG API Machinery]
### Feature
- #### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:
<!--
This section can be blank if this pull request does not require a release note.
When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.
See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files
Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
--> ([#119517](https://github.com/kubernetes/kubernetes/pull/119517), [@sanposhiho](https://github.com/sanposhiho)) [SIG Node, Scheduling and Testing]
- --interactive flag in kubectl delete will be visible to all users by default. ([#120416](https://github.com/kubernetes/kubernetes/pull/120416), [@ardaguclu](https://github.com/ardaguclu)) [SIG CLI and Testing]
- Add container filesystem to the ImageFsInfoResponse. ([#120914](https://github.com/kubernetes/kubernetes/pull/120914), [@kannon92](https://github.com/kannon92)) [SIG Node and Testing]
- Add job_pods_creation_total metrics for tracking Pods created by the Job controller labeled by events which triggered the Pod creation ([#121481](https://github.com/kubernetes/kubernetes/pull/121481), [@dejanzele](https://github.com/dejanzele)) [SIG Apps and Testing]
- Add multiplication functionality to Quantity. ([#117411](https://github.com/kubernetes/kubernetes/pull/117411), [@tenzen-y](https://github.com/tenzen-y)) [SIG API Machinery]
- Added a new `--init-only` command line flag to `kube-proxy`. Setting the flag makes `kube-proxy` perform its initial configuration that requires privileged mode, and then exit. The `--init-only` mode is intended to be executed in a privileged init container, so that the main container may run with a stricter `securityContext`. ([#120864](https://github.com/kubernetes/kubernetes/pull/120864), [@uablrek](https://github.com/uablrek)) [SIG Network and Scalability]
- Added new feature gate called "RuntimeClassInImageCriApi" to address kubelet changes needed for KEP 4216.
Noteable changes:
1. Populate new RuntimeHandler field in CRI's ImageSpec struct during image pulls from container runtimes.
2. Pass runtimeHandler field in RemoveImage() call to container runtime in kubelet's image garbage collection ([#121456](https://github.com/kubernetes/kubernetes/pull/121456), [@kiashok](https://github.com/kiashok)) [SIG Node and Windows]
- Adds `apiextensions_apiserver_update_ratcheting_time` metric for tracking time taken during requests by feature `CRDValidationRatcheting` ([#121462](https://github.com/kubernetes/kubernetes/pull/121462), [@alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery]
- Bump cel-go to v0.17.7 and introduce set ext library with new options. ([#121577](https://github.com/kubernetes/kubernetes/pull/121577), [@cici37](https://github.com/cici37)) [SIG API Machinery, Auth and Cloud Provider]
- Bump distroless-iptables to 0.4.1 based on Go 1.21.3 ([#121216](https://github.com/kubernetes/kubernetes/pull/121216), [@cpanato](https://github.com/cpanato)) [SIG Testing]
- CEL can now correctly handle a CRD openAPIV3Schema that has neither Properties nor AdditionalProperties. ([#121459](https://github.com/kubernetes/kubernetes/pull/121459), [@jiahuif](https://github.com/jiahuif)) [SIG API Machinery and Testing]
- CEL cost estimator no longer treats enums as unbounded strings when determining its length. Instead, the length is set to the longest possible enum value. ([#121085](https://github.com/kubernetes/kubernetes/pull/121085), [@jiahuif](https://github.com/jiahuif)) [SIG API Machinery]
- CRDValidationRatcheting: Adds support for ratcheting `x-kubernetes-validations` in schema ([#121016](https://github.com/kubernetes/kubernetes/pull/121016), [@alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery]
- CRI: support image pull per runtime class ([#121121](https://github.com/kubernetes/kubernetes/pull/121121), [@kiashok](https://github.com/kiashok)) [SIG Node and Windows]
- Calculate restartable init containers resource in pod autoscaler ([#120001](https://github.com/kubernetes/kubernetes/pull/120001), [@qingwave](https://github.com/qingwave)) [SIG Apps and Autoscaling]
- Certain requestBody params in the OpenAPI v3 are correctly marked as required ([#120735](https://github.com/kubernetes/kubernetes/pull/120735), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node and Storage]
- Client-side apply will use OpenAPI V3 by default ([#120707](https://github.com/kubernetes/kubernetes/pull/120707), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery and CLI]
- Cluster/gce: add webhook to replace PersistentVolumeLabel admission controller ([#121628](https://github.com/kubernetes/kubernetes/pull/121628), [@andrewsykim](https://github.com/andrewsykim)) [SIG Cloud Provider]
- Decouple TaintManager from NodeLifeCycleController (KEP-3902) ([#119208](https://github.com/kubernetes/kubernetes/pull/119208), [@atosatto](https://github.com/atosatto)) [SIG API Machinery, Apps, Instrumentation, Node, Scheduling and Testing]
- DevicePluginCDIDevices feature has been graduated to Beta and enabled by default in the Kubelet ([#121254](https://github.com/kubernetes/kubernetes/pull/121254), [@bart0sh](https://github.com/bart0sh)) [SIG Node]
- Dra: the scheduler plugin avoids additional scheduling attempts in some cases by falling back to SSA after a conflict ([#120534](https://github.com/kubernetes/kubernetes/pull/120534), [@pohly](https://github.com/pohly)) [SIG Node, Scheduling and Testing]
- Enable traces for KMSv2 encrypt/decrypt operations. ([#121095](https://github.com/kubernetes/kubernetes/pull/121095), [@aramase](https://github.com/aramase)) [SIG API Machinery, Architecture, Auth, Instrumentation and Testing]
- Etcd: build image for v3.5.9 ([#121567](https://github.com/kubernetes/kubernetes/pull/121567), [@mzaian](https://github.com/mzaian)) [SIG API Machinery]
- Fixes bugs in handling of server-side apply, create, and update API requests for objects containing duplicate items in keyed lists.
- A `create` or `update` API request with duplicate items in a keyed list no longer wipes out managedFields. Examples include env var entries with the same name, or port entries with the same containerPort in a pod spec.
- A server-side apply request that makes unrelated changes to an object which has duplicate items in a keyed list no longer fails, and leaves the existing duplicate items as-is.
- A server-side apply request that changes an object which has duplicate items in a keyed list, and modifies the duplicated item removes the duplicates and replaces them with the single item contained in the server-side apply request. ([#121575](https://github.com/kubernetes/kubernetes/pull/121575), [@apelisse](https://github.com/apelisse)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Storage and Testing]
- Graduate the `ReadWriteOncePod` feature gate to GA ([#121077](https://github.com/kubernetes/kubernetes/pull/121077), [@chrishenzie](https://github.com/chrishenzie)) [SIG Apps, Node, Scheduling, Storage and Testing]
- Introduce the job_finished_indexes_total metric for BackoffLimitPerIndex feature ([#121292](https://github.com/kubernetes/kubernetes/pull/121292), [@mimowo](https://github.com/mimowo)) [SIG Apps and Testing]
- KEP-4191- add support for split image filesystem in kubelet ([#120616](https://github.com/kubernetes/kubernetes/pull/120616), [@kannon92](https://github.com/kannon92)) [SIG Node and Testing]
- Kube-apiserver adds alpha support (guarded by the ServiceAccountTokenJTI feature gate) for adding a `jti` (JWT ID) claim to service account tokens it issues, adding an `authentication.kubernetes.io/credential-id` audit annotation in audit logs when the tokens are issued, and `authentication.kubernetes.io/credential-id` entry in the extra user info when the token is used to authenticate.
- kube-apiserver adds alpha support (guarded by the ServiceAccountTokenPodNodeInfo feature gate) for including the node name (and uid, if the node exists) as additional claims in service account tokens it issues which are bound to pods, and `authentication.kubernetes.io/node-name` and `authentication.kubernetes.io/node-uid` extra user info when the token is used to authenticate.
- kube-apiserver adds alpha support (guarded by the ServiceAccountTokenNodeBinding feature gate) for allowing TokenRequests that bind tokens directly to nodes, and (guarded by the ServiceAccountTokenNodeBindingValidation feature gate) for validating the node name and uid still exist when the token is used. ([#120780](https://github.com/kubernetes/kubernetes/pull/120780), [@munnerz](https://github.com/munnerz)) [SIG API Machinery, Apps, Auth, CLI and Testing]
- Kube-controller-manager: The `LegacyServiceAccountTokenCleanUp` feature gate is now beta and enabled by default. When enabled, legacy auto-generated service account token secrets are auto-labeled with a `kubernetes.io/legacy-token-invalid-since` label if the credentials have not been used in the time specified by `--legacy-service-account-token-clean-up-period` (defaulting to one year), **and** are referenced from the `.secrets` list of a ServiceAccount object, **and** are not referenced from pods. This label causes the authentication layer to reject use of the credentials. After being labeled as invalid, if the time specified by `--legacy-service-account-token-clean-up-period` (defaulting to one year) passes without the credential being used, the secret is automatically deleted. Secrets labeled as invalid which have not been auto-deleted yet can be re-activated by removing the `kubernetes.io/legacy-token-invalid-since` label. ([#120682](https://github.com/kubernetes/kubernetes/pull/120682), [@yt2985](https://github.com/yt2985)) [SIG Apps, Auth and Testing]
- Kube-scheduler implements scheduling hints for the NodeAffinity plugin.
The scheduling hints allow the scheduler to only retry scheduling a Pod
that was previously rejected by the NodeAffinity plugin if a new Node or a Node update matches the Pod's node affinity. ([#119155](https://github.com/kubernetes/kubernetes/pull/119155), [@carlory](https://github.com/carlory)) [SIG Scheduling]
- Kubeadm: Turn on FeatureGate `MergeCLIArgumentsWithConfig` to merge the config from flag and config file, otherwise, If the flag `--ignore-preflight-errors` is set from CLI, then the value from config file will be ignored. ([#119946](https://github.com/kubernetes/kubernetes/pull/119946), [@chendave](https://github.com/chendave)) [SIG Cluster Lifecycle]
- Kubeadm: allow deploying a kubelet that is 3 versions older than the version of kubeadm (N-3). This aligns with the recent change made by SIG Architecture that extends the support skew between the control plane and kubelets. Tolerate this new kubelet skew for the commands "init", "join" and "upgrade". Note that if the kubeadm user applies a control plane version that is older than the kubeadm version (N-1 maximum) then the skew between the kubelet and control plane would become a maximum of N-2. ([#120825](https://github.com/kubernetes/kubernetes/pull/120825), [@pacoxu](https://github.com/pacoxu)) [SIG Cluster Lifecycle]
- Kubelet allows pods to use the `net.ipv4.tcp_fin_timeout` , “net.ipv4.tcp_keepalive_intvl” and “net.ipv4.tcp_keepalive_probes“ sysctl by default; Pod Security admission allows this sysctl in v1.29+ versions of the baseline and restricted policies. ([#121240](https://github.com/kubernetes/kubernetes/pull/121240), [@HirazawaUi](https://github.com/HirazawaUi)) [SIG Auth and Node]
- Kubelet allows pods to use the `net.ipv4.tcp_keepalive_time` sysctl by default and the minimal kernel version is 4.5; Pod Security admission allows this sysctl in v1.29+ versions of the baseline and restricted policies. ([#118846](https://github.com/kubernetes/kubernetes/pull/118846), [@cyclinder](https://github.com/cyclinder)) [SIG Auth, Network and Node]
- Kubelet emits a metric for end-to-end pod startup latency including image pull. ([#121041](https://github.com/kubernetes/kubernetes/pull/121041), [@ruiwen-zhao](https://github.com/ruiwen-zhao)) [SIG Node]
- Kubernetes is now built with Go 1.21.3 ([#121149](https://github.com/kubernetes/kubernetes/pull/121149), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing]
- Make decoding etcd's response respect the timeout context. ([#121614](https://github.com/kubernetes/kubernetes/pull/121614), [@HirazawaUi](https://github.com/HirazawaUi)) [SIG API Machinery]
- Priority and Fairness feature is stable in 1.29, the feature gate will be removed in 1.31 ([#121638](https://github.com/kubernetes/kubernetes/pull/121638), [@tkashem](https://github.com/tkashem)) [SIG API Machinery and Testing]
- Promote PodHostIPs condition to beta. ([#120257](https://github.com/kubernetes/kubernetes/pull/120257), [@wzshiming](https://github.com/wzshiming)) [SIG Network, Node and Testing]
- Promote PodHostIPs condition to beta. ([#121477](https://github.com/kubernetes/kubernetes/pull/121477), [@wzshiming](https://github.com/wzshiming)) [SIG Network and Testing]
- Promote PodReplacementPolicy to beta. ([#121491](https://github.com/kubernetes/kubernetes/pull/121491), [@dejanzele](https://github.com/dejanzele)) [SIG Apps and Testing]
- Promotes plugin subcommand resolution feature to beta ([#120663](https://github.com/kubernetes/kubernetes/pull/120663), [@ardaguclu](https://github.com/ardaguclu)) [SIG CLI and Testing]
- Sidecar termination is now serialized and each sidecar container will receive a SIGTERM after all main containers and later starting sidecar containers have terminated. ([#120620](https://github.com/kubernetes/kubernetes/pull/120620), [@tzneal](https://github.com/tzneal)) [SIG Node and Testing]
- The CRD validation rule with feature gate `CustomResourceValidationExpressions` is promoted to GA. ([#121373](https://github.com/kubernetes/kubernetes/pull/121373), [@cici37](https://github.com/cici37)) [SIG API Machinery and Testing]
- The KMSv2 feature with feature gates `KMSv2` and `KMSv2KDF` are promoted to GA. The `KMSv1` feature gate is now disabled by default. ([#121485](https://github.com/kubernetes/kubernetes/pull/121485), [@ritazh](https://github.com/ritazh)) [SIG API Machinery, Auth and Testing]
- The SidecarContainers feature has graduated to beta and is enabled by default. ([#121579](https://github.com/kubernetes/kubernetes/pull/121579), [@gjkim42](https://github.com/gjkim42)) [SIG Node]
- Updated the generic apiserver library to produce an error if a new API server is configured with support for a data format other than JSON, YAML, or Protobuf. ([#121325](https://github.com/kubernetes/kubernetes/pull/121325), [@benluddy](https://github.com/benluddy)) [SIG API Machinery]
- ValidatingAdmissionPolicy now preserves types of composition variables, and raise type-related errors early. ([#121001](https://github.com/kubernetes/kubernetes/pull/121001), [@jiahuif](https://github.com/jiahuif)) [SIG API Machinery and Testing]
### Documentation
- When the Kubelet fails to assign CPUs to a Pod because there less available CPUs than the Pod requests, the error message changed from
"not enough cpus available to satisfy request" to "not enough cpus available to satisfy request: <num_requested> requested, only <num_available> available". ([#121059](https://github.com/kubernetes/kubernetes/pull/121059), [@matte21](https://github.com/matte21)) [SIG Node]
### Failing Test
- K8s.io/dynamic-resource-allocation: DRA drivers updating to this release are compatible with Kubernetes 1.27 and 1.28. ([#120868](https://github.com/kubernetes/kubernetes/pull/120868), [@pohly](https://github.com/pohly)) [SIG Node]
### Bug or Regression
- Add CAP_NET_RAW to netadmin debug profile and remove privileges when debugging nodes ([#118647](https://github.com/kubernetes/kubernetes/pull/118647), [@mochizuki875](https://github.com/mochizuki875)) [SIG CLI and Testing]
- Add a check: if a user attempts to create a static pod via the kubelet without specifying a name, they will get a visible validation error. ([#119522](https://github.com/kubernetes/kubernetes/pull/119522), [@YTGhost](https://github.com/YTGhost)) [SIG Node]
- Bugfix: OpenAPI spec no longer includes default of `{}` for certain fields where it did not make sense ([#120757](https://github.com/kubernetes/kubernetes/pull/120757), [@alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node and Storage]
- Changed kubelet logs from error to info for uncached partitions when using CRI stats provider ([#100448](https://github.com/kubernetes/kubernetes/pull/100448), [@saschagrunert](https://github.com/saschagrunert)) [SIG Node]
- Do not assign an empty value to the resource (CPU or memory) that not defined when stores the resources allocated to the pod in checkpoint ([#117615](https://github.com/kubernetes/kubernetes/pull/117615), [@aheng-ch](https://github.com/aheng-ch)) [SIG Node]
- Etcd: Update to v3.5.10 ([#121566](https://github.com/kubernetes/kubernetes/pull/121566), [@mzaian](https://github.com/mzaian)) [SIG API Machinery, Cloud Provider, Cluster Lifecycle, Etcd and Testing]
- Fix 121094 by re-introducing the readiness predicate for externalTrafficPolicy: Local services. ([#121116](https://github.com/kubernetes/kubernetes/pull/121116), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) [SIG Cloud Provider and Network]
- Fix panic in Job controller when podRecreationPolicy: Failed is used, and the number of terminating pods exceeds parallelism. ([#121147](https://github.com/kubernetes/kubernetes/pull/121147), [@kannon92](https://github.com/kannon92)) [SIG Apps]
- Fix systemLogQuery service name matching ([#120678](https://github.com/kubernetes/kubernetes/pull/120678), [@rothgar](https://github.com/rothgar)) [SIG Node]
- Fixed a 1.28.0 regression where kube-controller-manager can crash when StatefulSet with Parallel policy and PVC labels is scaled up. ([#121142](https://github.com/kubernetes/kubernetes/pull/121142), [@aleksandra-malinowska](https://github.com/aleksandra-malinowska)) [SIG Apps]
- Fixed a bug around restarting init containers in the right order relative to normal containers with SidecarContainers feature enabled. ([#120269](https://github.com/kubernetes/kubernetes/pull/120269), [@gjkim42](https://github.com/gjkim42)) [SIG Node and Testing]
- Fixed a bug where an API group's path was not unregistered from the API server's root paths when the group was deleted. ([#121283](https://github.com/kubernetes/kubernetes/pull/121283), [@tnqn](https://github.com/tnqn)) [SIG API Machinery and Testing]
- Fixed a bug where the CPU set allocated to an init container, with containerRestartPolicy of `Always`, were erroneously reused by a regular container. ([#119447](https://github.com/kubernetes/kubernetes/pull/119447), [@gjkim42](https://github.com/gjkim42)) [SIG Node and Testing]
- Fixed a bug where the device resources allocated to an init container, with containerRestartPolicy of `Always`, were erroneously reused by a regular container. ([#120461](https://github.com/kubernetes/kubernetes/pull/120461), [@gjkim42](https://github.com/gjkim42)) [SIG Node and Testing]
- Fixed a bug where the memory resources allocated to an init container, with containerRestartPolicy of `Always`, were erroneously reused by a regular container. ([#120715](https://github.com/kubernetes/kubernetes/pull/120715), [@gjkim42](https://github.com/gjkim42)) [SIG Node]
- Fixed a regression in default configurations, which enabled PodDisruptionConditions by default,
that prevented the control plane's pod garbage collector from deleting pods that contained duplicated field keys (env. variables with repeated keys or container ports). ([#121103](https://github.com/kubernetes/kubernetes/pull/121103), [@mimowo](https://github.com/mimowo)) [SIG Apps, Auth, Node, Scheduling and Testing]
- Fixed a regression in the Kubelet's behavior while creating a container when the `EventedPLEG` feature gate is enabled ([#120942](https://github.com/kubernetes/kubernetes/pull/120942), [@sairameshv](https://github.com/sairameshv)) [SIG Node]
- Fixed a regression since 1.27.0 in scheduler framework when running score plugins.
The `skippedScorePlugins` number might be greater than `enabledScorePlugins`,
so when initializing a slice the cap(len(skippedScorePlugins) - len(enabledScorePlugins)) is negative,
which is not allowed. ([#121632](https://github.com/kubernetes/kubernetes/pull/121632), [@kerthcet](https://github.com/kerthcet)) [SIG Scheduling]
- Fixed bug that kubelet resource metric `container_start_time_seconds` had timestamp equal to container start time. ([#120518](https://github.com/kubernetes/kubernetes/pull/120518), [@saschagrunert](https://github.com/saschagrunert)) [SIG Instrumentation, Node and Testing]
- Fixed inconsistency in the calculation of number of nodes that have an image, which affect the scoring in the ImageLocality plugin ([#116938](https://github.com/kubernetes/kubernetes/pull/116938), [@olderTaoist](https://github.com/olderTaoist)) [SIG Scheduling]
- Fixed some invalid and unimportant log calls. ([#121249](https://github.com/kubernetes/kubernetes/pull/121249), [@pohly](https://github.com/pohly)) [SIG Cloud Provider, Cluster Lifecycle and Testing]
- Fixed the bug that kubelet could't output logs after log file rotated when kubectl logs POD_NAME -f is running. ([#115702](https://github.com/kubernetes/kubernetes/pull/115702), [@xyz-li](https://github.com/xyz-li)) [SIG Node]
- Fixed the issue where pod with ordinal number lower than the rolling partitioning number was being deleted it was coming up with updated image. ([#120731](https://github.com/kubernetes/kubernetes/pull/120731), [@adilGhaffarDev](https://github.com/adilGhaffarDev)) [SIG Apps and Testing]
- Fixed tracking of terminating Pods in the Job status. The field was not updated unless there were other changes to apply ([#121342](https://github.com/kubernetes/kubernetes/pull/121342), [@dejanzele](https://github.com/dejanzele)) [SIG Apps and Testing]
- Fixes an issue where StatefulSet might not restart a pod after eviction or node failure. ([#121389](https://github.com/kubernetes/kubernetes/pull/121389), [@aleksandra-malinowska](https://github.com/aleksandra-malinowska)) [SIG Apps and Testing]
- Fixes calculating the requeue time in the cronjob controller, which results in properly handling failed/stuck jobs ([#121327](https://github.com/kubernetes/kubernetes/pull/121327), [@soltysh](https://github.com/soltysh)) [SIG Apps]
- Forbid sysctls for pod sharing the respective namespaces with the host when creating and update pod without such sysctls ([#118705](https://github.com/kubernetes/kubernetes/pull/118705), [@pacoxu](https://github.com/pacoxu)) [SIG Apps and Node]
- K8s.io/dynamic-resource-allocation/controller: ResourceClaimParameters and ResourceClassParameters validation errors were not visible on ResourceClaim, ResourceClass and Pod. ([#121065](https://github.com/kubernetes/kubernetes/pull/121065), [@byako](https://github.com/byako)) [SIG Node]
- Kube-proxy now reports its health more accurately in dual-stack clusters when there are problems with only one IP family. ([#118146](https://github.com/kubernetes/kubernetes/pull/118146), [@aroradaman](https://github.com/aroradaman)) [SIG Network and Windows]
- Metric buckets for pod_start_duration_seconds are changed to {0.5, 1, 2, 3, 4, 5, 6, 8, 10, 20, 30, 45, 60, 120, 180, 240, 300, 360, 480, 600, 900, 1200, 1800, 2700, 3600} ([#120680](https://github.com/kubernetes/kubernetes/pull/120680), [@ruiwen-zhao](https://github.com/ruiwen-zhao)) [SIG Instrumentation and Node]
- Mitigates http/2 DOS vulnerabilities for CVE-2023-44487 and CVE-2023-39325 for the API server when the client is unauthenticated. The mitigation may be disabled by setting the `UnauthenticatedHTTP2DOSMitigation` feature gate to `false` (it is enabled by default). An API server fronted by an L7 load balancer that already mitigates these http/2 attacks may choose to disable the kube-apiserver mitigation to avoid disrupting load balancer → kube-apiserver connections if http/2 requests from multiple clients share the same backend connection. An API server on a private network may opt to disable the kube-apiserver mitigation to prevent performance regressions for unauthenticated clients. Authenticated requests rely on the fix in golang.org/x/net v0.17.0 alone. https://issue.k8s.io/121197 tracks further mitigation of http/2 attacks by authenticated clients. ([#121120](https://github.com/kubernetes/kubernetes/pull/121120), [@enj](https://github.com/enj)) [SIG API Machinery]
- Registered metric `apiserver_request_body_size_bytes` to track the size distribution of requests by `resource` and `verb`. ([#120474](https://github.com/kubernetes/kubernetes/pull/120474), [@YaoC](https://github.com/YaoC)) [SIG API Machinery and Instrumentation]
- Update the CRI-O socket path, so users who configure kubelet to use a location like `/run/crio/crio.sock` don't see strange behaviour from CRI stats provider. ([#118704](https://github.com/kubernetes/kubernetes/pull/118704), [@dgl](https://github.com/dgl)) [SIG Node]
- Wait.PollUntilContextTimeout function, if immediate is true, the condition will be invoked before waiting and guarantees that the condition is invoked at least once and then wait a interval before executing again. ([#119762](https://github.com/kubernetes/kubernetes/pull/119762), [@AxeZhan](https://github.com/AxeZhan)) [SIG API Machinery]
### Other (Cleanup or Flake)
- Allow using lower and upper case feature flag value, the name has to match still ([#121441](https://github.com/kubernetes/kubernetes/pull/121441), [@soltysh](https://github.com/soltysh)) [SIG CLI]
- E2E storage tests: setting test tags like `[Slow]` via the DriverInfo.FeatureTag field is no longer supported. ([#121391](https://github.com/kubernetes/kubernetes/pull/121391), [@pohly](https://github.com/pohly)) [SIG Storage and Testing]
- EnqueueExtensions from plugins other than PreEnqueue, PreFilter, Filter, Reserve and Permit are ignored.
It reduces the number of kinds of cluster events the scheduler needs to subscribe/handle. ([#121571](https://github.com/kubernetes/kubernetes/pull/121571), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling]
- GetPodQOS(pod *core.Pod) function now returns the stored value from PodStatus.QOSClass, if set. To compute/evaluate the value of QOSClass from scratch, ComputePodQOS(pod *core.Pod) must be used. ([#119665](https://github.com/kubernetes/kubernetes/pull/119665), [@vinaykul](https://github.com/vinaykul)) [SIG API Machinery, Apps, CLI, Node, Scheduling and Testing]
- Graduate JobReadyPods to stable. The feature gate can no longer be disabled. ([#121302](https://github.com/kubernetes/kubernetes/pull/121302), [@stuton](https://github.com/stuton)) [SIG Apps and Testing]
- Kube-controller-manager's help will include controllers behind a feature gate in `--controllers` flag ([#120371](https://github.com/kubernetes/kubernetes/pull/120371), [@atiratree](https://github.com/atiratree)) [SIG API Machinery]
- Kubeadm: remove leftover ALPHA disclaimer that can be seen in the "kubeadm init phase certs" command help screen. The "certs" phase of "init" is not ALPHA. ([#121172](https://github.com/kubernetes/kubernetes/pull/121172), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle]
- Migrated the remainder of the scheduler to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#120933](https://github.com/kubernetes/kubernetes/pull/120933), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG Instrumentation, Scheduling and Testing]
- Previous versions of Kubernetes on Google Cloud required that workloads (e.g. Deployments, DaemonSets, etc.) which used PersistentDisk volumes were using them in read-only mode. This validation provided very little value at relatively host implementation cost, and will no longer be validated. If this is a problem for a specific use-case, please set the `SkipReadOnlyValidationGCE` gate to false to re-enable the validation, and file a kubernetes bug with details. ([#121083](https://github.com/kubernetes/kubernetes/pull/121083), [@thockin](https://github.com/thockin)) [SIG Apps]
- Remove GA featuregate about CSIMigrationvSphere in 1.29 ([#121291](https://github.com/kubernetes/kubernetes/pull/121291), [@bzsuni](https://github.com/bzsuni)) [SIG API Machinery, Node and Storage]
- Remove GA featuregate about ProbeTerminationGracePeriod in 1.29 ([#121257](https://github.com/kubernetes/kubernetes/pull/121257), [@bzsuni](https://github.com/bzsuni)) [SIG Node and Testing]
- Remove GA featuregate for JobTrackingWithFinalizers in 1.28 ([#119100](https://github.com/kubernetes/kubernetes/pull/119100), [@bzsuni](https://github.com/bzsuni)) [SIG Apps]
- Remove GAed feature gates OpenAPIV3 ([#121255](https://github.com/kubernetes/kubernetes/pull/121255), [@tukwila](https://github.com/tukwila)) [SIG API Machinery and Testing]
- Remove GAed feature gates SeccompDefault ([#121246](https://github.com/kubernetes/kubernetes/pull/121246), [@tukwila](https://github.com/tukwila)) [SIG Node]
- Remove GAed feature gates TopologyManager ([#121252](https://github.com/kubernetes/kubernetes/pull/121252), [@tukwila](https://github.com/tukwila)) [SIG Node]
- Removed the `CronJobTimeZone` feature gate (the feature is stable and always enabled)
- Removed the `JobMutableNodeSchedulingDirectives` feature gate (the feature is stable and always enabled)
- Removed the `LegacyServiceAccountTokenNoAutoGeneration` feature gate (the feature is stable and always enabled) ([#120192](https://github.com/kubernetes/kubernetes/pull/120192), [@SataQiu](https://github.com/SataQiu)) [SIG Apps, Auth and Scheduling]
- Removed the `DownwardAPIHugePages` feature gate (the feature is stable and always enabled) ([#120249](https://github.com/kubernetes/kubernetes/pull/120249), [@pacoxu](https://github.com/pacoxu)) [SIG Apps and Node]
- Removed the `GRPCContainerProbe` feature gate (the feature is stable and always enabled) ([#120248](https://github.com/kubernetes/kubernetes/pull/120248), [@pacoxu](https://github.com/pacoxu)) [SIG API Machinery, CLI and Node]
- Rename apiserver_request_body_sizes metric to apiserver_request_body_size_bytes ([#120503](https://github.com/kubernetes/kubernetes/pull/120503), [@dgrisonnet](https://github.com/dgrisonnet)) [SIG API Machinery]
- RetroactiveDefaultStorageClass feature gate that graduated to GA in 1.28 and was unconditionally enabled has been removed in v1.29. ([#120861](https://github.com/kubernetes/kubernetes/pull/120861), [@RomanBednar](https://github.com/RomanBednar)) [SIG Storage]
## Dependencies
### Added
- cloud.google.com/go/dataproc/v2: v2.0.1
- github.com/danwinship/knftables: [v0.0.13](https://github.com/danwinship/knftables/tree/v0.0.13)
- github.com/google/s2a-go: [v0.1.7](https://github.com/google/s2a-go/tree/v0.1.7)
- google.golang.org/genproto/googleapis/bytestream: e85fd2c
### Changed
- cloud.google.com/go/accessapproval: v1.6.0 → v1.7.1
- cloud.google.com/go/accesscontextmanager: v1.7.0 → v1.8.1
- cloud.google.com/go/aiplatform: v1.37.0 → v1.48.0
- cloud.google.com/go/analytics: v0.19.0 → v0.21.3
- cloud.google.com/go/apigateway: v1.5.0 → v1.6.1
- cloud.google.com/go/apigeeconnect: v1.5.0 → v1.6.1
- cloud.google.com/go/apigeeregistry: v0.6.0 → v0.7.1
- cloud.google.com/go/appengine: v1.7.1 → v1.8.1
- cloud.google.com/go/area120: v0.7.1 → v0.8.1
- cloud.google.com/go/artifactregistry: v1.13.0 → v1.14.1
- cloud.google.com/go/asset: v1.13.0 → v1.14.1
- cloud.google.com/go/assuredworkloads: v1.10.0 → v1.11.1
- cloud.google.com/go/automl: v1.12.0 → v1.13.1
- cloud.google.com/go/baremetalsolution: v0.5.0 → v1.1.1
- cloud.google.com/go/batch: v0.7.0 → v1.3.1
- cloud.google.com/go/beyondcorp: v0.5.0 → v1.0.0
- cloud.google.com/go/bigquery: v1.50.0 → v1.53.0
- cloud.google.com/go/billing: v1.13.0 → v1.16.0
- cloud.google.com/go/binaryauthorization: v1.5.0 → v1.6.1
- cloud.google.com/go/certificatemanager: v1.6.0 → v1.7.1
- cloud.google.com/go/channel: v1.12.0 → v1.16.0
- cloud.google.com/go/cloudbuild: v1.9.0 → v1.13.0
- cloud.google.com/go/clouddms: v1.5.0 → v1.6.1
- cloud.google.com/go/cloudtasks: v1.10.0 → v1.12.1
- cloud.google.com/go/compute: v1.19.0 → v1.23.0
- cloud.google.com/go/contactcenterinsights: v1.6.0 → v1.10.0
- cloud.google.com/go/container: v1.15.0 → v1.24.0
- cloud.google.com/go/containeranalysis: v0.9.0 → v0.10.1
- cloud.google.com/go/datacatalog: v1.13.0 → v1.16.0
- cloud.google.com/go/dataflow: v0.8.0 → v0.9.1
- cloud.google.com/go/dataform: v0.7.0 → v0.8.1
- cloud.google.com/go/datafusion: v1.6.0 → v1.7.1
- cloud.google.com/go/datalabeling: v0.7.0 → v0.8.1
- cloud.google.com/go/dataplex: v1.6.0 → v1.9.0
- cloud.google.com/go/dataqna: v0.7.0 → v0.8.1
- cloud.google.com/go/datastore: v1.11.0 → v1.13.0
- cloud.google.com/go/datastream: v1.7.0 → v1.10.0
- cloud.google.com/go/deploy: v1.8.0 → v1.13.0
- cloud.google.com/go/dialogflow: v1.32.0 → v1.40.0
- cloud.google.com/go/dlp: v1.9.0 → v1.10.1
- cloud.google.com/go/documentai: v1.18.0 → v1.22.0
- cloud.google.com/go/domains: v0.8.0 → v0.9.1
- cloud.google.com/go/edgecontainer: v1.0.0 → v1.1.1
- cloud.google.com/go/essentialcontacts: v1.5.0 → v1.6.2
- cloud.google.com/go/eventarc: v1.11.0 → v1.13.0
- cloud.google.com/go/filestore: v1.6.0 → v1.7.1
- cloud.google.com/go/firestore: v1.9.0 → v1.11.0
- cloud.google.com/go/functions: v1.13.0 → v1.15.1
- cloud.google.com/go/gkebackup: v0.4.0 → v1.3.0
- cloud.google.com/go/gkeconnect: v0.7.0 → v0.8.1
- cloud.google.com/go/gkehub: v0.12.0 → v0.14.1
- cloud.google.com/go/gkemulticloud: v0.5.0 → v1.0.0
- cloud.google.com/go/gsuiteaddons: v1.5.0 → v1.6.1
- cloud.google.com/go/iam: v0.13.0 → v1.1.1
- cloud.google.com/go/iap: v1.7.1 → v1.8.1
- cloud.google.com/go/ids: v1.3.0 → v1.4.1
- cloud.google.com/go/iot: v1.6.0 → v1.7.1
- cloud.google.com/go/kms: v1.10.1 → v1.15.0
- cloud.google.com/go/language: v1.9.0 → v1.10.1
- cloud.google.com/go/lifesciences: v0.8.0 → v0.9.1
- cloud.google.com/go/longrunning: v0.4.1 → v0.5.1
- cloud.google.com/go/managedidentities: v1.5.0 → v1.6.1
- cloud.google.com/go/maps: v0.7.0 → v1.4.0
- cloud.google.com/go/mediatranslation: v0.7.0 → v0.8.1
- cloud.google.com/go/memcache: v1.9.0 → v1.10.1
- cloud.google.com/go/metastore: v1.10.0 → v1.12.0
- cloud.google.com/go/monitoring: v1.13.0 → v1.15.1
- cloud.google.com/go/networkconnectivity: v1.11.0 → v1.12.1
- cloud.google.com/go/networkmanagement: v1.6.0 → v1.8.0
- cloud.google.com/go/networksecurity: v0.8.0 → v0.9.1
- cloud.google.com/go/notebooks: v1.8.0 → v1.9.1
- cloud.google.com/go/optimization: v1.3.1 → v1.4.1
- cloud.google.com/go/orchestration: v1.6.0 → v1.8.1
- cloud.google.com/go/orgpolicy: v1.10.0 → v1.11.1
- cloud.google.com/go/osconfig: v1.11.0 → v1.12.1
- cloud.google.com/go/oslogin: v1.9.0 → v1.10.1
- cloud.google.com/go/phishingprotection: v0.7.0 → v0.8.1
- cloud.google.com/go/policytroubleshooter: v1.6.0 → v1.8.0
- cloud.google.com/go/privatecatalog: v0.8.0 → v0.9.1
- cloud.google.com/go/pubsub: v1.30.0 → v1.33.0
- cloud.google.com/go/pubsublite: v1.7.0 → v1.8.1
- cloud.google.com/go/recaptchaenterprise/v2: v2.7.0 → v2.7.2
- cloud.google.com/go/recommendationengine: v0.7.0 → v0.8.1
- cloud.google.com/go/recommender: v1.9.0 → v1.10.1
- cloud.google.com/go/redis: v1.11.0 → v1.13.1
- cloud.google.com/go/resourcemanager: v1.7.0 → v1.9.1
- cloud.google.com/go/resourcesettings: v1.5.0 → v1.6.1
- cloud.google.com/go/retail: v1.12.0 → v1.14.1
- cloud.google.com/go/run: v0.9.0 → v1.2.0
- cloud.google.com/go/scheduler: v1.9.0 → v1.10.1
- cloud.google.com/go/secretmanager: v1.10.0 → v1.11.1
- cloud.google.com/go/security: v1.13.0 → v1.15.1
- cloud.google.com/go/securitycenter: v1.19.0 → v1.23.0
- cloud.google.com/go/servicedirectory: v1.9.0 → v1.11.0
- cloud.google.com/go/shell: v1.6.0 → v1.7.1
- cloud.google.com/go/spanner: v1.45.0 → v1.47.0
- cloud.google.com/go/speech: v1.15.0 → v1.19.0
- cloud.google.com/go/storagetransfer: v1.8.0 → v1.10.0
- cloud.google.com/go/talent: v1.5.0 → v1.6.2
- cloud.google.com/go/texttospeech: v1.6.0 → v1.7.1
- cloud.google.com/go/tpu: v1.5.0 → v1.6.1
- cloud.google.com/go/trace: v1.9.0 → v1.10.1
- cloud.google.com/go/translate: v1.7.0 → v1.8.2
- cloud.google.com/go/video: v1.15.0 → v1.19.0
- cloud.google.com/go/videointelligence: v1.10.0 → v1.11.1
- cloud.google.com/go/vision/v2: v2.7.0 → v2.7.2
- cloud.google.com/go/vmmigration: v1.6.0 → v1.7.1
- cloud.google.com/go/vmwareengine: v0.3.0 → v1.0.0
- cloud.google.com/go/vpcaccess: v1.6.0 → v1.7.1
- cloud.google.com/go/webrisk: v1.8.0 → v1.9.1
- cloud.google.com/go/websecurityscanner: v1.5.0 → v1.6.1
- cloud.google.com/go/workflows: v1.10.0 → v1.11.1
- cloud.google.com/go: v0.110.0 → v0.110.6
- github.com/alecthomas/template: [fb15b89 → a0175ee](https://github.com/alecthomas/template/compare/fb15b89...a0175ee)
- github.com/cncf/xds/go: [06c439d → e9ce688](https://github.com/cncf/xds/go/compare/06c439d...e9ce688)
- github.com/cyphar/filepath-securejoin: [v0.2.3 → v0.2.4](https://github.com/cyphar/filepath-securejoin/compare/v0.2.3...v0.2.4)
- github.com/docker/distribution: [v2.8.1+incompatible → v2.8.2+incompatible](https://github.com/docker/distribution/compare/v2.8.1...v2.8.2)
- github.com/docker/docker: [v20.10.21+incompatible → v20.10.24+incompatible](https://github.com/docker/docker/compare/v20.10.21...v20.10.24)
- github.com/envoyproxy/go-control-plane: [v0.10.3 → v0.11.1](https://github.com/envoyproxy/go-control-plane/compare/v0.10.3...v0.11.1)
- github.com/envoyproxy/protoc-gen-validate: [v0.9.1 → v1.0.2](https://github.com/envoyproxy/protoc-gen-validate/compare/v0.9.1...v1.0.2)
- github.com/fsnotify/fsnotify: [v1.6.0 → v1.7.0](https://github.com/fsnotify/fsnotify/compare/v1.6.0...v1.7.0)
- github.com/go-logr/logr: [v1.2.4 → v1.3.0](https://github.com/go-logr/logr/compare/v1.2.4...v1.3.0)
- github.com/godbus/dbus/v5: [v5.0.6 → v5.1.0](https://github.com/godbus/dbus/v5/compare/v5.0.6...v5.1.0)
- github.com/golang/glog: [v1.0.0 → v1.1.0](https://github.com/golang/glog/compare/v1.0.0...v1.1.0)
- github.com/google/cadvisor: [v0.47.3 → v0.48.1](https://github.com/google/cadvisor/compare/v0.47.3...v0.48.1)
- github.com/google/cel-go: [v0.17.6 → v0.17.7](https://github.com/google/cel-go/compare/v0.17.6...v0.17.7)
- github.com/google/go-cmp: [v0.5.9 → v0.6.0](https://github.com/google/go-cmp/compare/v0.5.9...v0.6.0)
- github.com/googleapis/gax-go/v2: [v2.7.1 → v2.11.0](https://github.com/googleapis/gax-go/v2/compare/v2.7.1...v2.11.0)
- github.com/grpc-ecosystem/grpc-gateway/v2: [v2.7.0 → v2.16.0](https://github.com/grpc-ecosystem/grpc-gateway/v2/compare/v2.7.0...v2.16.0)
- github.com/ishidawataru/sctp: [7c296d4 → 7ff4192](https://github.com/ishidawataru/sctp/compare/7c296d4...7ff4192)
- github.com/konsorten/go-windows-terminal-sequences: [v1.0.3 → v1.0.1](https://github.com/konsorten/go-windows-terminal-sequences/compare/v1.0.3...v1.0.1)
- github.com/onsi/gomega: [v1.28.0 → v1.29.0](https://github.com/onsi/gomega/compare/v1.28.0...v1.29.0)
- github.com/spf13/afero: [v1.2.2 → v1.1.2](https://github.com/spf13/afero/compare/v1.2.2...v1.1.2)
- github.com/stretchr/testify: [v1.8.2 → v1.8.4](https://github.com/stretchr/testify/compare/v1.8.2...v1.8.4)
- go.etcd.io/bbolt: v1.3.7 → v1.3.8
- go.etcd.io/etcd/api/v3: v3.5.9 → v3.5.10
- go.etcd.io/etcd/client/pkg/v3: v3.5.9 → v3.5.10
- go.etcd.io/etcd/client/v2: v2.305.9 → v2.305.10
- go.etcd.io/etcd/client/v3: v3.5.9 → v3.5.10
- go.etcd.io/etcd/pkg/v3: v3.5.9 → v3.5.10
- go.etcd.io/etcd/raft/v3: v3.5.9 → v3.5.10
- go.etcd.io/etcd/server/v3: v3.5.9 → v3.5.10
- go.opentelemetry.io/contrib/instrumentation/github.com/emicklei/go-restful/otelrestful: v0.35.0 → v0.42.0
- go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc: v0.35.0 → v0.42.0
- go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp: v0.35.1 → v0.44.0
- go.opentelemetry.io/contrib/propagators/b3: v1.10.0 → v1.17.0
- go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc: v1.10.0 → v1.19.0
- go.opentelemetry.io/otel/exporters/otlp/otlptrace: v1.10.0 → v1.19.0
- go.opentelemetry.io/otel/metric: v0.31.0 → v1.19.0
- go.opentelemetry.io/otel/sdk: v1.10.0 → v1.19.0
- go.opentelemetry.io/otel/trace: v1.10.0 → v1.19.0
- go.opentelemetry.io/otel: v1.10.0 → v1.19.0
- go.opentelemetry.io/proto/otlp: v0.19.0 → v1.0.0
- golang.org/x/crypto: v0.12.0 → v0.14.0
- golang.org/x/net: v0.14.0 → v0.17.0
- golang.org/x/oauth2: v0.8.0 → v0.10.0
- golang.org/x/sys: v0.12.0 → v0.13.0
- golang.org/x/term: v0.11.0 → v0.13.0
- golang.org/x/text: v0.12.0 → v0.13.0
- google.golang.org/api: v0.114.0 → v0.126.0
- google.golang.org/genproto/googleapis/api: dd9d682 → 23370e0
- google.golang.org/genproto/googleapis/rpc: 28d5490 → b8732ec
- google.golang.org/genproto: 0005af6 → f966b18
- google.golang.org/grpc: v1.54.0 → v1.58.3
- k8s.io/klog/v2: v2.100.1 → v2.110.1
- k8s.io/kube-openapi: d090da1 → 2dd684a
- sigs.k8s.io/structured-merge-diff/v4: v4.3.0 → v4.4.1
### Removed
- cloud.google.com/go/dataproc: v1.12.0
- cloud.google.com/go/gaming: v1.9.0
- github.com/blang/semver: [v3.5.1+incompatible](https://github.com/blang/semver/tree/v3.5.1)
- github.com/jmespath/go-jmespath/internal/testify: [v1.5.1](https://github.com/jmespath/go-jmespath/internal/testify/tree/v1.5.1)
- go.opentelemetry.io/otel/exporters/otlp/internal/retry: v1.10.0
# v1.29.0-alpha.2
## Downloads for v1.29.0-alpha.2
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes.tar.gz) | 138f47b2c53030e171d368d382c911048ce5d8387450e5e6717f09ac8cf6289b6c878046912130d58d7814509bbc45dbc19d6ee4f24404321ea18b24ebab2a36
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-src.tar.gz) | 73ab06309d6f6cbcb8a417c068367b670a04dcbe90574a7906201dd70b9c322cd052818114b746a4d61b7bce6115ae547eaafc955c41053898a315c968db2f36
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-client-darwin-amd64.tar.gz) | c9604fbb9e848a4b3dc85ee2836f74b4ccd321e4c72d22b2d4558eb0f0c3833bff35d0c36602c13c5c5c79e9233fda874bfa85433291ab3484cf61c9012ee515
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-client-darwin-arm64.tar.gz) | fed42ecbfc20b5f63ac48bbb9b73abc4b72aca76ac8bdd51b9ea6af053b1fc6a8e63b5e11f9d14c4814f03b49531da2536f1342cda2da03514c44ccf05c311b0
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-client-linux-386.tar.gz) | 93c61229d7b07a476296b5b800c853c8e984101d5077fc19a195673f7543e7d2eb2599311c1846c91ef1f7ae29c3e05b6f41b873e92a3429563e3d83900050da
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-client-linux-amd64.tar.gz) | 4260b49733f6b0967c504e2246b455b2348b487e84f7a019fda8b4a87d43d27a03e7ed55b505764c14f2079c4c3d71c68d77f981b604e13e7210680f45ee66e3
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-client-linux-arm.tar.gz) | 4e837fd2f55cbb5f93cdf60235511a85635485962f00e0378a95a7ff846eb86b7bf053203ab353b294131b2e2663d0e783dae79c18601d4d66f98a6e5152e96e
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-client-linux-arm64.tar.gz) | 6f3954d2adc289879984d18c2605110a7d5f0a5f6366233c25adf3a742f8dc1183e8a4d4747de8077af1045a259b150e0e86b27e10d683aa8decdc760ac6279b
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-client-linux-ppc64le.tar.gz) | 741b76827ff9e810e490d8698eb7620826a16e978e5c7744a1fa0e65124690cfc9601e7f1c8f50e77f25185ba3176789ddcb7d5caaddde66436c31658bacde1d
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-client-linux-s390x.tar.gz) | 0c635883e2f9caca03bcf3b42ba0b479f44c8cc2a3d5dd425b0fee278f3e884bef0e897fe51cbf00bb0bc061371805d9f9cbccf839477671f92e078c04728735
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-client-windows-386.tar.gz) | ebddbb358fd2d817908069eb66744dc62cae56ad470b1e36c6ebd0d2284e79ae5b9a5f8a86fef365f30b34e14093827ad736814241014f597e2ac88788102cf4
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-client-windows-amd64.tar.gz) | 01a451a809cd45e7916a3e982e2b94d372accab9dfe20667e95c10d56f9194b997721c0c219ff7ff97828b6466108eec6e57dcb33e3e3b0c5f770af1514a9f1a
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-client-windows-arm64.tar.gz) | 473ba648ffde41fd5b63374cc1595eb43b873808c6b0cc5e939628937f3f7fb36dba4b7c7c8ef03408d557442094ec22e12c03f40be137f9cc99761b4cc1a1f8
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-server-linux-amd64.tar.gz) | c3f7abcee3fdcf6f311b5de0bfe037318e646641c1ce311950d920252623cca285d1f1cef0e2d936c0f981edc1c725897a42aa9e03b77fe5f76f1090665d967f
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-server-linux-arm64.tar.gz) | 17614842df6bb528434b8b063b1d1c3efc8e4eff9cbc182f049d811f68e08514026fbb616199a3dee97e62ce2fd1eda0b9778d8e74040e645c482cfe6a18a8b4
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-server-linux-ppc64le.tar.gz) | 2f818035ef199a7745e24d2ce86abf6c52e351d7922885e264c5d07db3e0f21048c32db85f3044e01443abd87a45f92df52fda44e8df05000754b03f34132f2f
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-server-linux-s390x.tar.gz) | 96a34c768f347f23c46f990a8f6ddf3127b13f7a183453b92eb7bc27ce896767f31b38317a6ae5a11f2d4b459ec9564385f8abe61082a4165928edfee0c9765e
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-node-linux-amd64.tar.gz) | 66845cf86e32c19be9d339417a4772b9bcf51b2bf4d1ef5acc2e9eb006bbd19b3c036aa3721b3d8fe08b6fb82284ba25a6ecb5eb7b84f657cc968224d028f22c
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-node-linux-arm64.tar.gz) | 98902ee33242f9e78091433115804d54eafde24903a3515f0300f60c0273c7c0494666c221ce418d79e715f8ecf654f0edabc5b69765da26f83a812e963b5afb
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-node-linux-ppc64le.tar.gz) | 82f1213b5942c5c1576afadb4b066dfa1427c7709adf6ba636b9a52dfdb1b20f62b1cc0436b265e714fbee08c71d8786295d2439c10cc05bd58b2ab2a87611d4
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-node-linux-s390x.tar.gz) | 7cb8cb65195c5dd63329d02907cdbb0f5473066606c108f4516570f449623f93b1ca822d5a00fad063ec8630e956fa53a0ab530a8487bccb01810943847d4942
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.2/kubernetes-node-windows-amd64.tar.gz) | 1222e2d7dbaf7920e1ba927231cc7e275641cf0939be1520632353df6219bbcb3b49515d084e7f2320a2ff59b2de9fee252d8f5e9c48d7509f1174c6cb357b66
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.0-alpha.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.0-alpha.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.0-alpha.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.0-alpha.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.0-alpha.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.0-alpha.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-s390x)
## Changelog since v1.29.0-alpha.1
## Changes by Kind
### Feature
- Adds `apiserver_watch_list_duration_seconds` metrics. Which will measure response latency distribution in seconds for watch list requests broken by group, version, resource and scope ([#120490](https://github.com/kubernetes/kubernetes/pull/120490), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery and Instrumentation]
- Allow-list of metric labels can be configured by supplying a manifest using the --allow-metric-labels-manifest flag ([#118299](https://github.com/kubernetes/kubernetes/pull/118299), [@rexagod](https://github.com/rexagod)) [SIG Architecture and Instrumentation]
- Bump distroless-iptables to 0.3.3 based on Go 1.21.2 ([#121073](https://github.com/kubernetes/kubernetes/pull/121073), [@cpanato](https://github.com/cpanato)) [SIG Testing]
- Implements API for streaming for the etcd store implementation
When sendInitialEvents ListOption is set together with watch=true, it begins the watch stream with synthetic init events followed by a synthetic "Bookmark" after which the server continues streaming events. ([#119557](https://github.com/kubernetes/kubernetes/pull/119557), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery]
- Kubelet, when using cloud provider external, initializes temporary the node addresses using the --node-ip flag values if set, until the cloud provider overrides it. ([#121028](https://github.com/kubernetes/kubernetes/pull/121028), [@aojea](https://github.com/aojea)) [SIG Cloud Provider and Node]
- Kubernetes is now built with Go 1.21.2 ([#121021](https://github.com/kubernetes/kubernetes/pull/121021), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing]
- Migrated the volumebinding scheduler plugins to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#116803](https://github.com/kubernetes/kubernetes/pull/116803), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG Instrumentation, Scheduling and Storage]
- The kube-apiserver exposes four new metrics to inform about errors on the clusterIP and nodePort allocation logic ([#120843](https://github.com/kubernetes/kubernetes/pull/120843), [@aojea](https://github.com/aojea)) [SIG Instrumentation and Network]
### Failing Test
- K8s.io/dynamic-resource-allocation: DRA drivers updating to this release are compatible with Kubernetes 1.27 and 1.28. ([#120868](https://github.com/kubernetes/kubernetes/pull/120868), [@pohly](https://github.com/pohly)) [SIG Node]
### Bug or Regression
- Cluster-bootstrap: improve the security of the functions responsible for generation and validation of bootstrap tokens ([#120400](https://github.com/kubernetes/kubernetes/pull/120400), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle and Security]
- Do not fail volume attach or publish operation at kubelet if target path directory already exists on the node. ([#119735](https://github.com/kubernetes/kubernetes/pull/119735), [@akankshapanse](https://github.com/akankshapanse)) [SIG Storage]
- Fix regression with adding aggregated apiservices panicking and affected health check introduced in release v1.28.0 ([#120814](https://github.com/kubernetes/kubernetes/pull/120814), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery and Testing]
- Fixed a bug where containers would not start on cgroupv2 systems where swap is disabled. ([#120784](https://github.com/kubernetes/kubernetes/pull/120784), [@elezar](https://github.com/elezar)) [SIG Node]
- Fixed a regression in kube-proxy where it might refuse to start if given
single-stack IPv6 configuration options on a node that has both IPv4 and
IPv6 IPs. ([#121008](https://github.com/kubernetes/kubernetes/pull/121008), [@danwinship](https://github.com/danwinship)) [SIG Network]
- Fixed attaching volumes after detach errors. Now volumes that failed to detach are not treated as attached, Kubernetes will make sure they are fully attached before they can be used by pods. ([#120595](https://github.com/kubernetes/kubernetes/pull/120595), [@jsafrane](https://github.com/jsafrane)) [SIG Apps and Storage]
- Fixes a regression (CLIENTSET_PKG: unbound variable) when invoking deprecated generate-groups.sh script ([#120877](https://github.com/kubernetes/kubernetes/pull/120877), [@soltysh](https://github.com/soltysh)) [SIG API Machinery]
- K8s.io/dynamic-resource-allocation/controller: UnsuitableNodes did not handle a mix of allocated and unallocated claims correctly. ([#120338](https://github.com/kubernetes/kubernetes/pull/120338), [@pohly](https://github.com/pohly)) [SIG Node]
- K8s.io/dynamic-resource-allocation: handle a selected node which isn't listed as potential node ([#120871](https://github.com/kubernetes/kubernetes/pull/120871), [@pohly](https://github.com/pohly)) [SIG Node]
- Kubeadm: fix the bug that kubeadm always do CRI detection when --config is passed even if it is not required by the subcommand ([#120828](https://github.com/kubernetes/kubernetes/pull/120828), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle]
### Other (Cleanup or Flake)
- Client-go: k8s.io/client-go/tools events and record packages have new APIs for specifying a context and logger ([#120729](https://github.com/kubernetes/kubernetes/pull/120729), [@pohly](https://github.com/pohly)) [SIG API Machinery and Instrumentation]
- Deprecated the `--cloud-provider` and `--cloud-config` CLI parameters in kube-apiserver.
These parameters will be removed in a future release. ([#120903](https://github.com/kubernetes/kubernetes/pull/120903), [@dims](https://github.com/dims)) [SIG API Machinery]
## Dependencies
### Added
_Nothing has changed._
### Changed
- github.com/emicklei/go-restful/v3: [v3.9.0 → v3.11.0](https://github.com/emicklei/go-restful/v3/compare/v3.9.0...v3.11.0)
- github.com/onsi/ginkgo/v2: [v2.9.4 → v2.13.0](https://github.com/onsi/ginkgo/v2/compare/v2.9.4...v2.13.0)
- github.com/onsi/gomega: [v1.27.6 → v1.28.0](https://github.com/onsi/gomega/compare/v1.27.6...v1.28.0)
- golang.org/x/crypto: v0.11.0 → v0.12.0
- golang.org/x/mod: v0.10.0 → v0.12.0
- golang.org/x/net: v0.13.0 → v0.14.0
- golang.org/x/sync: v0.2.0 → v0.3.0
- golang.org/x/sys: v0.10.0 → v0.12.0
- golang.org/x/term: v0.10.0 → v0.11.0
- golang.org/x/text: v0.11.0 → v0.12.0
- golang.org/x/tools: v0.8.0 → v0.12.0
### Removed
_Nothing has changed._
# v1.29.0-alpha.1
## Downloads for v1.29.0-alpha.1
### Source Code
filename | sha512 hash
-------- | -----------
[kubernetes.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes.tar.gz) | 107062e8da7c416206f18b4376e9e0c2ca97b37c720a047f2bc6cf8a1bdc2b41e84defd0a29794d9562f3957932c0786a5647450b41d2850a9b328826bb3248d
[kubernetes-src.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-src.tar.gz) | 8182774faa5547f496642fdad7e2617a4d07d75af8ddf85fb8246087ddffab596528ffde29500adc9945d4e263fce766927ed81396a11f88876b3fa76628a371
### Client Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-client-darwin-amd64.tar.gz) | ac9a08cd98af5eb27f8dde895510db536098dd52ee89682e7f103c793cb99cddcd992e3a349d526854caaa27970aa1ef964db4cc27d1009576fb604bf0c1cdf1
[kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-client-darwin-arm64.tar.gz) | 28744076618dcd7eca4175726d7f3ac67fe94f08f1b6ca4373b134a6402c0f5203f1146d79a211443c751b2f2825df3507166fc3c5e40a55d545c3e5d2a48e56
[kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-client-linux-386.tar.gz) | 0207a2571b6d0e6e55f36af9d2ed27f31eacfb23f2f54dd2eb8fbc38ef5b033edb24fb9a5ece7e7020fd921a9c841fff435512d12421bfa13294cc9c297eb877
[kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-client-linux-amd64.tar.gz) | 57fc39ba259ae61b88c23fd136904395abc23c44f4b4db3e2922827ec7e6def92bc77364de3e2f6b54b27bb4b5e42e9cf4d1c0aa6d12c4a5a17788d9f996d9ad
[kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-client-linux-arm.tar.gz) | 53a54d3fbda46162139a90616d708727c23d3aae0a2618197df5ac443ac3d49980a62034e3f2514f1a1622e4ce5f6e821d2124a61a9e63ce6d29268b33292949
[kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-client-linux-arm64.tar.gz) | ee3ca4626c802168db71ad55c1d8b45c03ec774c146dd6da245e5bb26bf7fd6728a477f1ad0c5094967a0423f94e35e4458c6716f3abe005e8fc55ae354174cf
[kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-client-linux-ppc64le.tar.gz) | 60cd35076dd4afb9005349003031fa9f1802a2a120fbbe842d6fd061a1bca39baabcbb18fb4b6610a5ca626fc64e1d780c7aadb203d674697905489187a415ce
[kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-client-linux-s390x.tar.gz) | 68fdd0fc35dfd6fae0d25d7834270c94b16ae860fccc4253e7c347ce165d10cadc190e8b320fd2c4afd508afc6c10f246b8a5f0148ca1b1d56f7b2843cc39d30
[kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-client-windows-386.tar.gz) | 0c5d3dbfaaffa81726945510c972cc15895ea87bcd43b798675465fdadaa4d2d9597cb4fc6baee9ee719c919d1f46a9390c15cb0da60250f41eb4fcc3337b337
[kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-client-windows-amd64.tar.gz) | 2e519867cbc793ea1c1e45f040de81b49c70b9b42fac072ac5cac36e8de71f0dddd0c64354631bcb2b3af36a0f377333c0cd885c2df36ef8cd7e6c8fd5628aa4
[kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-client-windows-arm64.tar.gz) | 1a80cad80c1c9f753a38e6c951b771b0df820455141f40ba44e227f6acc81b59454f8dbff12e83c61bf647eaa1ff98944930969a99c96a087a35921f4e6ac968
### Server Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-server-linux-amd64.tar.gz) | c74a3f7bdd16095fb366b4313e50984f2ee7cb99c77ad2bcccea066756ce6e0fc45f4528b79c8cb7e6370430ee2d03fa6bc10ca87a59d8684a59e1ebd3524afd
[kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-server-linux-arm64.tar.gz) | b6844b5769fd5687525dcedca42c7bb036f6acad65d3de3c8cda46dbbe0ac23c289fdb7fbf15f1c37184498d6a1fb018e41e1c97ded4581f045ad2039e3ddec2
[kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-server-linux-ppc64le.tar.gz) | a15eb2db4821454974920a987bb1e73bc4ee638b845b07f35cab55dcf482c142d3cdaed347bfa0452d5311b3d9152463a3dae1d176b6101ed081ec594e0d526c
[kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-server-linux-s390x.tar.gz) | 60e24d8b4902821b436b5adebd6594ef0db79802d64787a1424aa6536873e2d749dfc6ebc2eb81db3240c925500a3e927ee7385188f866c28123736459e19b7b
### Node Binaries
filename | sha512 hash
-------- | -----------
[kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-node-linux-amd64.tar.gz) | 44832c7b90c88e7ca70737bad8d50ee8ba434ee7a94940f9d45beda9e9aadc7e2c973b65fcb986216229796a5807dae2470dbcf1ade5c075d86011eefe21509b
[kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-node-linux-arm64.tar.gz) | a13862d9bae0ff358377afc60f5222490a8e6bb7197d4a7d568edd4f150348f7a3dc7342129cd2d5c5353d2d43349b97c854df3e8886a8d52aedb95c634e3b5a
[kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-node-linux-ppc64le.tar.gz) | 57348f82bb4db8c230d8dffdef513ed75d7b267b226a5d15b3deb9783f8ed56fe40f8ce018ab34c28f9f8210b2e41b0f55d185dcdbaf912dd57e2ea78f8d3c53
[kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-node-linux-s390x.tar.gz) | 2013eb4746e818cf336e0fee37650df98c19876030397803abce9531730eb0b95e6284f5a2abdd2b97090a67d07fd7a9c74c84fc7b4b83f0bce04a6dc9ad2555
[kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.29.0-alpha.1/kubernetes-node-windows-amd64.tar.gz) | 3a4d63e2117cdbebc655e674bb017e246c263e893fc0ca3e8dc0091d6d9f96c9f0756c0fa8b45ba461502ae432f908ea922c21378b82ff3990b271f42eedc138
### Container Images
All container images are available as manifest lists and support the described
architectures. It is also possible to pull a specific architecture directly by
adding the "-$ARCH" suffix to the container image name.
name | architectures
---- | -------------
[registry.k8s.io/conformance:v1.29.0-alpha.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x)
[registry.k8s.io/kube-apiserver:v1.29.0-alpha.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x)
[registry.k8s.io/kube-controller-manager:v1.29.0-alpha.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x)
[registry.k8s.io/kube-proxy:v1.29.0-alpha.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x)
[registry.k8s.io/kube-scheduler:v1.29.0-alpha.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x)
[registry.k8s.io/kubectl:v1.29.0-alpha.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kubectl-s390x)
## Changelog since v1.28.0
## Changes by Kind
### Deprecation
- #### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:
<!--
This section can be blank if this pull request does not require a release note.
When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.
See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files
Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
--> ([#119495](https://github.com/kubernetes/kubernetes/pull/119495), [@bzsuni](https://github.com/bzsuni)) [SIG API Machinery]
### API Change
- Added a new `ipMode` field to the `.status` of Services where `type` is set to `LoadBalancer`.
The new field is behind the `LoadBalancerIPMode` feature gate. ([#119937](https://github.com/kubernetes/kubernetes/pull/119937), [@RyanAoh](https://github.com/RyanAoh)) [SIG API Machinery, Apps, Cloud Provider, Network and Testing]
- Fixed a bug where CEL expressions in CRD validation rules would incorrectly compute a high estimated cost for functions that return strings, lists or maps.
The incorrect cost was evident when the result of a function was used in subsequent operations. ([#119800](https://github.com/kubernetes/kubernetes/pull/119800), [@jpbetz](https://github.com/jpbetz)) [SIG API Machinery, Auth and Cloud Provider]
- Go API: the ResourceRequirements struct needs to be replaced with VolumeResourceRequirements for use with volumes. ([#118653](https://github.com/kubernetes/kubernetes/pull/118653), [@pohly](https://github.com/pohly)) [SIG API Machinery, Apps, Auth, Node, Scheduling, Storage and Testing]
- Kube-apiserver: adds --authentication-config flag for reading AuthenticationConfiguration files. --authentication-config flag is mutually exclusive with the existing --oidc-* flags. ([#119142](https://github.com/kubernetes/kubernetes/pull/119142), [@aramase](https://github.com/aramase)) [SIG API Machinery, Auth and Testing]
- Kube-scheduler component config (KubeSchedulerConfiguration) kubescheduler.config.k8s.io/v1beta3 is removed in v1.29. Migrate kube-scheduler configuration files to kubescheduler.config.k8s.io/v1. ([#119994](https://github.com/kubernetes/kubernetes/pull/119994), [@SataQiu](https://github.com/SataQiu)) [SIG Scheduling and Testing]
- Mark the onPodConditions field as optional in Job's pod failure policy. ([#120204](https://github.com/kubernetes/kubernetes/pull/120204), [@mimowo](https://github.com/mimowo)) [SIG API Machinery and Apps]
- Retry NodeStageVolume calls if CSI node driver is not running ([#120330](https://github.com/kubernetes/kubernetes/pull/120330), [@rohitssingh](https://github.com/rohitssingh)) [SIG Apps, Storage and Testing]
- The kube-scheduler `selectorSpread` plugin has been removed, please use the `podTopologySpread` plugin instead. ([#117720](https://github.com/kubernetes/kubernetes/pull/117720), [@kerthcet](https://github.com/kerthcet)) [SIG Scheduling]
### Feature
- --sync-frequency will not affect the update interval of volumes that use ConfigMaps or Secrets when the configMapAndSecretChangeDetectionStrategy is set to Cache. The update interval is only affected by node.alpha.kubernetes.io/ttl node annotation." ([#120255](https://github.com/kubernetes/kubernetes/pull/120255), [@likakuli](https://github.com/likakuli)) [SIG Node]
- Add a new scheduler metric, `pod_scheduling_sli_duration_seconds`, and start the deprecation for `pod_scheduling_duration_seconds`. ([#119049](https://github.com/kubernetes/kubernetes/pull/119049), [@helayoty](https://github.com/helayoty)) [SIG Instrumentation, Scheduling and Testing]
- Added apiserver_envelope_encryption_dek_cache_filled to measure number of records in data encryption key(DEK) cache. ([#119878](https://github.com/kubernetes/kubernetes/pull/119878), [@ritazh](https://github.com/ritazh)) [SIG API Machinery and Auth]
- Added kubectl node drain helper callbacks `OnPodDeletionOrEvictionStarted` and `OnPodDeletionOrEvictionFailed`; people extending `kubectl` can use these new callbacks for more granularity.
- Deprecated the `OnPodDeletedOrEvicted` node drain helper callback. ([#117502](https://github.com/kubernetes/kubernetes/pull/117502), [@adilGhaffarDev](https://github.com/adilGhaffarDev)) [SIG CLI]
- Adding apiserver identity to the following metrics:
apiserver_envelope_encryption_key_id_hash_total, apiserver_envelope_encryption_key_id_hash_last_timestamp_seconds, apiserver_envelope_encryption_key_id_hash_status_last_timestamp_seconds, apiserver_encryption_config_controller_automatic_reload_failures_total, apiserver_encryption_config_controller_automatic_reload_success_total, apiserver_encryption_config_controller_automatic_reload_last_timestamp_seconds
Fix bug to surface events for the following metrics: apiserver_encryption_config_controller_automatic_reload_failures_total, apiserver_encryption_config_controller_automatic_reload_last_timestamp_seconds, apiserver_encryption_config_controller_automatic_reload_success_total ([#120438](https://github.com/kubernetes/kubernetes/pull/120438), [@ritazh](https://github.com/ritazh)) [SIG API Machinery, Auth, Instrumentation and Testing]
- Bump distroless-iptables to 0.3.2 based on Go 1.21.1 ([#120527](https://github.com/kubernetes/kubernetes/pull/120527), [@cpanato](https://github.com/cpanato)) [SIG Testing]
- Changed `kubectl help` to display basic details for subcommands from plugins ([#116752](https://github.com/kubernetes/kubernetes/pull/116752), [@xvzf](https://github.com/xvzf)) [SIG CLI]
- Changed the `KMSv2KDF` feature gate to be enabled by default. ([#120433](https://github.com/kubernetes/kubernetes/pull/120433), [@enj](https://github.com/enj)) [SIG API Machinery, Auth and Testing]
- Graduated the following kubelet resource metrics to **general availability**:
- `container_cpu_usage_seconds_total`
- `container_memory_working_set_bytes`
- `container_start_time_seconds`
- `node_cpu_usage_seconds_total`
- `node_memory_working_set_bytes`
- `pod_cpu_usage_seconds_total`
- `pod_memory_working_set_bytes`
- `resource_scrape_error`
Deprecated (renamed) `scrape_error` in favor of `resource_scrape_error` ([#116897](https://github.com/kubernetes/kubernetes/pull/116897), [@Richabanker](https://github.com/Richabanker)) [SIG Architecture, Instrumentation, Node and Testing]
- Graduation API List chunking (aka pagination) feature to stable ([#119503](https://github.com/kubernetes/kubernetes/pull/119503), [@wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery, Cloud Provider and Testing]
- Implements API for streaming for the etcd store implementation
When sendInitialEvents ListOption is set together with watch=true, it begins the watch stream with synthetic init events followed by a synthetic "Bookmark" after which the server continues streaming events. ([#119557](https://github.com/kubernetes/kubernetes/pull/119557), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery]
- Improve memory usage of kube-scheduler by dropping the `.metadata.managedFields` field that kube-scheduler doesn't require. ([#119556](https://github.com/kubernetes/kubernetes/pull/119556), [@linxiulei](https://github.com/linxiulei)) [SIG Scheduling]
- In a scheduler with Permit plugins, when a Pod is rejected during WaitOnPermit, the scheduler records the plugin.
The scheduler will use the record to honor cluster events and queueing hints registered for the plugin, to inform whether to retry the pod. ([#119785](https://github.com/kubernetes/kubernetes/pull/119785), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling and Testing]
- In tree cloud providers are now switched off by default. Please use DisableCloudProviders and DisableKubeletCloudCredentialProvider feature flags if you still need this functionality. ([#117503](https://github.com/kubernetes/kubernetes/pull/117503), [@dims](https://github.com/dims)) [SIG API Machinery, Cloud Provider and Testing]
- Introduce new apiserver metric apiserver_flowcontrol_current_inqueue_seats. This metric is analogous to `apiserver_flowcontrol_current_inqueue_requests` but tracks totals seats as each request can take more than 1 seat. ([#119385](https://github.com/kubernetes/kubernetes/pull/119385), [@andrewsykim](https://github.com/andrewsykim)) [SIG API Machinery]
- Kube-proxy don't panic on exit when the Node object changes its PodCIDR ([#120375](https://github.com/kubernetes/kubernetes/pull/120375), [@pegasas](https://github.com/pegasas)) [SIG Network]
- Kube-proxy will only install the DROP rules for invalid conntrack states if the nf_conntrack_tcp_be_liberal is not set. ([#120412](https://github.com/kubernetes/kubernetes/pull/120412), [@aojea](https://github.com/aojea)) [SIG Network]
- Kubeadm: add validation to verify that the CertificateKey is a valid hex encoded AES key ([#120064](https://github.com/kubernetes/kubernetes/pull/120064), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle]
- Kubeadm: promoted feature gate `EtcdLearnerMode` to beta. Learner mode for joining etcd members is now enabled by default. ([#120228](https://github.com/kubernetes/kubernetes/pull/120228), [@pacoxu](https://github.com/pacoxu)) [SIG Cluster Lifecycle]
- Kubelet exposes latency metrics of different stages of the node startup. ([#118568](https://github.com/kubernetes/kubernetes/pull/118568), [@qiutongs](https://github.com/qiutongs)) [SIG Instrumentation, Node and Scalability]
- Kubernetes is now built with Go 1.21.1 ([#120493](https://github.com/kubernetes/kubernetes/pull/120493), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing]
- Kubernetes is now built with go 1.21.0 ([#118996](https://github.com/kubernetes/kubernetes/pull/118996), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing]
- List the pods using <PVC> as an ephemeral storage volume in "Used by:" part of the output of `kubectl describe pvc <PVC>` command. ([#120427](https://github.com/kubernetes/kubernetes/pull/120427), [@MaGaroo](https://github.com/MaGaroo)) [SIG CLI]
- Migrated the nodevolumelimits scheduler plugin to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#116884](https://github.com/kubernetes/kubernetes/pull/116884), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG Instrumentation, Node, Scheduling, Storage and Testing]
- Promote ServiceNodePortStaticSubrange to stable and lock to default ([#120233](https://github.com/kubernetes/kubernetes/pull/120233), [@xuzhenglun](https://github.com/xuzhenglun)) [SIG Network]
- QueueingHint got error in its returning value. If QueueingHint returns error, the scheduler logs the error and treats the event as QueueAfterBackoff so that the Pod wouldn't be stuck in the unschedulable pod pool. ([#119290](https://github.com/kubernetes/kubernetes/pull/119290), [@carlory](https://github.com/carlory)) [SIG Node, Scheduling and Testing]
- Remove /livez livezchecks for KMS v1 and v2 to ensure KMS health does not cause kube-apiserver restart. KMS health checks are still in place as a healthz and readiness checks. ([#120583](https://github.com/kubernetes/kubernetes/pull/120583), [@ritazh](https://github.com/ritazh)) [SIG API Machinery, Auth and Testing]
- The CloudDualStackNodeIPs feature is now beta, meaning that when using
an external cloud provider that has been updated to support the feature,
you can pass comma-separated dual-stack `--node-ips` to kubelet and have
the cloud provider take both IPs into account. ([#120275](https://github.com/kubernetes/kubernetes/pull/120275), [@danwinship](https://github.com/danwinship)) [SIG API Machinery, Cloud Provider and Network]
- The Dockerfile for the kubectl image has been updated with the addition of a specific base image and essential utilities (bash and jq). ([#119592](https://github.com/kubernetes/kubernetes/pull/119592), [@rayandas](https://github.com/rayandas)) [SIG CLI, Node, Release and Testing]
- Use of secret-based service account tokens now adds an `authentication.k8s.io/legacy-token-autogenerated-secret` or `authentication.k8s.io/legacy-token-manual-secret` audit annotation containing the name of the secret used. ([#118598](https://github.com/kubernetes/kubernetes/pull/118598), [@yuanchen8911](https://github.com/yuanchen8911)) [SIG Auth, Instrumentation and Testing]
- Volume_zone plugin will consider beta labels as GA labels during the scheduling process.Therefore, if the values of the labels are the same, PVs with beta labels can also be scheduled to nodes with GA labels. ([#118923](https://github.com/kubernetes/kubernetes/pull/118923), [@AxeZhan](https://github.com/AxeZhan)) [SIG Scheduling]
### Documentation
- Added descriptions and examples for the situation of using kubectl rollout restart without specifying a particular deployment. ([#120118](https://github.com/kubernetes/kubernetes/pull/120118), [@Ithrael](https://github.com/Ithrael)) [SIG CLI]
### Failing Test
- DRA: when the scheduler has to deallocate a claim after a node became unsuitable for a pod, it might have needed more attempts than really necessary. ([#120428](https://github.com/kubernetes/kubernetes/pull/120428), [@pohly](https://github.com/pohly)) [SIG Node and Scheduling]
- E2e framework: retrying after intermittent apiserver failures was fixed in WaitForPodsResponding ([#120559](https://github.com/kubernetes/kubernetes/pull/120559), [@pohly](https://github.com/pohly)) [SIG Testing]
- KCM specific args can be passed with `/cluster` script, without affecting CCM. New variable name: `KUBE_CONTROLLER_MANAGER_TEST_ARGS`. ([#120524](https://github.com/kubernetes/kubernetes/pull/120524), [@jprzychodzen](https://github.com/jprzychodzen)) [SIG Cloud Provider]
- This contains the modified windows kubeproxy testcases with mock implementation ([#120105](https://github.com/kubernetes/kubernetes/pull/120105), [@princepereira](https://github.com/princepereira)) [SIG Network and Windows]
### Bug or Regression
- Added a redundant process to remove tracking finalizers from Pods that belong to Jobs. The process kicks in after the control plane marks a Job as finished ([#119944](https://github.com/kubernetes/kubernetes/pull/119944), [@Sharpz7](https://github.com/Sharpz7)) [SIG Apps]
- Allow specifying ExternalTrafficPolicy for Services with ExternalIPs. ([#119150](https://github.com/kubernetes/kubernetes/pull/119150), [@tnqn](https://github.com/tnqn)) [SIG API Machinery, Apps, CLI, Cloud Provider, Network, Release and Testing]
- Exclude nodes from daemonset rolling update if the scheduling constraints are not met. This eliminates the problem of rolling update stuck of daemonset with tolerations. ([#119317](https://github.com/kubernetes/kubernetes/pull/119317), [@mochizuki875](https://github.com/mochizuki875)) [SIG Apps and Testing]
- Fix OpenAPI v3 not being cleaned up after deleting APIServices ([#120108](https://github.com/kubernetes/kubernetes/pull/120108), [@tnqn](https://github.com/tnqn)) [SIG API Machinery and Testing]
- Fix a 1.28 regression in scheduler: a pod with concurrent events could incorrectly get moved to the unschedulable queue where it could got stuck until the next periodic purging after 5 minutes if there was no other event for it. ([#120413](https://github.com/kubernetes/kubernetes/pull/120413), [@pohly](https://github.com/pohly)) [SIG Scheduling]
- Fix a bug in cronjob controller where already created jobs may be missing from the status. ([#120649](https://github.com/kubernetes/kubernetes/pull/120649), [@andrewsykim](https://github.com/andrewsykim)) [SIG Apps]
- Fix a concurrent map access in TopologyCache's `HasPopulatedHints` method. ([#118189](https://github.com/kubernetes/kubernetes/pull/118189), [@Miciah](https://github.com/Miciah)) [SIG Apps and Network]
- Fix kubectl events doesn't filter events by GroupVersion for resource with full name. ([#120119](https://github.com/kubernetes/kubernetes/pull/120119), [@Ithrael](https://github.com/Ithrael)) [SIG CLI and Testing]
- Fixed CEL estimated cost of `replace()` to handle a zero length replacement string correctly.
Previously this would cause the estimated cost to be higher than it should be. ([#120097](https://github.com/kubernetes/kubernetes/pull/120097), [@jpbetz](https://github.com/jpbetz)) [SIG API Machinery]
- Fixed a 1.26 regression scheduling bug by ensuring that preemption is skipped when a PreFilter plugin returns `UnschedulableAndUnresolvable` ([#119778](https://github.com/kubernetes/kubernetes/pull/119778), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling and Testing]
- Fixed a 1.27 scheduling regression that PostFilter plugin may not function if previous PreFilter plugins return Skip ([#119769](https://github.com/kubernetes/kubernetes/pull/119769), [@Huang-Wei](https://github.com/Huang-Wei)) [SIG Scheduling and Testing]
- Fixed a 1.28 regression around restarting init containers in the right order relative to normal containers ([#120281](https://github.com/kubernetes/kubernetes/pull/120281), [@gjkim42](https://github.com/gjkim42)) [SIG Node and Testing]
- Fixed a regression in default 1.27 configurations in kube-apiserver: fixed the AggregatedDiscoveryEndpoint feature (beta in 1.27+) to successfully fetch discovery information from aggregated API servers that do not check `Accept` headers when serving the `/apis` endpoint ([#119870](https://github.com/kubernetes/kubernetes/pull/119870), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery]
- Fixed an issue where a CronJob could fail to clean up Jobs when the ResourceQuota for Jobs had been reached. ([#119776](https://github.com/kubernetes/kubernetes/pull/119776), [@ASverdlov](https://github.com/ASverdlov)) [SIG Apps]
- Fixes a 1.28 regression handling negative index json patches ([#120327](https://github.com/kubernetes/kubernetes/pull/120327), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node and Storage]
- Fixes a bug where Services using finalizers may hold onto ClusterIP and/or NodePort allocated resources for longer than expected if the finalizer is removed using the status subresource ([#120623](https://github.com/kubernetes/kubernetes/pull/120623), [@aojea](https://github.com/aojea)) [SIG Network and Testing]
- Fixes an issue where StatefulSet might not restart a pod after eviction or node failure. ([#120398](https://github.com/kubernetes/kubernetes/pull/120398), [@aleksandra-malinowska](https://github.com/aleksandra-malinowska)) [SIG Apps]
- Fixes an issue with the garbagecollection controller registering duplicate event handlers if discovery requests fail. ([#117992](https://github.com/kubernetes/kubernetes/pull/117992), [@liggitt](https://github.com/liggitt)) [SIG API Machinery and Apps]
- Fixes the bug when images pinned by the container runtime can be garbage collected by kubelet ([#119986](https://github.com/kubernetes/kubernetes/pull/119986), [@ruiwen-zhao](https://github.com/ruiwen-zhao)) [SIG Node]
- Fixing issue with incremental id generation for loadbalancer and endpoint in Kubeproxy mock test framework. ([#120723](https://github.com/kubernetes/kubernetes/pull/120723), [@princepereira](https://github.com/princepereira)) [SIG Network and Windows]
- If a watch with the `progressNotify` option set is to be created, and the registry hasn't provided a `newFunc`, return an error. ([#120212](https://github.com/kubernetes/kubernetes/pull/120212), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery]
- Improved handling of jsonpath expressions for kubectl wait --for. It is now possible to use simple filter expressions which match on a field's content. ([#118748](https://github.com/kubernetes/kubernetes/pull/118748), [@andreaskaris](https://github.com/andreaskaris)) [SIG CLI and Testing]
- Incorporating feedback on PR #119341 ([#120087](https://github.com/kubernetes/kubernetes/pull/120087), [@divyasri537](https://github.com/divyasri537)) [SIG API Machinery]
- Kubeadm: Use universal deserializer to decode static pod. ([#120549](https://github.com/kubernetes/kubernetes/pull/120549), [@pacoxu](https://github.com/pacoxu)) [SIG Cluster Lifecycle]
- Kubeadm: fix nil pointer when etcd member is already removed ([#119753](https://github.com/kubernetes/kubernetes/pull/119753), [@pacoxu](https://github.com/pacoxu)) [SIG Cluster Lifecycle]
- Kubeadm: fix the bug that `--image-repository` flag is missing for some init phase sub-commands ([#120072](https://github.com/kubernetes/kubernetes/pull/120072), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle]
- Kubeadm: improve the logic that checks whether a systemd service exists. ([#120514](https://github.com/kubernetes/kubernetes/pull/120514), [@fengxsong](https://github.com/fengxsong)) [SIG Cluster Lifecycle]
- Kubeadm: print the default component configs for `reset` and `join` is now not supported ([#119346](https://github.com/kubernetes/kubernetes/pull/119346), [@chendave](https://github.com/chendave)) [SIG Cluster Lifecycle]
- Kubeadm: remove 'system:masters' organization from etcd/healthcheck-client certificate. ([#119859](https://github.com/kubernetes/kubernetes/pull/119859), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle]
- Kubectl prune v2: Switch annotation from `contains-group-resources` to `contains-group-kinds`,
because this is what we defined in the KEP and is clearer to end-users. Although the functionality is
in alpha, we will recognize the prior annotation; this migration support will be removed in beta/GA. ([#118942](https://github.com/kubernetes/kubernetes/pull/118942), [@justinsb](https://github.com/justinsb)) [SIG CLI]
- Kubectl will not print events if --show-events=false argument is passed to describe PVC subcommand. ([#120380](https://github.com/kubernetes/kubernetes/pull/120380), [@MaGaroo](https://github.com/MaGaroo)) [SIG CLI]
- More accurate requeueing in scheduling queue for Pods rejected by the temporal failure (e.g., temporal failure on kube-apiserver.) ([#119105](https://github.com/kubernetes/kubernetes/pull/119105), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling and Testing]
- No-op and GC related updates to cluster trust bundles no longer require attest authorization when the ClusterTrustBundleAttest plugin is enabled. ([#120779](https://github.com/kubernetes/kubernetes/pull/120779), [@enj](https://github.com/enj)) [SIG Auth]
- Reintroduce resourcequota.NewMonitor constructor for other consumers ([#120777](https://github.com/kubernetes/kubernetes/pull/120777), [@atiratree](https://github.com/atiratree)) [SIG Apps]
- Scheduler: Fix field apiVersion is missing from events reported from taint manager ([#114095](https://github.com/kubernetes/kubernetes/pull/114095), [@aimuz](https://github.com/aimuz)) [SIG Apps, Node and Scheduling]
- Service Controller: update load balancer hosts after node's ProviderID is updated ([#120492](https://github.com/kubernetes/kubernetes/pull/120492), [@cezarygerard](https://github.com/cezarygerard)) [SIG Cloud Provider and Network]
- Setting the `status.loadBalancer` of a Service whose `spec.type` is not `"LoadBalancer"` was previously allowed, but any update to the `metadata` or `spec` would wipe that field. Setting this field is no longer permitted unless `spec.type` is `"LoadBalancer"`. In the very unlikely event that this has unexpected impact, you can enable the `AllowServiceLBStatusOnNonLB` feature gate, which will restore the previous behavior. If you do need to set this, please file an issue with the Kubernetes project to help contributors understand why you need it. ([#119789](https://github.com/kubernetes/kubernetes/pull/119789), [@thockin](https://github.com/thockin)) [SIG Apps and Testing]
- Sometimes, the scheduler incorrectly placed a pod in the "unschedulable" queue instead of the "backoff" queue. This happened when some plugin previously declared the pod as "unschedulable" and then in a later attempt encounters some other error. Scheduling of that pod then got delayed by up to five minutes, after which periodic flushing moved the pod back into the "active" queue. ([#120334](https://github.com/kubernetes/kubernetes/pull/120334), [@pohly](https://github.com/pohly)) [SIG Scheduling]
- The `--bind-address` parameter in kube-proxy is misleading, no port is opened with this address. Instead it is translated internally to "nodeIP". The nodeIPs for both families are now taken from the Node object if `--bind-address` is unspecified or set to the "any" address (0.0.0.0 or ::). It is recommended to leave `--bind-address` unspecified, and in particular avoid to set it to localhost (127.0.0.1 or ::1) ([#119525](https://github.com/kubernetes/kubernetes/pull/119525), [@uablrek](https://github.com/uablrek)) [SIG Network and Scalability]
### Other (Cleanup or Flake)
- Add context to "caches populated" log messages. ([#119796](https://github.com/kubernetes/kubernetes/pull/119796), [@sttts](https://github.com/sttts)) [SIG API Machinery]
- Add download the cni binary for the corresponding arch in local-up-cluster.sh ([#120312](https://github.com/kubernetes/kubernetes/pull/120312), [@HirazawaUi](https://github.com/HirazawaUi)) [SIG Network and Node]
- Changes behavior of kube-proxy by allowing to set sysctl values lower than the existing one. ([#120448](https://github.com/kubernetes/kubernetes/pull/120448), [@aroradaman](https://github.com/aroradaman)) [SIG Network]
- Clean up kube-apiserver http logs for impersonated requests. ([#119795](https://github.com/kubernetes/kubernetes/pull/119795), [@sttts](https://github.com/sttts)) [SIG API Machinery]
- Dynamic resource allocation: avoid creating a new gRPC connection for every call of prepare/unprepare resource(s) ([#118619](https://github.com/kubernetes/kubernetes/pull/118619), [@TommyStarK](https://github.com/TommyStarK)) [SIG Node]
- Fixes an issue where the vsphere cloud provider will not trust a certificate if:
- The issuer of the certificate is unknown (x509.UnknownAuthorityError)
- The requested name does not match the set of authorized names (x509.HostnameError)
- The error surfaced after attempting a connection contains one of the substrings: "certificate is not trusted" or "certificate signed by unknown authority" ([#120736](https://github.com/kubernetes/kubernetes/pull/120736), [@MadhavJivrajani](https://github.com/MadhavJivrajani)) [SIG Architecture and Cloud Provider]
- Fixes bug where Adding GroupVersion log line is constantly repeated without any group version changes ([#119825](https://github.com/kubernetes/kubernetes/pull/119825), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery]
- Generated ResourceClaim names are now more readable because of an additional hyphen before the random suffix (`<pod name>-<claim name>-<random suffix>` ). ([#120336](https://github.com/kubernetes/kubernetes/pull/120336), [@pohly](https://github.com/pohly)) [SIG Apps and Node]
- Improve memory usage of kube-controller-manager by dropping the `.metadata.managedFields` field that kube-controller-manager doesn't require. ([#118455](https://github.com/kubernetes/kubernetes/pull/118455), [@linxiulei](https://github.com/linxiulei)) [SIG API Machinery and Cloud Provider]
- Kubeadm: remove 'system:masters' organization from apiserver-etcd-client certificate ([#120521](https://github.com/kubernetes/kubernetes/pull/120521), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle]
- Kubeadm: updated warning message when swap space is detected. When swap is active on Linux, kubeadm explains that swap is supported for cgroup v2 only and is beta but disabled by default. ([#120198](https://github.com/kubernetes/kubernetes/pull/120198), [@pacoxu](https://github.com/pacoxu)) [SIG Cluster Lifecycle]
- Makefile and scripts now respect GOTOOLCHAIN and otherwise ensure ./.go-version is used ([#120279](https://github.com/kubernetes/kubernetes/pull/120279), [@BenTheElder](https://github.com/BenTheElder)) [SIG Release]
- Optimized NodeUnschedulable Filter to avoid unnecessary calculations ([#119399](https://github.com/kubernetes/kubernetes/pull/119399), [@wackxu](https://github.com/wackxu)) [SIG Scheduling]
- Previously, the pod name and namespace were eliminated in the event log message. This PR attempts to add the preemptor pod UID in the preemption event message logs for easier debugging and safer transparency. ([#119971](https://github.com/kubernetes/kubernetes/pull/119971), [@kwakubiney](https://github.com/kwakubiney)) [SIG Scheduling]
- Promote to conformance a test that verify that Services only forward traffic on the port and protocol specified. ([#120069](https://github.com/kubernetes/kubernetes/pull/120069), [@aojea](https://github.com/aojea)) [SIG Architecture, Network and Testing]
- Remove ephemeral container legacy server support for the server versions prior to 1.22 ([#119537](https://github.com/kubernetes/kubernetes/pull/119537), [@ardaguclu](https://github.com/ardaguclu)) [SIG CLI]
- Scheduler: handling of unschedulable pods because a ResourceClass is missing is a bit more efficient and no longer relies on periodic retries ([#120213](https://github.com/kubernetes/kubernetes/pull/120213), [@pohly](https://github.com/pohly)) [SIG Node, Scheduling and Testing]
- Set the resolution for the job_controller_job_sync_duration_seconds metric from 4ms to 1min ([#120577](https://github.com/kubernetes/kubernetes/pull/120577), [@alculquicondor](https://github.com/alculquicondor)) [SIG Apps and Instrumentation]
- Statefulset should wait for new replicas in tests when removing .start.ordinal ([#119761](https://github.com/kubernetes/kubernetes/pull/119761), [@soltysh](https://github.com/soltysh)) [SIG Apps and Testing]
- The `horizontalpodautoscaling` and `clusterrole-aggregation` controllers now assume the `autoscaling/v1` and `rbac.authorization.k8s.io/v1` APIs are available. If you disable those APIs and do not want to run those controllers, exclude them by passing `--controllers=-horizontalpodautoscaling` or `--controllers=-clusterrole-aggregation` to `kube-controller-manager`. ([#117977](https://github.com/kubernetes/kubernetes/pull/117977), [@liggitt](https://github.com/liggitt)) [SIG API Machinery and Cloud Provider]
- The metrics controlled by the ComponentSLIs feature-gate and served at /metrics/slis are now GA and unconditionally enabled. The feature-gate will be removed in 1.31. ([#120574](https://github.com/kubernetes/kubernetes/pull/120574), [@logicalhan](https://github.com/logicalhan)) [SIG API Machinery, Architecture, Cloud Provider, Instrumentation, Network, Node and Scheduling]
- Updated CNI plugins to v1.3.0. ([#119969](https://github.com/kubernetes/kubernetes/pull/119969), [@saschagrunert](https://github.com/saschagrunert)) [SIG Cloud Provider, Node and Testing]
- Updated cri-tools to v1.28.0. ([#119933](https://github.com/kubernetes/kubernetes/pull/119933), [@saschagrunert](https://github.com/saschagrunert)) [SIG Cloud Provider]
- Updated distroless-iptables to use registry.k8s.io/build-image/distroless-iptables:v0.3.1 ([#120352](https://github.com/kubernetes/kubernetes/pull/120352), [@saschagrunert](https://github.com/saschagrunert)) [SIG Release and Testing]
- Upgrade coredns to v1.11.1 ([#120116](https://github.com/kubernetes/kubernetes/pull/120116), [@tukwila](https://github.com/tukwila)) [SIG Cloud Provider and Cluster Lifecycle]
- ValidatingAdmissionPolicy and ValidatingAdmissionPolicyBinding objects are persisted in etcd using the v1beta1 version. Remove alpha objects or disable the alpha ValidatingAdmissionPolicy feature in a 1.27 server before upgrading to a 1.28 server with the beta feature and API enabled. ([#120018](https://github.com/kubernetes/kubernetes/pull/120018), [@liggitt](https://github.com/liggitt)) [SIG API Machinery and Testing]
- Yes, kubectl will not support the "/swagger-2.0.0.pb-v1" endpoint that has been long deprecated ([#119410](https://github.com/kubernetes/kubernetes/pull/119410), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery]
## Dependencies
### Added
- github.com/distribution/reference: [v0.5.0](https://github.com/distribution/reference/tree/v0.5.0)
### Changed
- github.com/coredns/corefile-migration: [v1.0.20 → v1.0.21](https://github.com/coredns/corefile-migration/compare/v1.0.20...v1.0.21)
- github.com/docker/distribution: [v2.8.2+incompatible → v2.8.1+incompatible](https://github.com/docker/distribution/compare/v2.8.2...v2.8.1)
- github.com/evanphx/json-patch: [v5.6.0+incompatible → v4.12.0+incompatible](https://github.com/evanphx/json-patch/compare/v5.6.0...v4.12.0)
- github.com/google/cel-go: [v0.16.0 → v0.17.6](https://github.com/google/cel-go/compare/v0.16.0...v0.17.6)
- github.com/gorilla/websocket: [v1.4.2 → v1.5.0](https://github.com/gorilla/websocket/compare/v1.4.2...v1.5.0)
- github.com/opencontainers/runc: [v1.1.7 → v1.1.9](https://github.com/opencontainers/runc/compare/v1.1.7...v1.1.9)
- github.com/opencontainers/selinux: [v1.10.0 → v1.11.0](https://github.com/opencontainers/selinux/compare/v1.10.0...v1.11.0)
- github.com/vmware/govmomi: [v0.30.0 → v0.30.6](https://github.com/vmware/govmomi/compare/v0.30.0...v0.30.6)
- google.golang.org/protobuf: v1.30.0 → v1.31.0
- k8s.io/gengo: c0856e2 → 9cce18d
- k8s.io/kube-openapi: 2695361 → d090da1
- k8s.io/utils: d93618c → 3b25d92
- sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.1.2 → v0.28.0
- sigs.k8s.io/structured-merge-diff/v4: v4.2.3 → v4.3.0
### Removed
_Nothing has changed._
Codebase Browser
kubernetes