-
Notifications
You must be signed in to change notification settings - Fork 183
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: George Jenkins <[email protected]>
- Loading branch information
Showing
1 changed file
with
136 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,136 @@ | ||
| --- | ||
| hip: 9999 | ||
| title: "Utilize Server Side Apply for installs/upgrades" | ||
| authors: [ "George Jenkins <[email protected]>" ] | ||
| created: "2023-07-07" | ||
| type: "feature" | ||
| status: "draft" | ||
| --- | ||
|
|
||
| ## Abstract | ||
|
|
||
| <!-- A short (~200 word) description of the technical issue being addressed. --> | ||
| Helm 3 introduced the [three-way strategic merge and patch](https://helm.sh/docs/faq/changes_since_helm2/#improved-upgrade-strategy-3-way-strategic-merge-patches) methology. Allowing Helm to update resources which may have been modified by other processes. | ||
|
|
||
| Kubernetes now offers [Server-Side Apply](https://kubernetes.io/docs/reference/using-api/server-side-apply/) (SSA), that has serveral advantages over the client-side apply (CSA) methods that Helm, and e.g. Kubectl traditionally incorporated. | ||
|
|
||
| This HIP proposes enabling Helm to utilize Kubernete's Server-Side Apply for Helm resource management. But limits scope to incorporating SSA as an opt-in feature only. And defers removing / changing existing usaging of the (client-side) three-way strategic merge patch processes (without opt-in). | ||
|
|
||
| ## Motivation | ||
|
|
||
| <!-- Clearly explain why the existing design is inadequate to address the problem | ||
| that the HIP solves. --> | ||
|
|
||
| The primary motivation for Helm to switch to Server-Side Apply is to gain the benefits of Kubernetes managing resource upgrades where the resource may be modified by other processes in addition to Helm. The following describes the advantages of SSA from Kubernetes perspective, where SSA generally is considered to be superior: | ||
|
|
||
| https://kubernetes.io/blog/2022/10/20/advanced-server-side-apply/ | ||
|
|
||
| Helm supporting SSA will allow users to realize the benefits of SSA. And longer term goals (out of scope for this HIP) might be for Helm to utilize SSA by default ([similar to kubectl](https://github.com/kubernetes/enhancements/tree/master/keps/sig-cli/3805-ssa-default)). Finally later as a code mantainence benefit, eventually drop support for the strategic-merge patch client-side apply process/code. | ||
|
|
||
| ## Rationale | ||
|
|
||
| <!-- Describe why particular design decisions were made. --> | ||
|
|
||
| Due to the backwards compatibility concerns documented below, the decision was made to provide opt-in support only. And even when a chart is deployed with SSA enabled, Helm should mantain full compatibility with reverting back to using CSA for upgrades/rollbacks. Additionally it was desirable to mimic kubectl's cli interface/flags. As a way for Helm to be consistent with an existing kubectl functionality. | ||
|
|
||
| ## Specification | ||
|
|
||
| <!-- Describe the syntax and semantics of any new feature. --> | ||
|
|
||
| ### General | ||
|
|
||
| Helm will add a new flag to the install, upgrade and rollback sub-commands, modelled after the like `kubectl` flag. And add a corresponding field to the respective install/upgrade/rollback SDK API objects: | ||
| `--server-side=false|true|auto` | ||
|
|
||
| Helm will default to `false` for installs, retaining its existing behaviour. When `true`, Helm will submit resources to the Kubernetes API server with SSA enabled. When `auto`, Helm will default to CSA on installs (`false`), and for upgrades and rollbacks enable SSA based on the whether SSA was utilized for the prior operation (`auto`). The usage of SSA for a given release will be stored in the release's manifest. | ||
|
|
||
| If a user deploys a chart with a new version of Helm with SSA enabled, then upgrades the chart with an older version of Helm. It is expected that the older Helm will be able to upgrade the chart with CSA. And the new manifest object will omit the indication of SSA usage from the releases manifest (and so futher upgrades/rollbacks will continue to use CSA by default). | ||
|
|
||
| In a future version of Helm 3 cli, it might be desirable to omit a warning if a user is not using SSA (in order to prompt the user to enabling SSA). | ||
|
|
||
| #### Custom resources | ||
|
|
||
| Helm will utilize SSA for custom resources, which previously it replaced (TODO: reference). As SSA allows custom resources to be correctly merged based on metadata within the CRD. Or the API server will use the default approach <https://kubernetes.io/docs/reference/using-api/server-side-apply/#custom-resources>. | ||
|
|
||
| #### Conflicts and forcing | ||
|
|
||
| It is possible when Helm upgrades a chart, that there will be a field conflict with another field manager. In this case, Helm should report the error to the user. | ||
|
|
||
| Helm will also add a new flag `--force-conflicts`, and corresponding field to the `upgrade` and `rollback` commands. Which will tell the apply to ignore/override field conflicts (and so thus make Helm the field manager). | ||
|
|
||
| (Of note, Helm upgrade already includes the `--force` flag, which tells Helm to replace objects <TODO: confirm this, going from memory>. However, given this different semantics, it is thought a new flag is required) | ||
|
|
||
| ### Implementation | ||
|
|
||
| <TODO: how do we execute SSA> | ||
| - field manager name | ||
|
|
||
| https://github.com/kubernetes/kubectl/blob/197123726db24c61aa0f78d1f0ba6e91a2ec2f35/pkg/cmd/apply/apply.go#L248 | ||
| https://github.com/kubernetes/kubectl/blob/197123726db24c61aa0f78d1f0ba6e91a2ec2f35/pkg/cmd/apply/apply.go#L564-L657 | ||
|
|
||
| #### Special | ||
|
|
||
| - `--dry-run=client` doesn't work with SSA enabled | ||
| - If the user attempts to use SSA with a cluster which does not support it (unlikely: SSA [went GA](https://kubernetes.io/blog/2021/08/06/server-side-apply-ga/) in Kubernetes v1.22), Helm should error. | ||
|
|
||
|
|
||
| ## Backwards compatibility | ||
|
|
||
| <!-- Describe potential impact and severity on pre-existing code. --> | ||
|
|
||
| There are thought to be three main compatibility concerns for Helm switching to SSA: | ||
| 1. The differences between resource manifests managed with SSA vs the existing three-way strategic merge process | ||
| 2. Issues with a user enabling/disabling SSA over the lifetime of a release e.g. environment differences or older Helm clients which don't understand SSA | ||
| 3. Requirement of PATCH <TODO: check if this is new> | ||
|
|
||
| For 1., this HIP limits Helm to opt-in to server side apply, allowing users to verify SSA's behaviour matches their expectations. For 2., the user can mitigate by ensuring SSA is consistantly used within their environment. But the expectation is that SSA should generally produce equivalient Kubernetes objects as CSA. | ||
|
|
||
| Additionally as noted, SSA isn't compatible with very old Kubernetes versions (Kubernetes v1.17 and prior) | ||
|
|
||
|
|
||
| ## Security implications | ||
|
|
||
| <!-- How could a malicious user take advantage of this new feature? --> | ||
|
|
||
| There should not be any additional security concerns. Server-side apply and client=side apply should be eqivalient security-wise. | ||
|
|
||
| ## How to teach this | ||
|
|
||
| <!-- How to teach users, new and experienced, how to apply the HIP to their work. --> | ||
|
|
||
| Documentation for how Helm does resource upgrades, and the advantages of SSA. Also documentation for the `--server-side=false|true|auto` and `--force-conflicts=false|true` flags, and the defaults for install/upgrades/rollbacks. And the corresponding documentation for the field(s) SDK API objects. | ||
|
|
||
| ## Reference implementation | ||
|
|
||
| <!-- Link to any existing implementation and details about its state, e.g. | ||
| proof-of-concept. --> | ||
|
|
||
| Kubectl: | ||
| <https://github.com/kubernetes/kubectl/blob/197123726db24c61aa0f78d1f0ba6e91a2ec2f35/pkg/cmd/apply/apply.go#L248> | ||
| <https://github.com/kubernetes/kubectl/blob/197123726db24c61aa0f78d1f0ba6e91a2ec2f35/pkg/cmd/apply/apply.go#L564-L657> | ||
|
|
||
|
|
||
| ## Rejected ideas | ||
|
|
||
| <!-- Why certain ideas that were brought while discussing this HIP were not | ||
| ultimately pursued. --> | ||
|
|
||
| - Opting users into SSA as default (defer to later) | ||
| - Forcing upgrades to be follow install time CSA or SSA setting: since clients can always use an old Helm version. Or may want to "upgrade" an existing release with SSA. | ||
| - Supporting multiple field managers within the same release | ||
|
|
||
| ## Open issues | ||
|
|
||
| <!-- Any points that are still being decided/discussed. --> | ||
|
|
||
| - How to transistion SSA to being default | ||
| - How to remove support for client-side apply | ||
|
|
||
| ## References | ||
|
|
||
| <!-- A collection of URLs or materials used as references through the HIP. --> | ||
|
|
||
| - <https://kubernetes.io/docs/reference/using-api/server-side-apply/> | ||
| - <https://github.com/kubernetes/enhancements/tree/master/keps/sig-cli/3805-ssa-default#summary> | ||
| - <https://github.com/kubernetes/community/blob/master/contributors/devel/sig-api-machinery/strategic-merge-patch.md> | ||
| - <https://github.com/kubernetes-sigs/structured-merge-diff> |