Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: 📝 document CRDs technical choices #1321

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

docs: 📝 document CRDs technical choices #1321

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
244f0b8
chore: :memo: document CRDs technical choices
darkweaver87 Jan 27, 2025
b2c1c17
fix: apply suggestions from @mloiseleur's code review
darkweaver87 Jan 29, 2025
a80e636
fix: apply suggestions from @mloiseleur's code review
darkweaver87 Feb 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 116 additions & 23 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,15 @@ Starting with v28.x, this chart now bootstraps Traefik Proxy version 3 as a Kube
using Custom Resources `IngressRoute`: <https://doc.traefik.io/traefik/v3.0/routing/providers/kubernetes-crd/>.

It's possible to use this chart with Traefik Proxy v2 using v27.x
This chart support policy is aligned with [upstream support policy](https://doc.traefik.io/traefik/deprecation/releases/) of Traefik Proxy.
This chart support policy is aligned
with [upstream support policy](https://doc.traefik.io/traefik/deprecation/releases/) of Traefik Proxy.

See [Migration guide from v2 to v3](https://doc.traefik.io/traefik/v3.0/migration/v2-to-v3/) and upgrading section of this chart on CRDs.
See [Migration guide from v2 to v3](https://doc.traefik.io/traefik/v3.0/migration/v2-to-v3/) and upgrading section of
this chart on CRDs.

Starting with v34.x, to work around [Helm caveats](https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations), it's possible to use an additional Chart dedicated to CRDs: **traefik-crds**.
Starting with v34.x, to work
around [Helm caveats](https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations),
it's possible to use an additional Chart dedicated to CRDs: **traefik-crds**.

### Philosophy

Expand All @@ -24,12 +28,14 @@ to avoid integrating any third party solutions nor any specific use cases.

Accordingly, the encouraged approach to fulfill your needs:

1. Override the default Traefik configuration values ([yaml file or cli](https://helm.sh/docs/chart_template_guide/values_files/))
1. Override the default Traefik configuration
values ([yaml file or cli](https://helm.sh/docs/chart_template_guide/values_files/))
2. Append your own configurations (`kubectl apply -f myconf.yaml`)

[Examples](https://github.com/traefik/traefik-helm-chart/blob/master/EXAMPLES.md) of common usage are provided.

If needed, one may use [extraObjects](./traefik/tests/values/extra.yaml) or extend this HelmChart [as a Subchart](https://helm.sh/docs/chart_template_guide/subcharts_and_globals/).
If needed, one may use [extraObjects](./traefik/tests/values/extra.yaml) or extend this
HelmChart [as a Subchart](https://helm.sh/docs/chart_template_guide/subcharts_and_globals/).
darkweaver87 marked this conversation as resolved.
Show resolved Hide resolved

## Installing

Expand All @@ -40,23 +46,25 @@ If needed, one may use [extraObjects](./traefik/tests/values/extra.yaml) or exte

### Kubernetes Version Support

Due to changes in CRD version support, the following versions of the chart are usable and supported on the following Kubernetes versions:
Due to changes in CRD version support, the following versions of the chart are usable and supported on the following
Kubernetes versions:

| | Kubernetes v1.15 and below | Kubernetes v1.16-v1.21 | Kubernetes v1.22 and above |
|-------------------------|-----------------------------|------------------------|----------------------------|
| Chart v9.20.2 and below | [x] | [x] | |
| Chart v10.0.0 and above | | [x] | [x] |
| Chart v22.0.0 and above | | | [x] |
| | Kubernetes v1.15 and below | Kubernetes v1.16-v1.21 | Kubernetes v1.22 and above |
|-------------------------|----------------------------|------------------------|----------------------------|
| Chart v9.20.2 and below | [x] | [x] | |
| Chart v10.0.0 and above | | [x] | [x] |
| Chart v22.0.0 and above | | | [x] |

### CRDs Support of Traefik Proxy

Due to changes in API Group of Traefik CRDs from `containo.us` to `traefik.io`, this Chart install CRDs needed by default Traefik Proxy version, following this table:
Due to changes in API Group of Traefik CRDs from `containo.us` to `traefik.io`, this Chart install CRDs needed by
default Traefik Proxy version, following this table:

| | `containo.us` | `traefik.io` |
|-------------------------|-----------------------------|------------------------|
| Chart v22.0.0 and below | [x] | |
| Chart v23.0.0 and above | [x] | [x] |
| Chart v28.0.0 and above | | [x] |
| | `containo.us` | `traefik.io` |
|-------------------------|---------------|--------------|
| Chart v22.0.0 and below | [x] | |
| Chart v23.0.0 and above | [x] | [x] |
| Chart v28.0.0 and above | | [x] |

### Deploying

Expand All @@ -81,6 +89,88 @@ helm install -f myvalues.yaml traefik traefik/traefik

#### With additional CRDs chart

<details>
darkweaver87 marked this conversation as resolved.
Show resolved Hide resolved
<summary>Few words on technical choices about CRDs implementation</summary>
Some Traefik Helm chart users asked for help in managing CRDs installed by this chart (cf. [#1141](https://github.com/traefik/traefik-helm-chart/issues/1141), [#1209](https://github.com/traefik/traefik-helm-chart/issues/1209)).
Helm doesn't support CRDs upgrades (cf. [HIP-0011](https://github.com/helm/community/blob/main/hips/hip-0011.md) for details).

The objectives are the following:

1. Support the nominal installation case following official Helm GuideLines
2. Stay conservative about CRDs to protect resource removal (that's actually one of the reasons why helm doesn't support)
CRDs upgrades)
3. Allow users to install multiple instances of Traefik chart along with helm managed CRDs

Several implementations have been experimented. Here are pros and cons of each:

<table>
<thead>
<tr>
<td>solution</td>
<td>pros</td>
<td>cons</td>
</tr>
</thead>
<tbody>
<tr>
<td>templatized CRDs within Traefik helm chart</td>
<td>
<ul>
<li>simple</li>
<li>users can specify only install a subset of CRDs</li>
<li>users don't have to bother with CRDs upgrades</li>
</ul>
</td>
<td>
<ul>
<li><code>--skip-crds</code> will be inefficient and can lost users</li>
<li>the first installation fails are CRDs are not rendered first by helm</li>
<li>when installing multiple instances, CRDs are attached to one instance</li>
</ul>
</td>
</tr>
<tr>
<td>seperated CRDs chart as main chart dependency</td>
<td>
<ul>
<li>users can specify only install a subset of CRDs</li>
<li>users don't have to bother with CRDs upgrades</li>
<li>CRDs are versioned aside from main chart</li>
<li>users can install CRDs along with multiple instances of main chart</li>
</ul>
</td>
<td>
<ul>
<li><code>--skip-crds</code> will be inefficient and can lost users</li>
<li>the first installation fails are CRDs are not rendered first by helm (helm doesn't respect dependency order)</li>
<li>when installing multiple instances, CRDs are attached to one instance</li>
</ul>
</td>
</tr>
<tr>
<td>seperated CRDs chart</td>
<td>
<ul>
<li>users can specify only install a subset of CRDs</li>
<li>users don't have to bother with CRDs upgrades</li>
<li>CRDs are versioned aside from main chart</li>
<li>users can install CRDs along with multiple instances of main chart</li>
</ul>
</td>
<td>
<ul>
<li><code>--skip-crds</code> will be inefficient and can lost users</li>
<li>the first installation fails are CRDs are not rendered first by helm</li>
</ul>
</td>
</tr>
</tbody>
</table>

Consequently, we decided the last option was the less disruptive.

</details>

```bash
helm install traefik-crds traefik/traefik-crds
helm install traefik traefik/traefik --skip-crds
Expand All @@ -98,7 +188,8 @@ New major version indicates that there is an incompatible breaking change.
### A standard installation

When using Helm native management for CRDs, user **MUST** upgrade CRDs before calling _helm upgrade_ command.
CRDs are **not** updated by Helm. See [HIP-0011](https://github.com/helm/community/blob/main/hips/hip-0011.md) for details.
CRDs are **not** updated by Helm. See [HIP-0011](https://github.com/helm/community/blob/main/hips/hip-0011.md) for
details.

```bash
# Update repository
Expand All @@ -122,7 +213,6 @@ kubectl get customresourcedefinitions.apiextensions.k8s.io -o name | grep gatewa
helm install traefik-crds traefik/traefik-crds
```


### An installation with additional CRDs chart

```bash
Expand All @@ -138,7 +228,8 @@ helm upgrade traefik traefik/traefik

### Upgrade up to 27.X

When upgrading on Traefik Proxy v2 version, one need to stay at Traefik Helm Chart v27.x. The command to upgrade to the latest Traefik Proxy v2 CRD is:
When upgrading on Traefik Proxy v2 version, one need to stay at Traefik Helm Chart v27.x. The command to upgrade to the
latest Traefik Proxy v2 CRD is:

```bash
kubectl apply --server-side --force-conflicts -k https://github.com/traefik/traefik-helm-chart/traefik/crds/?ref=v27
Expand All @@ -165,8 +256,9 @@ On HTTP/3, if you want to avoid this issue, you can set
`ports.websecure.http3.advertisedPort` to an other value than `443`

If you were previously using HTTP/3, you should update your values as follows:
- Replace the old value (`true`) of `ports.websecure.http3` with a key `enabled: true`
- Remove `experimental.http3.enabled=true` entry

- Replace the old value (`true`) of `ports.websecure.http3` with a key `enabled: true`
- Remove `experimental.http3.enabled=true` entry

### Upgrading from 16.x to 17.x

Expand All @@ -182,7 +274,8 @@ order to be able to upgrade.
You may also upgrade by deploying another Traefik to a different namespace and
removing after your first Traefik.

Alternatively, since version 20.3.0 of this chart, you may set `instanceLabelOverride` to the previous value of that label.
Alternatively, since version 20.3.0 of this chart, you may set `instanceLabelOverride` to the previous value of that
label.
This will override the new `Release.Name-Release.Namespace` pattern to avoid any (longer) downtime.

## Contributing
Expand Down
Loading