From 14dc385a1d268349b8cbe9da19ce8a6285a7c2d4 Mon Sep 17 00:00:00 2001 From: Stephan Feurer Date: Fri, 15 Dec 2023 09:17:12 +0100 Subject: [PATCH] Add Exoscale architecture reference documentation --- .../images/ocp4-architecture-exoscale.svg | 4 + .../references/exoscale/architecture.adoc | 107 ++++++++++++++++++ .../architecture/glossary-general.adoc | 14 ++- .../ROOT/partials/architecture/storage.adoc | 9 ++ docs/modules/ROOT/partials/nav.adoc | 2 + 5 files changed, 135 insertions(+), 1 deletion(-) create mode 100644 docs/modules/ROOT/assets/images/ocp4-architecture-exoscale.svg create mode 100644 docs/modules/ROOT/pages/references/exoscale/architecture.adoc diff --git a/docs/modules/ROOT/assets/images/ocp4-architecture-exoscale.svg b/docs/modules/ROOT/assets/images/ocp4-architecture-exoscale.svg new file mode 100644 index 00000000..92850bb4 --- /dev/null +++ b/docs/modules/ROOT/assets/images/ocp4-architecture-exoscale.svg @@ -0,0 +1,4 @@ + + + +
Exoscale Zone
Exoscale Zone
Cluster Machine Network (/24, Cluster Security Group)
Cluster Machine Network (/24, Cluster Security Group)
OpenShift 4 Control plane nodes
3x 4vCPU/16GiB RAM
OpenShift 4 Control pla...
OpenShift 4 infrastructure nodes
4x 4vCPU/16GiB RAM
OpenShift 4 infrastruct...
OpenShift 4 worker nodes
3x 4vCPU/16GiB RAM
OpenShift 4 worker node...
Pod Network – Cilium VXLAN – 10.128.0.0/14
Pod Network – Cilium VXLAN – 10.128.0.0/14
Kubernetes API
Kubernetes API
Scheduler
Scheduler
Controllers
Controllers
OpenShift API
OpenShift API
etcd
etcd
Ingress Router
Ingress Router
Monitoring
Monitoring
Registry
Registry
Logging
Logging
...
...
Application A
Application A
Application B
Application B
Application C
Application C
Service Network – Cilium eBPF rules – 172.30.0.0/16
Service Network – Cilium eBPF rules – 172.30.0.0/16
Load Balancer Security Group
Load Balancer Security Group
DNS:
api.cluster.cust.vshnmanaged.net
DNS:...
DNS:
ingress.cluster.cust.vshnmanaged.net
*.apps.cluster.cust.vshnmanaged.net
DNS:...
Firewall rules:

ANY       -> 80/tcp   -> Ingress VIP*
ANY       -> 443/tcp  -> Ingress VIP*
ANY       -> 6443/tcp -> API VIP

*Should use PROXY protocol to
 preserve source IPs
Firewall rules:...
Other customer networks
Other customer networks
Other customer systems
Other customer sys...
Other customer systems
Other customer sys...
VSHN services

- LDAP / SSO
- GitLab
- Project Syn API
- Project Syn Vault
- acme-dns
- central metrics store
VSHN services...
Red Hat services

- OpenShift Update Service
- Container registries
- NTP servers
Red Hat services...
3rd party services

- OpsGenie
- Passbolt
- Let's Encrypt
- Container registries
3rd party services...
managed by VSHN
managed by VSHN
managed by Exoscale
managed by Exoscale
managed by Customer, optional
managed by Customer, optional
6443/tcp
6443/tcp
80/tcp 443/tcp
80/tcp 443/tcp
Load Balancer
Load Balancer
API IP
API IP
Ingress IP
Ingress IP Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/modules/ROOT/pages/references/exoscale/architecture.adoc b/docs/modules/ROOT/pages/references/exoscale/architecture.adoc new file mode 100644 index 00000000..8e2f333f --- /dev/null +++ b/docs/modules/ROOT/pages/references/exoscale/architecture.adoc @@ -0,0 +1,107 @@ +:infra-type: Exoscale +:infra-svg: ocp4-architecture-exoscale.svg += APPUiO Managed OpenShift 4 on {infra-type} + +== Architecture overview + +include::partial$architecture/overview.adoc[] + +== {infra-type} requirements + +APPUiO Managed OpenShift 4 on {infra-type} needs a https://docs.openshift.com/container-platform/4.14/installing/installing_bare_metal/installing-bare-metal.html#installation-load-balancing-user-infra_installing-bare-metal[Load Balancer setup] that must meet the following requirements: + +1. API load balancer: Provides a common endpoint to interact with the OpenShift and Kubernetes. + +2. Ingress load balancer: Provides an ingress point for application traffic flowing in from outside the cluster. + +See the https://docs.openshift.com/container-platform/latest/installing/installing_bare_metal/installing-bare-metal.html#installation-requirements-user-infra_installing-bare-metal[upstream documentation] for details on {infra-type} requirements. + + +== Networking + +=== Security Groups + +On {infra-type} APPUiO Managed OpenShift 4 uses public ips for each node in the cluster. +See https://kb.vshn.ch/oc4/explanations/exoscale/limitations.html#_private_networks[Limitations] of the {infra-type} environment. + +The individual VMs are placed in https://community.exoscale.com/documentation/compute/security-groups[Security Groups] to restrict access and isolate the nodes from the public internet. + +=== Virtual IPs + +To expose applications and the Kubernetes API outside the cluster, APPUiO Managed OpenShift 4 manages two floating IPs: + +1. The "API VIP" for the Kubernetes and OpenShift API. +APPUiO Managed OpenShift 4 uses a public floating IP as the API VIP. +2. The "Ingress VIP" for the OpenShift Ingress Router +APPUiO Managed OpenShift 4 uses a public floating IP as the Ingress VIP. + +APPUiO Managed OpenShift 4 uses two Load Balancer instances to manage the API and ingress VIPs and distributes traffic to the master / infrastructure nodes. + +=== Pod and service networks + +include::partial$architecture/networking-pods.adoc[] + +=== Exposing the cluster + +On {infra-type} infrastructure two Load Balancer instances provide ingress to the cluster. +The Load Balancer setup exposes two public IPs: + +1. A public IP for the API. +Traffic to port `6443/tcp` on this IP must be forwarded to the control plane nodes in the machine network. +The forwarding of this traffic must happen transparently. +In particular, no TLS interception can be performed as the Kubernetes API depends on mutual TLS authentication. +VSHN will manage a DNS record pointing to this IP. +2. A public IP for HTTP(s) ingress. +Traffic to ports `80/tcp` and `443/tcp` on this IP must be forwarded to the infrastructure nodes in the machine network. +The PROXY protocol should be enabled to preserve source IPs. +Forwarding should happen transparently in TCP mode. +VSHN will manage a wildcard DNS record pointing to this IP. +Additional DNS records can be pointed to this IP by the customer. + +=== External services + +include::partial$architecture/networking-external.adoc[] + +== Storage + +include::partial$architecture/storage.adoc[] + +== Glossary + +=== Components {infra-type} + +[cols="1,3,1"] +|=== +|Name|Description|provided by + +|Security Group +a|Exoscale Security Groups provide a modular way to define and compose firewall rules. + +Security Groups hold two different types of information: +* A list of rules to apply to traffic +* A list of member instances in the security group which allows using groups as traffic sources or destinations in rules + +See https://community.exoscale.com/documentation/compute/security-groups[Upstream Documentation]. + +|{infra-type} + +|S3 compatible storage +a|Various OpenShift components require S3 compatible storage. +This storage is provided by {infra-type}. + +The main APPUiO Managed OpenShift 4 components that use object storage are + +* OpenShift integrated image registry +* OpenShift logging stack +* APPUiO Managed cluster backups +|{infra-type} + +|=== + +=== Components General + +include::partial$architecture/glossary-general.adoc[] + +=== Other terms + +include::partial$architecture/glossary-others.adoc[] diff --git a/docs/modules/ROOT/partials/architecture/glossary-general.adoc b/docs/modules/ROOT/partials/architecture/glossary-general.adoc index becd3b1e..5ee39d03 100644 --- a/docs/modules/ROOT/partials/architecture/glossary-general.adoc +++ b/docs/modules/ROOT/partials/architecture/glossary-general.adoc @@ -32,9 +32,21 @@ If the service network IP range conflicts with existing subnets, the service net | VSHN / Cilium |DNS -|The APPUiO Managed OpenShift 4 cluster's base DNS records are defined and managed by VSHN. +a|The APPUiO Managed OpenShift 4 cluster's base DNS records are defined and managed by VSHN. All records must be publicly resolvable. To expose applications under a customer domain, a CNAME target is provided. | VSHN + +ifeval::["{infra-type}" == "Exoscale"] +|Storage Cluster +a|The APPUiO Managed Storage Cluster offers advanced cloud-native storage capabilities for APPUiO Managed OpenShift 4. + +This product is based on https://rook.io/[Rook] and uses https://ceph.io/en/[Ceph] as it’s underlying storage technology. + +See https://products.vshn.ch/appuio/managed/storage_cluster.html[APPUiO Managed Storage Cluster] product page for more details. + +| VSHN / Rook +endif::[] + |=== diff --git a/docs/modules/ROOT/partials/architecture/storage.adoc b/docs/modules/ROOT/partials/architecture/storage.adoc index 5a35120e..30b45c9b 100644 --- a/docs/modules/ROOT/partials/architecture/storage.adoc +++ b/docs/modules/ROOT/partials/architecture/storage.adoc @@ -19,7 +19,14 @@ They're allocated dynamically based on requests from workloads (applications or These block devices are automatically attached to the VM hosting the application container. They're deleted when the corresponding Kubernetes `PersistentVolume` resource is deleted. +ifeval::["{infra-type}" != "Exoscale"] The {infra-type} CSI driver is the in-cluster component which is responsible for allocating, attaching and deleting the persistent volume block devices. +endif::[] + +ifeval::["{infra-type}" == "Exoscale"] +NOTE: On the {infra-type} environment there is no Kubernetes integration available for such block devices. +https://products.vshn.ch/appuio/managed/storage_cluster.html[APPUiO Managed Storage Cluster] is available as and addon to the cluster offering if such functionality is required. +endif::[] These devices hold application data, but backups are usually done from within the cluster. @@ -29,6 +36,8 @@ Various OpenShift components, such as the integrated image registry, the logging The customer or vSphere infrastructure operator must provide S3 compatible object storage. Most modern storage solutions offer some object storage functionality. +ifeval::["{infra-type}" != "Exoscale"] If https://products.vshn.ch/appcat/index.html[VSHN's Application Catalog (AppCat)] offering is required on the cluster, the object storage must support automatic bucket creation via an AppCat-supported provisioner. NOTE: If no object storage is available, we can use external object storage as a fallback. +endif::[] diff --git a/docs/modules/ROOT/partials/nav.adoc b/docs/modules/ROOT/partials/nav.adoc index 87b19e7c..749d9893 100644 --- a/docs/modules/ROOT/partials/nav.adoc +++ b/docs/modules/ROOT/partials/nav.adoc @@ -14,6 +14,7 @@ ** xref:oc4:ROOT:references/architecture/single_sign_on.adoc[] ** Exoscale +*** xref:oc4:ROOT:references/exoscale/architecture.adoc[Architecture] *** xref:oc4:ROOT:explanations/exoscale/limitations.adoc[Limitations] ** Google Cloud Platform @@ -39,6 +40,7 @@ *** xref:oc4:ROOT:how-tos/cloudscale/decommission.adoc[Decommissioning] ** Exoscale +*** xref:oc4:ROOT:references/exoscale/architecture.adoc[Architecture] *** xref:oc4:ROOT:references/exoscale/config.adoc[Configuration] *** xref:oc4:ROOT:how-tos/exoscale/install.adoc[Install] // Node management