Initial E2E Tests #387

jfaltermeier · 2024-12-18T14:08:35Z

Use squash on merge

This PR establishes a foundation for implementing E2E tests.

It has a matrix GitHub workflow that combines different configurations: Kubernetes versions, ephemeral vs. persistent storage, subdomain vs. path-based routing, and Keycloak enablement.

Currently, the tests are just checking login/logout and starting a session, but this provides a base that can be extended with additional tests.

It also fixes an issue with the MinikubePersistentVolumeCreator that caused an error.

Test Run: https://github.com/eclipse-theia/theia-cloud/actions/runs/12413770653

Contributed on behalf of STMicroelectronics

This will through an error otherwise

.github/workflows/e2e-tests.yml

This reverts commit d86d7d6.

lucas-koehler

Hi @jfaltermeier , Thanks for providing this vast improvement in testing 🥳

The changes already look pretty good to me. I only have a few small remarks inline.

lucas-koehler · 2025-01-13T11:48:59Z

.github/workflows/e2e-tests.yml

+          path: "./theia-cloud-helm"
+
+      - name: Setup Minikube
+        uses: manusa/actions-setup-minikube@92af4db914ab207f837251cd53eb7060e6477614


Minor: add comment which version this commit hash resolves to.

lucas-koehler · 2025-01-13T11:53:00Z

.github/workflows/e2e-tests.yml

+
+      - name: Enable Minikube Addons
+        run: |
+          minikube addons enable dashboard


What is the dashboard needed for in E2E tests?

Good point, I always enable it locally for debugging, so this is how I copied the instruction here as well. But it shouldn't be required, unless it also has some side-effect, I'll disable it and trigger a test run.

lucas-koehler · 2025-01-13T12:06:21Z

node/e2e-tests/src/constats.ts

I think the file name contains a typo and the file should be named constants.ts?

lucas-koehler · 2025-01-13T13:53:11Z

terraform/ci-configurations/e2e_tests.tf

+
+  name             = "theia-cloud-crds"
+  chart            = "../../../theia-cloud-helm/charts/theia-cloud-crds"
+  namespace        = "theiacloud"


Small detail: AFAIK since we aligned all namings, our default namespace name is theia-cloud. Should we adapt this here for the sake of consistency?
If yes, also adapt below

lucas-koehler · 2025-01-13T13:55:05Z

terraform/ci-configurations/run-locally.md

+kubectl -n ingress-nginx delete pod -l app.kubernetes.io/name=ingress-nginx
+```
+
+Adapt your environment so that all docker images are built in minikube. Build all Theia Cloud docker images + Demos with tag `minikube-ci-e2e`, e.g. `theiacloud/theia-cloud-service:minikube-ci-e2e`.


We should add a note that the docker build commands need to be executed in the repository root to avoid confusioon

.github/workflows/e2e-tests.yml

jfaltermeier · 2025-01-24T14:01:28Z

ephemeral false, keycloak true tests seem to fail atm, I'll check

jfaltermeier · 2025-01-24T14:59:38Z

Locally I can reproduce the errors. Nice to see that the tests are helping :D

14:56:59.220 [-1717433286-pool-6-thread-1] ERROR org.eclipse.theia.cloud.operator.handler.session.LazySessionHandler - [fdc4ef7b-a0ee-4d1e-aaf5-963c27bd3219] An unexpected exception occurred while adding Session: name=ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session version=3190 value=SessionSpec [name=ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session, appDefinition=theia-cloud-monitor-theia, [email protected], workspace=ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io]
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://10.96.0.1:443/api/v1/namespaces/theia-cloud/services. Message: Service "session-foo-theia-cloud-monit-8e3bcf8dedab" is invalid: metadata.labels: Invalid value: "ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session": must be no more than 63 characters. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=metadata.labels, message=Invalid value: "ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session": must be no more than 63 characters, reason=FieldValueInvalid, additionalProperties={})], group=null, kind=Service, name=session-foo-theia-cloud-monit-8e3bcf8dedab, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Service "session-foo-theia-cloud-monit-8e3bcf8dedab" is invalid: metadata.labels: Invalid value: "ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session": must be no more than 63 characters, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).
	at io.fabric8.kubernetes.client.KubernetesClientException.copyAsCause(KubernetesClientException.java:238) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.waitForResult(OperationSupport.java:507) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleResponse(OperationSupport.java:524) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleCreate(OperationSupport.java:340) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.handleCreate(BaseOperation.java:754) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.handleCreate(BaseOperation.java:98) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.CreateOnlyResourceOperation.create(CreateOnlyResourceOperation.java:42) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.create(BaseOperation.java:1155) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.create(BaseOperation.java:98) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at org.eclipse.theia.cloud.operator.util.K8sUtil.loadAndCreateTypeWithOwnerReference(K8sUtil.java:198) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at org.eclipse.theia.cloud.operator.util.K8sUtil.loadAndCreateServiceWithOwnerReference(K8sUtil.java:125) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at org.eclipse.theia.cloud.operator.handler.session.LazySessionHandler.createAndApplyService(LazySessionHandler.java:394) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at org.eclipse.theia.cloud.operator.handler.session.LazySessionHandler.doSessionAdded(LazySessionHandler.java:208) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at org.eclipse.theia.cloud.operator.handler.session.LazySessionHandler.sessionAdded(LazySessionHandler.java:98) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at org.eclipse.theia.cloud.operator.BasicTheiaCloudOperator.handleSessionEvent(BasicTheiaCloudOperator.java:188) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at org.eclipse.theia.cloud.operator.util.SpecWatch.eventReceived(SpecWatch.java:86) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at org.eclipse.theia.cloud.operator.util.SpecWatch.eventReceived(SpecWatch.java:32) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.utils.internal.WatcherToggle.eventReceived(WatcherToggle.java:48) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager$SerialWatcher.lambda$eventReceived$0(AbstractWatchManager.java:70) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.utils.internal.SerialExecutor.lambda$execute$0(SerialExecutor.java:57) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
	at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://10.96.0.1:443/api/v1/namespaces/theia-cloud/services. Message: Service "session-foo-theia-cloud-monit-8e3bcf8dedab" is invalid: metadata.labels: Invalid value: "ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session": must be no more than 63 characters. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=metadata.labels, message=Invalid value: "ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session": must be no more than 63 characters, reason=FieldValueInvalid, additionalProperties={})], group=null, kind=Service, name=session-foo-theia-cloud-monit-8e3bcf8dedab, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Service "session-foo-theia-cloud-monit-8e3bcf8dedab" is invalid: metadata.labels: Invalid value: "ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session": must be no more than 63 characters, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:660) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:640) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.assertResponseCode(OperationSupport.java:589) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.lambda$handleResponse$0(OperationSupport.java:549) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(Unknown Source) ~[?:?]
	at java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[?:?]
	at java.util.concurrent.CompletableFuture.complete(Unknown Source) ~[?:?]
	at io.fabric8.kubernetes.client.http.StandardHttpClient.lambda$completeOrCancel$10(StandardHttpClient.java:141) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source) ~[?:?]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(Unknown Source) ~[?:?]
	at java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[?:?]
	at java.util.concurrent.CompletableFuture.complete(Unknown Source) ~[?:?]
	at io.fabric8.kubernetes.client.http.ByteArrayBodyHandler.onBodyDone(ByteArrayBodyHandler.java:51) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source) ~[?:?]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(Unknown Source) ~[?:?]
	at java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[?:?]
	at java.util.concurrent.CompletableFuture.complete(Unknown Source) ~[?:?]
	at io.fabric8.kubernetes.client.okhttp.OkHttpClientImpl$OkHttpAsyncBody.doConsume(OkHttpClientImpl.java:136) ~[defaultoperator-1.1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	... 3 more
14:56:59.236 [-1717433286-pool-6-thread-3] TRACE org.eclipse.theia.cloud.operator.util.SpecWatch - [session-watch-a7c2700f-0a4b-4111-9303-d1160c89c0ec] Session 168f36ef-4e6b-449f-a840-8e3bcf8dedab : received an event: MODIFIED
14:56:59.236 [-1717433286-pool-6-thread-3] TRACE org.eclipse.theia.cloud.operator.util.SpecWatch - [session-watch-a7c2700f-0a4b-4111-9303-d1160c89c0ec] Session 168f36ef-4e6b-449f-a840-8e3bcf8dedab : already known. Check if outdated event
14:56:59.236 [-1717433286-pool-6-thread-3] TRACE org.eclipse.theia.cloud.operator.util.SpecWatch - [session-watch-a7c2700f-0a4b-4111-9303-d1160c89c0ec] Session 168f36ef-4e6b-449f-a840-8e3bcf8dedab : event is NOT outdated. Handle event
14:56:59.236 [-1717433286-pool-6-thread-1] TRACE org.eclipse.theia.cloud.operator.util.SpecWatch - [session-watch-6d0c4a5a-3192-4d75-9b69-e6dcbe25703a] Session 168f36ef-4e6b-449f-a840-8e3bcf8dedab : received an event: MODIFIED
14:56:59.236 [-1717433286-pool-6-thread-1] TRACE org.eclipse.theia.cloud.operator.util.SpecWatch - [session-watch-6d0c4a5a-3192-4d75-9b69-e6dcbe25703a] Session 168f36ef-4e6b-449f-a840-8e3bcf8dedab : already known. Check if outdated event
14:56:59.236 [-1717433286-pool-6-thread-1] TRACE org.eclipse.theia.cloud.operator.util.SpecWatch - [session-watch-6d0c4a5a-3192-4d75-9b69-e6dcbe25703a] Session 168f36ef-4e6b-449f-a840-8e3bcf8dedab : event is NOT outdated. Handle event
14:57:52.492 [pool-7-thread-1] DEBUG org.eclipse.theia.cloud.operator.plugins.MonitorActivityTracker - Pinging sessions: []
14:57:52.514 [pool-5-thread-1] TRACE org.eclipse.theia.cloud.operator.BasicTheiaCloudOperator - [timeout-d1b10702-0648-49cb-a991-d08a3fc7a6a2] App Definition watch was idle for 109445 ms
14:57:52.514 [pool-5-thread-1] TRACE org.eclipse.theia.cloud.operator.BasicTheiaCloudOperator - [timeout-d1b10702-0648-49cb-a991-d08a3fc7a6a2] Workspace watch was idle for 53537 ms
14:57:52.514 [pool-5-thread-1] TRACE org.eclipse.theia.cloud.operator.BasicTheiaCloudOperator - [timeout-d1b10702-0648-49cb-a991-d08a3fc7a6a2] Session watch was idle for 53278 ms
14:57:52.518 [pool-4-thread-1] TRACE org.eclipse.theia.cloud.operator.BasicTheiaCloudOperator - [f835c09d-4cba-455a-90f6-e2e3c869172b] Checking ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session. minutesSinceLastActivity: 0. limit: 6
14:57:52.519 [pool-4-thread-1] TRACE org.eclipse.theia.cloud.operator.BasicTheiaCloudOperator - [timeout-f835c09d-4cba-455a-90f6-e2e3c869172b] Session ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session will keep running until the limit of 6 is hit.

Looks like some labels may grow too long now.

I think it tries to apply this:

apiVersion: "v1"
kind: "Service"
metadata:
  labels:
    app: "ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-sess"
    theia-cloud.io/app-definition: "theia-cloud-monitor-theia"
    app.kubernetes.io/part-of: "theia-cloud"
    theia-cloud.io/user: "foo_at_theia_cloud_io"
    theia-cloud.io/session: "ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session"
    app.kubernetes.io/component: "session"
  name: "session-foo-theia-cloud-monit-8e3bcf8dedab"
  namespace: "theia-cloud"
  ownerReferences:
  - apiVersion: "theia.cloud/v1beta8"
    kind: "Session"
    name: "ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session"
    uid: "168f36ef-4e6b-449f-a840-8e3bcf8dedab"
spec:
  ports:
  - name: "http"
    port: 3000
    protocol: "TCP"
    targetPort: "web"
  selector:
    app: "ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-sess"
  type: "ClusterIP"

Session spec for reference:

 apiVersion: theia.cloud/v1beta8
kind: Session
metadata:
  creationTimestamp: '2025-01-24T15:01:59Z'
  generation: 1
  managedFields:
    - apiVersion: theia.cloud/v1beta8
      fieldsType: FieldsV1
      fieldsV1:
        f:spec:
          .: {}
          f:appDefinition: {}
          f:envVars: {}
          f:envVarsFromConfigMaps: {}
          f:envVarsFromSecrets: {}
          f:name: {}
          f:sessionSecret: {}
          f:user: {}
          f:workspace: {}
      manager: fabric8-kubernetes-client
      operation: Update
      time: '2025-01-24T15:01:59Z'
    - apiVersion: theia.cloud/v1beta8
      fieldsType: FieldsV1
      fieldsV1:
        f:status:
          .: {}
          f:lastActivity: {}
          f:operatorMessage: {}
          f:operatorStatus: {}
      manager: fabric8-kubernetes-client
      operation: Update
      subresource: status
      time: '2025-01-24T15:01:59Z'
  name: ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session
  namespace: theia-cloud
  resourceVersion: '3833'
  uid: 7e4b093a-88fe-484f-a5de-3b2f8429cf44
spec:
  appDefinition: theia-cloud-monitor-theia
  envVars: {}
  envVarsFromConfigMaps: []
  envVarsFromSecrets: []
  name: ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io-session
  sessionSecret: f027708d-4f38-43b6-b25a-82a124c7ad4e
  user: [email protected]
  workspace: ws-asdfghjkl-theia-cloud-monitor-theia-foo-theia-cloud-io
status:
  lastActivity: 0
  operatorMessage: >-
    Unexpected error. Please check the logs for correlationId:
    cf0acace-6de9-4242-9c21-ab8641302626
  operatorStatus: ERROR

jfaltermeier added 4 commits December 18, 2024 13:31

E2E Testing

2014e84

Contributed on behalf of STMicroelectronics

Dont' add workspace as owner reference on persistent volume

f3b94bb

This will through an error otherwise

Update triggers and tested kubernetes versions

cfe21cf

Trigger Test run

d86d7d6

jfaltermeier commented Dec 19, 2024

View reviewed changes

.github/workflows/e2e-tests.yml Outdated Show resolved Hide resolved

jfaltermeier force-pushed the e2e-testing branch from 6d54450 to 27e2119 Compare December 19, 2024 13:12

Refactor test code

c4232a9

jfaltermeier force-pushed the e2e-testing branch from 27e2119 to c4232a9 Compare December 19, 2024 13:35

jfaltermeier added 2 commits December 19, 2024 14:40

Revert "Trigger Test run"

30a1935

This reverts commit d86d7d6.

Change tests to fixed version number

510739a

jfaltermeier marked this pull request as ready for review December 19, 2024 15:06

jfaltermeier requested review from sgraband and lucas-koehler December 19, 2024 15:07

lucas-koehler removed the request for review from sgraband January 13, 2025 11:44

lucas-koehler requested changes Jan 13, 2025

View reviewed changes

jfaltermeier added 5 commits January 24, 2025 13:51

Merge branch 'main' into x/jf/e2e-testing

2b9ee2d

Trigger Test run

99f3cc7

Update Kubernetes Versions

2e9f09b

Adress Review Comments

d1c8c2f

Fix namespace

de43d40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial E2E Tests #387

Initial E2E Tests #387

jfaltermeier commented Dec 18, 2024 •

edited

Loading

lucas-koehler left a comment

lucas-koehler Jan 13, 2025

lucas-koehler Jan 13, 2025

jfaltermeier Jan 24, 2025

lucas-koehler Jan 13, 2025

lucas-koehler Jan 13, 2025

lucas-koehler Jan 13, 2025

jfaltermeier commented Jan 24, 2025

jfaltermeier commented Jan 24, 2025 •

edited

Loading

Initial E2E Tests #387

Are you sure you want to change the base?

Initial E2E Tests #387

Conversation

jfaltermeier commented Dec 18, 2024 • edited Loading

lucas-koehler left a comment

Choose a reason for hiding this comment

lucas-koehler Jan 13, 2025

Choose a reason for hiding this comment

lucas-koehler Jan 13, 2025

Choose a reason for hiding this comment

jfaltermeier Jan 24, 2025

Choose a reason for hiding this comment

lucas-koehler Jan 13, 2025

Choose a reason for hiding this comment

lucas-koehler Jan 13, 2025

Choose a reason for hiding this comment

lucas-koehler Jan 13, 2025

Choose a reason for hiding this comment

jfaltermeier commented Jan 24, 2025

jfaltermeier commented Jan 24, 2025 • edited Loading

jfaltermeier commented Dec 18, 2024 •

edited

Loading

jfaltermeier commented Jan 24, 2025 •

edited

Loading