Validate test script against a ocp+proxy setup environment

[prietyc123]

/kind user-story
/area testing

User Story

As a user I want to run odo test script inside a cluster which has proxy setup

Acceptance Criteria

• odo test should run successfully under such scenario

Scope Of Issue:

Verify the steps manually

• Create a flexy job (ocp+proxy)
• Configure the cluster using the make target (make configure-installer....)
• Run the test script

Links

• Feature Request: NA
• Related issue: NA

/kind user-story

[zhengxiaomei123]

Hi @prietyc123 , We did some validation on disconnected env before. About the proxy environment, Is there any difference or special requirements ? Thanks.

[kandarpamalipeddi]

Hi @prietyc123 & @zhengxiaomei123

Following are some of the changes implemented in ODO to make it work for PowerVS cluster (Proxy enabled cluster):

PowerVS is proxy based cluster.

Initially, we used to set the HTTP_PROXY and HTTPS_PROXY variables with odo create command.

-               helper.CmdShouldPass("odo", "create", "--s2i", cmpType, srcType+"-app", "--project", project, "--context", context, "--app", appName)
+               helper.CmdShouldPass("odo", "create", "--s2i", cmpType, srcType+"-app", "--project", project, "--context", context, "--app", appName, "--env", "HTTP_PROXY=<http_proxy_value>", "--env", "HTTPS_PROXY=<http_proxy_value>")

We observed that, odo doesn't allow --env flag with --starter. Hence starter projects failed to execute.

We added changes, to run odo config set after odo create.
With this approach, most of the things works. But odo create … --now will fail.

To take care of proxy environment, thought of implementing code changes inside odo.
There are 2 paths of odo push:

• With --starter (devfile): thought of pushing HTTP_PROXY and HTTPS_PROXY values to containers created by ODO.

	diff --git a/pkg/devfile/adapters/kubernetes/component/adapter.go b/pkg/devfile/adapters/kubernetes/component/adapter.go
	index b3b195e70..d5864abb3 100644
	--- a/pkg/devfile/adapters/kubernetes/component/adapter.go
	+++ b/pkg/devfile/adapters/kubernetes/component/adapter.go
	@@ -284,6 +284,9 @@ func (a Adapter) createOrUpdateComponent(componentExists bool, ei envinfo.EnvSpe
	 		return fmt.Errorf("No valid components found in the devfile")
	 	}
	 
	+	// Add Enviroment variables if available
	+	utils.UpdateProxyVariablesIfAvailable(&containers)
	+
	 	// Add the project volume before generating init containers
	 	utils.AddOdoProjectVolume(&containers)
	
	diff --git a/pkg/devfile/adapters/kubernetes/utils/utils.go b/pkg/devfile/adapters/kubernetes/utils/utils.go
	index a33bcbceb..7d535e472 100644
	--- a/pkg/devfile/adapters/kubernetes/utils/utils.go
	+++ b/pkg/devfile/adapters/kubernetes/utils/utils.go
	@@ -2,6 +2,8 @@ package utils
	 
	 import (
	 	"fmt"
	+	"net/url"
	+	"os"
	 	"strconv"
	 	"strings"
	 
	@@ -81,6 +83,62 @@ func isEnvPresent(EnvVars []corev1.EnvVar, envVarName string) bool {
	 	return isPresent
	 }
	 
	+func UpdateProxyVariablesIfAvailable(containers *[]corev1.Container) {
	+	httpProxy := os.Getenv("Http_Proxy")
	+	httpsProxy := os.Getenv("Https_Proxy")
	+	if httpProxy == "<no value>" || httpsProxy == "<no value>" || httpProxy == "" || httpsProxy == "" {
	+		return
	+	}
	+
	+	httpProxyURL, err := url.Parse(httpProxy)
	+	if err != nil {
	+		return
	+	}
	+
	+	httpsProxyURL, err := url.Parse(httpsProxy)
	+	if err != nil {
	+		return
	+	}
	+
	+	mavenProxyOptions := fmt.Sprintf("-DproxySet=true -Dhttp.proxyHost=%s -Dhttp.proxyPort=%s -Dhttps.proxyHost=%s -Dhttps.proxyPort=%s",
	+		httpProxyURL.Hostname(), httpProxyURL.Port(), httpsProxyURL.Hostname(), httpsProxyURL.Port())
	+
	+	for i, container := range *containers {
	+		isHTTPProxyEnvAvailable := false
	+		isHTTPSProxyEnvAvailable := false
	+		for _, env := range container.Env {
	+			if env.Name == "HTTP_PROXY" {
	+				isHTTPProxyEnvAvailable = true
	+			}
	+			if env.Name == "HTTPS_PROXY" {
	+				isHTTPSProxyEnvAvailable = true
	+			}
	+		}
	+		if !isHTTPProxyEnvAvailable {
	+			fmt.Println("[KP-DBG] Adding HTTP_PROXY as ", httpProxy)
	+			container.Env = append(container.Env, corev1.EnvVar{
	+				Name:  "HTTP_PROXY",
	+				Value: httpProxy,
	+			})
	+		}
	+		if !isHTTPSProxyEnvAvailable {
	+			fmt.Println("[KP-DBG] Adding HTTPS_PROXY as ", httpsProxy)
	+			container.Env = append(container.Env, corev1.EnvVar{
	+				Name:  "HTTPS_PROXY",
	+				Value: httpsProxy,
	+			})
	+		}
	+
	+		fmt.Println("[KP-DBG] Adding MAVEN_OPTS as ", mavenProxyOptions)
	+		container.Env = append(container.Env, corev1.EnvVar{
	+			Name:  "MAVEN_OPTS",
	+			Value: mavenProxyOptions,
	+		})
	+
	+		(*containers)[i] = container
	+	}
	+}
	+
	 // AddOdoProjectVolume adds the odo project volume to the containers
	 func AddOdoProjectVolume(containers *[]corev1.Container) {
	 	for i, container := range *containers {

• With --s2i : Adding Proxy environment variables to config.yaml file.

	diff --git a/pkg/config/config.go b/pkg/config/config.go
	index 651bc56f4..dbfe9ada3 100644
	--- a/pkg/config/config.go
	+++ b/pkg/config/config.go
	@@ -6,6 +6,7 @@ import (
	 	"net/url"
	 	"os"
	 	"path/filepath"
	+	"sort"
	 	"strconv"
	 	"strings"
	 
	@@ -361,6 +362,9 @@ func (lci *LocalConfigInfo) GetComponentSettings() ComponentSettings {
	 
	 // SetComponentSettings sets the componentSettings from to the local config and writes to the file
	 func (lci *LocalConfigInfo) SetComponentSettings(cs ComponentSettings) error {
	+	proxyEnvs := getProxyDetailsIfAvailable()
	+	cs.Envs = proxyEnvs.Merge(cs.Envs)
	+	// Sorting is required to keep env variables in an order. To make test case to pass.
	+	sort.Slice(cs.Envs, func(i, j int) bool { return cs.Envs[i].Name < cs.Envs[j].Name })
	 	lci.componentSettings = cs
	 	return lci.writeToFile()
	 }
	@@ -470,6 +474,33 @@ func (lc *LocalConfig) GetMaxCPU() string {
	 	return util.GetStringOrEmpty(lc.componentSettings.MaxCPU)
	 }
	 
	+func getProxyDetailsIfAvailable() (envList EnvVarList) {
	+	httpProxy := os.Getenv("Http_Proxy")
	+	httpsProxy := os.Getenv("Https_Proxy")
	+	if httpProxy == "<no value>" || httpsProxy == "<no value>" || httpProxy == "" || httpsProxy == "" {
	+		return
	+	}
	+
	+	httpProxyURL, err := url.Parse(httpProxy)
	+	if err != nil {
	+		return
	+	}
	+
	+	httpsProxyURL, err := url.Parse(httpsProxy)
	+	if err != nil {
	+		return
	+	}
	+
	+	mavenProxyOptions := fmt.Sprintf("-DproxySet=true -Dhttp.proxyHost=%s -Dhttp.proxyPort=%s -Dhttps.proxyHost=%s -Dhttps.proxyPort=%s",
	+		httpProxyURL.Hostname(), httpProxyURL.Port(), httpsProxyURL.Hostname(), httpsProxyURL.Port())
	+
	+	envList = append(envList, EnvVar{Name: "HTTP_PROXY", Value: httpProxy})
	+	envList = append(envList, EnvVar{Name: "HTTPS_PROXY", Value: httpsProxy})
	+	envList = append(envList, EnvVar{Name: "MAVEN_OPTS", Value: mavenProxyOptions})
	+
	+	return envList
	+}
	+
	 // GetEnvs returns the Envs, returns empty if nil
	 func (lc *LocalConfig) GetEnvs() EnvVarList {
	 	if lc.componentSettings.Envs == nil {

Notes:

1. Adding MAVEN_OPTS to make sure, there won't be any issues with JAVA projects compiled with maven.
2. Code cleanup is very much required.

With the above changes, all tests are working fine.

Second part of this scenario is getting proxy variables:

Initially tried of getting proxy details inside the odo code using below code:

       type Proxy struct {
              HttpProxy  string `json:"httpProxy"`
              HttpsProxy string `json:"httpsProxy"`
              NoProxy    string `json:"noProxy"`
       }

       proxyDetails := Proxy{}
       coreGet := c.kubeClient.KubeClient.CoreV1().RESTClient().Get()
       rawProxyValues, err := coreGet.AbsPath("/apis/config.openshift.io/v1/proxies").Do().Raw()
       if err != nil {
               // when using Minishift (or plain 'oc cluster up' for that matter) with OKD 3.11, the version endpoint is missing...
               klog.V(3).Infof("Unable to get OpenShift Version - endpoint '/apis/config.openshift.io/v1/proxies' doesn't exist")
               return err
       }
       proxyJSON := map[string]*json.RawMessage{}
       err = json.Unmarshal([]byte(rawProxyValues), &proxyJSON)
       if err != nil {
               klog.V(3).Infof("Failed to parse json data")
               return err
       }
       proxyItems := []map[string]*json.RawMessage{}
       err = json.Unmarshal(*(proxyJSON["items"]), &proxyItems)
       if err != nil {
               klog.V(3).Infof("Failed to parse Proxy items")
               return err
       }
       for _, proxy := range proxyItems {
               if string(*proxy["kind"]) == "\"Proxy\"" {
                       if err = json.Unmarshal(*proxy["status"], &proxyDetails); err != nil {
                               klog.V(3).Infof("Failed to parse status of the proxy")
                               return err
                       }
                       break
               }
       }

Issue with this approach is REST call to /apis/config.openshift.io/v1/proxies can be done by admin users.
Getting error with developer user. All the tests run as developers, so not able to get the proxy details via odo code. To fix this added following lines into scripts/openshiftci-presubmit-all-test.sh

export Http_Proxy=$(oc get proxy -o go-template='{{range .items}}{{.spec.httpProxy}}{{"\n"}}{{end}}')
export Https_Proxy=$(oc get proxy -o go-template='{{range .items}}{{.spec.httpsProxy}}{{"\n"}}{{end}}')

Please let me know your views on the above changes.

[kandarpamalipeddi]

Hi @prietyc123 @dharmit
any update on this issue ?
Please let me know do you have any plans on this issue.

[kandarpamalipeddi]

Hi @prietyc123 @dharmit
any update on this issue ?
Please let me know do you have any plans on this issue.

@dharmit @girishramnani @prietyc123
any update on this ?

[dharmit]

@kandarpamalipeddi at the moment, this isn't a priority. Is this blocking you folks in any way?

@prietyc123 when do you plan to pick this up from the QE side?

[kandarpamalipeddi]

@dharmit, @prietyc123

We are not blocked with this.
Currently our tests are working fine with the work around I mentioned in the Comment.

Do you guys have any plans to implement code changes to make odo work on Proxy based cluster ? Or this story is to track the testing status of the Odo on proxy based cluster ?

[prietyc123]

Do you guys have any plans to implement code changes to make odo work on Proxy based cluster ? Or this story is to track the testing status of the Odo on proxy based cluster ?

@kandarpamalipeddi From testing end this is not the priority currently. Before this issue I guess we need to focus on #4409 as part of QE plan. But again its of low priority currently. As you said work arounds going well for you, sounds like its not a blocker from your end too.
As far as code change is concerned, I am pretty sure that's also not a priority from Dev end. However I would like someone from Dev to chime in and update the status.
@dharmit @kadel @feloy WDYT about comment

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale