-
Notifications
You must be signed in to change notification settings - Fork 459
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MinIOJob does not pull value from secret #2233
Comments
@JamesAtIntegratnIO Could you post the job pod yaml? |
minio-add-user-podapiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2024-07-22T23:07:38Z"
generateName: minio-user-creation-job-add-admin-user-
labels:
batch.kubernetes.io/controller-uid: 33c3d771-5fcd-4900-986a-1d37c28e7589
batch.kubernetes.io/job-name: minio-user-creation-job-add-admin-user
controller-uid: 33c3d771-5fcd-4900-986a-1d37c28e7589
job-name: minio-user-creation-job-add-admin-user
job.min.io/job-name: add-admin-user
name: minio-user-creation-job-add-admin-user-rrws2
namespace: minio-tenant
ownerReferences:
- apiVersion: batch/v1
blockOwnerDeletion: true
controller: true
kind: Job
name: minio-user-creation-job-add-admin-user
uid: 33c3d771-5fcd-4900-986a-1d37c28e7589
resourceVersion: "360229"
uid: 17ed03b3-5d16-4326-9925-b3bf14883675
spec:
containers:
- command:
- mc
- admin
- user
- add
- myminio
- ${USERNAME}
- ${PASSWORD}
envFrom:
- secretRef:
name: minio-user-creation-job-job-secret
- secretRef:
name: minio-tenant-user
image: quay.io/minio/mc:RELEASE.2024-07-11T18-01-28Z
imagePullPolicy: IfNotPresent
name: mc
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /.mc
name: config-dir
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-kvctb
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: monitoring-2
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: OnFailure
schedulerName: default-scheduler
securityContext: {}
serviceAccount: mc-job-sa
serviceAccountName: mc-job-sa
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- emptyDir: {}
name: config-dir
- name: kube-api-access-kvctb
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:41Z"
status: "False"
type: PodReadyToStartContainers
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:38Z"
reason: PodCompleted
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:38Z"
reason: PodCompleted
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:38Z"
reason: PodCompleted
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:38Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: containerd://bd5f82e5630239606c0d8329f6f9fc86ea6e8126608a5d654932c8ae36ee81c7
image: quay.io/minio/mc:RELEASE.2024-07-11T18-01-28Z
imageID: quay.io/minio/mc@sha256:b48f3b90fa33f8c6f3a9c365e3c8b8c3e9e2b062ed1f8e1dae458025866dcae9
lastState: {}
name: mc
ready: false
restartCount: 0
started: false
state:
terminated:
containerID: containerd://bd5f82e5630239606c0d8329f6f9fc86ea6e8126608a5d654932c8ae36ee81c7
exitCode: 0
finishedAt: "2024-07-22T23:07:40Z"
reason: Completed
startedAt: "2024-07-22T23:07:39Z"
hostIP: 10.0.5.102
hostIPs:
- ip: 10.0.5.102
phase: Succeeded
podIP: 10.244.1.52
podIPs:
- ip: 10.244.1.52
qosClass: BestEffort
startTime: "2024-07-22T23:07:38Z" admin-policy-createapiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2024-07-22T23:07:38Z"
generateName: minio-user-creation-job-admin-policy-create-
labels:
batch.kubernetes.io/controller-uid: f7ce2818-7977-4177-8bbf-0ed66789d1fb
batch.kubernetes.io/job-name: minio-user-creation-job-admin-policy-create
controller-uid: f7ce2818-7977-4177-8bbf-0ed66789d1fb
job-name: minio-user-creation-job-admin-policy-create
job.min.io/job-name: admin-policy-create
name: minio-user-creation-job-admin-policy-create-zfddw
namespace: minio-tenant
ownerReferences:
- apiVersion: batch/v1
blockOwnerDeletion: true
controller: true
kind: Job
name: minio-user-creation-job-admin-policy-create
uid: f7ce2818-7977-4177-8bbf-0ed66789d1fb
resourceVersion: "360233"
uid: ad8b4d4f-15ed-470f-95ff-ea2572b67a06
spec:
containers:
- command:
- mc
- admin
- policy
- create
- myminio
- admin-access
- /temp/policy.json
envFrom:
- secretRef:
name: minio-user-creation-job-job-secret
image: quay.io/minio/mc:RELEASE.2024-07-11T18-01-28Z
imagePullPolicy: IfNotPresent
name: mc
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /.mc
name: config-dir
- mountPath: /temp
name: policy
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-lttmt
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: monitoring-2
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: OnFailure
schedulerName: default-scheduler
securityContext: {}
serviceAccount: mc-job-sa
serviceAccountName: mc-job-sa
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- emptyDir: {}
name: config-dir
- configMap:
defaultMode: 420
items:
- key: policy.json
path: policy.json
name: mc-admin-policy
name: policy
- name: kube-api-access-lttmt
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:41Z"
status: "False"
type: PodReadyToStartContainers
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:38Z"
reason: PodCompleted
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:38Z"
reason: PodCompleted
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:38Z"
reason: PodCompleted
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:38Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: containerd://cfdff9e54913610ca896e78192cf87a1bc5f712ba521c317618a66705aecc7e3
image: quay.io/minio/mc:RELEASE.2024-07-11T18-01-28Z
imageID: quay.io/minio/mc@sha256:b48f3b90fa33f8c6f3a9c365e3c8b8c3e9e2b062ed1f8e1dae458025866dcae9
lastState: {}
name: mc
ready: false
restartCount: 0
started: false
state:
terminated:
containerID: containerd://cfdff9e54913610ca896e78192cf87a1bc5f712ba521c317618a66705aecc7e3
exitCode: 0
finishedAt: "2024-07-22T23:07:40Z"
reason: Completed
startedAt: "2024-07-22T23:07:39Z"
hostIP: 10.0.5.102
hostIPs:
- ip: 10.0.5.102
phase: Succeeded
podIP: 10.244.1.53
podIPs:
- ip: 10.244.1.53
qosClass: BestEffort
startTime: "2024-07-22T23:07:38Z" admin-policy-attachapiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2024-07-22T23:07:43Z"
finalizers:
- batch.kubernetes.io/job-tracking
generateName: minio-user-creation-job-admin-policy-attach-
labels:
batch.kubernetes.io/controller-uid: b5d586c7-8783-43c5-8d51-c3aba876435f
batch.kubernetes.io/job-name: minio-user-creation-job-admin-policy-attach
controller-uid: b5d586c7-8783-43c5-8d51-c3aba876435f
job-name: minio-user-creation-job-admin-policy-attach
job.min.io/job-name: admin-policy-attach
name: minio-user-creation-job-admin-policy-attach-x4lp7
namespace: minio-tenant
ownerReferences:
- apiVersion: batch/v1
blockOwnerDeletion: true
controller: true
kind: Job
name: minio-user-creation-job-admin-policy-attach
uid: b5d586c7-8783-43c5-8d51-c3aba876435f
resourceVersion: "360731"
uid: 256459ce-2bb4-492c-9aa1-6bef915f2156
spec:
containers:
- command:
- mc
- admin
- policy
- attach
- myminio
- admin-access
- --user
- ${USERNAME}
envFrom:
- secretRef:
name: minio-user-creation-job-job-secret
- secretRef:
name: minio-tenant-user
image: quay.io/minio/mc:RELEASE.2024-07-11T18-01-28Z
imagePullPolicy: IfNotPresent
name: mc
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /.mc
name: config-dir
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-qwlps
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: monitoring-2
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: OnFailure
schedulerName: default-scheduler
securityContext: {}
serviceAccount: mc-job-sa
serviceAccountName: mc-job-sa
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- emptyDir: {}
name: config-dir
- name: kube-api-access-qwlps
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:45Z"
status: "True"
type: PodReadyToStartContainers
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:43Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:09:17Z"
message: 'containers with unready status: [mc]'
reason: ContainersNotReady
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:09:17Z"
message: 'containers with unready status: [mc]'
reason: ContainersNotReady
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2024-07-22T23:07:43Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: containerd://a8b4f49adc6c08c1213ad6a5c47a1251ebf95a3ff23326bc6a0fd38006748658
image: quay.io/minio/mc:RELEASE.2024-07-11T18-01-28Z
imageID: quay.io/minio/mc@sha256:b48f3b90fa33f8c6f3a9c365e3c8b8c3e9e2b062ed1f8e1dae458025866dcae9
lastState:
terminated:
containerID: containerd://a8b4f49adc6c08c1213ad6a5c47a1251ebf95a3ff23326bc6a0fd38006748658
exitCode: 1
finishedAt: "2024-07-22T23:09:16Z"
reason: Error
startedAt: "2024-07-22T23:09:16Z"
name: mc
ready: false
restartCount: 4
started: false
state:
waiting:
message: back-off 1m20s restarting failed container=mc pod=minio-user-creation-job-admin-policy-attach-x4lp7_minio-tenant(256459ce-2bb4-492c-9aa1-6bef915f2156)
reason: CrashLoopBackOff
hostIP: 10.0.5.102
hostIPs:
- ip: 10.0.5.102
phase: Running
podIP: 10.244.1.54
podIPs:
- ip: 10.244.1.54
qosClass: BestEffort
startTime: "2024-07-22T23:07:43Z" secret details
boboysdadda@GameRig:~/projects/gitops-homelab$ kubectl -n minio-tenant get secrets minio-tenant-user -o yaml
apiVersion: v1
data:
PASSWORD: anVzdGljZTEyMw==
USERNAME: YWRtaW4=
kind: Secret
metadata:
annotations:
operator.1password.io/item-path: vaults/dont/need/this/path
operator.1password.io/item-version: "5"
creationTimestamp: "2024-07-22T20:43:24Z"
labels:
argocd.argoproj.io/instance: minio-tenant
name: minio-tenant-user
namespace: minio-tenant
ownerReferences:
- apiVersion: onepassword.com/v1
kind: OnePasswordItem
name: minio-tenant-user
uid: b45bd8fb-86f8-4ec0-98e5-57bfd56e755e
resourceVersion: "324633"
uid: beb153b8-dd20-42bd-9c1b-9078a513faac
type: Opaque |
Please share all the job pod logs @JamesAtIntegratnIO |
add-user
policy create
policy attach
The error you are seeing in the above policy attach pod has been corrected. That was from running the job multiple times without deleting the policy attachment. Obviously it can't attach a policy that is already attached. But it would be nice if it handled that error better rather than failing the pod. polic attach without fail apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2024-07-23T00:19:38Z"
generateName: minio-user-creation-job-admin-policy-attach-
labels:
batch.kubernetes.io/controller-uid: 2805397a-cc4a-4ee7-ad1a-5dc994940400
batch.kubernetes.io/job-name: minio-user-creation-job-admin-policy-attach
controller-uid: 2805397a-cc4a-4ee7-ad1a-5dc994940400
job-name: minio-user-creation-job-admin-policy-attach
job.min.io/job-name: admin-policy-attach
name: minio-user-creation-job-admin-policy-attach-9z4cb
namespace: minio-tenant
ownerReferences:
- apiVersion: batch/v1
blockOwnerDeletion: true
controller: true
kind: Job
name: minio-user-creation-job-admin-policy-attach
uid: 2805397a-cc4a-4ee7-ad1a-5dc994940400
resourceVersion: "378228"
uid: 8926e715-2039-42fd-9ea5-1e49668757d6
spec:
containers:
- command:
- mc
- admin
- policy
- attach
- myminio
- admin-access
- --user
- ${USERNAME}
env:
- name: USERNAME
valueFrom:
secretKeyRef:
key: USERNAME
name: minio-tenant-user
- name: PASSWORD
valueFrom:
secretKeyRef:
key: PASSWORD
name: minio-tenant-user
envFrom:
- secretRef:
name: minio-user-creation-job-job-secret
image: quay.io/minio/mc:RELEASE.2024-07-11T18-01-28Z
imagePullPolicy: IfNotPresent
name: mc
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /.mc
name: config-dir
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-tc4nm
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: monitoring-2
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: OnFailure
schedulerName: default-scheduler
securityContext: {}
serviceAccount: mc-job-sa
serviceAccountName: mc-job-sa
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- emptyDir: {}
name: config-dir
- name: kube-api-access-tc4nm
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2024-07-23T00:19:41Z"
status: "False"
type: PodReadyToStartContainers
- lastProbeTime: null
lastTransitionTime: "2024-07-23T00:19:38Z"
reason: PodCompleted
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2024-07-23T00:19:40Z"
reason: PodCompleted
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2024-07-23T00:19:40Z"
reason: PodCompleted
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2024-07-23T00:19:38Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: containerd://ed635b458ca66e9eb4b455223dcb3b423ad4c28fae597173a0df76f60e22c73a
image: quay.io/minio/mc:RELEASE.2024-07-11T18-01-28Z
imageID: quay.io/minio/mc@sha256:b48f3b90fa33f8c6f3a9c365e3c8b8c3e9e2b062ed1f8e1dae458025866dcae9
lastState: {}
name: mc
ready: false
restartCount: 0
started: false
state:
terminated:
containerID: containerd://ed635b458ca66e9eb4b455223dcb3b423ad4c28fae597173a0df76f60e22c73a
exitCode: 0
finishedAt: "2024-07-23T00:19:39Z"
reason: Completed
startedAt: "2024-07-23T00:19:39Z"
hostIP: 10.0.5.102
hostIPs:
- ip: 10.0.5.102
phase: Succeeded
podIP: 10.244.1.69
podIPs:
- ip: 10.244.1.69
qosClass: BestEffort
startTime: "2024-07-23T00:19:38Z" |
@JamesAtIntegratnIO Could you use |
I'm in the minio slack if you'd like to chat through this if its easier. Same name |
output of user create
|
@JamesAtIntegratnIO Checked. doc error. Should use |
That did it. Thanks. |
I'm attempting to create an admin user with a MinIOJob, but the secrets are never used. Instead it creates a user of
${USERNAME}
Expected Behavior
secrets reference and env vars parsed for values
Current Behavior
env vars are not parsed and user of ${USERNAME} is created
Possible Solution
Steps to Reproduce (for bugs)
job.yaml
job.yaml
secret.yaml
Tenant Volume info
Context
Regression
Your Environment
minio-operator
):uname -a
):The text was updated successfully, but these errors were encountered: