argo service account has no permission to create configmap

[jingkkkkai]

Pre-requisites

• I have double-checked my configuration
• I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
• I have searched existing issues and could not find a match for this bug
• I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

in our case, we encountered error when creating a workflow contains a large size of raw data input artifacts
the error message is shown as following:

configmaps is forbidden: User "system:serviceaccount:our-ns:argo" cannot create resource "configmaps" in API group "" in the namespace "our-ns"

based on the argo codebase,
when creating a pod, the workflow-controller checks the length of each template’s environment variables.
Ref: workflowpod.go#L424

If it exceeds the maxEnvVarLen (131072), the environment variables are passed to the pod via a ConfigMap instead.
Ref: workflowpod.go#L431

However, it seems that the service account used by the workflow-controller, ‘argo,’ doesn’t have permission to create ConfigMaps.
Ref: workflow-controller-rbac/workflow-controller-role.yaml#L30

Can anyone help with this issue?

Version(s)

v3.6.2

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflow that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: large-input-artifact-
spec:
  entrypoint: main
  templates:
  - name: main
    inputs:
      artifacts:
      - name: large-input
        path: /tmp/input-data.txt
        raw:
          data: |
            Lorem ipsum dolor sit amet, consectetur adipiscing elit. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Lorem ipsum dolor sit amet, consectetur adipiscing elit. 
            <<Repeat until more than 131072 characters>>
            
    container:
      image: alpine:3.16
      command: [sh, -c]
      args:
        - |
          echo "Processing large input artifact...";
          cat /tmp/input-data.txt | wc -c;
          echo "Data successfully processed.";

Logs from the workflow controller

kubectl logs -n argo deploy/workflow-controller | grep ${workflow}

Logs from in your workflow's wait container

kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

[jswxstw]

Your analysis is correct; this issue is indeed caused by the default enabling of offloading template environment to ConfigMap by #12325.

[shuangkun]

It looks like this permission is missing, maybe we need to add documentation. Strange，TestWorkflowWithLongArguments can pass.

[jswxstw]

It looks like this permission is missing, maybe we need to add documentation. Strange，TestWorkflowWithLongArguments can pass.

@shuangkun It seems that the unit tests do not have permission control. I also tested it locally, TestWorkflowWithLongArguments also works well.