Backend: Opt-in OIDC JSON Web Token (JWT)

Context

Competitive analysis

This feature will bring us in par with GitHub action OIDC, Circle CI and Jenkins OIDC plugin which allows authenticating your build jobs to different cloud providers using a JWT token, GitLab feature is currently in Alpha, and could be moved to GA once we complete this work.

Background

Currently, whenever a user configures the JWT token it becomes available to all jobs in the pipeline. Tokens are sensitive information and should not be exposed to all jobs without a reason. We would like to prevent this from happening, and allow users to opt-in the token per job basis

Requirements

For any existing or new users:

• JWT token should not be exposed by default
• Users need to opt-in the JWT token in their YML configuration per job basis
• Users should be able to configure their audience claim (aud:)
• aud can be either a string or an array of strings

How does JWT work?

https://drive.google.com/file/d/12k82v78CKp46wZm4J0uk1iHRoKPTYU2c/view

Proposal

Add configurable JWTs

A new id_tokens keyword will allow users to create JWTs that have a custom aud. They will only be available in the job in which they are defined and they can be accessed using a custom variable name. The syntax will look like:

job_name:
  id_tokens:
    CLOUD_PROVIDER_ID_TOKEN: # or any other variable name
      aud:
        - "AWS"
        - "GCK"
    VAULT_ID_TOKEN: # or any other variable name
      aud: "Vault"
  script:
    - auth into AWS with $CLOUD_PROVIDER_ID_TOKEN
    - auth into GCK with $CLOUD_PROVIDER_ID_TOKEN
    - auth into Vault with $VAULT_ID_TOKEN

Add an opt_in_jwt toggle

Add an opt_in_jwt toggle to project CI settings that disables the injection of the JWT into every job when true. It will be false by default. This toggle will default to true in %16.0.

When the toggle is true, CI_JOB_JWT, CI_JOB_JWT_V1, and CI_JOB_JWT_V2 will no longer be created. Jobs with Vault integrations using the secrets keyword must also have an id_token defined. The first id_token will be used to authenticate with Vault

auth_job:
  id_tokens:
    VAULT_JWT_1:  
      aud: 'Vault'
    VAULT_JWT_2:
      aud: 'Another Vault'
  secrets:
    STAGING_DATABASE_PASSWORD:
      vault: staging/db/password@ops # authenticates with VAULT_JWT_1

Optional token sub-keyword

When more than one ID token is defined, the user can specify which ID token should be used to authenticate with Vault using a new token sub-keyword.

auth_job:
  id_tokens:
    VAULT_JWT_1:
      aud: 'Vault'
    VAULT_JWT_2:
      aud: 'Another Vault'
  secrets:
    STAGING_DATABASE_PASSWORD:
      token: $VAULT_JWT_2
      vault: staging/db/password@ops

Implementation Table

Title	MR Link	Done?
Spike	!94403 (closed)	✅
Add id_tokens field to ci_builds_metadata	!94878 (merged)	✅
Pass first ID token to the runner	!95254 (merged)	✅
Add id_tokens keyword to .gitlab-ci.yml	!103391 (merged)	✅
Add token keyword to .gitlab-ci.yml	!103801 (merged)
Document id_tokens keyword	!106052 (merged)	✅