Bug: moving project variables to separate resource combined with custom environment results in error

[TowardsDeath]

Description

When moving environment variables from vercel_project.environment to a separate resource vercel_project_environment_variables, the apply will error:

│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to
│ module.vercel.vercel_project_environment_variables.web_app, provider
│ "module.vercel.provider[\"registry.terraform.io/vercel/vercel\"]" produced
│ an unexpected new value: .variables: planned set element
│ cty.ObjectVal(map[string]cty.Value{"comment":cty.UnknownVal(cty.String),
│ "custom_environment_ids":cty.SetVal([]cty.Value{cty.StringVal("env_MDG5cRT4IrTNeP7u9dBWJgQoMPRF")}),
│ "git_branch":cty.NullVal(cty.String), "id":cty.UnknownVal(cty.String),
│ "key":cty.StringVal("VAR_NAME"),
│ "sensitive":cty.UnknownVal(cty.Bool),
│ "target":cty.UnknownVal(cty.Set(cty.String)),
│ "value":cty.StringVal("http://staging.domain.com/")}) does
│ not correlate with any element in actual.
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.

Current behaviour

The custom environment is created, but the plan results in the above error. Re-running plan does not help:

module.vercel.vercel_project_environment_variables.web_app is tainted, so must be replaced

and then proceeds to try to delete and recreate all environment variables, which fails again with the above error.

Expected behaviour

I would expect the custom environment to be created without errors.

Context

I had a project with environment variables in vercel_project.environment. (Please don't attack my terrible Terraform code to split an env file into separate variables.)

Code

data "aws_ssm_parameter" "web_app_development_env_file" {
  name = "/vercel/web-app/development/ENV_FILE"
}

data "aws_ssm_parameter" "web_app_preview_env_file" {
  name = "/vercel/web-app/development/ENV_FILE"
}

data "aws_ssm_parameter" "web_app_production_env_file" {
  name = "/vercel/web-app/production/ENV_FILE"
}

resource "vercel_project" "web_app" {
  name                       = "web-app"
  framework                  = "nextjs"
  team_id                    = var.team_id
  serverless_function_region = var.serverless_function_region
  skew_protection = "12 hours"

  environment = concat([
    for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_development_env_file.value)) : {
      key    = split("=", line)[0]
      value  = split("=", line)[1]
      target = ["development"]
    }
    ], [
    for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_preview_env_file.value)) : {
      key    = split("=", line)[0]
      value  = split("=", line)[1]
      target = ["preview"]
    }
    ], [
    for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_production_env_file.value)) : {
      key    = split("=", line)[0]
      value  = split("=", line)[1]
      target = ["production"]
    }
  ])

  vercel_authentication = {
    deployment_type = "standard_protection"
  }
}

resource "vercel_project_deployment_retention" "web_app" {
  project_id            = vercel_project.web_app.id
  team_id               = vercel_project.web_app.team_id
  expiration_preview    = "3m"
  expiration_production = "unlimited"
  expiration_canceled   = "1m"
  expiration_errored    = "1m"
}

resource "vercel_project_domain" "web_app_staging" {
  project_id = vercel_project.web_app.id
  domain     = "staging.domain.com"
}

resource "vercel_project_domain" "web_app_production" {
  project_id = vercel_project.web_app.id
  domain     = "domain.com"
}

I wanted to add a custom environment. In order to prevent a circular reference between the project and the custom environment, I had to move the variables to a separate vercel_project_environment_variables resource.

Code after adding custom environment and moving variables

data "aws_ssm_parameter" "web_app_development_env_file" {
  name = "/vercel/web-app/development/ENV_FILE"
}

data "aws_ssm_parameter" "web_app_preview_env_file" {
  name = "/vercel/web-app/preview/ENV_FILE"
}

data "aws_ssm_parameter" "web_app_staging_env_file" {
  name = "/vercel/web-app/staging/ENV_FILE"
}

data "aws_ssm_parameter" "web_app_production_env_file" {
  name = "/vercel/web-app/production/ENV_FILE"
}

resource "vercel_project" "web_app" {
  name                       = "web-app"
  framework                  = "nextjs"
  team_id                    = var.team_id
  serverless_function_region = var.serverless_function_region
  skew_protection = "12 hours"

  vercel_authentication = {
    deployment_type = "standard_protection"
  }
}

resource "vercel_custom_environment" "web_app_staging" {
  project_id  = vercel_project.web_app.id
  name        = "web-app-staging"
  description = "Long-running testing environment in case people want to try out stuff without an explicit branch deployment."
}

resource "vercel_project_environment_variables" "web_app" {
  project_id = vercel_project.web_app.id
  variables = concat([
    for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_development_env_file.value)) : {
      key    = split("=", line)[0]
      value  = split("=", line)[1]
      target = ["development"]
    }
    ], [
    for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_preview_env_file.value)) : {
      key    = split("=", line)[0]
      value  = split("=", line)[1]
      target = ["preview"]
    }
    ], [
    for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_staging_env_file.value)) : {
      key   = split("=", line)[0]
      value = split("=", line)[1]
      custom_environment_ids = [
        vercel_custom_environment.web_app_staging.id
      ]
    }
    ], [
    for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_production_env_file.value)) : {
      key    = split("=", line)[0]
      value  = split("=", line)[1]
      target = ["production"]
    }
  ])
}

resource "vercel_project_deployment_retention" "web_app" {
  project_id            = vercel_project.web_app.id
  team_id               = vercel_project.web_app.team_id
  expiration_preview    = "3m"
  expiration_production = "unlimited"
  expiration_canceled   = "1m"
  expiration_errored    = "1m"
}

resource "vercel_project_domain" "web_app_staging" {
  project_id            = vercel_project.web_app.id
  domain                = "staging.domain.com"
  custom_environment_id = vercel_custom_environment.web_app_staging.id
}

resource "vercel_project_domain" "web_app_production" {
  project_id = vercel_project.web_app.id
  domain     = "domain.com"
}

Diff

data "aws_ssm_parameter" "web_app_development_env_file" {
  name = "/vercel/web-app/development/ENV_FILE"
}

data "aws_ssm_parameter" "web_app_preview_env_file" {
  name = "/vercel/web-app/preview/ENV_FILE"
}

data "aws_ssm_parameter" "web_app_staging_env_file" {
  name = "/vercel/web-app/staging/ENV_FILE"
}

data "aws_ssm_parameter" "web_app_production_env_file" {
  name = "/vercel/web-app/production/ENV_FILE"
}

resource "vercel_project" "web_app" {
  name                       = "web-app"
  framework                  = "nextjs"
  team_id                    = var.team_id
  serverless_function_region = var.serverless_function_region
  skew_protection = "12 hours"

- environment = concat([
-    for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_development_env_file.value)) : {
-      key    = split("=", line)[0]
-      value  = split("=", line)[1]
-      target = ["development"]
-    }
-    ], [
-    for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_preview_env_file.value)) : {
-      key    = split("=", line)[0]
-      value  = split("=", line)[1]
-      target = ["preview"]
-    }
-    ], [
-    for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_production_env_file.value)) : {
-      key    = split("=", line)[0]
-      value  = split("=", line)[1]
-      target = ["production"]
-    }
- ])

  vercel_authentication = {
    deployment_type = "standard_protection"
  }
}

+ resource "vercel_custom_environment" "web_app_staging" {
+   project_id  = vercel_project.web_app.id
+   name        = "web-app-staging"
+   description = "Long-running testing environment in case people want to try out stuff without an explicit branch deployment."
+ }
+ 
+ resource "vercel_project_environment_variables" "web_app" {
+   project_id = vercel_project.web_app.id
+   variables = concat([
+     for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_development_env_file.value)) : {
+       key    = split("=", line)[0]
+       value  = split("=", line)[1]
+       target = ["development"]
+     }
+     ], [
+     for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_preview_env_file.value)) : {
+       key    = split("=", line)[0]
+       value  = split("=", line)[1]
+       target = ["preview"]
+     }
+     ], [
+     for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_staging_env_file.value)) : {
+       key   = split("=", line)[0]
+       value = split("=", line)[1]
+       custom_environment_ids = [
+         vercel_custom_environment.web_app_staging.id
+       ]
+     }
+     ], [
+     for line in split("\n", nonsensitive(data.aws_ssm_parameter.web_app_production_env_file.value)) : {
+       key    = split("=", line)[0]
+       value  = split("=", line)[1]
+       target = ["production"]
+     }
+   ])
+ }

resource "vercel_project_deployment_retention" "web_app" {
  project_id            = vercel_project.web_app.id
  team_id               = vercel_project.web_app.team_id
  expiration_preview    = "3m"
  expiration_production = "unlimited"
  expiration_canceled   = "1m"
  expiration_errored    = "1m"
}

resource "vercel_project_domain" "web_app_staging" {
  project_id            = vercel_project.web_app.id
  domain                = "staging.domain.com"
+ custom_environment_id = vercel_custom_environment.web_app_staging.id
}

resource "vercel_project_domain" "web_app_production" {
  project_id = vercel_project.web_app.id
  domain     = "domain.com"
}

The plan seemed fine:

Plan log

Terraform will perform the following actions:
  # module.vercel.vercel_custom_environment.web_app_staging will be created
  + resource "vercel_custom_environment" "web_app_staging" {
      + branch_tracking = (known after apply)
      + description     = "Long-running testing environment in case people want to try out stuff without an explicit branch deployment."
      + id              = (known after apply)
      + name            = "web-app-staging"
      + project_id      = "<id>"
      + team_id         = (known after apply)
    }
  # module.vercel.vercel_project.web_app will be updated in-place
  ~ resource "vercel_project" "web_app" {
      - environment                                       = (sensitive value) -> null
        id                                                = "<id>"
        name                                              = "web-app"
      + protection_bypass_for_automation_secret           = (sensitive value)
        # (15 unchanged attributes hidden)
    }
  # module.vercel.vercel_project_domain.web_app_staging will be updated in-place
  ~ resource "vercel_project_domain" "web_app_staging" {
      + custom_environment_id = (known after apply)
        id                    = "staging.domain.com"
        # (3 unchanged attributes hidden)
    }
  # module.vercel.vercel_project_environment_variables.web_app will be created
  + resource "vercel_project_environment_variables" "web_app" {
      + project_id = "<id>"
      + team_id    = (known after apply)
      + variables  = [
          + {
              + comment                = (known after apply)
              + custom_environment_ids = (known after apply)
              + id                     = (known after apply)
              + key                    = "NEXT_PUBLIC_BACKEND_ENDPOINT"
              + sensitive              = (known after apply)
              + target                 = [
                  + "development",
                ]
              + value                  = "http://localhost:8081/"
            },
          + {
              + comment                = (known after apply)
              + custom_environment_ids = (known after apply)
              + id                     = (known after apply)
              + key                    = "NEXT_PUBLIC_BACKEND_ENDPOINT"
              + sensitive              = (known after apply)
              + target                 = [
                  + "preview",
                ]
              + value                  = "http://staging.domain.com/"
            },
          + {
              + comment                = (known after apply)
              + custom_environment_ids = (known after apply)
              + id                     = (known after apply)
              + key                    = "NEXT_PUBLIC_BACKEND_ENDPOINT"
              + sensitive              = (known after apply)
              + target                 = [
                  + "production",
                ]
              + value                  = "http://domain.com/"
            },/
          + {
              + comment                = (known after apply)
              + custom_environment_ids = [
                  + (known after apply),
                ]
              + id                     = (known after apply)
              + key                    = "NEXT_PUBLIC_BACKEND_ENDPOINT"
              + sensitive              = (known after apply)
              + target                 = (known after apply)
              + value                  = "http://staging.domain.com/"
            },
        ]
    }

But when applying, the above error occurs.

Reproducing

My Terraform experience is limited, so the above code and logs are the best I can do right now. If you need more info or logs, please let me know.

[dglsparsons]

Hi @TowardsDeath, Thanks for raising this. I think it's unrelated to moving env vars. I shall look into the error and let you know!

[TowardsDeath]

Thanks for your quick reply. I also don't think it's related to moving the env vars, since the error complains about custom_environment_ids, but that was the scenario where I encountered the problem.
Maybe this helps as well: if I remove all references to the vercel_custom_environment, the apply succeeds.

Also weird (I think): if I remove the custom_environment_id from the vercel_project_domain, the domain doesn't get updated, but gets removed entirely, even though the plan says it will be updated in place.

[dglsparsons]

Thanks for raising these. Sounds like some odd issues!