API call exceeded rate limit due to too many requests - TF can not complete due to Okta errors

[ruzickap]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

❯ terraform -v
Terraform v1.5.7
on darwin_arm64

Affected Resource(s)

• all

Expected Behavior

I would like to create large amount of objectc in Okta using okta_group, okta_group_owner, okta_app_group_assignment, okta_group_ownerresources and read osme data resources likeokta_user, okta_app`.

Unfortunately I'm hitting the Okta API limits: Error returned: Get "https://myorg.okta.com/api/v1/groups/xxxxxxxxxxxx/owners": the API returned an error: API call exceeded rate limit due to too many requests., giving up after 6 attempt(s)

I tried to set the Okta provider limits parameter to some values but I have never been able to configure it to avoid the Okta API limit errors + TF erros:

provider "okta" {
  org_name = "myorg"
  base_url = "okta.com"

  # max_retries      = 10
  # max_wait_seconds = 600
  # min_wait_seconds = 60
  # max_api_capacity = 5
  # max_api_capacity = 1 - terribly slow but doesn't help
  # max_api_capacity = 10

  max_retries      = 30
  backoff          = true
  min_wait_seconds = 60
  max_wait_seconds = 600
}

-> Is there any way how I can configure the terraform to put some necessary delays between queries (or anything else) which will allow me to complete the Terraform task?
-> I do not care - how much time (hours) the TF will be running - I need to successfully create the defined objects without errors.

Can this be done in the Admin UI?

Yes

Can this be done in the actual API call?

Yes

Actual Behavior

Terraform can not complete successfully the change because of the Okta API limits.

Steps to Reproduce

1. terraform apply

│ Error: Error retrieving list group owners
│
│   with okta_group_owner.product_ids_dev2_global_admin["12491"],
│   on okta-dev2.tf line 29, in resource "okta_group_owner" "product_ids_dev2_global_admin":
│   29: resource "okta_group_owner" "product_ids_dev2_global_admin" {
│
│ Error returned: Get "https://myorg.okta.com/api/v1/groups/xxxxxxxxxxxx/owners": the API returned an error: API call exceeded rate limit due to too many requests., giving up after 6 attempt(s)
╵
╷
│ Error: Error retrieving list group owners
│
│   with okta_group_owner.product_ids_dev2_global_admin["17515"],
│   on okta-dev2.tf line 29, in resource "okta_group_owner" "product_ids_dev2_global_admin":
│   29: resource "okta_group_owner" "product_ids_dev2_global_admin" {
│
│ Error returned: Get "https://myorg.okta.com/api/v1/groups/xxxxxxxxxxxx/owners": the API returned an error: API call exceeded rate limit due to too many requests., giving up after 6 attempt(s)
╵
╷
│ Error: Error retrieving list group owners
│
│   with okta_group_owner.product_ids_dev2_global_admin["13218"],
│   on okta-dev2.tf line 29, in resource "okta_group_owner" "product_ids_dev2_global_admin":
│   29: resource "okta_group_owner" "product_ids_dev2_global_admin" {
│
│ Error returned: Get "https://myorg.okta.com/api/v1/groups/xxxxxxxxxxxx/owners": the API returned an error: API call exceeded rate limit due to too many requests., giving up after 6 attempt(s)
╵

[crewbacca2]

Also seeing this error when we deployed 100 group owner resources in one component.

[duytiennguyen-okta]

@ruzickap you can reduce the number of thread with -parallelism=? if you do not care about the run time

[ruzickap]

Thank you.
The disadvantage of -parallelism=? parameter is, that it will affect the whole terraform "stack" (not only Okta provider) which will really slows down the Terraform execution.

Anyway I guess by default the parallelism is set to 10.
I tried -parallelism=3 and still getting the "API call exceeded rate limit" error.