Splitting the state for frequently changing resources

[edikmkoyan]

Terraform Version

Next version

Use Cases

When the number of resources becomes too big, the plan and application take too long as many API requests are sent to update the local state file and sync with AWS for example.

Attempted Solutions

We didn't try to code it. Everyone knows the HCL code base needs refactoring, but who does that?

Proposal

We propose gathering statistics on all resource changes relative to the total number of state changes. Based on these metrics, it may be beneficial to suggest splitting the HCL and state files. We would call this metric change-ratio.

If frequently and infrequently changing resources are part of the same connected graph, the edges of the graph can be converted into data sources and output variables, as is typically done in Terraform for similar cases.
Most probably serial can not be used as it does not increment by 1, maybe revision parameter should be added the the state file on the top and resource levels. Then for every resource change-ratio can be connected, and a new terrafrom split command can split the state and the HCL files by creating two dictionaries, data sources and output variables.

This approach will help minimize unnecessary API calls, reducing time spent on operations that result in minimal or no changes.

References

No response

[edikmkoyan]

https://discuss.hashicorp.com/t/terraform-state-schema/48660/3

[crw]

Thanks for this feature request! If you are viewing this issue and would like to indicate your interest, please use the 👍 reaction on the issue description to upvote this issue. We also welcome additional use case descriptions. Thanks again!