Make consolidateAfter configurable by disruption reason

[yagonobre]

Description

What problem are you trying to solve?

We are trying to better control consolidation and avoid pods being moved around too frequently. To achieve this, we started using consolidateAfter. However, this has the side effect of keeping empty nodes in our cluster for longer, leading to additional costs.

It would be ideal if we had the option to configure consolidateAfter based on the disruption reason, allowing us better control over the process. We understand that the Karpenter API is now stable, but what we are proposing is to add an optional consolidateAfter configuration per disruption reason, which would take higher priority than the global setting.

Here is a api suggestion:

spec:
  disruption:
    budgets:
    - nodes: "2%"
      schedule: "0 21 * * *"
      duration: 12h
      reasons:
      - Drifted
    - nodes: "100%"
      consolidateAfter: 15m
      reasons:
      - Empty
    - nodes: "2%"
      reasons:
      - Underutilized
    consolidateAfter: 12h

Her we should have Drifted and Unterutilized Nodes consolidated after 12h but Empty nodes after 15m.

How important is this feature to you?

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request

[k8s-ci-robot]

This issue is currently awaiting triage.

If Karpenter contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.