PD Request from TiDB Should Honor Query max_execution_time

[HaoW30]

Enhancement

Currently, TiDB queries are terminated when they exceed max_execution_time, but the PD requests generated by these queries, such as GetRegion calls, do not adhere to the same timeout constraint.

The default PD RPC timeout (3 seconds) is relatively long compared to the max_execution_time for time-sensitive OLTP workloads. In some cases, even after TiDB has terminated the queries, their PD requests continue to retry on the PD, leading to unnecessary resource consumption. In extreme scenarios, we've observed that PD can become overwhelmed by a flood of these retried requests from TiDB.

As an enhancement, PD requests like GetRegion calls should honor the same timeout as the originating TiDB queries, enforced via the Go context. This would ensure that resources consumed by PD RPC calls are promptly released when the parent queries are terminated.

[HaoW30]

We have a similar ask for tso requests #51893