[NEW] CPU throttling

[lschmidtcavalcante-sc]

The problem/use-case that the feature addresses

The goal is to prevent node loss due to heavy load on CPU. For example, due to a sudden traffic change the CPU can become overloaded putting the cluster's nodes at risk.

Description of the feature

Periodically monitor the CPU usage, if it is above a given threshold then respond to commands as THROTTLED: it alleviates CPU usage by not processing the command and gives the client an opportunity to act accordingly.

[xbasel]

If the concern is that high CPU usage puts cluster nodes at risk (and even generally preventing ADMIN clients from time constrained operations), would a more aggressive cluster management approach be a better solution? For example, implementing QoS at the ae-loop level could prioritize critical operations and prevent overload more effectively. Also, if the engine returns THROTTLED, clients might abuse it by resending or reconnecting in a busy loop, potentially making the problem worse. Would blocking clients for some time be a better alternative?"

[sarthakaggarwal97]

Also, if the engine returns THROTTLED, clients might abuse it by resending or reconnecting in a busy loop, potentially making the problem worse. Would blocking clients for some time be a better alternative?"

Sending THROTTLED seems synonymous to HTTP 429s, which is a common practice. This way clients can be allowed to have their retry mechanisms in place imo. Although, blocking a client till the duress reduces could be a self healing action by valkey engine.