Autoscaling is failing in 1.27 clusters

[okozachenko1203]

To support 1.27, we need to bump autoscaler chart to 9.29.1.

[okozachenko1203]

Autoscaler1.27 is failing to get the list of machinepools because of perm lack. kubernetes/autoscaler@2c7c6c4

[okozachenko1203]

@mnaser

• i think we have to consider the version upgrade for not only autoscaler but also other addons (like csi, ccm, cni, etc) which are maintained by mcapi driver.
• Also i have another concern. It is that the available versions supported by mcapi could be limited by those addons' version compatibility matrix. To mitigate that, we could need to keep multiple versions of addons in mcapi.

What do you think?

[fnpanic]

Checking for depracted APIs can be done with pluto,KubePug,KubeNoTrouble and kubent. This will help prevent this problems in the future and can be integrated in the CI workflow.

Checking the deprecation guide is a good idea anyway before declaring a version as stable:

https://kubernetes.io/docs/reference/using-api/deprecation-guide/

[mnaser]

@okozachenko1203 i think this is the use case to look at this:

#74

if we implement the addons to be managed, this means we can use helm charts to deploy them and we can just use helm values to manage state/etc.

perhaps we can work on that effort first, and then we start managing those cluster addons at that point using the provider, and potentially even start moving the autoscaler into the cluster itself too and manage it via an addon provider.