Inline blob mount producing error message claiming blobfuse2 is not in $PATH on Azure stack Azure Linux node #1830
Comments
|
@sindreal have you tried other example, e.g. option#1, option#2? |
|
I see where is the problem, have you set |
|
The other examples also fails, and AKS on Azure Local is just renamed AKS on Azure Stack, which lacks access to this extension, hence the use of Helm. |
|
pls provide the csi driver logs first: https://github.com/kubernetes-sigs/blob-csi-driver/blob/master/docs/csi-debug.md#case2-volume-mountunmount-failed, the blobfuse2 and blobfuse-proxy install logs are in |
|
kubectl logs daemonset/csi-blob-node -c blob -n kube-system |
|
Finally got some time to continue troubleshooting. First of all, when including the following helm values: It failed to install the helmrelease properly, and printed the following error message: After removing the "cloud: AzureStackCloud" it installed successfully, but I still get the original error message on the pod itself: Is the cloud value needed? What does it do? Sinde reinstall, the output from "kubectl logs daemonset/csi-blob-node -c blob -n kube-system" is quite small, but I couldn't find a smoking gun: |
|
what is the output of following commands: |
|
kubectl describe ds csi-blob-node -n kube-system: kubectl logs daemonset/csi-blob-node -c install-blobfuse-proxy -n kube-system |
this is Azure Linux VM, you need to install blobfuse2 on all nodes manually |
|
It's AKS nodes on Azure Local (new name for Azure Stack), I'm not supposed to do manual tasks on those, as they may rotate etc. I'm assuming that the "cloud: AzureStackCloud" value does something related to this? If so, maybe follow that trail instead. |
|
|
andyzhangx
changed the title
|
Okay, now I'm slightly confused. |
|
Even without the helm value set, it seems like "--enable-blobfuse-proxy=true" is set in "kubectl describe ds csi-blob-node -n kube-system" And yet, Is that the correct way to disable it, and to check if it's actually disabled? |
|
just run |
|
Okay, I did that and set it to false. It worked, and the pod got its blob inline mount working, so far so good. But I also need the correct helm value to avoid having to manually edit anything. |
|
just set |
|
@andyzhangx I'm installing helm charts via flux, so there's an extra step, but I can't seem to get it to correctly pick up that value. My helmrelease yaml definition includes: And when running And yet https://github.com/kubernetes-sigs/blob-csi-driver/tree/master/charts#latest-chart-configuration also shows node.enableBlobfuseProxy to be false as default, so it's a bit weird that I can't disable it without manually editing it after install. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Doing a simple inline blob mount test following the example at:
https://github.com/kubernetes-sigs/blob-csi-driver/blob/master/deploy/example/e2e_usage.md#option3-inline-volume
What happened:
Pod won't start, due to the following error mesage:
Warning FailedMount 14s (x35 over 55m) kubelet MountVolume.SetUp failed for volume "volume-inline-blob-1" : rpc error: code = Internal desc = Mount failed with error: rpc error: code = Unknown desc = exec: "blobfuse2": executable file not found in $PATH , output:
Please refer to http://aka.ms/blobmounterror for possible causes and solutions for mount errors.
What you expected to happen:
That the pod started with the blob volume mounted.
Anything else we need to know?:
All csi-blob-node-xxxxx nodes seems to have blobfuse2 in their $PATH, and I can successfully mount the blob manually using blobfuse2 from the csi-blob-node. The secret is also present in the same namespace as the pod.
Environment:
kubectl version): 1.29.4The text was updated successfully, but these errors were encountered: