Issue: For GKE cluster version 1.34.1-gke.3899001 (sidecar mounter image version 1.21.9) and later affected versions, Cloud...

For GKE cluster version 1.34.1-gke.3899001 (sidecar mounter image version 1.21.9) and later affected versions, Cloud Storage FUSE volumes might fail to mount if the GKE metadata service isn't ready when the Cloud Storage FUSE sidecar initiates.

When this issue occurs, you might see the following error:

MountVolume.SetUp failed for volume "volume-name" : rpc error: code = Internal desc = the sidecar container terminated due to Error, exit code: 255

Additionally, the gcsfuse-sidecar container displays the following error:

Failed to fetch identity pool and identity provider details required for bucket access check, got error Failed to set up metadata service: failed to get project: Get "http://X.X.X.X/computeMetadata/v1/project/project-id": dial tcp 169.254.169.254:80: connect: connection refused for identity pool PROJECT_ID.svc.id.goog and identity provider https://container.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/clusters/CLUSTER_NAME

Mitigation

To resolve this issue, perform one of the following mitigations:

Upgrade your cluster to one of the following fixed GKE versions:

• 1.34.8-gke.1218000 or later • 1.35.3-gke.2347000 or later • 1.36.0-gke.1266000 or later.

Create an init container in your Pod that validates metadata service availability.

Manually inject the sidecar to ensure the sidecar is blocked by an init container.

For more information, see the Cloud Storage FUSE CSI driver troubleshooting guide.