Sidecar cannot reach storage

[bisgaard-itis]

Placeholder for https://z43.manuscript.com/f/cases/191098/service-failed-sidecar-cannot-reach-storage

[matusdrobuliak66]

https://git.speag.com/oSparc/osparc-infra/-/issues/incident/47

[YuryHrytsuk]

It is awaiting @GitHK changes in dynamic-sidecar. As soon as, dynamic-sidecar uses the new endpoint and all works, we can close it

[YuryHrytsuk]

The fix is merged. We will observe it for a couple of days before closing

[sanderegg]

Just a remark on this "fix":

• spaghetti code: Dynamic-sidecar --> storage uses https,
• all other services use regular http
  As much as I understand the need for this fix I think the solution is half backed and will only increase code complexity and understanding.

I would be more than happy that at least references to this issue are set in the osparc-simcore code because otherwise it makes no sense.
@GitHK @YuryHrytsuk

[GitHK]

@sanderegg I'll drop a refenrece to the issue in one of my PRs

[YuryHrytsuk]

Just a remark on this "fix":

* spaghetti code: Dynamic-sidecar --> storage uses https,

* all other services use regular http
  As much as I understand the need for this fix I think the solution is half backed and will only increase code complexity and understanding.

I would be more than happy that at least references to this issue are set in the osparc-simcore code because otherwise it makes no sense. @GitHK @YuryHrytsuk

This is true. The fix doesn't seem to be the best solution. On the other hand I don't see any other solution to the problem. And the main issue is that we still don't fully understand the root of the problem (also because it is not easy to reproduce).

I don't feel that any of us will dive deep into docker for a couple of days (or a week) to find out the root cause.

Any proposals how to get away from current position?

@sanderegg @GitHK @mrnicegyu11

[mrnicegyu11]

One invasive suggestion to improve the current state and at least reestablish code homogenity:

• We decide the storage microservice is now publicly accessible
• We really mean it and make its openapi specs public
• We enforce all simcore-microservices to talk to storage via a public route by explicitly changing the docker-internal DNS name of storage so all microservices talking via http/internally will fail

[YuryHrytsuk]

After this solution was introduced to master deployment, storage failures never occurred. I consider this issue to be fixed as of now