Allow redirecting to another host to download package.

[derekkraan]

I am currently developing a private registry. It was already working with Mix, but uploading and consuming packages with Rebar3 has been a challenge. I have got uploading working with a PR to rebar3_hex, and this PR is intended to address difficulties with downloading a package using rebar3.

My package registry works by validating that the user may access the package version they are attempting to access, and then sending a 302 directing the client to my CDN, with a signed URL. This currently fails at the "check_hostname" step, since rebar3 attempts to verify that the hostname matches the original (pre-redirect) URL. (I am redirecting from codecodeship.com to a "shared" domain on Fastly.)

I have "fixed" this issue by simply removing the check_hostname option from the SSL verify function. I will admit right away that I am not sure of the further security implications of this change. But it seems like this is locking the request to just the originally requested hostname, and blocking redirects to other hostnames outright. I know (from watching Bram Verburg speak a number of times) that SSL handling is a minefield.

What I can tell you for certain is that every other package manager I have implemented so far is handling this case out of the box with no issues (Hex, NPM, Cargo, RubyGems, Nuget).

Your feedback is appreciated.

[ferd]

This is essentially removing all TLS checks and is a bad idea for security. That check makes sure that the certificate chain matches the domain requested in the query in the CA bundle we have.

Rebar3 supports mirrors or private registries that are layered, and will seek any package, through all of them, in order until they eventually hit a registry with the package they want. The hash version from the lock file will determine whether the registry has the proper package version, and signatures will be compared to make sure the proper copy is the one being fetched.

You can see what a configuration for this looks like in rebar3 repos:

$rebar3 repos
Repos:
[#{name => <<"hexpm">>,
   http_adapter => {rebar_httpc_adapter,#{profile => rebar}},
   http_user_agent_fragment =>
       <<"(rebar3/3.22.0+build.5279.ref1b6297dd) (httpc)">>,
   http_etag => undefined,repo_verify_origin => true,repo_name => <<"hexpm">>,
   repo_url => <<"https://repo.hex.pm">>,api_organization => undefined,
   api_repository => undefined,repo_organization => undefined,
   api_url => <<"https://hex.pm/api">>,repo_verify => true,
   api_key => undefined,http_headers => #{},repo_key => undefined,
   repo_public_key =>
       <<"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEApqREcFDt5vV21JVe2QNB\nEdvzk6w36aNFhVGWN5toNJRjRJ6m4hIuG4KaXtDWVLjnvct6MYMfqhC79HAGwyF+\nIqR6Q6a5bbFSsImgBJwz1oadoVKD6ZNetAuCIK84cjMrEFRkELtEIPNHblCzUkkM\n3rS9+DPlnfG8hBvGi6tvQIuZmXGCxF/73hU0/MyGhbmEjIKRtG6b0sJYKelRLTPW\nXgK7s5pESgiwf2YC/2MGDXjAJfpfCd0RpLdvd4eRiXtVlE9qO9bND94E7PgQ/xqZ\nJ1i2xWFndWa6nfFnRxZmCStCOZWYYPlaxr+FZceFbpMwzTNs4g3d4tLNUcbKAIH4\n0wIDAQAB\n-----END PUBLIC KEY-----">>,
   tarball_max_size => 8388608,tarball_max_uncompressed_size => 67108864}]

Pretty much all fields there are optional so you can put in your own, and can be specified in the {hex, [{repos, $LIST_FROM_ABOVE_EXAMPLE}]}. format in your rebar.config file (possibly the global one at ~/.config/rebar3/rebar.config to be convenient).

You can also use these options to use private organizations on hex, or to publish to various repos with distinct keys (see https://hexdocs.pm/rebar3_hex/rebar3_hex_organization.html for these points).

I realize now that https://hex.pm/docs/rebar3-private alludes to this functionality, but does not explain how to make it work.

[derekkraan]

Hey @ferd, thanks for taking the time to comment. Despite the somewhat missing documentation, I did manage to figure out, largely through reading the code, how to configure rebar3 to make uploading and downloading work.

Where I am struggling, specifically, is in this hostname check. Rebar3 doesn't seem to want to deal with a 302 redirect in the same way that any other package manager client is doing today.

If the change I am proposing removes all TLS checks, is there another way to reconfigure this part of the code to allow 302s to another domain (for package download), while still maintaining the checks that keep the client secure?

[ferd]

I'd need to see the request and the domain going there, but that check fails specifically when doing a TLS call and getting a certificate for the wrong domain name. If the endpoint that does the 302 redirection serves the proper cert (or one that is not self-signed), then it should be okay.

But if what you have is something like:

• you call whatever.domain and it replies with a self-signed cert
• you call whatever.domain and it replies with a cert for other.domain
• you call whatever.domain and it replies with the right cert, but forwards you to a CDN with a mismatched cert

These can all fail validation checks.

Removing the domain check, no matter what you do, essentially says "let any man-in-the-middle respond with whatever they want here, don't check that they are who I expect them to be."

I don't really know what other package managers are doing, but if they let you 302 with the wrong certs, they're either dangerous, or they use another signing mechanism with known keys inside the protocol do an extra layer of validation. We can also do the latter, but safer TLS validation isn't a bad thing.

I can possibly help debug the specific chain of calls you want to happen, but I'll probably want more details about the operations and the domains that this goes through. I don't think "just don't check TLS, it works elsewhere" is workable because it absolutely depends on where and how the internal protocols are doing further checks for trust.

[tsloughter]

Curious what mix is doing?

[ferd]

I guess one issue would be that the redirect is to a distinct domain but the check remains on the same one as well.

[derekkraan]

@ferd I believe this is what is happening.

httpc has autoredirect set to true by default, and we are not overriding this in rebar3. Therefor the SSL opts gets set only once, and it is checking subsequent certs not on the redirected URL, but on the URL of the original request.

A possible solution could be to change autoredirect to false and handle redirects explicitly in the code. Then the URI check on the certificate could be set for each step in the redirect chain.

[derekkraan]

Updated this PR with a POC for that approach. This doesn't disable any checks, but handles redirects manually. I have tested this and it also serves to fix the redirect issue. (Although it is incomplete in its handling of 3xx responses)

I'm not really convinced that this is the solution though. More just trying to push the conversation forward.

[ferd]

I do believe this is a better approach than disabling checks altogether yeah.

The other option would be to modernize validation. I think the current validation function was added in R15, way before OTP had any mechanism for TLS validation. Starting in OTP-21, we can pass something like this:

    {customize_hostname_check, [
        {match_fun, public_key:pkix_verify_hostname_match_fun(https)}
    ]}

to httpc options to get cert validation going. Starting with OTP-25, we can use the OS's store for CA certs, but I think we can start relying on this in one major version (starting with OTP-27). Then starting with OTP-26 (which we can use starting in OTP-28), the checks are done by default.

https://erlef.github.io/security-wg/secure_coding_and_deployment_hardening/inets has the details on what to do here. Chances are that right now we can use the public_key validation check and keep passing in our certifi CA bundle. It might work better for the redirect mechanism without erroring out.

[derekkraan]

Closing in favour of #2805