Secret Resources

[mitchdenny]

This PR adds two new resource types (parent + child). SecretStoreResource and SecretResource. The SecretStoreResource resource models the concept of a secure location where secrets are stored. SecretResource represents the actual string value of the resource.

Here is some example usage:

var builder = DistributedApplication.CreateBuilder(args);
var secrets= builder.AddSecretStore("secrets");
var apikey = store.AddSecret("apikey");
var app = builder.AddProject<Projects.MyApp>("myapp")
         .WithEnvironment("API_KEY", apikey);

When published, this is the manifest that will be produced:

{
  "resources": {
    "screts": {
      "type": "secrets.store.v0"
    },
    "apikey": {
      "type": "secrets.secret.v0",
      "parent": "secretstore",
      "value": "{apikey.inputs.value}",
      "inputs": {
        "value": {
          "type": "string",
          "secret": "true"
        }
      }
    },
    "myapp": {
      "type": "project.v0",
      "path": "../MyApp/MyApp.csproj",
      "env": {
        "OTEL_DOTNET_EXPERIMENTAL_OTLP_EMIT_EXCEPTION_LOG_ATTRIBUTES": "true",
        "OTEL_DOTNET_EXPERIMENTAL_OTLP_EMIT_EVENT_LOG_ATTRIBUTES": "true",
        "API_KEY": "{apikey.value}",
      },
      "bindings": {
        "http": {
          "scheme": "http",
          "protocol": "tcp",
          "transport": "http"
        },
        "https": {
          "scheme": "https",
          "protocol": "tcp",
          "transport": "http"
        }
      }
    }
  }
}

When run locally, the setting will be pulled from the AppHost's own configuration. The configuration might look like this:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "Secrets": {
    "secrets": {
      "apikey": "fakeapikey"
    }
  }
}

Related:
#1478
#1151
#830
#770
#166

[davidfowl]

Shouldn't this come from user secrets in practice?

[mitchdenny]

Yep. That's what I'm thinking.

[davidfowl]

Would you rewrite the container db passwords as secret resources? What about connection strings? (like app insights)?

[mitchdenny]

I think we should do this optionally for some resources. Absolutely for those things like Open AI which need a secret.

I want to avoid needing to use secrets directly if the resource type doesn't need it. But if someone is using a container and they want to operationalize it in production then they'll probably want key secrets managed externally to the ephemeral container.

[mitchdenny]

@davidfowl I think I'd like to tackle this secrets work in two phases. I think what I have here represents the first phase (just the resource type and the prompting mechanism). We should get this in so that tool authors can do the integration work for preview 3.

Following that I want to tackle how we might better handle secrets with pre-existing resource types. Here are some thoughts for that which probably belong in a separate PR:

APIs such as AddPostgresContainer(...) can optionally take an IResourceBuilder<SecretResource> as an argument (example):

var builder = DistributedApplication.CreateBuilder(args);
var secret = builder.AddSecretStore("mysecrets").AddSecret("postgrespassword");
builder.AddPostgresContainer("postgres", password: secret);

This is probably pretty straight forward. The more interesting question is what do we do when you do this (existing API):

var builder = DistributedApplication.CreateBuilder(args);
builder.AddPostgresContainer("postgres", password: "somerandompassword");

What I am thinking here is that this API call would result in a secret store/secret being created behind the scenes so that even if someone doesn't explicitly setup a secret store that one will be created for them to store these kinds of secrets. In these circumstances, the manifest would actually be slightly different in that the default value would be supplied literally:

{
  "resources": {
    "defaultsecretstore": {
      "type": "secrets.store.v0"
    },
    "postgrespassword": { // name generated
      "type": "secrets.secret.v0",
      "parent": "defaultsecretstore",
      "value": "{postgrespassword.inputs.value}",
      "inputs": {
        "value": {
          "type": "string",
          "secret": "true",
          "default": "somerandompassword"
        }
      }
    }
    "postgrescontainer": {
      "type": "container.v0",
      "image": "postgres:latest",
      "env": {
        "POSTGRES_HOST_AUTH_METHOD": "scram-sha-256",
        "POSTGRES_INITDB_ARGS": "--auth-host=scram-sha-256 --auth-local=scram-sha-256",
        "POSTGRES_PASSWORD": "{postgrespassword.value}"
      },
      "bindings": {
        "tcp": {
          "scheme": "tcp",
          "protocol": "tcp",
          "transport": "tcp",
          "containerPort": 5432
        }
      },
      "connectionString": "Host={postgrescontainer.bindings.tcp.host};Port={postgrescontainer.bindings.tcp.port};Username=postgres;Password={postgrespassword.value};",
    },
  }
}

[mitchdenny]

@prom3theu5 would love to get your input here as well!

[prom3theu5]

I like it!

Very little change really required on my end just another strategy for value substitution to implement.

Quick suggestion. Maybe drop secret: true from the input value, as the resource type is a secret.v0 so that's kind of implicit isn't it?

How will generated change with this?
Factory for the default value?

[davidfowl]

Looks good. Undo the sample update and write a test instead (to make sure these secret values come from configuration when the manifest isn't being used).

[ellismg]

I think the way these values are consumed by other resources is going to be interesting with respect to interpolated strings. ACA makes it easy to say: "Inject this value from KeyVault into the process with this environment variable name", but I'm not sure it's easy to say: "interpolate this secret with this literal".

If someone references the connectionString property of the container resource in their env block, what is the expectation? Does the tool create another secret to represent this (and if so, does that mean it snapshots the value of the existing secret?) Or does it pull the secret and do the interpolation and bake the value in as a literal value instead of a reference to keyvault?

[mitchdenny]

Yeah that is a good observation. Realistically, I think that if the only content in the string is the {mysecret.value} then you would reference the KeyVault secret. But if you had something like: "Database=foo;Server=blah;Password={mysecret.value}" it is going to be problematic.

I think in that situation you'd probably read the value from KeyVault at deployment time but then have to interpolate the string and store the environment variable as a secret in ACA. But you'd probably want to warn if you did that?

[mitchdenny]

Quick suggestion. Maybe drop secret: true from the input value, as the resource type is a secret.v0 so that's kind of implicit isn't it?

I'd prefer not to conflate secret resources with input secrets. Whilst they will necessarily overlap a lot, I want to make sure that we are always signalling when a particular value needs to be handled with care in terms of logging etc.