RegisterHostOutputs

[chsienki]

Background and Motivation

This is a re-working of a previous proposal: #63291

The basic idea is to allow generators to specify a 'host specific' output. That is, something that doesn't contribute to the compilation, or even do anything other than appear in the run result for the generator.

Razor would like to use this to store the intermediate state that was used to generate documents, which is needed for the tooling.

It would also be useful in testing scenarios. We have lots of places today where we want to check something was called, or some operation produced some output, and we 'smuggle' the result out by adding some source with a content like // {result.ToString()}. Host outputs would allow us an easy mechanism to do this sort of result-based testing.

Proposed API

Implementation PR: #74750

namespace Microsoft.CodeAnalysis
{
    public readonly partial struct IncrementalGeneratorInitializationContext
    {
+       public void RegisterHostOutput<TSource>(IncrementalValueProvider<TSource> source, Action<HostOutputProductionContext, TSource> action);
+
+       public void RegisterHostOutput<TSource>(IncrementalValuesProvider<TSource> source, Action<HostOutputProductionContext, TSource> action);
    }

+   public readonly struct HostOutputProductionContext
+   {
+        /// <summary>
+        /// Adds a host specific output
+        /// </summary>
+        /// <param name="name">The name of the output to be added.</param>
+        /// <param name="value">The output to be added.</param>
+        /// <remarks>
+        /// A host output has no defined use. It does not contribute to the final compilation in any way. Any outputs registered 
+        /// here are made available via the <see cref="GeneratorRunResult.HostOutputs"/> collection, and it is up the host to
+        /// decide how to use them. A host may also disable these outputs altogether if they are not needed. The generator driver
+        /// otherwise makes no guarantees about how the outputs are used, other than that they will be present if the host has
+        /// requested they be produced.
+        /// </remarks>
+        public void AddOutput(string name, object value) => Outputs.Add((name, value));
+
+        /// <summary>
+        /// A <see cref="System.Threading.CancellationToken"/> that can be checked to see if producing the output should be cancelled.
+        /// </summary>
+        public CancellationToken CancellationToken { get; }
+   }

    public readonly struct GeneratorRunResult
    {
+        /// <summary>
+        /// A collection of items added via <see cref="HostOutputProductionContext.AddOutput(string,  object)"/>.
+        /// </summary>
+       public ImmutableArray<(string, object)> HostOutputs { get; }
    }
}

Usage Examples

public void Initialize(IncrementalGeneratorInitializationContext context)
{
    var provider = context.CompilationProvider.Select(/* .. */);

    // make outputs available to other consumers
    context.RegisterHostOutput(provider, (context, value) =>
    {
        context.AddOutput("key", value);
    });

    // NOTE: can still do more with the provider
    var provider2 = provider.Select(/* .. */);

    // regular source output
    context.RegisterSourceOutput(provider2, (context, value) => /* .. */);
}

Alternative Designs

In the previous design discussion we talked about if it should be object or string. We decided on string because there were worries about things like serialization and how we would handle that. This proposal switches back to object as the seralization overhead for Razor is going to be far too much and we've explicitly designed co-hosting to live in the same process.

For third party usage, I think we should just make it clear that it's entirely up to the host what happens with these outputs, and there is no defined expectation.

Risks

[333fred]

API Review

• What if we just used the existing named step tracking infrastructure?
  • This is a generator testing component, not a strong guarantee of a contract.
  • Better to have an API that intentionally defines an output component, rather than exposing internal generator details
• Would this be related to supplemental file generation?
  • We don't think so. Those would be tied to semantics of being a file, and this should not be.
• By switching to object, we are saying that we won't be serializing anything here.
  • Yup. People want to use host outputs in VS are going to have to come talk to us
• Arguments to using a ImmutableDictionary<string, object>:
  • Making it appear to be key/value without it actually being a dictionary is a bit odd.
  • What would the ordering be if we did ImmutableArray<(string, object)>?
  • Making it a dictionary will match behavior around file hint names

Conclusion: API is approved. We will use an ImmutableDictionary<string, object> for the host outputs, rather than an array.

[chsienki]

Implemented via #74750