Add Support for Dynamic OpenSSL Engines

[andyhopp]

Background and Motivation

Currently, the System.Cryptography classes under non-Windows platforms such as Linux, MacOS, et. Al. use a native interop layer for invoking OpenSSL APIs to perform cryptography functions. This layer currently does expose any support for dynamic engines, which prevents the use of many hardware security modules. An API should be provided that allows a developer to enumerate, load, initialize, and configure which cryptographic functions the selected dynamic engine will perform on these platforms.

Proposed API

namespace System.Security.Cryptography.OpenSsl {
    // internal Interop methods class removed
    internal class DynamicEngineHandle : Microsoft.Win32.SafeHandles.SafeHandleZeroOrMinusOneIsInvalid {
        public DynamicEngineHandle();
        public bool ReleaseHandle();
    }
    public class DynamicEngineRegistry {
        public IEnumerable<DynamicEngine> GetDynamicEngines();
        public DynamicEngine LoadDynamicEngine(string id);
    }
    [Flags]
    public enum EngineDefaults
    {
        None = 0,
        RSA = 1,
        DSA = 2,
        DH = 4,
        RandomNumberGeneration = 8,
        ECDH = 0x10,
        ECDSA = 0x20,
        Ciphers = 0x40,
        Digests = 0x80,
        Store = 0x0100,
        PublicKeyMethods = 0x0200,
        PublicKeyAsn1Methods = 0x0400,
        All = 0xFFFF
    }
    public class DynamicEngine: IDisposable {
        internal DynamicEngine(DynamicEngineHandle);
        public string Name { get; }
        public string Id { get; }
        public void Initialize();
        public void SetDefaults(EngineDefaults defaults);
        public void Finish();
        public void Dispose();
    }
}

Class Descriptions

DynamicEngineRegistry

Provides an API for enumerating installed OpenSSL dynamic engines as well as loading a specfied engine by ID.

GetDynamicEngines

Returns an IEnumerable<DynamicEngine> that can be iterated over to discover the set of installed OpenSSL dynamic engines.

LoadDynamicEngine

Returns a reference to a DynamicEngine instance corresponding to the supplied engine ID (if it exists).

DynamicEngine

Provides an API for initializing, configuring, and releasing an OpenSSL dynamic engine.

ID

A property which returns the ID of the OpenSSL dynamic engine represented by this class.

Name

A property which returns the Name of the OpenSSL dynamic engine represented by this class.

Initialize

Prepares the OpenSSL dynamic engine represented by this class for use.

Finish

Unloads the OpenSSL dynamic engine represented by this class.

SetDefaults

Configures which cryptographic functions for which the OpenSSL dynamic engine represented by this class will be used.

Usage Examples

var engineRegistry = new DynamicEngineRegistry();
using (var engine = engineRegistry.LoadDynamicEngine("myhsm"))
{
    engine.Initialize();
    Console.WriteLine($"Engine {engine.Id} initialized");
    engine.SetDefaults(EngineDefaults.RSA);
    Console.WriteLine($"Loading certificate...");
    using (var store = new X509Store(StoreName.My, StoreLocation.CurrentUser))
    {
        store.Open(OpenFlags.ReadOnly);
        foreach (var certificate in store.Certificates.Find(X509FindType.FindBySubjectName, subjectName, true))
        {
            certificateToUse = certificate;
            break;
        }
    }
    // Perform cryptographic functions
    // ...
} // Dispose calls Finish();

Alternative Designs

It was considered to have the methods on the DynamicEngineRegistry implemented as static methods on the DynamicEngine class, but that would pose difficulties implementing unit tests.

It was considered to have the constructor of the DynamicEngine class invoke the engine initialization method, but this would have required a "registry entry" class for representing the installed engines without performing initialization.

The names for the EngineDefaults enumeration were originally the names of the #defines in engines.h (e.g. ENGINE_METHOD_RSA), but have been changed to reflect the idioms of other C# APIs.

Risks

This API wraps DllImport calls to OpenSSL's Engine_* APIs which allocates handles to unmanaged memory structures, several of which utilize reference counting. Care will need to be taken to prevent memory leaks due to handles not being properly released. Therefore, the use of SafeHandle-derived wrappers for said handles is critical.

Additional Notes

A draft version of this API has been implemented and successfully tested on Ubuntu, Fedora, and Centos.

[ghost]

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq
Notify danmosemsft if you want to be subscribed.

[Wraith2]

This would expose abstracted implementation details to users and appears to have no windows equivalent.

[vcsjones]

@Wraith2 there is a bit of that already with OpenSSL / CNG. If this were to move forward, I could see it living in the System.Security.Cryptography.OpenSsl package.

[andyhopp]

@vcsjones is correct; the intent would be for this implementation to be in the System.Security.Cryptography.OpenSsl assembly, which is only used on non-Windows platforms.

[Wraith2]

Do users directly consume the S.S.C.OpenSsl then? I had thought it was an implementation used through the abstractions in S.S.C

[vcsjones]

Do users directly consume the S.S.C.OpenSsl then?

They can if they want to by referencing that package to gain access to a few OpenSSL-specific APIs. Most developers will probably consume the opaque types like RSA which don't have any platform specific APIs. Obviously it is encouraged to use the non-platform specific APIs since they should just work. However types like RSAOpenSsl and RSACng exist so you can do things specific to that platform.

[andyhopp]

@Wraith2 you are correct; currently, users do not normally directly consume the classes in the SS.C.OpenSsl namespace since S.S.C abstracts away the crypto functionality. However, this abstraction is a bit too abstract; it prevents users from leveraging HSMs and/or smart cards that use dynamic engines. As @vcsjones pointed out, the closest analogue to the proposed API are the S.S.C classes for CNG providers, which are Windows-only implementations that are still accessible to users if/when needed. This API would provide users with the necessary surface to perform that platform-specific configuration when needed.

[Wraith2]

If possible it would be good to provide additional platform abstract apis for access to those functions that lights up on capable host systems (desktop Linux, windows etc) rather than expose the user to multiple platform specific versions and force them to do the abstraction themselves.

[andyhopp]

@Wraith2 I had looked into that, but the platform differences between CNG and OpenSSL appear to be too stark. Where the Windows implementation is able to automatically determine the correct CNG provider to use when loading the keys, OpenSSL expects that context to have been provided explicitly by loading/initializing the module in advance (see: https://www.openssl.org/docs/man1.0.2/man3/engine.html).

[andyhopp]

Hello! Is there anything else I can do to help move this along?

[benlongo]

OpenSSL 3.0 is moving to providers instead of engines, but I think the same principle applies.

[krwq]

Note ENGINE part has been done in #55356 (ships with .NET 8) - providers work is tracked separately but was not completed mostly due to lack of time #89167 and fact that those OpenSSL APIs are not yet FIPS approved. I'll close this issue since this is already tracked elsewhere. Please reopen if you think this hasn't been addressed and add any details in the remaining issues.