Best practices & definitions for working with Java modules

[hendrikebbers]

The issues #4239 and #4222 refactored the repository to use Java modules for all our modules. While this is already a really good step forward there are some details in the Java module topic that need to be discussed. This epic issue will collect all the points and issues that need to be defined.

Status Quo

Today the src/main/java (+ src/main/resources) content of all gradle modules in the project is defined as Java modules. Based on that you can find a module-info.java file in each src/main/java folder. If any of that modules will break with the definition of the java module definition or has a dependency that does not support Java modules our build will break. From my point of view this is not a problem but a feature. You can think about the Java module support that we have in our project as a full support (we create so called "explicit modules“).

Our dependencies

Some libs does not go that far and are only so called „automatic modules“. An automatic module only needs an unique module name that is defined by the Automatic-Module-Name property in the MANIFEST.MF file of the lib. Sadly several of our dependencies does not even have that minimal support. Next to this some dependencies even have a bigger problem: With Java modules (independent from explicit or automatic modules) no 2 modules are allowed to export the same packages. This „split package“ problem is sadly a common problem when migration libraries to the java module system.

Happily we solved all that issues for our dependencies. For some libs that had the „split package“ problem you can find alternative dependencies that simply migrate some Jars into a single on (example: grpc/grpc-java#3522 (comment)). Next to this we are using a gradle plugin (https://github.com/gradlex-org/extra-java-module-info) that automatically converts libraries to automatic modules. You can find the usage at buildSrc/src/main/kotlin/com.hedera.hashgraph.jpms-modules.gradle.kts

Our tests

For tests we added the support for test fixtures. This is a general gradle feature (https://docs.gradle.org/current/userguide/java_testing.html#sec:java_test_fixtures) and is not specific to java modules. By using test fixtures utility functionality for tests can be extracted and shared with several modules. Currently we have defined a test fixtures set for the gradle hedera-mono-service module. The source set of the test fixture can be found at hedera-node/hedera-mono-service/src/testFixtures/java/. Since this set contains a module-info.java file the test fixture is handled as a java module.

The unit tests of all new modules (hedera-*-service(-impl)) even have a module-info.java as part of all unit test sets (under src/test/java/). Based on that unit tests can not share the same (base) package as the module that should be tested.

For all other modules the tests (unit, integration, end-to-end) does not define java modules.

Limitations and issues

As said several of our initial dependencies does not support the java module standard. While the gradle plugin that we use is really helpful it is only a workaround. In a best case all our dependencies are at least automatic modules. Based on that I created some PRs for libraries that we are depend on to add support for automatic modules:

• using compile only for annotation dependencies that are used for static code analysis apache/incubator-tuweni#450
• Automatic-Module-Name added terl/lazysodium-java#122
• 7903440: JMH: Add automatic module names openjdk/jmh#90
• Automatic module name added offbynull/portmapper#48

We should decide if we want to continue to work on that to get all our dependencies support the java module system (at least as automatic modules).

But even automatic modules have some downsides. Based on the definition of java modules all automatic modules will be fully exposed to a module if that module requires at minimum 1 automatic module. More information can be found in the following issues: gradlex-org/extra-java-module-info#38. We should define how we want to handle automatic modules in future.

In general we should define how we handle tests (unit tests, integration tests & end-to-end test) and test fixtures in combination with Java modules. From my point of view this is a complex topic since every possible solutions has some benefits, downsides and pitfalls. I will add a more detailed description to the issue in new future. For a good support of test fixtures with modules I created gradlex-org/java-module-testing#4