Android: add aar dependencies support, create proper build pipeline, add Jetpack Compose example

[0xnm]

This PR changes a lot of things in the Android build support (mainly focusing on creating proper build pipeline). In details:

• aar dependencies processing
• Manifest merging
• Setting min/target/compile SDK properties in the build script properties instead of manifest
• Proper work with aapt2 at the compile/linkstages
• R classes generation for the libraries (hacky though)
• Proguard definitions collection. NB: there is no support of obfuscation in this PR, Proguard here is supported only for the main DEX packing.
• Clear separation between debug and release signing configs (release key shouldn't be a part of the required setup; user should be able to bootstrap project quickly and create release key only when it is really needed).
• Changes to the Android build script APIs
• Introduction of the debug vs release flags for aapt2 and dexing
• Plus it adds Jetpack Compose sample

This PR doesn't add yet support of the transitive Android modules, this is a work to be done in the future.

The code quality is maybe meh, but I expect many pieces here to be changed in the future to be better and have wider support of the different build options/flags.

The most important part this PR does is mentioned above: it brings a proper pipeline. I added many flags/comments for the future work, allowing contributors to focus on the specific parts.

One thing to mention as well is that in the current state the incremental builds will have terrible performance. This is the area which requires a massive time investment to be on-par with AGP to support only processing of the files in the pipeline which really changed. Because of this the output types of certain tasks will probably change in the future or they will be split into more granular tasks.

I would like also to mention other things which should be improved:

• Things like def bundleToolVersion = "1.17.2" should be pre-defined in the Android SDK module and they shouldn't be required for writing build script, because:
  • in this particular example bundle is not needed for APK modules or lib modules;
  • these things are really technical and it is not easy to get versions list without knowing artifact coordinates
  • ideally Mill should be able to fetch latest versions of such tooling by itself
• Project structure in the Android examples created so far is very different from project structure created by Android Studio, this may bring additional complexity migrating existing projects. Maybe it is worth to create something similar to MavenModule?
• Successful APK/bundle compilation doesn't guarantee that everything is alright, app may crash at runtime because something wasn't packed correctly. It is needed to add integration tests support on the Android emulator as a part of CI pipeline
• IMPORTANT: Release key tasks for getting key alias/pass/store pass are leaking sensitive credentials, because outputs will be serialized to JSON. Sensitive data should never be exposed (let it be disk or console output).
• Maybe it is worth to create mill.androidlib?
• I think it is not worth to focus on Java samples right now, given that Kotlin is the preferred language for the Android development.
• I don't thing creating many traits like AndroidAppBundle is a good idea, because it is just a way of packaging. AGP has only 2 main entrypoints: Application and Library, the rest is their configuration options. Bazel also has application and library as main entrypoints for rules (see https://github.com/bazelbuild/rules_android/tree/main/rules).
• Default build tools version should be inferred from compileSdk version (e.g compileSdk is 35, then buildToolsVersion should be 35.0.0 by default), but in the current layout it is not possible to do implicitly, so both versions should be specified.

[0xnm]

Android examples are failing, because work with emulator is flaky - it seems that, for example, it starts very slow, causing timeout. This is not related to the changes in this PR.

[lihaoyi]

I think it looks reasonable for me, @vaslabs would be great if you could give this a review pass (whenever you're available) before I merge it

[sideeffffect]

In Scala, it is customary to achieve this via

trait AndroidAppModule extends JavaModule { self =>

or

trait AndroidAppModule extends JavaModule { parent =>

if you will

[vaslabs]

I wonder if we should have similar behaviour with gradle/android studio and store this in ~/.android

[vaslabs]

I was also playing with instant APK feature, it could solve this problem

[vaslabs]

LGTM, nice, only left one comment but in general we can keep working on any rough edges in subsequent PRs. I've also tested it locally!

[lihaoyi]

Thanks @vaslabs ! Let me kick CI a few times to try and get a green before merging

[lihaoyi]

@0xnm example.android.kotlinlib[1-hello-kotlin].local.test seems to be failing pretty persistently in CI; does it work on your machine?

[vaslabs]

@0xnm example.android.kotlinlib[1-hello-kotlin].local.test seems to be failing pretty persistently in CI; does it work on your machine?

indeed, when ran concurrently

[3166]   keytool error: java.lang.Exception: Key pair not generated, alias <mill-android> already exists

this was already mentioned in one of the methods. Perhaps a quick fix, if not adding more work for a central debug keystore, we can have an option to have different aliases for different apps.

In general we need to find a way to test these usecases that rely on shared resources (I had the same issue with the emulator which right now also causes a lot of flakiness) without conflicts .

[0xnm]

example.android.kotlinlib[1-hello-kotlin].local.test seems to be failing pretty persistently in CI; does it work on your machine?

yes, it did.

I guess there is some flakiness, I will try to address it as a part of this PR, hold on with the merge.

[lihaoyi]

If it's concurrency related we could try setting -j1 in the android build github action

mill/.github/workflows/run-tests.yml

Line 80 in d2df820

millargs: "'example.android.__.local.test'"

 millargs: "-j1 'example.android.__.local.test'" 

[0xnm]

Indeed, the concurrency of the tests run is the issue. The problem here is that AVD/Emulator is the shared resource: it can only be a single instance of emulator on the particular port; it can be only the single instance of AVD with the given name.

It means, that if certain Android example tests are running in parallel, they can mess up with each other, because they are managing the same AVD and the same emulator. They (or even better Android hadling logic in Mill) should cover "non-happy" cases (like AVD with the given name already exists when trying to create it, etc.).

For the moment I've added -j1 flag and it helped, execution is more stable (although there is still a timeout, but at least it shows that [1-hello-kotlin] runs successfully, timeout in the latest run is for [1-hello-world]), but not the way proposed - mill.testargs are supplied to the selective.run / selective.resolve and this command is greedy, it treats everything after if as a part of its arguments).

[lihaoyi]

Apart from -j1, we also have Task.Command(exclusive = true), which we can use to mark tasks as running single-threaded even if the other tasks run in parallel

[vaslabs]

I've configured each task (one in kotlin on in java) that uses the emulator to use a different one and on different port. but I think the issue is that both need 1.5GB of ram and it cannot be reduced. I've seen in my branch that if the job is run on its own, it usually passes.
I think on one of subsequent PRs, we can change to use a common emulator that is set up before the action runs

[lihaoyi]

I'm going to go ahead and merge this, since example.android.javalib[1-hello-world].local.test seems pretty busted in main anyway, so this PR doesn't seem to make things appreciable worse. We can fix the flakiness or breakage in a follow up