Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update no jitpack buld for monorepo #23325

Conversation

mchowning
Copy link
Contributor

@mchowning mchowning commented Jun 19, 2020

Description

This PR addresses two issues that were found in the no-jitpack WPAndroid builds with the monorepo:

  1. The bundle would not get included in the apk, so the app would crash upon loading the Gutenberg editor; and
  2. The build was always rebuilding the bundle even when there were no changes.

In addition, while investigating the always-rebuilding issue, I discovered a problem where the bundle would not get properly generated when a build was run with the clean task. That is also fixed in this PR.

How has this been tested?

For all of these tests:

  1. the WPAndroid app must be built with wp.BUILD_GUTENBERG_FROM_SOURCE either (a) missing or (b) set to false in the gradle.properties file. These changes should have no effect when that variable is set to true; and
  2. The bundler should not be running on the computer. We want to rely solely on the bundle that is included in the apk when testing this PR.

This is just a proposed set of commands that should cover all of the edge cases I have encountered, but it's not a bad idea to try different combinations since that could catch something I missed. All these commands should be run from the monorepo branch for WPAndroid, but with this fix_no_jitpack_build branch checked out for gutenberg.

  1. Clean the project: ./gradlew clean
  2. Build and install: ./gradlew installWasabiDebug
  3. Run the app and confirm you can open the Gutenberg editor without crashing (establishes that the bundle is being included in the apk)
  4. Build the app again with the same command ./gradlew installWasabiDebug
  5. Observe that this build takes under 1 minute whereas the first build took multiple minutes because this build is not regenerating the bundle file (establishes that the build detected there were no changes that would affect the bundle file)
  6. Edit any file within the gutenberg-mobile or gutenberg projects that should alter the resulting bundle other than a file within a node_modules/ folder (the build is just checking the package.json and package-lock.json files for dependency changes).
  7. Build the app again with the same command ./gradlew installWasabiDebug
  8. Observe that the app takes multiple minute to build this time because it detected the change and is regenerating the bundle (establishes that the build detected there were no changes that would affect the bundle file).
  9. Run clean, build, and install in a single command: ./gradlew clean installWasabiDebug
  10. Run the app and confirm you can open the Gutenberg editor without crashing (establishes does not just check whether the bundle file exists at the beginning of the build, but instead checks it after the "clean" task has run).
  11. Try building the Vanilla variant. Ideally, do a release build, but if your environment isn't set up for that a debug build is fine: ./gradlew installVanillaRelease
  12. Run the app and confirm you can open the Gutenberg editor without crashing.

Checklist:

  • My code is tested.
  • My code follows the WordPress code style.
  • My code follows the accessibility standards.
  • My code has proper inline documentation.
  • I've included developer documentation if appropriate.
  • I've updated all React Native files affected by any refactorings/renamings in this PR.

Also added a check to insure that this kind of mistake gets caught
quickly in the future since otherwise it only reveals itself at runtime.
@mchowning mchowning added the Mobile App - i.e. Android or iOS Native mobile impl of the block editor. (Note: used in scripts, ping mobile folks to change) label Jun 19, 2020
@mchowning mchowning requested review from hypest and cameronvoell June 19, 2020 19:06
@github-actions
Copy link

github-actions bot commented Jun 19, 2020

Size Change: 0 B

Total Size: 1.12 MB

ℹ️ View Unchanged
Filename Size Change
build/a11y/index.js 1.14 kB 0 B
build/annotations/index.js 3.62 kB 0 B
build/api-fetch/index.js 3.4 kB 0 B
build/autop/index.js 2.83 kB 0 B
build/blob/index.js 620 B 0 B
build/block-directory/index.js 7.26 kB 0 B
build/block-directory/style-rtl.css 937 B 0 B
build/block-directory/style.css 937 B 0 B
build/block-editor/index.js 106 kB 0 B
build/block-editor/style-rtl.css 10.7 kB 0 B
build/block-editor/style.css 10.7 kB 0 B
build/block-library/editor-rtl.css 7.86 kB 0 B
build/block-library/editor.css 7.87 kB 0 B
build/block-library/index.js 129 kB 0 B
build/block-library/style-rtl.css 8 kB 0 B
build/block-library/style.css 8.01 kB 0 B
build/block-library/theme-rtl.css 730 B 0 B
build/block-library/theme.css 732 B 0 B
build/block-serialization-default-parser/index.js 1.88 kB 0 B
build/block-serialization-spec-parser/index.js 3.1 kB 0 B
build/blocks/index.js 48.1 kB 0 B
build/components/index.js 196 kB 0 B
build/components/style-rtl.css 15.9 kB 0 B
build/components/style.css 15.8 kB 0 B
build/compose/index.js 9.6 kB 0 B
build/core-data/index.js 11.4 kB 0 B
build/data-controls/index.js 1.29 kB 0 B
build/data/index.js 8.44 kB 0 B
build/date/index.js 5.47 kB 0 B
build/deprecated/index.js 772 B 0 B
build/dom-ready/index.js 569 B 0 B
build/dom/index.js 3.17 kB 0 B
build/edit-navigation/index.js 8.26 kB 0 B
build/edit-navigation/style-rtl.css 1.02 kB 0 B
build/edit-navigation/style.css 1.02 kB 0 B
build/edit-post/index.js 303 kB 0 B
build/edit-post/style-rtl.css 5.5 kB 0 B
build/edit-post/style.css 5.5 kB 0 B
build/edit-site/index.js 16.6 kB 0 B
build/edit-site/style-rtl.css 3.03 kB 0 B
build/edit-site/style.css 3.03 kB 0 B
build/edit-widgets/index.js 9.33 kB 0 B
build/edit-widgets/style-rtl.css 2.43 kB 0 B
build/edit-widgets/style.css 2.43 kB 0 B
build/editor/editor-styles-rtl.css 468 B 0 B
build/editor/editor-styles.css 469 B 0 B
build/editor/index.js 44.8 kB 0 B
build/editor/style-rtl.css 3.8 kB 0 B
build/editor/style.css 3.8 kB 0 B
build/element/index.js 4.65 kB 0 B
build/escape-html/index.js 733 B 0 B
build/format-library/index.js 7.72 kB 0 B
build/format-library/style-rtl.css 542 B 0 B
build/format-library/style.css 543 B 0 B
build/hooks/index.js 2.13 kB 0 B
build/html-entities/index.js 622 B 0 B
build/i18n/index.js 3.56 kB 0 B
build/is-shallow-equal/index.js 711 B 0 B
build/keyboard-shortcuts/index.js 2.51 kB 0 B
build/keycodes/index.js 1.94 kB 0 B
build/list-reusable-blocks/index.js 3.12 kB 0 B
build/list-reusable-blocks/style-rtl.css 446 B 0 B
build/list-reusable-blocks/style.css 447 B 0 B
build/media-utils/index.js 5.29 kB 0 B
build/notices/index.js 1.79 kB 0 B
build/nux/index.js 3.4 kB 0 B
build/nux/style-rtl.css 662 B 0 B
build/nux/style.css 657 B 0 B
build/plugins/index.js 2.56 kB 0 B
build/primitives/index.js 1.5 kB 0 B
build/priority-queue/index.js 788 B 0 B
build/redux-routine/index.js 2.85 kB 0 B
build/rich-text/index.js 14 kB 0 B
build/server-side-render/index.js 2.68 kB 0 B
build/shortcode/index.js 1.7 kB 0 B
build/token-list/index.js 1.28 kB 0 B
build/url/index.js 4.06 kB 0 B
build/viewport/index.js 1.85 kB 0 B
build/warning/index.js 1.14 kB 0 B
build/wordcount/index.js 1.17 kB 0 B

compressed-size-action

}
}

def isRelevantFile = { it.name.endsWithAny('.js', '.css') || it.name ==~ /package(-lock)?\.json/ }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check has been working for us so far, but I wouldn't be surprised at all if there's a filetype that we've missed. Let me know if you can think of any other "relevant" files we should be checking for changes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should include ".scss", or just make it ends with "css" (without the ".")?

Copy link
Contributor Author

@mchowning mchowning Jun 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Good catch. I made the update in afc4d52. Went with adding ".scss" just for the sake of being super-explicit.

inputs.files(inputFiles())
def dirs = [jsRootDir]
file(jsRootDir).eachDirRecurse { dir ->
def isRelevantDir = !['react-native-bridge/android/build/intermediates',
Copy link
Contributor Author

@mchowning mchowning Jun 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm excluding react-native-bridge/android/build/intermediates here because these files were getting generated after a clean, which was throwing off the up-to-date check here since they are js files that are being generated:

build/intermediates/library_assets/debug/out/gutenberg-web-single-block/inject-css.js
build/intermediates/library_assets/debug/out/gutenberg-web-single-block/prevent-autosaves.js
build/intermediates/library_assets/debug/out/gutenberg-web-single-block/editor-style-overrides.css

I feel like we shouldn't need to check these for changes since they're generated files, but I'm not really sure what these are, so I wanted to call this out specifically in case it is something we shouldn't be excluding.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea these are related to unsupported blocks, and I think this is correct that we don't need to watch for changes on these generated files

@cameronvoell
Copy link
Member

cameronvoell commented Jun 19, 2020

This is definitely looking better. One thing I'm seeing, is that the second build after making a change doesn't seem to skip the npm_ci / buildJSBundle task as expected. See steps below:

  1. Verify we're in a state with builds working with ./gradlew installWasabiDebug sub 1 minute because we are not detecting any changes.
  2. Make a change to a .js file or something to trigger the extra bundle tasks (longer build)
  3. Then the next ./gradlew installWasabiDebug take a few minutes as expected because it is running the extra tasks. Build works successfully.
  4. Now we run ./gradlew installWasabiDebug a second time without making any changes, on this second run after a change, I am seeing the npm_ci task and buildJSBundle tasks run again, making the build take a few minutes. I expected it to already be sub 1 minute without this extra task.
  5. On the third run after the change was made, we are now returning to the sub 1 minute build.

@mchowning
Copy link
Contributor Author

mchowning commented Jun 19, 2020

One thing I'm seeing, is that the second build after making a change doesn't seem to skip the npm_ci / buildJSBundle task as expected.

I'm seeing a couple of things that could be causing this.

1. Package.json formatting changes get undone

One that I just realized is that npm install will update the formatting in the package.json file, so if the "change" we make is to change the formatting in that file, then the first build sees the change, but subsequently fixes it, and then the second build sees that the formatting has changed, so it redoes everything.

2. Package-lock.json is getting modified by the build

A second one is that I am seeing a change being introduced in gutenberg-mobile/package-lock.json. The line:

"version": "npm:[email protected]",

is being changed to:

"version": "npm:[email protected]",

during the build. So the first build sees the original line, then the second build sees that it changed and redoes everything. The third build finally sees no changes for the first time and gives us a fast(er) build.

I'm not seeing the long-second-build issue with these steps:

  1. Change something other than package.json
  2. Insure that gutenberg-mobile/package-lock.json has no changes.
  3. Run ./gradlew installWasabiDebug and see the slow build
  4. Observe that there are now changes to gutenberg-mobile/package-lock.json
  5. Revert the package-lock.json changes
  6. Run ./gradlew installWasabiDebug and see a fast build.

If undoing the change to package-lock.json resolves the issue for you too, I think the issue is with what npm is doing and not with this PR.

@mchowning
Copy link
Contributor Author

If undoing the change to package-lock.json resolves the issue for you too, I think the issue is with what npm is doing and not with this PR.

Or we could probably just stop checking the package-lock.json files for changes since I believe checking the package.json files should be sufficient.

@mchowning
Copy link
Contributor Author

mchowning commented Jun 19, 2020

Pushing 2 updates:

  • 26be253 ensures that we are ignoring changes under the bundle/ directory, including its subdirectories.
  • 47fc9a6 no longer checks package-lock.json files. Instead, we're back to just checking the package.json files. That's what we did before, and it seems like that should cover us as long as we rebuild anytime a package.json file changes.

@mchowning mchowning force-pushed the fix_no_jitpack_build branch from 5a60bfe to 0d0bc36 Compare June 20, 2020 01:47
@mchowning mchowning force-pushed the fix_no_jitpack_build branch from 0d0bc36 to 47fc9a6 Compare June 20, 2020 01:53
@hypest
Copy link
Contributor

hypest commented Jun 22, 2020

I can confirm that multiple invocations of ./gradlew installWasabiDebug show that only the first can be long, actually building the JS bundle.

Also, I can confirm that the non-from-source-mode WPAndroid app runs OK and the editor launches OK too without Metro running.

While at it @mchowning , what do you think about revising the "Building the Gutenberg Mobile JS bundle" message in this PR? With all the build optimizations in place, that message is a bit misleading. We could prolly change it to "Executing nested Gutenberg Mobile build tasks..." or something.

@mchowning
Copy link
Contributor Author

With all the build optimizations in place, that message is a bit misleading. We could prolly change it to "Executing nested Gutenberg Mobile build tasks..." or something.

Certainly up for improving that message, but I feel like if I saw "Executing nested Gutenberg Mobile build tasks..." there's a decent chance I would be misled into thinking that it was a from-source build (wp.BUILD_GUTENBERG_FROM_SOURCE=true). I'm trying to see if I can think of of a better message.

You mentioned that the current message is a bit misleading "with all the build optimizations in place" @hypest . Would you mind explaining a bit how it is misleading? I think that might help me think of some better wording for the message.

@hypest
Copy link
Contributor

hypest commented Jun 22, 2020

You mentioned that the current message is a bit misleading "with all the build optimizations in place" @hypest . Would you mind explaining a bit how it is misleading? I think that might help me think of some better wording for the message.

Oh, so, the current one ("Building the Gutenberg Mobile JS bundle") to me reads like the JS is actually being built, and the JS/RN tools being run. I would expect to see that printed only if the JS bundle is (re)built.

if I saw "Executing nested Gutenberg Mobile build tasks..." there's a decent chance I would be misled into thinking that it was a from-source build (wp.BUILD_GUTENBERG_FROM_SOURCE=true). I'm trying to see if I can think of of a better message.

Fair.
Actually, since we always do the "build" unless it has been suppressed, perhaps we should remove the message and only print it when suppressed?

Note, this is not a blocker and we can work on it in a follow up PR after the monorepo things get merged.

@mchowning
Copy link
Contributor Author

Regarding the "Building the Gutenberg Mobile JS bundle" message, I don't feel strongly but unless you have a suggestion you feel good or strongly about @hypest , let's just leave this to be addressed later. I haven't been able to come up with a message I like yet, and there's a decent chance this will be changing a bit anyway in the near future due to jitpack, so that will provide a good opportunity to improve the messaging.

@mchowning mchowning force-pushed the fix_no_jitpack_build branch from 90aa339 to 857f407 Compare June 22, 2020 16:50
Copy link
Contributor

@hypest hypest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fixes @mchowning !

✅ 3. Run the app and confirm you can open the Gutenberg editor without crashing (establishes that the bundle is being included in the apk)

First ^ build took 7m 9s for me.

✅ 5. Observe that this build takes under 1 minute whereas the first build took multiple minutes because this build is not regenerating the bundle file (establishes that the build detected there were no changes that would affect the bundle file)

Second build took 28s for me 🎉

✅ 6. Edit any file within the gutenberg-mobile or gutenberg projects that should alter the resulting bundle other than a file within a node_modules/ folder (the build is just checking the package.json and package-lock.json files for dependency changes).

Added a comment in the root src/index.js file.

✅ Observe that the app takes multiple minute to build this time because it detected the change and is regenerating the bundle (establishes that the build detected there were no changes that would affect the bundle file).

^ took 2m 7s for me.

✅ 10. Run the app and confirm you can open the Gutenberg editor without crashing (establishes does not just check whether the bundle file exists at the beginning of the build, but instead checks it after the "clean" task has run).

^ took 2m 54s for me and app/editor ran OK on device.

✅ 12. Run the app and confirm you can open the Gutenberg editor without crashing.

LGTM! :shipit:

@hypest
Copy link
Contributor

hypest commented Jun 22, 2020

Will merge with "preserve commits" now.

@hypest hypest merged commit 0cddbf0 into rnmobile/monorepo-merge-master-and-gbmobile-develop Jun 22, 2020
@hypest hypest deleted the fix_no_jitpack_build branch June 22, 2020 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Mobile App - i.e. Android or iOS Native mobile impl of the block editor. (Note: used in scripts, ping mobile folks to change)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants