Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gcc: update ARM branch to latest backport #89857

Closed
wants to merge 2 commits into from

Conversation

fxcoudert
Copy link
Member

I've updated the ARM backport to GCC 11.2.0. This should allow to build a bottle on ARM Monterey, now.

@carlocab carlocab added CI-long-timeout [DEPRECATED] Use longer GitHub Actions CI timeout. CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. CI-no-bottles Merge without publishing bottles CI-skip-dependents Pass --skip-dependents to brew test-bot. CI-skip-recursive-dependents Pass --skip-recursive-dependents to brew test-bot. and removed CI-skip-recursive-dependents Pass --skip-recursive-dependents to brew test-bot. CI-skip-dependents Pass --skip-dependents to brew test-bot. CI-no-bottles Merge without publishing bottles labels Nov 23, 2021
@carlocab
Copy link
Member

Oops; I thought we could skip publishing bottles and not test dependents. I forgot this affects ARM Big Sur. Do we not want a revision bump?

@carlocab carlocab added the CI-linux-self-hosted Build on Linux self-hosted runner label Nov 23, 2021
@fxcoudert
Copy link
Member Author

Regarding revision: maybe you're right, I hesitated and don't know what's best. On the other hand, if there has been no recent complaint about GCC being broken, it means the current bottles were doing a good job (even on Monterey), doesn't it?

@carlocab
Copy link
Member

Hmmmm, maybe. I guess we can always bump the revision later if needed, but there's no taking them back when we've already bumped it.

@fxcoudert
Copy link
Member Author

fxcoudert commented Nov 23, 2021

Failure in stage1 libgomp, because stage1 gcc is complaining about finding libgcc_s.1.1.dylib.
This was passing for me locally before (built outside of Homebrew), so I'll have to understand where the different comes from.

configure:4074: ./conftest
dyld[78308]: Library not loaded: @rpath/libgcc_s.1.1.dylib
  Referenced from: /private/tmp/gcc-20211123-41856-gy4s74/gcc-gcc-11.2.0-arm-20211123/build/aarch64-apple-darwin21/libgomp/conftest
  Reason: tried: '/usr/local/lib/libgcc_s.1.1.dylib' (no such file), '/usr/lib/libgcc_s.1.1.dylib' (no such file)
/private/tmp/gcc-20211123-41856-gy4s74/gcc-gcc-11.2.0-arm-20211123/libgomp/configure: line 4076: 78308 Abort trap: 6           ./conftest$ac_cv_exeext

Upstream issue: iains/gcc-darwin-arm64#50

@fxcoudert
Copy link
Member Author

Found the problem, fixed it, let's try again.

@carlocab
Copy link
Member

Monterey ARM build now testing dependents 🎉

@fxcoudert
Copy link
Member Author

  • On Big Sur (ARM), no regression
  • On Monterey (ARM), bottling fine, two dependents fail: libgccjit and root.

I'm updating libgccjit as part of the same PR, to keep it in sync with gcc.

root failure will be expected until we can bottle it, but we must bottle dependents first, and for that we need gcc. So it will have to be like that for a while.

@carlocab
Copy link
Member

Linux job died while trying to start up. Restarting.

@SMillerDev
Copy link
Member

@dawidd6 another instance of the issue I mentioned yesterday: https://github.com/Homebrew/homebrew-core/runs/4312017375?check_suite_focus=true

@carlocab
Copy link
Member

carlocab commented Nov 24, 2021

Yea, it won't go away. I'm logging on to the runner and deleting that directory. (That won't break anything, will it? Well, not like it's working right now...)

Edit:

/home/actions/actions-runner/_work/homebrew-core/homebrew-core is just a symlink to brew --repository homebrew/core, so I imagine it should be fine.

carlocab added a commit to carlocab/homebrew-core that referenced this pull request Nov 24, 2021
This should hopefully fix the `brew test` failure:

    Library not loaded: @rpath/libcsnd6.6.0.dylib

Seen at Homebrew#89857.
@carlocab carlocab mentioned this pull request Nov 24, 2021
6 tasks
iMichka pushed a commit that referenced this pull request Nov 24, 2021
This should hopefully fix the `brew test` failure:

    Library not loaded: @rpath/libcsnd6.6.0.dylib

Seen at #89857.
@carlocab
Copy link
Member

We have some formulae that look like they can be bottled on ARM Big Sur when this is merged:

and on ARM Monterey:

augustus
datetime-fortran
fastme
giza
imake
kim-api
lapack
mpich
packmol
plplot

@fxcoudert
Copy link
Member Author

On ARM Monterey we have:

brew install --build-from-source arrayfire
brew test --retry --verbose astrometry-net
brew install --build-from-source blaze
brew test --retry --verbose csound
brew linkage --cached --test --strict dlib
brew install --build-from-source dungeon
brew install --build-from-source dynare
brew linkage --cached --test --strict field3d
brew linkage --cached --test --strict gmic
brew linkage --cached --test --strict grace
brew install --build-from-source hdf5-mpi
brew install --build-from-source [email protected]
brew install --build-from-source [email protected]
brew linkage --cached --test --strict igraph
brew install --build-from-source itpp
brew install --build-from-source kahip
brew install --build-from-source libtensorflow
brew install --build-from-source mpdviz
brew install --build-from-source mrbayes
brew install --build-from-source mvtools
brew linkage --cached --test --strict nco
brew install --build-from-source nwchem
brew install --build-from-source openfast
brew linkage --cached --test --strict osmcoastline
brew install --build-from-source raxml-ng
brew test --retry --verbose root
brew install --build-from-source sratoolkit
brew install --build-from-source veclibfort

which I think is not too bad :)

@carlocab Where is your list from? I think we have a lot more formulas to bottle once gcc is in on ARM Monterey, including open-mpi, openblas… and then all those that depend on that (numpy, scipy, etc)

@fxcoudert
Copy link
Member Author

On Linux, though, we have Error: 129 failed steps! 😢

@carlocab
Copy link
Member

carlocab commented Nov 25, 2021

The list is on the CI summary page: https://github.com/Homebrew/homebrew-core/actions/runs/1499347701

I didn't have the annotations set up correctly (fixed in Homebrew/homebrew-test-bot#730), so they're not showing up correctly. However the formulae I listed are all successful attempts at source builds (with passing brew linkage and brew test).

Ah, I see why my list is incomplete: https://github.com/Homebrew/homebrew-test-bot/blob/f213e518037d54b43317db570e4e5ea8cb26e335/lib/tests/formulae_dependents.rb#L214-L220 it checks that all dependencies are bottled, but that won't be true about GCC obviously.

Homebrew/homebrew-test-bot#732

carlocab added a commit to carlocab/homebrew-test-bot that referenced this pull request Nov 25, 2021
We might be running in a PR where a formula is bottled for the first
time (e.g. Homebrew/homebrew-core#89857), so requiring that all
dependencies are bottled is unnecessarily restrictive.
@carlocab
Copy link
Member

On Linux, though, we have Error: 129 failed steps! 😢

The Java-related ones are probably from a miscompiled openjdk. Should be fixed in #89642.

@carlocab
Copy link
Member

All the Linux brew test failures are Java-dependents, except for:

beagle
bnfc
fauna-shell
mrbayes
mx
pegtl

fauna-shell fails from what looks like a deadlock. (It holds up brew test forever until you kill it manually, which I did yesterday.)

@Bo98
Copy link
Member

Bo98 commented Nov 25, 2021

Regarding revision: maybe you're right, I hesitated and don't know what's best.

An argument could be that this is effectively a version bump from 11.1.0 to 11.2.0 on arm64 so users may want the changes.

On the other hand, if there has been no recent complaint about GCC being broken, it means the current bottles were doing a good job (even on Monterey), doesn't it?

Probably. It will have been mostly limited to things like -mmacosx-version-min=12 not working I think, which will affect people though not many will be doing that so early.

The default SDK selection could be important for those building against system libcurl, sqlite3 or expat (which were updated in Monterey), but we don't force people to reinstall when they upgrade macOS so that doesn't really matter for now.

@iMichka
Copy link
Member

iMichka commented Nov 25, 2021

I'm fine to merge this as-is for Linux. I am working on the openjdk failures. Once these are gone this should give us more visibility about what remains to be fixed, and I'll go through the list over the next pull requests. It's a slow process because some of these take time to be fixed.

@fxcoudert
Copy link
Member Author

Any idea why the Intel Monterey node so much slower than all other runners?

@Bo98
Copy link
Member

Bo98 commented Nov 25, 2021

That's odd - it's definitely not stuck as I see it just moved on to opencv@3 right now. Might get a better idea when it's finished and we can see timestamps of each step.

@carlocab
Copy link
Member

Any idea why the Intel Monterey node so much slower than all other runners?

test-bot attempts a source build for every formula that has no bottle but has bottled dependencies. We could probably tweak that behaviour.

https://github.com/Homebrew/homebrew-test-bot/blob/f213e518037d54b43317db570e4e5ea8cb26e335/lib/tests/formulae_dependents.rb#L79-L86

@BrewTestBot
Copy link
Member

:shipit: @fxcoudert has triggered a merge.

@fxcoudert fxcoudert deleted the gcc branch November 25, 2021 13:57
carlocab added a commit to carlocab/homebrew-test-bot that referenced this pull request Nov 25, 2021
I trialled this behaviour in Homebrew/homebrew-core#89857, and it adds a
significant amount of time to the CI run. I'm still interested in
automating the detection of formulae that should be bottled, but I think
it requires a more nuanced approach.

In the meantime, let's restore the old behaviour that dependents are
built from source only when requested via a flag.

Closes Homebrew#731.
@carlocab
Copy link
Member

@Homebrew/core: in case you haven't noticed yet, we now have annotations from some [*] CI failures. See the "Files" tab in this PR for an example.

Note that annotations will show up even for formulae that the PR doesn't modify, so you'll need to scroll down past the modified formulae to see them, as you'll see here. There are no annotations on the mobile app in case you're reviewing PRs on that. You can see all the annotations on the workflow summary page. There should actually be more in this PR but I think GitHub gave up after the first 50 or so.

There are three types of annotations: errors (red), warnings (yellow), and notices (black). Errors are failures we shouldn't ignore and warnings are errors we can ignore but should try to do something about. Notices will be rare. Currently they will only show up for formulae that should have bottles but don't.

Let me know if you have any thoughts about how to make these more useful.

[*] Primarily brew install, brew linkage, and brew test. No brew audit or brew livecheck yet, but those can easily be added.

@github-actions github-actions bot added the outdated PR was locked due to age label Dec 28, 2021
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 28, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
CI-linux-self-hosted Build on Linux self-hosted runner CI-long-timeout [DEPRECATED] Use longer GitHub Actions CI timeout. CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. outdated PR was locked due to age
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants