[Python] Cannot generate code from proto during installation with protobuf 3.20.2

[Abacn]

apache/beam#23227 raising here and investigating at the same time

What version of protobuf and what language are you using?
Version: v3.20.2
which is released on Sept 13th.

Language: Python
What operating system (Linux, Windows, ...) and version?
Linux/Windows/MacOS

What runtime / compiler are you using (e.g., python version or gcc version)
Python3.7/3.8/3.9

What did you do?

Observed on Apache Beam which relies on protobuf to generate code during installation: apache/beam#23227

Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

What did you expect to see

Code generated

What did you see instead?

Code not generated and causing installation fail.

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).

Anything else we should know about your project / environment

[deannagarcia]

I'm sorry about this issue, I'm looking into the problem now. What operating system/architecture are you using?

[Abacn]

Thanks for attention. We experienced test failure on all test platforms (Linux/Windows/MacOS): https://github.com/apache/beam/actions/runs/3049863679 so this is not platform specific. apache/beam#23227 had more details

[deannagarcia]

I believe the issue here was that we had not released all the artifacts. I uploaded the missing artifacts, and am closing the issue but definitely reopen the issue if you still see this error.

[Abacn]

Thanks @deannagarcia. Validated tests now passing with 3.20.2

[AnandInguva]

Hi, the same issue is happening again for Python 3.10 on MacOS

Failed run: https://github.com/apache/beam/actions/runs/3230139132/jobs/5288244948. Downloads protobuf-3.20.2-py2.py3-none-any.whl (162 kB) on MacOS - October 11

Successful run: https://github.com/apache/beam/actions/runs/3093512313/jobs/5005959441. Downloads protobuf-3.20.2-cp310-cp310-macosx_10_9_universal2.whl(1.1 MB) - This one was run on September 20.

[deannagarcia]

Can you open a new bug with a simple reproduction case? Make sure you are redownloading the protobuf package since some of them have changed.

[AnandInguva]

Can you open a new bug with a simple reproduction case? Make sure you are redownloading the protobuf package since some of them have changed.

Sure I will.

I have one question. I can't find the wheels for MacOS on Python 3.10. Is it intended or is the artifact missing?

[haberman]

We intentionally removed the macOS universal2 wheel for 3.20.3, because it was broken for M1 machines (see: #10443).

It was unfortunately not feasible to just fix the universal2 wheel, because it was built by an old build system that was unmaintainable and not well understood. There was no clear explanation for what would have changed since 3.20.2 that would break the universal2 wheel for M1, and no clear path to a fix. So we pulled the universal2 wheel, since the none-any wheel should provide equivalent functionality (albeit with much worse performance).

So anyone still on the 3.20.x line will need to either use the none-any wheel or upgrade to Protobuf 4.21.x, which does not have this problem.

Sorry for the inconvenience.