Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-18322: [Python] Add PYARROW_WITH_FLIGHT to PyArrow C++ cmake #14642

Merged
merged 1 commit into from
Nov 15, 2022

Conversation

AlenkaF
Copy link
Member

@AlenkaF AlenkaF commented Nov 15, 2022

This PR adds the use of PYARROW_WITH_FLIGHT environment variable when building PyArrow C++. With this change PyArrow will be able to build successfully even when Arrow C++ is installed without Arrow Flight (example, JFrog artifactory) but ARROW_FLIGHT flag is not registered to be set to off.
Note: PYARROW_WITH_FLIGHT=0 is needed when building PyArrow.

I tested the change on my regular build with Arrow Flight but with removed libarrow_flight* libs. Without this patch I get:

CMake Error at /Users/alenkafrim/repos/arrow/dist/lib/cmake/ArrowFlight/ArrowFlightTargets.cmake:93 (message):
  The imported target "ArrowFlight::arrow_flight_shared" references the file

     "/Users/alenkafrim/repos/arrow/dist/lib/libarrow_flight.1100.0.0.dylib"

  but this file does not exist.  Possible reasons include:

  * The file was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and contained

     "/Users/alenkafrim/repos/arrow/dist/lib/cmake/ArrowFlight/ArrowFlightTargets.cmake"

  but not all the files it references.

Call Stack (most recent call first):
  /Users/alenkafrim/repos/arrow/dist/lib/cmake/ArrowFlight/ArrowFlightConfig.cmake:56 (include)
  CMakeLists.txt:262 (find_package)

With this patch PyArrow builds succesfully.

@github-actions
Copy link

@AlenkaF AlenkaF changed the title ARROW-18322: [Python] source installation from pip without flight ARROW-18322: [Python] Add PYARROW_WITH_FLIGHT to PyArrow C++ cmake Nov 15, 2022
@AlenkaF AlenkaF requested a review from kou November 15, 2022 10:06
Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Thanks!

@kou kou merged commit 8f43a9f into apache:master Nov 15, 2022
kou pushed a commit that referenced this pull request Nov 15, 2022
…14642)

This PR adds the use of `PYARROW_WITH_FLIGHT` environment variable when building PyArrow C++. With this change PyArrow will be able to build successfully even when Arrow C++ is installed without Arrow Flight ([example](https://github.com/OSGeo/gdal/blob/58ab718542a7b5ceb94e3f24a5acc0b18ac1efb2/docker/ubuntu-full/Dockerfile#L223-L231), JFrog artifactory) but `ARROW_FLIGHT` flag is not registered to be set to off.
Note: `PYARROW_WITH_FLIGHT=0` is needed when building PyArrow.

I tested the change on my regular build with Arrow Flight but with removed `libarrow_flight*` libs. Without this patch I get:
```
CMake Error at /Users/alenkafrim/repos/arrow/dist/lib/cmake/ArrowFlight/ArrowFlightTargets.cmake:93 (message):
  The imported target "ArrowFlight::arrow_flight_shared" references the file

     "/Users/alenkafrim/repos/arrow/dist/lib/libarrow_flight.1100.0.0.dylib"

  but this file does not exist.  Possible reasons include:

  * The file was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and contained

     "/Users/alenkafrim/repos/arrow/dist/lib/cmake/ArrowFlight/ArrowFlightTargets.cmake"

  but not all the files it references.

Call Stack (most recent call first):
  /Users/alenkafrim/repos/arrow/dist/lib/cmake/ArrowFlight/ArrowFlightConfig.cmake:56 (include)
  CMakeLists.txt:262 (find_package)
```

With this patch PyArrow builds succesfully.

Authored-by: Alenka Frim <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
@AlenkaF AlenkaF deleted the ARROW-18322 branch November 16, 2022 05:10
@ursabot
Copy link

ursabot commented Nov 16, 2022

Benchmark runs are scheduled for baseline = 77f099f and contender = 8f43a9f. 8f43a9f is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.6% ⬆️0.03%] test-mac-arm
[Finished ⬇️0.82% ⬆️0.82%] ursa-i9-9960x
[Finished ⬇️1.5% ⬆️0.03%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 8f43a9fc ec2-t3-xlarge-us-east-2
[Finished] 8f43a9fc test-mac-arm
[Finished] 8f43a9fc ursa-i9-9960x
[Finished] 8f43a9fc ursa-thinkcentre-m75q
[Finished] 77f099fb ec2-t3-xlarge-us-east-2
[Finished] 77f099fb test-mac-arm
[Finished] 77f099fb ursa-i9-9960x
[Finished] 77f099fb ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@ursabot
Copy link

ursabot commented Nov 16, 2022

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants