Skip to content

Add arm64 mac and linux wheels #954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
elprans merged 2 commits into from
Sep 27, 2022
Merged

Add arm64 mac and linux wheels #954

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0f98dc0
Add arm64 mac and linux wheels
ddelange Sep 17, 2022
50277d9
Maximize parallellization (#2)
ddelange Sep 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Maximize parallellization (#2) 
* Maximize parallellization

* Bump cibuildwheel==2.10.2

* Prepare for merge
  • Loading branch information
@ddelange
ddelange authored Sep 25, 2022
commit 50277d94af86294b3496965eef8fcba1201f78b3
52 changes: 29 additions & 23 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:
submodules: true

- name: Set up Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4

- name: Build source distribution
run: |
Expand All @@ -68,28 +68,35 @@ jobs:
name: dist
path: dist/*.tar.*

build-wheels:
build-wheels-matrix:
needs: validate-release-request
runs-on: ubuntu-latest
outputs:
include: ${{ steps.set-matrix.outputs.include }}
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v4
- run: pip install cibuildwheel==2.10.2
- id: set-matrix
run: |
MATRIX_INCLUDE=$(
{
cibuildwheel --print-build-identifiers --platform linux --arch x86_64,aarch64 | grep cp | jq -Rc '{"only": inputs, "os": "ubuntu-latest"}' \
&& cibuildwheel --print-build-identifiers --platform macos --arch x86_64,arm64 | grep cp | jq -Rc '{"only": inputs, "os": "macos-latest"}' \
&& cibuildwheel --print-build-identifiers --platform windows --arch x86,AMD64 | grep cp | jq -Rc '{"only": inputs, "os": "windows-latest"}'
} | jq -sc
)
echo ::set-output name=include::"$MATRIX_INCLUDE"

build-wheels:
needs: build-wheels-matrix
runs-on: ${{ matrix.os }}
continue-on-error: true
name: Build ${{ matrix.only }}

strategy:
matrix:
include:
- {os: macos-latest, arch: x86_64, build: "*"}
- {os: macos-latest, arch: arm64, build: "*"}
- {os: windows-latest, arch: AMD64, build: "*"}
- {os: windows-latest, arch: x86, build: "*"}
- {os: ubuntu-latest, arch: x86_64, build: "*"}
- {os: ubuntu-latest, arch: aarch64, build: "*[61]-manylinux*"}
- {os: ubuntu-latest, arch: aarch64, build: "*[72]-manylinux*"}
- {os: ubuntu-latest, arch: aarch64, build: "*[83]-manylinux*"}
- {os: ubuntu-latest, arch: aarch64, build: "*[94]-manylinux*"}
- {os: ubuntu-latest, arch: aarch64, build: "*[05]-manylinux*"}
- {os: ubuntu-latest, arch: aarch64, build: "*[61]-musllinux*"}
- {os: ubuntu-latest, arch: aarch64, build: "*[72]-musllinux*"}
- {os: ubuntu-latest, arch: aarch64, build: "*[83]-musllinux*"}
- {os: ubuntu-latest, arch: aarch64, build: "*[94]-musllinux*"}
- {os: ubuntu-latest, arch: aarch64, build: "*[05]-musllinux*"}
include: ${{ fromJson(needs.build-wheels-matrix.outputs.include) }}

defaults:
run:
Expand All @@ -108,12 +115,11 @@ jobs:
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v2

- uses: pypa/cibuildwheel@v2.10.0
- uses: pypa/cibuildwheel@v2.10.2
with:
only: ${{ matrix.only }}
env:
CIBW_BUILD_VERBOSITY: 1
CIBW_ARCHS: ${{ matrix.arch }}
CIBW_BUILD: ${{ matrix.build }} # py versions from setup.py -> python_requires
CIBW_SKIP: pp*
CIBW_MANYLINUX_X86_64_IMAGE: manylinux_2_28
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux_2_28
Copy link
Copy Markdown
Contributor Author

@ddelange ddelange Sep 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could remove these two completely, defaulting to manylinux_2_21 (which is built on centos RHEL7), also wrt backward compatibility ref giampaolo/psutil#2103 (comment)

Though on centos, CI also has postgres installation script issues (I started out with it), something about a bad GPG key also mentioned on the developer's 3rd party yum repo's issue tracker. Thought about doing a yum search (or smth) to download the exact URL for the respective postgres versions on the official repo.

In the end, I happily accepted gg when I saw the green ticks on CI as it is now.

Martin.Obiols.-.Can.t.stop.laughing.at.this.1255206061159714819.mp4

Copy link
Copy Markdown
Member

@elprans elprans Sep 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should try and keep compatibility with 2_21 at least.

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

Copy link
Copy Markdown
Contributor Author

@ddelange ddelange Sep 27, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, I think this thread is irrelevant, the current state of this PR (so building on almalinux8) already produces wheels that are backward compatible all the way to 2_17 ref https://github.com/ddelange/asyncpg/actions/runs/3121931647/jobs/5063553875#step:4:950

Copy link
Copy Markdown
Contributor Author

@ddelange ddelange Sep 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elprans I think in this respect we're good to go 👍


Expand All @@ -137,7 +143,7 @@ jobs:
submodules: true

- name: Set up Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: 3.8

Expand Down