Skip to content

👷 Build wheels for riscv64#88

Merged
weiji14 merged 15 commits intomainfrom
riscv64
Apr 8, 2026
Merged

👷 Build wheels for riscv64#88
weiji14 merged 15 commits intomainfrom
riscv64

Conversation

@weiji14
Copy link
Copy Markdown
Owner

@weiji14 weiji14 commented Mar 30, 2026

Support RISC-V instruction set architecture. Build on manylinux 2_39.

Tested that emulation works, but want to see if can get native builds via https://github.com/riseproject-dev/riscv-runner-images

@weiji14 weiji14 added this to the 0.1.0 milestone Mar 30, 2026
@weiji14 weiji14 self-assigned this Mar 30, 2026
Support RISC-V instruction set architecture. Build on manylinux 2_31.
@weiji14
Copy link
Copy Markdown
Owner Author

weiji14 commented Mar 30, 2026

Getting this error on actions/setup-python:

  Version 3.13 was not found in the local cache
  Error: The version '3.13' with architecture 'riscv64' was not found for Ubuntu 24.04.
  The list of all available versions can be found here: https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json

See issue at actions/setup-python#1288

and also an updated versions-manifest.json file with riscv64 arch.
weiji14 added 2 commits March 31, 2026 13:57
Specifically, the quay.io/repository/pypa/manylinux_2_39_riscv64 image which has linux/riscv64 docker builds.
Not much CUDA binaries for RISC-V yet.
@weiji14
Copy link
Copy Markdown
Owner Author

weiji14 commented Mar 31, 2026

Getting further along with some changes to maturin-action at PyO3/maturin-action#429. Now hitting into this error at https://github.com/weiji14/cog3pio/actions/runs/23776298510/job/69278840811?pr=88#step:5:221:

    × No solution found when resolving dependencies:
    ╰─▶ Because only the following versions of sccache are available:
            sccache<=0.10.0
            sccache==0.14.0
        and sccache>=0.10.0 has no wheels with a matching platform tag (e.g.,
        `manylinux_2_39_riscv64`), we can conclude that sccache>=0.10.0 cannot
        be used.
        And because you require sccache>=0.10.0, we can conclude that your
        requirements are unsatisfiable.
        hint: Wheels are available for `sccache` (v0.14.0) on the following
        platforms: `manylinux_2_17_aarch64`, `manylinux_2_17_i686`,
        `manylinux_2_17_x86_64`, `manylinux2014_aarch64`, `manylinux2014_i686`,
        `manylinux2014_x86_64`, `macosx_10_12_x86_64`, `macosx_11_0_arm64`,
        `win32`, `win_amd64`, `win_arm64`
  Error: The process '/usr/bin/docker' failed with exit code 1
      at ExecState._setResult (file:///home/runner/_work/_actions/weiji14/maturin-action/cd95c89b1d7a6b5d9377c6f26b76efdbaf516d8d/dist/index.js:44149:25)
      at ExecState.CheckComplete (file:///home/runner/_work/_actions/weiji14/maturin-action/cd95c89b1d7a6b5d9377c6f26b76efdbaf516d8d/dist/index.js:44132:18)
      at ChildProcess.<anonymous> (file:///home/runner/_work/_actions/weiji14/maturin-action/cd95c89b1d7a6b5d9377c6f26b76efdbaf516d8d/dist/index.js:44028:27)
      at ChildProcess.emit (node:events:508:28)
      at maybeClose (node:internal/child_process:1101:16)
      at ChildProcess._handle.onexit (node:internal/child_process:305:5)

Seems like there are no riscv64 wheels for sccache yet at https://pypi.org/project/sccache/0.14.0/#files. Could just disable sccache for now, or look into not using uv to install sccache, or get riscv64 wheels built 🤔 Edit: wheels are available now at https://pypi.org/project/sccache/0.14.0/#sccache-0.14.0-py3-none-manylinux_2_31_riscv64.whl

@weiji14
Copy link
Copy Markdown
Owner Author

weiji14 commented Mar 31, 2026

Woohoo, maturin build compile on RISC-V finished in just under 30min at https://github.com/weiji14/cog3pio/actions/runs/23780042947/job/69290397422?pr=88#step:5:761!

   Compiling cog3pio v0.1.0-beta.1 (/home/runner/_work/cog3pio/cog3pio)
    Finished `release` profile [optimized] target(s) in 24m 17s
📦 Wheel is eligible for a higher priority tag. You requested manylinux_2_39 but this wheel is eligible for manylinux_2_34
📦 Built wheel for abi3 Python ≥ 3.13 to dist/cog3pio-0.1.0b1-cp313-abi3-manylinux_2_39_riscv64.whl

That was for the GIL version though. The non free-threaded build I'll fix shortly.

@weiji14
Copy link
Copy Markdown
Owner Author

weiji14 commented Mar 31, 2026

Ok, confirmed that both non-free-threaded and free-threaded wheel builds on RISC-V works~. Both taking ~5min with sccache now enabled 🚀

Now just need to handle pytest not working on uraimo/run-on-arch-action. Actually could just copy all the apt install stuff and run thing natively, or get pre-built wheels from https://github.com/gounthar/riscv64-python-wheels 🤔

and make the RISC-V pytest checks run natively without going through QEMU.
@weiji14
Copy link
Copy Markdown
Owner Author

weiji14 commented Mar 31, 2026

and... pytest still doesn't work because the venv cannot be created with python3 -m venv .venv. Goes back to the actions/setup-python issue above in some sense. Error is at https://github.com/weiji14/cog3pio/actions/runs/23783180648/job/69300206400?pr=88#step:8:23:

Error: Command '['/home/runner/_work/cog3pio/cog3pio/.venv/bin/python3', '-m', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

There is a workaround listed at riseproject-dev/riscv-runner-images#28 which I could try, but it involves a bootstrap script...


Edit: venv creation works after riseproject-dev/riscv-runner-images#36, so reran CI. Now hitting into another error during install of the wheels at https://github.com/weiji14/cog3pio/actions/runs/23783180648/job/70155686708?pr=88#step:8:37:

Looking in links: dist
0 location(s) to search for versions of cog3pio:
Skipping link: none of the wheel's tags (cp313-cp313t-manylinux_2_39_riscv64) are compatible (run pip debug --verbose to show compatible tags): file:///home/runner/_work/cog3pio/cog3pio/dist/cog3pio-0.1.0b1-cp313-cp313t-manylinux_2_39_riscv64.whl
Skipping link: none of the wheel's tags (cp313-abi3-manylinux_2_39_riscv64) are compatible (run pip debug --verbose to show compatible tags): file:///home/runner/_work/cog3pio/cog3pio/dist/cog3pio-0.1.0b1-cp313-abi3-manylinux_2_39_riscv64.whl
Given no hashes to check 0 links for project 'cog3pio': discarding no candidates
ERROR: Could not find a version that satisfies the requirement cog3pio (from versions: none)
ERROR: No matching distribution found for cog3pio
Exception information:
Traceback (most recent call last):
  File "/home/runner/_work/cog3pio/cog3pio/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/home/runner/_work/cog3pio/cog3pio/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 174, in _add_to_criteria
    raise RequirementsConflicted(criterion)
pip._vendor.resolvelib.resolvers.RequirementsConflicted: Requirements conflict: SpecifierRequirement('cog3pio')

Maybe something I need to change about the manylinux_2_39 tag? There was that 📦 Wheel is eligible for a higher priority tag. You requested manylinux_2_39 but this wheel is eligible for manylinux_2_34 warning above during the wheel build step 🤔 Edit: no, it's because we're using the fallback system Python 3.12, but the wheel is Python 3.13+. Need to wait for actions/setup-python#1289 or ...

Get RISC-V builds of Python from https://github.com/riseproject-dev/python-versions, to get Python 3.13 instead of Python 3.12 system fallback on runner.
Patches 2bc61d1, got the order swapped. Also set PIP_PRE=1 to try and get `numpy-2.5.0.dev0-cp313-cp313-linux_riscv64.whl` from the extra-index-url.
@weiji14
Copy link
Copy Markdown
Owner Author

weiji14 commented Apr 7, 2026

Managed to pull the NumPy RISC-V wheel from https://gounthar.github.io/riscv64-python-wheels/simple/, but some other dependencies aren't compiled for RISC-V yet.

Looking in indexes: https://pypi.org/simple, https://gounthar.github.io/riscv64-python-wheels/simple/
Requirement already satisfied: cog3pio in ./.venv/lib/python3.13/site-packages (0.1.0b1)
Collecting mypy
  Downloading mypy-1.20.0-py3-none-any.whl.metadata (2.3 kB)
Collecting pytest
  Downloading pytest-9.0.2-py3-none-any.whl.metadata (7.6 kB)
Collecting numpy>=2.0 (from cog3pio)
  Downloading https://github.com/gounthar/riscv64-python-wheels/releases/download/v2026.04.06-cp313/numpy-2.5.0.dev0-cp313-cp313-linux_riscv64.whl (5.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.9/5.9 MB 9.6 MB/s  0:00:00
Collecting xarray>=2024.10.0 (from cog3pio)
  Downloading xarray-2026.2.0-py3-none-any.whl.metadata (12 kB)
Collecting typing_extensions>=4.6.0 (from mypy)
  Downloading typing_extensions-4.15.0-py3-none-any.whl.metadata (3.3 kB)
Collecting mypy_extensions>=1.0.0 (from mypy)
  Downloading mypy_extensions-1.1.0-py3-none-any.whl.metadata (1.1 kB)
Collecting pathspec>=1.0.0 (from mypy)
  Downloading pathspec-1.0.4-py3-none-any.whl.metadata (13 kB)
Collecting librt>=0.8.0 (from mypy)
  Downloading librt-0.8.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl.metadata (1.3 kB)
Collecting iniconfig>=1.0.1 (from pytest)
  Downloading iniconfig-2.3.0-py3-none-any.whl.metadata (2.5 kB)
Collecting packaging>=22 (from pytest)
  Downloading packaging-26.0-py3-none-any.whl.metadata (3.3 kB)
Collecting pluggy<2,>=1.5 (from pytest)
  Downloading pluggy-1.6.0-py3-none-any.whl.metadata (4.8 kB)
Collecting pygments>=2.7.2 (from pytest)
  Downloading pygments-2.20.0-py3-none-any.whl.metadata (2.5 kB)
Collecting pandas>=2.2 (from xarray>=2024.10.0->cog3pio)
  Downloading pandas-3.0.2.tar.gz (4.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 8.8 MB/s  0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [192 lines of output]
      + meson setup /tmp/pip-install-3quhlfwl/pandas_46b9b52635dc46139ff6465f997e808e /tmp/pip-install-3quhlfwl/pandas_46b9b52635dc46139ff6465f997e808e/.mesonpy-1_syvab6 -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=/tmp/pip-install-3quhlfwl/pandas_46b9b52635dc46139ff6465f997e808e/.mesonpy-1_syvab6/meson-python-native-file.ini
      The Meson build system
      Version: 1.11.0.rc3
      Source dir: /tmp/pip-install-3quhlfwl/pandas_46b9b52635dc46139ff6465f997e808e
      Build dir: /tmp/pip-install-3quhlfwl/pandas_46b9b52635dc46139ff6465f997e808e/.mesonpy-1_syvab6
      Build type: native build
      Project name: pandas
      Project version: 3.0.2
      C compiler for the host machine: /home/runner/.cargo/bin/sccache cc (gcc 13.3.0 "cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0")
      C linker for the host machine: cc ld.bfd 2.42
      C++ compiler for the host machine: /home/runner/.cargo/bin/sccache c++ (gcc 13.3.0 "c++ (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0")
      C++ linker for the host machine: c++ ld.bfd 2.42
      
      ../meson.build:2:0: ERROR: Compiler cython cannot compile programs.
      
      A full log can be found at /tmp/pip-install-3quhlfwl/pandas_46b9b52635dc46139ff6465f997e808e/.mesonpy-1_syvab6/meson-logs/meson-log.txt

It is time to remove xarray as a dependency, so pandas won't need to be compiled anymore. See #89

@weiji14
Copy link
Copy Markdown
Owner Author

weiji14 commented Apr 8, 2026

Nice, RISC-V native builds and tests are working 🎉

Details
Successfully installed cog3pio-0.1.0b1
Removed build tracker: '/tmp/pip-build-tracker-of50e7h6'
Looking in indexes: https://pypi.org/simple, https://gitlab.com/api/v4/projects/56254198/packages/pypi/simple/
Requirement already satisfied: cog3pio in ./.venv/lib/python3.13/site-packages (0.1.0b1)
Collecting mypy
  Downloading mypy-1.20.0-py3-none-any.whl.metadata (2.3 kB)
Collecting pytest
  Downloading pytest-9.0.3-py3-none-any.whl.metadata (7.6 kB)
Collecting numpy>=2.0 (from cog3pio)
  Downloading https://gitlab.com/api/v4/projects/56254198/packages/pypi/files/3dfbbda758b81fbbd59accdccab2b3debeb3b15b67e2d1b382f07d0f5db575f0/numpy-2.4.3-cp313-cp313-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl (11.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.7/11.7 MB 9.1 MB/s  0:00:01
Collecting typing_extensions>=4.6.0 (from mypy)
  Downloading typing_extensions-4.15.0-py3-none-any.whl.metadata (3.3 kB)
Collecting mypy_extensions>=1.0.0 (from mypy)
  Downloading mypy_extensions-1.1.0-py3-none-any.whl.metadata (1.1 kB)
Collecting pathspec>=1.0.0 (from mypy)
  Downloading pathspec-1.0.4-py3-none-any.whl.metadata (13 kB)
Collecting librt>=0.8.0 (from mypy)
  Downloading librt-0.8.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl.metadata (1.3 kB)
Collecting iniconfig>=1.0.1 (from pytest)
  Downloading iniconfig-2.3.0-py3-none-any.whl.metadata (2.5 kB)
Collecting packaging>=22 (from pytest)
  Downloading packaging-26.0-py3-none-any.whl.metadata (3.3 kB)
Collecting pluggy<2,>=1.5 (from pytest)
  Downloading pluggy-1.6.0-py3-none-any.whl.metadata (4.8 kB)
Collecting pygments>=2.7.2 (from pytest)
  Downloading pygments-2.20.0-py3-none-any.whl.metadata (2.5 kB)
Downloading mypy-1.20.0-py3-none-any.whl (2.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 9.0 MB/s  0:00:00
Downloading pytest-9.0.3-py3-none-any.whl (375 kB)
Downloading pluggy-1.6.0-py3-none-any.whl (20 kB)
Downloading iniconfig-2.3.0-py3-none-any.whl (7.5 kB)
Downloading librt-0.8.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl (218 kB)
Downloading mypy_extensions-1.1.0-py3-none-any.whl (5.0 kB)
Downloading packaging-26.0-py3-none-any.whl (74 kB)
Downloading pathspec-1.0.4-py3-none-any.whl (55 kB)
Downloading pygments-2.20.0-py3-none-any.whl (1.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 7.0 MB/s  0:00:00
Downloading typing_extensions-4.15.0-py3-none-any.whl (44 kB)
Installing collected packages: typing_extensions, pygments, pluggy, pathspec, packaging, numpy, mypy_extensions, librt, iniconfig, pytest, mypy

Successfully installed iniconfig-2.3.0 librt-0.8.1 mypy-1.20.0 mypy_extensions-1.1.0 numpy-2.4.3 packaging-26.0 pathspec-1.0.4 pluggy-1.6.0 pygments-2.20.0 pytest-9.0.3 typing_extensions-4.15.0

Notice:  A new release of pip is available: 25.3 -> 26.0.1
Notice:  To update, run: pip install --upgrade pip
============================= test session starts ==============================
platform linux -- Python 3.13.12, pytest-9.0.3, pluggy-1.6.0 -- /home/runner/_work/cog3pio/cog3pio/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /home/runner/_work/cog3pio/cog3pio
configfile: pyproject.toml
collecting ... collected 12 items / 1 skipped

python/tests/test_cog3pio.py::test_doc PASSED                            [  8%]
python/tests/test_cog3pio.py::test_version PASSED                        [ 16%]
python/tests/test_io_geotiff.py::test_read_geotiff_local PASSED          [ 25%]
python/tests/test_io_geotiff.py::test_read_geotiff_remote PASSED         [ 33%]
python/tests/test_io_geotiff.py::test_read_geotiff_multi_band PASSED     [ 41%]
python/tests/test_io_geotiff.py::test_read_geotiff_invalid_filepath PASSED [ 50%]
python/tests/test_io_geotiff.py::test_read_geotiff_invalid_remote_url PASSED [ 58%]
python/tests/test_io_geotiff.py::test_read_geotiff_missing_url PASSED    [ 66%]
python/tests/test_io_geotiff.py::test_read_geotiff_unsupported_colortype PASSED [ 75%]
python/tests/test_io_geotiff.py::test_read_geotiff_unsupported_dtype PASSED [ 83%]
python/tests/test_io_geotiff.py::test_cogReader_to_dlpack PASSED         [ 91%]
python/tests/test_io_geotiff.py::test_cogreader_xy_coords PASSED         [100%]

=============================== warnings summary ===============================
...
==================================== PASSES ====================================
================== 12 passed, 1 skipped, 3 warnings in 6.39s ===================

There are a few hacks that will need undo-ing, but will merge this once I confirm the linter checks pass.

@weiji14 weiji14 marked this pull request as ready for review April 8, 2026 02:58
@weiji14 weiji14 enabled auto-merge (squash) April 8, 2026 03:12
@weiji14 weiji14 merged commit 2ad0728 into main Apr 8, 2026
23 checks passed
@weiji14 weiji14 deleted the riscv64 branch April 8, 2026 03:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant