Conversation
Support RISC-V instruction set architecture. Build on manylinux 2_31.
|
Getting this error on actions/setup-python: See issue at actions/setup-python#1288 |
11827af to
72bfcc1
Compare
and also an updated versions-manifest.json file with riscv64 arch.
Specifically, the quay.io/repository/pypa/manylinux_2_39_riscv64 image which has linux/riscv64 docker builds.
Not much CUDA binaries for RISC-V yet.
|
Getting further along with some changes to maturin-action at PyO3/maturin-action#429. Now hitting into this error at https://github.com/weiji14/cog3pio/actions/runs/23776298510/job/69278840811?pr=88#step:5:221: Seems like there are no riscv64 wheels for sccache yet at https://pypi.org/project/sccache/0.14.0/#files. Could just disable sccache for now, or look into not using |
|
Woohoo, That was for the GIL version though. The non free-threaded build I'll fix shortly. |
Fork no longer needed since PyO3/maturin-action#429 has been merged.
|
Ok, confirmed that both non-free-threaded and free-threaded wheel builds on RISC-V works~. Both taking ~5min with sccache now enabled 🚀 Now just need to handle pytest not working on |
and make the RISC-V pytest checks run natively without going through QEMU.
|
and... pytest still doesn't work because the venv cannot be created with There is a workaround listed at riseproject-dev/riscv-runner-images#28 which I could try, but it involves a bootstrap script... Edit: venv creation works after riseproject-dev/riscv-runner-images#36, so reran CI. Now hitting into another error during install of the wheels at https://github.com/weiji14/cog3pio/actions/runs/23783180648/job/70155686708?pr=88#step:8:37: Looking in links: dist
0 location(s) to search for versions of cog3pio:
Skipping link: none of the wheel's tags (cp313-cp313t-manylinux_2_39_riscv64) are compatible (run pip debug --verbose to show compatible tags): file:///home/runner/_work/cog3pio/cog3pio/dist/cog3pio-0.1.0b1-cp313-cp313t-manylinux_2_39_riscv64.whl
Skipping link: none of the wheel's tags (cp313-abi3-manylinux_2_39_riscv64) are compatible (run pip debug --verbose to show compatible tags): file:///home/runner/_work/cog3pio/cog3pio/dist/cog3pio-0.1.0b1-cp313-abi3-manylinux_2_39_riscv64.whl
Given no hashes to check 0 links for project 'cog3pio': discarding no candidates
ERROR: Could not find a version that satisfies the requirement cog3pio (from versions: none)
ERROR: No matching distribution found for cog3pio
Exception information:
Traceback (most recent call last):
File "/home/runner/_work/cog3pio/cog3pio/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "/home/runner/_work/cog3pio/cog3pio/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 174, in _add_to_criteria
raise RequirementsConflicted(criterion)
pip._vendor.resolvelib.resolvers.RequirementsConflicted: Requirements conflict: SpecifierRequirement('cog3pio')
|
Get RISC-V builds of Python from https://github.com/riseproject-dev/python-versions, to get Python 3.13 instead of Python 3.12 system fallback on runner.
Patches 2bc61d1, got the order swapped. Also set PIP_PRE=1 to try and get `numpy-2.5.0.dev0-cp313-cp313-linux_riscv64.whl` from the extra-index-url.
|
Managed to pull the NumPy RISC-V wheel from https://gounthar.github.io/riscv64-python-wheels/simple/, but some other dependencies aren't compiled for RISC-V yet. Looking in indexes: https://pypi.org/simple, https://gounthar.github.io/riscv64-python-wheels/simple/
Requirement already satisfied: cog3pio in ./.venv/lib/python3.13/site-packages (0.1.0b1)
Collecting mypy
Downloading mypy-1.20.0-py3-none-any.whl.metadata (2.3 kB)
Collecting pytest
Downloading pytest-9.0.2-py3-none-any.whl.metadata (7.6 kB)
Collecting numpy>=2.0 (from cog3pio)
Downloading https://github.com/gounthar/riscv64-python-wheels/releases/download/v2026.04.06-cp313/numpy-2.5.0.dev0-cp313-cp313-linux_riscv64.whl (5.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.9/5.9 MB 9.6 MB/s 0:00:00
Collecting xarray>=2024.10.0 (from cog3pio)
Downloading xarray-2026.2.0-py3-none-any.whl.metadata (12 kB)
Collecting typing_extensions>=4.6.0 (from mypy)
Downloading typing_extensions-4.15.0-py3-none-any.whl.metadata (3.3 kB)
Collecting mypy_extensions>=1.0.0 (from mypy)
Downloading mypy_extensions-1.1.0-py3-none-any.whl.metadata (1.1 kB)
Collecting pathspec>=1.0.0 (from mypy)
Downloading pathspec-1.0.4-py3-none-any.whl.metadata (13 kB)
Collecting librt>=0.8.0 (from mypy)
Downloading librt-0.8.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl.metadata (1.3 kB)
Collecting iniconfig>=1.0.1 (from pytest)
Downloading iniconfig-2.3.0-py3-none-any.whl.metadata (2.5 kB)
Collecting packaging>=22 (from pytest)
Downloading packaging-26.0-py3-none-any.whl.metadata (3.3 kB)
Collecting pluggy<2,>=1.5 (from pytest)
Downloading pluggy-1.6.0-py3-none-any.whl.metadata (4.8 kB)
Collecting pygments>=2.7.2 (from pytest)
Downloading pygments-2.20.0-py3-none-any.whl.metadata (2.5 kB)
Collecting pandas>=2.2 (from xarray>=2024.10.0->cog3pio)
Downloading pandas-3.0.2.tar.gz (4.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 8.8 MB/s 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [192 lines of output]
+ meson setup /tmp/pip-install-3quhlfwl/pandas_46b9b52635dc46139ff6465f997e808e /tmp/pip-install-3quhlfwl/pandas_46b9b52635dc46139ff6465f997e808e/.mesonpy-1_syvab6 -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=/tmp/pip-install-3quhlfwl/pandas_46b9b52635dc46139ff6465f997e808e/.mesonpy-1_syvab6/meson-python-native-file.ini
The Meson build system
Version: 1.11.0.rc3
Source dir: /tmp/pip-install-3quhlfwl/pandas_46b9b52635dc46139ff6465f997e808e
Build dir: /tmp/pip-install-3quhlfwl/pandas_46b9b52635dc46139ff6465f997e808e/.mesonpy-1_syvab6
Build type: native build
Project name: pandas
Project version: 3.0.2
C compiler for the host machine: /home/runner/.cargo/bin/sccache cc (gcc 13.3.0 "cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0")
C linker for the host machine: cc ld.bfd 2.42
C++ compiler for the host machine: /home/runner/.cargo/bin/sccache c++ (gcc 13.3.0 "c++ (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0")
C++ linker for the host machine: c++ ld.bfd 2.42
../meson.build:2:0: ERROR: Compiler cython cannot compile programs.
A full log can be found at /tmp/pip-install-3quhlfwl/pandas_46b9b52635dc46139ff6465f997e808e/.mesonpy-1_syvab6/meson-logs/meson-log.txtIt is time to remove |
Find a source where libopenblas is packaged inside the numpy wheel. Xref https://riseproject.gitlab.io/python/wheel_builder/packages/numpy.html
|
Nice, RISC-V native builds and tests are working 🎉 DetailsSuccessfully installed cog3pio-0.1.0b1
Removed build tracker: '/tmp/pip-build-tracker-of50e7h6'
Looking in indexes: https://pypi.org/simple, https://gitlab.com/api/v4/projects/56254198/packages/pypi/simple/
Requirement already satisfied: cog3pio in ./.venv/lib/python3.13/site-packages (0.1.0b1)
Collecting mypy
Downloading mypy-1.20.0-py3-none-any.whl.metadata (2.3 kB)
Collecting pytest
Downloading pytest-9.0.3-py3-none-any.whl.metadata (7.6 kB)
Collecting numpy>=2.0 (from cog3pio)
Downloading https://gitlab.com/api/v4/projects/56254198/packages/pypi/files/3dfbbda758b81fbbd59accdccab2b3debeb3b15b67e2d1b382f07d0f5db575f0/numpy-2.4.3-cp313-cp313-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl (11.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.7/11.7 MB 9.1 MB/s 0:00:01
Collecting typing_extensions>=4.6.0 (from mypy)
Downloading typing_extensions-4.15.0-py3-none-any.whl.metadata (3.3 kB)
Collecting mypy_extensions>=1.0.0 (from mypy)
Downloading mypy_extensions-1.1.0-py3-none-any.whl.metadata (1.1 kB)
Collecting pathspec>=1.0.0 (from mypy)
Downloading pathspec-1.0.4-py3-none-any.whl.metadata (13 kB)
Collecting librt>=0.8.0 (from mypy)
Downloading librt-0.8.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl.metadata (1.3 kB)
Collecting iniconfig>=1.0.1 (from pytest)
Downloading iniconfig-2.3.0-py3-none-any.whl.metadata (2.5 kB)
Collecting packaging>=22 (from pytest)
Downloading packaging-26.0-py3-none-any.whl.metadata (3.3 kB)
Collecting pluggy<2,>=1.5 (from pytest)
Downloading pluggy-1.6.0-py3-none-any.whl.metadata (4.8 kB)
Collecting pygments>=2.7.2 (from pytest)
Downloading pygments-2.20.0-py3-none-any.whl.metadata (2.5 kB)
Downloading mypy-1.20.0-py3-none-any.whl (2.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 9.0 MB/s 0:00:00
Downloading pytest-9.0.3-py3-none-any.whl (375 kB)
Downloading pluggy-1.6.0-py3-none-any.whl (20 kB)
Downloading iniconfig-2.3.0-py3-none-any.whl (7.5 kB)
Downloading librt-0.8.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl (218 kB)
Downloading mypy_extensions-1.1.0-py3-none-any.whl (5.0 kB)
Downloading packaging-26.0-py3-none-any.whl (74 kB)
Downloading pathspec-1.0.4-py3-none-any.whl (55 kB)
Downloading pygments-2.20.0-py3-none-any.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 7.0 MB/s 0:00:00
Downloading typing_extensions-4.15.0-py3-none-any.whl (44 kB)
Installing collected packages: typing_extensions, pygments, pluggy, pathspec, packaging, numpy, mypy_extensions, librt, iniconfig, pytest, mypy
Successfully installed iniconfig-2.3.0 librt-0.8.1 mypy-1.20.0 mypy_extensions-1.1.0 numpy-2.4.3 packaging-26.0 pathspec-1.0.4 pluggy-1.6.0 pygments-2.20.0 pytest-9.0.3 typing_extensions-4.15.0
Notice: A new release of pip is available: 25.3 -> 26.0.1
Notice: To update, run: pip install --upgrade pip
============================= test session starts ==============================
platform linux -- Python 3.13.12, pytest-9.0.3, pluggy-1.6.0 -- /home/runner/_work/cog3pio/cog3pio/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /home/runner/_work/cog3pio/cog3pio
configfile: pyproject.toml
collecting ... collected 12 items / 1 skipped
python/tests/test_cog3pio.py::test_doc PASSED [ 8%]
python/tests/test_cog3pio.py::test_version PASSED [ 16%]
python/tests/test_io_geotiff.py::test_read_geotiff_local PASSED [ 25%]
python/tests/test_io_geotiff.py::test_read_geotiff_remote PASSED [ 33%]
python/tests/test_io_geotiff.py::test_read_geotiff_multi_band PASSED [ 41%]
python/tests/test_io_geotiff.py::test_read_geotiff_invalid_filepath PASSED [ 50%]
python/tests/test_io_geotiff.py::test_read_geotiff_invalid_remote_url PASSED [ 58%]
python/tests/test_io_geotiff.py::test_read_geotiff_missing_url PASSED [ 66%]
python/tests/test_io_geotiff.py::test_read_geotiff_unsupported_colortype PASSED [ 75%]
python/tests/test_io_geotiff.py::test_read_geotiff_unsupported_dtype PASSED [ 83%]
python/tests/test_io_geotiff.py::test_cogReader_to_dlpack PASSED [ 91%]
python/tests/test_io_geotiff.py::test_cogreader_xy_coords PASSED [100%]
=============================== warnings summary ===============================
...
==================================== PASSES ====================================
================== 12 passed, 1 skipped, 3 warnings in 6.39s ===================There are a few hacks that will need undo-ing, but will merge this once I confirm the linter checks pass. |
Support RISC-V instruction set architecture. Build on manylinux 2_39.
Tested that emulation works, but want to see if can get native builds via https://github.com/riseproject-dev/riscv-runner-images