feat!: make windows use venvs#3680
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors Python virtual environment (venv) creation to support platform-specific requirements, particularly for Windows, including the addition of a venv_bin_files attribute to PyRuntimeInfo for including DLLs and PDBs. The changes also update the bootstrap templates to handle Windows-specific pathing, pyvenv.cfg generation, and zip-app execution. Feedback from the review points out a global hardcoding of build_zip_enabled that should be restricted to Windows, redundant logic in the Unix venv creation path, and a bug in the bootstrap template where a function was incorrectly evaluated as a variable. Additionally, minor issues regarding PEP 8 formatting and a placeholder version string in the documentation were noted.
…into sys.py.win.venv
…python into sys.py.win.venv
Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>
…python into sys.py.win.venv
|
Ok, i think this is finally ready. Changelog updated. I've been fiddling with it the last couple days, but don't see much left to poke. The two main breaking changes are:
Switching to a venv is certainly going to break something, but I've no idea what. There weren't many test failures relating to using a venv itself. |
This makes Windows use a lite venv to better align it with the unixy
implementations. A lite venv is one that has the interpreter and
pyvenv.cfg file, but site-packages isn't populated.
Unfortunately, it has to recreate the venv at runtime because Bazel
incorrectly canonicalizes relative symlinks on Windows. This isn't
ideal, however, Windows builds were already going through a zip-unzip
process, so creating a handful of symlinks seems like an improvement
overall.
A particularity of venvs on Windows is that various supporting
.dllfiles must be in the same directory as
python.exe. A newattribute (and associated provider plumbing) is added to
py_runtimeto capture these extra files.
Note that this requires symlink support be enabled in Bazel and
Windows. If both aren't enabled, then weird errors will occur because
junctions (instead of symlinks) are created that point to files
(junctions can only point to directories).
Zipapp support for these venvs is also added. This required adding
PyExecutableInfo.venv_interpreter_symlinksto keep track of relativesymlinks that need to be created. Tracking them separately is needed
because Bazel has a bug where relative symlinks are made absolute
on Windows.
The internal plumbing for these symlinks is kept relatively generic
to support using such symlinks in site-packages in a future change.
Along the way...
previously broken because the launcher doesn't know how to look
inside the zip file.
RULES_PYTHON_EXTRACT_ROOTsupport when manually passinga zipapp to Python.
failure.