Documentation and readability

Author: ddelange

.github/workflows/main.yml

@@ -89,7 +89,9 @@ jobs:
         only: ${{ matrix.only }}
       env:
         CIBW_BUILD_VERBOSITY: 1
+        # add compiled libmagic to the build directory (to include in the wheel)
         CIBW_BEFORE_BUILD: bash "add_libmagic.sh"
+        # simple smoke test run on each wheel: this is an HLS MP4 video, only recognised in recent versions of libmagic
         CIBW_TEST_COMMAND: python -c "import magic; assert magic.Magic(mime=True).from_buffer(b'\x00\x00\x00\x1cftypiso5\x00\x00\x00\x01isomiso5hlsf\x00\x00') == 'video/mp4'"
 
     - uses: actions/upload-artifact@v4

README.md

@@ -2,11 +2,11 @@
 [![PyPI version](https://badge.fury.io/py/python-magic.svg)](https://badge.fury.io/py/python-magic)
 [![Build Status](https://travis-ci.org/ahupp/python-magic.svg?branch=master)](https://travis-ci.org/ahupp/python-magic) [![Join the chat at https://gitter.im/ahupp/python-magic](https://badges.gitter.im/ahupp/python-magic.svg)](https://gitter.im/ahupp/python-magic?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 
-python-magic is a Python interface to the libmagic file type
-identification library.  libmagic identifies file types by checking
+[python-magic](https://github.com/ahupp/python-magic) is a Python interface to the libmagic file type
+identification library. libmagic identifies file types by checking
 their headers according to a predefined list of file types. This
 functionality is exposed to the command line by the Unix command
-`file`.
+[`file`](https://www.darwinsys.com/file/).
 
 ## Usage
 
@@ -43,26 +43,25 @@ You can also combine the flag options:
 
 ## Installation
 
-The current stable version of python-magic is available on PyPI and
-can be installed by running `pip install python-magic`.
+This module is a simple [CDLL](https://docs.python.org/3/library/ctypes.html) wrapper around the libmagic C library.
+The current stable version of python-magic is available on [PyPI](http://pypi.python.org/pypi/python-magic/)
+and can be installed by running `pip install python-magic`.
 
-Other sources:
+Compiled libmagic and the magic database come bundled in the wheels on PyPI.
+You can use your own `magic.mgc` database by setting the `MAGIC`
+environment variable, or by using `magic.Magic(magic_file='path/to/magic.mgc')`.
+If you want to compile your own libmagic, circumvent the wheels
+by installing from source: `pip install python-magic --no-binary python-magic`.
 
-- PyPI: http://pypi.python.org/pypi/python-magic/
-- GitHub: https://github.com/ahupp/python-magic
-
-This module is a simple wrapper around the libmagic C library, and
-comes bundled in the wheels on PyPI.
-If you want to use your own (custom) libmagic installation,
-circumvent the wheels by running `pip install python-magic --no-binary python-magic`.
-For systems not supported by the wheels, libmagic
-needs to be installed before installing python-magic:
+For systems not supported by the wheels, pip installs from source,
+requiring libmagic to be available before installing python-magic:
 
 ### Linux
 
 The Linux wheels should run on most systems out of the box.
 
-Depending on your system and CPU architecture, there might be no compatible wheel uploaded. However, precompiled libmagic might still be available for your system:
+Depending on your system and CPU architecture, there might be no compatible wheel uploaded.
+However, precompiled libmagic might still be available for your system:
 
 ```sh
 # Debian/Ubuntu
@@ -75,13 +74,15 @@ yum install file-libs
 
 ### Windows
 
-The DLLs that are bundled in the Windows wheels are compiled by @julian-r and hosted at https://github.com/julian-r/file-windows/releases.
+The DLLs that are bundled in the Windows wheels are compiled by @julian-r
+and are hosted at https://github.com/julian-r/file-windows/releases.
 
 For ARM64 Windows, you'll need to compile libmagic from source.
 
 ### OSX
 
-The Mac wheels are compiled on GitHub Actions using `macos-11` runners. For older Macs, you'll need to install libmagic from source:
+The Mac wheels are compiled with maximum backward compatibility.
+For older Macs, you'll need to install libmagic from source:
 
 ```sh
 # homebrew
@@ -95,7 +96,7 @@ port install file
 - 'MagicException: could not find any magic files!': some
   installations of libmagic do not correctly point to their magic
   database file.  Try specifying the path to the file explicitly in the
-  constructor: `magic.Magic(magic_file="path_to_magic_file")`.
+  constructor: `magic.Magic(magic_file='path/to/magic.mgc')`.
 
 - 'WindowsError: [Error 193] %1 is not a valid Win32 application':
   Attempting to run the 32-bit libmagic DLL in a 64-bit build of
@@ -105,7 +106,6 @@ port install file
 - 'WindowsError: exception: access violation writing 0x00000000 ' This may indicate you are mixing
   Windows Python and Cygwin Python. Make sure your libmagic and python builds are consistent.
 
-
 ## Bug Reports
 
 python-magic is a thin layer over the libmagic C library.

add_libmagic.sh

@@ -3,6 +3,9 @@
 set -euxo pipefail
 
 install_source() {
+    # install from source
+    # https://www.darwinsys.com/file/
+    # https://github.com/file/file/blob/FILE5_45/INSTALL#L51
     version="file-5.45"
     (
         tmpfile="$(mktemp)" &&
@@ -34,21 +37,33 @@ install_precompiled() {
     elif [ -n "$(which yum)" ]; then
         yum install file-libs
     else
-        python -c 'import platform, sysconfig, io, zipfile, urllib.request; assert platform.system() == "Windows"; machine = "x86" if sysconfig.get_platform() == "win32" else "x64"; print(machine); zipfile.ZipFile(io.BytesIO(urllib.request.urlopen(f"https://github.com/julian-r/file-windows/releases/download/v5.44/file_5.44-build104-vs2022-{machine}.zip").read())).extractall(".")' &&
+        # windows (no install, just download into current working directory)
+        python <<EOF
+import platform, sysconfig, io, zipfile, urllib.request
+assert platform.system() == "Windows"
+machine = "x86" if sysconfig.get_platform() == "win32" else "x64"
+url = f"https://github.com/julian-r/file-windows/releases/download/v5.44/file_5.44-build104-vs2022-{machine}.zip"
+print("Downloading", url)
+zipfile.ZipFile(io.BytesIO(urllib.request.urlopen(url).read())).extractall(".")
+EOF
+        # check what was copied
         ls -ltra
     fi
 }
 
 copy_libmagic() {
     # on cibuildwheel, the lib needs to exist in the project before running setup.py
     # copy lib into the magic dir, regardless of platform
+    # this python command relies on current working directory containing `./magic/loader.py`
     libmagic_path="$(python -c 'from magic.loader import load_lib; print(load_lib()._name)')" &&
     cp "${libmagic_path}" "magic" &&
-    # only on linux: additionally copy compiled db into magic dir
+    # only on linux: additionally copy compiled db into magic dir (prefer the one installed by install_source)
     ( ( cp "/usr/local/share/misc/magic.mgc" "magic" || cp "/usr/share/misc/magic.mgc" "magic" ) || true ) &&
     # check what was copied
     ls -ltra magic
 }
 
+# prefer a recent build from source
 install_source || install_precompiled
+# files to be copied into the wheel
 copy_libmagic

magic/loader.py

@@ -11,8 +11,11 @@ def _lib_candidates():
     if sys.platform == 'darwin':
 
         paths = [
+            # libmagic bundled in the wheel
             here,
+            # libmagic in the current working directory
             os.path.abspath('.'),
+            # libmagic in other common sources like homebrew
             '/opt/local/lib',
             '/usr/local/lib',
             '/opt/homebrew/lib',
@@ -26,28 +29,34 @@ def _lib_candidates():
         prefixes = ['libmagic', 'magic1', 'magic-1', 'cygmagic-1', 'libmagic-1', 'msys-magic-1']
 
         for i in prefixes:
-            # find_library searches in %PATH% but not the current directory,
-            # so look for both
+            # libmagic bundled in the wheel
             yield os.path.join(here, '%s.dll' % i)
+            # libmagic in the current working directory
             yield os.path.join(os.path.abspath('.'), '%s.dll' % i)
+            # find_library searches in %PATH% but not the current directory
             yield find_library(i)
 
     elif sys.platform == 'linux':
 
         prefixes = ['libmagic.so.1', 'libmagic.so']
 
         for i in prefixes:
+            # libmagic bundled in the wheel
             yield os.path.join(here, i)
+            # libmagic in the current working directory
             yield os.path.join(os.path.abspath('.'), i)
+            # libmagic install from source default destination path
             yield os.path.join('/usr/local/lib', i)
             # on some linux systems (musl/alpine), find_library('magic') returns None
-            # first try ldconfig with backup string in case of error
+            # first try finding libmagic using ldconfig
+            # otherwise fall back to /usr/lib/
             yield subprocess.check_output(
                 "( ldconfig -p | grep '{0}' | grep -o '/.*' ) || echo '/usr/lib/{0}'".format(i),
                 shell=True,
                 universal_newlines=True,
             ).strip()
 
+    # fallback
     yield find_library('magic')