Packaging changes for PGO and rework to make it as close as possible to upstream#5948
Open
EvgeniyPatlan wants to merge 11 commits into
Open
Packaging changes for PGO and rework to make it as close as possible to upstream#5948EvgeniyPatlan wants to merge 11 commits into
EvgeniyPatlan wants to merge 11 commits into
Conversation
…lags Align Percona Server RPM and DEB packaging (build-ps/) with MySQL upstream modern patterns while preserving Percona-specific features, and consolidate duplicated cmake invocations across both the custom build-ps/ packaging and the inherited packaging/ tree. build-ps RPM: - Template the spec via build-ps/percona-server.spec.in + CMakeLists.txt (CONFIGURE_FILE @var@ substitution) - Factor telemetry setup/cleanup into helper scripts (rpm/, debian/extra/) - Keep PGO infrastructure wired through --define "with_pgo 1" - Drop legacy rpm/filter-provides.sh, filter-requires.sh, mysql.init - Enable PGO by default in percona-server-9.0_builder.sh build-ps DEB: - Upgrade debhelper compat 9 -> 13 (removed debian/compat) - Split packages: add percona-server-client-plugins, percona-server-server-core, percona-server-client-core - Template debian/rules via rules.in + CONFIGURE_FILE - Use --list-missing with debian/not-installed - Factor telemetry into helper scripts shared with RPM cmake flag consolidation (build-ps + inherited packaging/): - build-ps/percona-server.spec.in: introduce %{cmake_common_flags} macro plus %{cmake_fido_flags} / %{cmake_lto_flags} sub-macros; reduce three duplicated cmake invocations (debug, release, PGO) to single-macro references. Fix latent bug: duplicate -DWITH_INNODB_MEMCACHED=1. - build-ps/debian/rules, rules.in: introduce CMAKE_COMMON_FLAGS Make variable; reduce three duplicated blocks to shared-variable references. Fix latent bug: PGO-rebuild block was missing -DWITH_ARCHIVE_STORAGE_ENGINE, -DWITH_BLACKHOLE_STORAGE_ENGINE, -DWITH_FEDERATED_STORAGE_ENGINE. - packaging/rpm-oel/mysql.spec.in: same %{cmake_common_flags} pattern applied to the inherited upstream spec. Fix three latent bugs in PGO block: hardcoded cmake3 -> %{cmake3}, inconsistent -DWITH_AUTHENTICATION_FIDO=0 / -DWITH_KEYRING_OCI=0 removed, empty %if 0%{?ssl_default} branch now emits -DWITH_AUTHENTICATION_CLIENT_PLUGINS=1. - packaging/deb-in/rules.in, deb_debug.cmake: introduce CMAKE_COMMON_FLAGS. Fix two latent bugs: duplicate -DMYSQL_MAINTAINER_MODE=0 and redundant -DINSTALL_DOCDIR (implied by INSTALL_LAYOUT=DEB).
The abseil-cpp-20230802.1 bundled library was reorganized between MySQL 9.6.0 and 9.7.0. The %files list explicitly enumerates each .so for percona-server-server-core, so a stale list trips rpmbuild's "File not found in BUILDROOT" error. Remove 6 abseil libs that no longer exist in 9.7.0: libabsl_bad_any_cast_impl.so libabsl_bad_optional_access.so libabsl_bad_variant_access.so libabsl_flags.so libabsl_low_level_hash.so libabsl_random_internal_pool_urbg.so Add 11 new abseil libs introduced in 9.7.0: libabsl_decode_rust_punycode.so libabsl_demangle_rust.so libabsl_hashtable_profiler.so libabsl_log_internal_fnmatch.so libabsl_log_internal_structured_proto.so libabsl_poison.so libabsl_profile_builder.so libabsl_random_internal_entropy_pool.so libabsl_tracing_internal.so libabsl_utf8_for_code_point.so libabsl_vlog_config_internal.so The mysqlrouter/private/libabsl_*.so glob already auto-handles router- side abseil churn, so no change needed there.
…ke3 fallback
Two bugs in get_sources() on the release-9.7.0-1-1 Jenkins build:
1. Lines 223 / 226: `[ ${PERCONAFT_REPO} = 0 ]` / `[ ${TOKUBACKUP_REPO} = 0 ]`
When PERCONAFT_REPO / TOKUBACKUP_REPO are empty or unset (the
normal case for non-TokuDB builds) the unquoted substitution
collapses the test to literal `[ = 0 ]` and bash reports:
./ps_builder.sh: line 223: [: =: unary operator expected
./ps_builder.sh: line 226: [: =: unary operator expected
Quote both sides + add `:-` default so the test evaluates as
`[ "" = "0" ]` (false) rather than producing a syntax error.
2. Line 269: `cmake . -DWITH_SSL=system ...`
PS 9.x requires CMake >= 3.17.5 (top-level CMakeLists.txt:112).
Some release build agents still ship CMake 3.16.3 as the default
`cmake`, so the configure step aborts:
CMake Error at CMakeLists.txt:112 (CMAKE_MINIMUM_REQUIRED):
CMake 3.17.5 or higher is required. You are running version 3.16.3
With cmake failing, no Makefile is generated, `make dist` then
reports `No rule to make target 'dist'`, and EXPORTED_TAR / PSDIR
end up empty. Every later step cascades into corruption:
`tar xzf` with no arg, rsync targets resolve to absolute `/storage/...`
paths, `cd ${PSDIR}` becomes `cd` (home dir), and the subsequent
sed of `build-ps/debian/rules` fails with "No such file".
On hosts that have both `cmake` (old) and `cmake3` (EPEL /
backports alternative), prefer `cmake3` when the default `cmake`
is older than 3.17.5. Same pattern build-ps/pxc_builder.sh
already uses for PXC on EL7.
This does NOT help hosts where neither `cmake` nor `cmake3` is
>= 3.17.5 — those need either a base-image cmake bump or a
cmake install in install_deps. Note: the make dist step legitimately
needs cmake to copy bison-generated sql_yacc.{h,cc} /
sql_hints.yy.{h,cc} into the tarball and to autoreconf UDF/, so
replacing it with plain `git archive` would silently drop those
pre-generated files. Keeping the cmake+make-dist path and just
choosing a sufficient cmake binary is the right answer.
Port of the release-8.4.9-9-1 fix (commit 1a2cb62) to the 9.x builder. build_deb() carried the identical MySQL 5.6/5.7-era block that forces CC=gcc-4.7 / gcc-4.8 for any DEBIAN_VERSION not in a hardcoded allowlist: if [ ${DEBIAN_VERSION} != trusty -a ... != trixie ]; then gcc47=$(which gcc-4.7 ...) if [ -x "${gcc47}" ]; then export CC=gcc-4.7 ... else export CC=gcc-4.8 ... fi On the 8.4.9-9-1 line this made the deb build fail at override_dh_auto_configure: CMake Error: Could not find compiler set in environment variable CC: gcc-4.8. The 9.x deb build will hit the same failure the moment it runs on an agent whose distro codename is not in the allowlist. Same reasoning as the 8.4 fix: * MySQL 9.x requires a C++20 toolchain (gcc >= 10); gcc-4.x cannot build it even if installed. * Modern agents don't ship gcc-4.7/4.8. * Allowlisted distros already build with the system default compiler (block skipped for them) — proving the default path is correct. * The exported USE_THIS_GCC_VERSION is never consumed. Remove the block; let dpkg-buildpackage / cmake use the agent's default toolchain, consistent with the allowlisted distros and with the 8.4.9-9-1 line.
Bump extra/libkmip gitlink 44607d8 -> 27c5898 to pick up Percona-Lab/libkmip PR #30 ("PS-10970 fix oracle8 compilation"). 27c5898 sits directly on top of the previously-pinned 44607d8 (44607d8 is its parent), so this brings exactly one commit and nothing else: kmipclient/CMakeLists.txt | 3 ++- kmipclient/src/NetClientOpenSSL.cpp | 35 ++++++++++++++++-- kmipcore/include/kmipcore/kmipcore_version.hpp | 2 +- The libkmip change adds OpenSSL-version-guarded helpers so the KMIP TLS client compiles on Oracle Linux 8 (OpenSSL 1.1.1, which lacks TLS_method()/SSL_get0_peer_certificate() — these are 1.1.0 / 3.0 APIs respectively) and links Threads::Threads. Compile-time guards only; no behavior change on newer OpenSSL. INTERIM PIN: 27c5898 is the head of the still-OPEN PR branch PS-10970-fix-oracle8-compilation, not a commit on libkmip master. Once PR #30 merges, re-bump this gitlink to the resulting master commit (merge/squash SHA will differ). The PR branch is a real branch on Percona-Lab/libkmip so `git submodule update` can fetch the SHA in CI.
Port of release-8.4.9-9-1 commit 3c2a6ba to the 9.x builder. percona-server-9.0_builder.sh carried the identical noble/trixie guard around libtirpc-dev: if [ x${DIST} = xnoble -o x"${DIST}" = xtrixie ]; then apt-get -y install libtirpc-dev gsasl-common fi glibc >= 2.26 removed the built-in Sun RPC implementation, so every modern Debian/Ubuntu needs libtirpc for rpc/rpc.h. cmake/rpc.cmake hard-FATAL_ERRORs if system rpc/rpc.h is missing (no bundled-tirpc fallback despite extra/tirpc/ shipping in the tree). The 9.x deb build fails configure with "Could not find rpc/rpc.h" on any agent whose DIST is not noble/trixie (e.g. the aarch64 agent that hit this on the 8.4.9-9-1 line). Hoist `apt-get -y install libtirpc-dev` out of the guard into the unconditional deb dependency list. The noble/trixie block is left intact (it also installs gsasl-common); the duplicate libtirpc-dev install there is a harmless idempotent apt no-op. As with the 8.4 fix: libtirpc-dev is still absent from debian/control Build-Depends; adding it there would be the more correct long-term fix so dpkg-buildpackage enforces it independently of this script. Left as a follow-up.
Port of the release-9.6.0-2 fix (commit 32536d6a239) to the 9.7 line.
release-9.7.0-1-1 carries the identical server/client -core split and
the same deep exact (= ${binary:Version}) cross-pin chain:
server -> common(=), client(=), server-core(=)
client -> common(=), client-core(=), client-plugins(=)
client-core -> client-plugins(=)
This makes the whole set installable only if every split member
exists in the repo at the exact same revision. A non-atomic /
mixed-revision experimental repo then can't satisfy the chain:
percona-server-server : Depends: percona-server-server-core
(= <ver>) but it is not installable
release-9.5.0-1 (pre-split) only exact-pinned packages that were
always co-published from one source build (server->common,
server->client, client->common) and had no -core/-plugins packages
at all.
Restore the 9.5 dependency shape while keeping the split:
- Keep exact (= ${binary:Version}) on server->common, server->client,
client->common (one-source-build, always co-published, ABI-coupled
— lockstep there is correct and never broke).
- Drop the exact version qualifier on the NEW split-internal pins
(no 9.5 equivalent; the ones a mixed-revision repo can't satisfy):
percona-server-client-core: client-plugins(=) -> client-plugins
percona-server-client: client-core(=) -> client-core
client-plugins(=) -> client-plugins
percona-server-server: server-core(=) -> server-core
Identical 4-line change to 32536d6a239 on release-9.6.0-2.
release-8.4.9-9-1 needs no port: it has no -core split (already the
monolithic 9.5-style control).
…parens)
The bundled private libs (libabsl_*, libicu*, libprotobuf-lite) were
shipping as unsatisfiable auto-Requires on EVERY rpm distro:
nothing provides libabsl_base.so()(64bit) needed by
percona-server-server-9.6.0-1.<rev>.<dist>.x86_64
... (libicu*.so.77.1, libprotobuf-lite.so.24.4.0, etc.)
Root cause is NOT stale packages (this reproduced from a build done
at the very commit that carries the filter, 7c74f95). The
__requires_exclude regex never compiled, so rpm silently applied
ZERO requires filtering on all distros.
%global _privateperl perl\\\\((GD|hostnames|lib::mtr|lib::v1|mtr_|My::)
%global __requires_exclude ^((%{_privatelibs})|(%{_privateperl}))
rpm collapses one backslash level in %global values, so `perl\\\\(`
becomes `perl\\(` in the regex = literal `perl` + literal `\` + an
*unescaped* group-open `(` that is never closed. The resulting
__requires_exclude (`^((libabsl_...)|(perl\\((GD|...)))`) has an
unbalanced parenthesis -> POSIX regcomp fails -> rpm's dependency
filter is a no-op -> the private-lib auto-Requires are emitted with
no matching Provides (correctly suppressed by
__provides_exclude_from), making the package uninstallable.
Fix: `perl\\\\(` -> `perl\\(`. After rpm's one-level collapse this
yields `perl\(` = a *literal* `perl(`, the (GD|...) group is the only
real group and is properly closed, the full __requires_exclude
balances and compiles. Verified the fixed regex compiles and:
- libabsl_base.so()(64bit) -> excluded
- libicui18n.so.77.1()(64bit) -> excluded
- libprotobuf-lite.so.24.4.0()(64bit) -> excluded
- perl(My::SafeProcess), perl(GD) -> excluded
- libc.so.6()(64bit), libssl.so.3 -> kept (real deps preserved)
Distro-independent (%global, no %if guard) so one rebuild fixes
every rpm target.
…match
9.7 build on Ubuntu 26.04 (resolute) fails when linking libkmip's
demo binaries with:
lto1: fatal error: bytecode stream in file '../libkmip/src/libkmip.a'
generated with LTO version 11.3 instead of the expected 15.1
/usr/bin/x86_64-linux-gnu-ld.bfd: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
Root cause is an install_deps per-distro toolchain gap.
percona-server-9.0_builder.sh has explicit branches for
bionic/bookworm/noble/trixie that pick a matching gcc, and an `else`
fallback that installs gcc-11 + update-alternatives'es it for gcc/g++
(via slave) and cc. Resolute (Ubuntu 26.04 LTS) is not in any
explicit branch so it falls into the `else`. That has two problems:
1. Forcing gcc-11 on a 2026 LTS is wrong — resolute's native
toolchain is gcc-15, and gcc-11 is just an older compiler
dragged in for compat with much older distros.
2. The `else` only updates the `gcc`, `g++` (via slave) and `cc`
alternatives — it does NOT touch `/usr/bin/c++`. On resolute
`/usr/bin/c++` is part of the gcc-15 install and stays at
g++-15. Result: a mixed-compiler build — C compiles via gcc-11
(cc -> gcc-11 via the alternative), C++ link via g++-15
(untouched `c++`). libkmip's C sources go into libkmip.a with
LTO bytecode 11.3, then linking the C++ demo with g++-15 calls
lto1 (15.1) on libkmip.a -> version mismatch -> link aborts.
Add an explicit resolute branch that installs gcc-15 + g++-15 and
points all four alternatives (gcc, g++, cc, c++) at the gcc-15
toolchain. Resolute is then on a single-version toolchain matching
its native default, and the LTO bytecode is uniformly 15.1.
Side note: the missing `c++` alternative is a latent gap in the
else branch too. Not fixing it generically here to keep the diff
minimal; the resolute branch sets all four explicitly so the
problem doesn't recur on this specific distro.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.