From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 4A1741FF0BC for ; Mon, 16 Nov 2026 22:48:52 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id B90342156F; Mon, 16 Nov 2026 22:48:46 +0100 (CET) Message-ID: Date: Mon, 16 Nov 2026 22:48:40 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta Subject: Re: [PATCH pve-kernel v4 1/2] d/scripts: correctly compare namespaced ABI exports To: =?UTF-8?B?57WQ5Y+L?= , pve-devel@lists.proxmox.com References: <20261024052438.1274-1-miagetegorann@gmail.com> <20261024052438.1274-2-miagetegorann@gmail.com> Content-Language: en-US From: Thomas Lamprecht Autocrypt: addr=t.lamprecht@proxmox.com; keydata= xsFNBFsLjcYBEACsaQP6uTtw/xHTUCKF4VD4/Wfg7gGn47+OfCKJQAD+Oyb3HSBkjclopC5J uXsB1vVOfqVYE6PO8FlD2L5nxgT3SWkc6Ka634G/yGDU3ZC3C/7NcDVKhSBI5E0ww4Qj8s9w OQRloemb5LOBkJNEUshkWRTHHOmk6QqFB/qBPW2COpAx6oyxVUvBCgm/1S0dAZ9gfkvpqFSD 90B5j3bL6i9FIv3YGUCgz6Ue3f7u+HsEAew6TMtlt90XV3vT4M2IOuECG/pXwTy7NtmHaBQ7 UJBcwSOpDEweNob50+9B4KbnVn1ydx+K6UnEcGDvUWBkREccvuExvupYYYQ5dIhRFf3fkS4+ wMlyAFh8PQUgauod+vqs45FJaSgTqIALSBsEHKEs6IoTXtnnpbhu3p6XBin4hunwoBFiyYt6 YHLAM1yLfCyX510DFzX/Ze2hLqatqzY5Wa7NIXqYYelz7tXiuCLHP84+sV6JtEkeSUCuOiUY virj6nT/nJK8m0BzdR6FgGtNxp7RVXFRz/+mwijJVLpFsyG1i0Hmv2zTn3h2nyGK/I6yhFNt dX69y5hbo6LAsRjLUvZeHXpTU4TrpN/WiCjJblbj5um5eEr4yhcwhVmG102puTtuCECsDucZ jpKpUqzXlpLbzG/dp9dXFH3MivvfuaHrg3MtjXY1i+/Oxyp5iwARAQABzTNUaG9tYXMgTGFt cHJlY2h0IChBdXRoLTQpIDx0LmxhbXByZWNodEBwcm94bW94LmNvbT7CwY4EEwEIADgWIQQO R4qbEl/pah9K6VrTZCM6gDZWBgUCWwuNxgIbAwULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAK CRDTZCM6gDZWBm/jD/4+6JB2s67eaqoP6x9VGaXNGJPCscwzLuxDTCG90G9FYu29VcXtubH/ bPwsyBbNUQpqTm/s4XboU2qpS5ykCuTjqavrcP33tdkYfGcItj2xMipJ1i3TWvpikQVsX42R G64wovLs/dvpTYphRZkg5DwhgTmy3mRkmofFCTa+//MOcNOORltemp984tWjpR3bUJETNWpF sKGZHa3N4kCNxb7A+VMsJZ/1gN3jbQbQG7GkJtnHlWkw9rKCYqBtWrnrHa4UAvSa9M/XCIAB FThFGqZI1ojdVlv5gd6b/nWxfOPrLlSxbUo5FZ1i/ycj7/24nznW1V4ykG9iUld4uYUY86bB UGSjew1KYp9FmvKiwEoB+zxNnuEQfS7/Bj1X9nxizgweiHIyFsRqgogTvLh403QMSGNSoArk tqkorf1U+VhEncIn4H3KksJF0njZKfilrieOO7Vuot1xKr9QnYrZzJ7m7ZxJ/JfKGaRHXkE1 feMmrvZD1AtdUATZkoeQtTOpMu4r6IQRfSdwm/CkppZXfDe50DJxAMDWwfK2rr2bVkNg/yZI tKLBS0YgRTIynkvv0h8d9dIjiicw3RMeYXyqOnSWVva2r+tl+JBaenr8YTQw0zARrhC0mttu cIZGnVEvQuDwib57QLqMjQaC1gazKHvhA15H5MNxUhwm229UmdH3KM7BTQRbC43GARAAyTkR D6KRJ9Xa2fVMh+6f186q0M3ni+5tsaVhUiykxjsPgkuWXWW9MbLpYXkzX6h/RIEKlo2BGA95 QwG5+Ya2Bo3g7FGJHAkXY6loq7DgMp5/TVQ8phsSv3WxPTJLCBq6vNBamp5hda4cfXFUymsy HsJy4dtgkrPQ/bnsdFDCRUuhJHopnAzKHN8APXpKU6xV5e3GE4LwFsDhNHfH/m9+2yO/trcD txSFpyftbK2gaMERHgA8SKkzRhiwRTt9w5idOfpJVkYRsgvuSGZ0pcD4kLCOIFrer5xXudk6 NgJc36XkFRMnwqrL/bB4k6Pi2u5leyqcXSLyBgeHsZJxg6Lcr2LZ35+8RQGPOw9C0ItmRjtY ZpGKPlSxjxA1WHT2YlF9CEt3nx7c4C3thHHtqBra6BGPyW8rvtq4zRqZRLPmZ0kt/kiMPhTM 8wZAlObbATVrUMcZ/uNjRv2vU9O5aTAD9E5r1B0dlqKgxyoImUWB0JgpILADaT3VybDd3C8X s6Jt8MytUP+1cEWt9VKo4vY4Jh5vwrJUDLJvzpN+TsYCZPNVj18+jf9uGRaoK6W++DdMAr5l gQiwsNgf9372dbMI7pt2gnT5/YdG+ZHnIIlXC6OUonA1Ro/Itg90Q7iQySnKKkqqnWVc+qO9 GJbzcGykxD6EQtCSlurt3/5IXTA7t6sAEQEAAcLBdgQYAQgAIBYhBA5HipsSX+lqH0rpWtNk IzqANlYGBQJbC43GAhsMAAoJENNkIzqANlYGD1sP/ikKgHgcspEKqDED9gQrTBvipH85si0j /Jwu/tBtnYjLgKLh2cjv1JkgYYjb3DyZa1pLsIv6rGnPX9bH9IN03nqirC/Q1Y1lnbNTynPk IflgvsJjoTNZjgu1wUdQlBgL/JhUp1sIYID11jZphgzfDgp/E6ve/8xE2HMAnf4zAfJaKgD0 F+fL1DlcdYUditAiYEuN40Ns/abKs8I1MYx7Yglu3RzJfBzV4t86DAR+OvuF9v188WrFwXCS RSf4DmJ8tntyNej+DVGUnmKHupLQJO7uqCKB/1HLlMKc5G3GLoGqJliHjUHUAXNzinlpE2Vj C78pxpwxRNg2ilE3AhPoAXrY5qED5PLE9sLnmQ9AzRcMMJUXjTNEDxEYbF55SdGBHHOAcZtA kEQKub86e+GHA+Z8oXQSGeSGOkqHi7zfgW1UexddTvaRwE6AyZ6FxTApm8wq8NT2cryWPWTF BDSGB3ujWHMM8ERRYJPcBSjTvt0GcEqnd+OSGgxTkGOdufn51oz82zfpVo1t+J/FNz6MRMcg 8nEC+uKvgzH1nujxJ5pRCBOquFZaGn/p71Yr0oVitkttLKblFsqwa+10Lt6HBxm+2+VLp4Ja 0WZNncZciz3V3cuArpan/ZhhyiWYV5FD0pOXPCJIx7WS9PTtxiv0AOS4ScWEUmBxyhFeOpYa DrEx In-Reply-To: <20261024052438.1274-2-miagetegorann@gmail.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1794865720714 X-SPAM-LEVEL: Spam detection results: 0 AWL -1.098 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: KWU3C3D5AV4NAR35KM7DAP5JIWTQLPQA X-Message-ID-Hash: KWU3C3D5AV4NAR35KM7DAP5JIWTQLPQA X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Am 24.10.26 um 07:25 schrieb 結友: > abi-generate emits a different field order for namespaced exports. Parse > both layouts explicitly so the module name does not become part of the > symbol name. Otherwise, building hv_vmbus into vmlinux falsely reports > three existing namespaced exports as missing. > > Compare the actual export type and namespace, preserving hash and missing > symbol checks and symbol/module blacklists. Reject added or changed > namespace requirements, malformed entries and duplicate symbols. Fix the > export-type diagnostic's printf argument separator. > > Add 12 ABI regression tests, including end-to-end abi-generate output, > relocation, hash changes, GPL restrictions, namespaces and skip handling. > Document the check and test command, and ignore Python bytecode caches. > > All 12 tests pass under WSL2 Ubuntu 24.04. Rechecking the existing > 7.0.14-26-pve build artifacts with SKIPABI=0 preserves all 32,726 symbols, > their CRCs and export types, with only the expected 47 VMBus relocations. > > Signed-off-by: 結友 > --- > .gitignore | 1 + > README | 8 ++ > debian/scripts/abi-check | 56 ++++++++++--- > debian/tests/test_abi_check.py | 140 +++++++++++++++++++++++++++++++++ > 4 files changed, 192 insertions(+), 13 deletions(-) > create mode 100644 debian/tests/test_abi_check.py > > diff --git a/.gitignore b/.gitignore > index 295de11..aa1d4e1 100644 > --- a/.gitignore > +++ b/.gitignore > @@ -3,6 +3,7 @@ > *.deb > *.dsc > *.prepared > +__pycache__/ > /abi-tmp* > /config*.org > /proxmox-kernel-[0-9]*/ > diff --git a/README b/README > index 9dd855a..00564c8 100644 > --- a/README > +++ b/README > @@ -167,6 +167,14 @@ variable in the Makefile, rebuilding the packages and running 'make abiupdate' > (the 'abiupdate' target in 'Makefile' contains the steps for consistently > updating the repository). > > +The ABI check reads both ordinary and namespaced exports. Moving an export > +between a module and vmlinux is reported without failing the check. Missing > +symbols, hash changes and stricter export or namespace requirements fail it > +unless an applicable blacklist or the ABI skip mode permits them. > + > +Run the ABI checker regression tests on a Debian build host with > +'python3 -B -m unittest discover -s debian/tests -v'. > + > The source package-name is proxmox-kernel-MAJ.MIN, the version of the package > defaults to MAJ.MIN.PATCH-KREL - but this can be adapted to contain a suffix > for test-packages (e.g. ~test01). The version in the Makefile needs to be equal > diff --git a/debian/scripts/abi-check b/debian/scripts/abi-check > index 9987d04..ff000c3 100755 > --- a/debian/scripts/abi-check > +++ b/debian/scripts/abi-check > @@ -89,17 +89,35 @@ sub is_ignored($$) { > return 0; > } > > +# abi-generate emits four fields for ordinary exports, and five fields for > +# namespaced exports (namespace, type, hash, symbol, module). In the latter > +# format the trailing module must not become part of the symbol name. > +sub parse_symbol { > + my ($line, $file, $number) = @_; > + chomp $line; > + $line =~ s/\r$//; > + > + if ($line =~ /^(EXPORT_SYMBOL(?:_[A-Z0-9_]+)?)\s+(\S+)\s+(0x[0-9a-f]+)\s+(\S+)$/) { > + return ($4, $1, $2, $3, ''); > + } > + if ($line =~ /^(\S+)\s+(EXPORT_SYMBOL(?:_[A-Z0-9_]+)?)\s+(0x[0-9a-f]+)\s+(\S+)\s+(\S+)$/) { > + return ($4, $2, $5, $3, $1); > + } > + die "EE: $file:$number: malformed ABI entry: '$line'\n"; > +} > + > # Read new syms first > print " Reading new symbols ($abistr)..."; > $count = 0; > open(my $NEW_FH, '<', $abinew) or die "Could not open $abinew - $!"; > while (<$NEW_FH>) { > - chomp; > - m/^(\S+)\s(.+)\s(0x[0-9a-f]+)\s(.+)$/; > - $symbols{$4}{'type'} = $1; > - $symbols{$4}{'loc'} = $2; > - $symbols{$4}{'hash'} = $3; > - $module_syms{$2} = 0; > + my ($sym, $type, $loc, $hash, $namespace) = parse_symbol($_, $abinew, $.); > + die "EE: $abinew:$.: duplicate symbol '$sym'\n" if defined($symbols{$sym}{'type'}); > + $symbols{$sym}{'type'} = $type; > + $symbols{$sym}{'loc'} = $loc; > + $symbols{$sym}{'hash'} = $hash; > + $symbols{$sym}{'namespace'} = $namespace; > + $module_syms{$loc} = 0; > $count++; > } > close($NEW_FH); > @@ -110,11 +128,12 @@ print " Reading old symbols..."; > $count = 0; > open(my $OLD_FH, '<', $abiold) or die "Could not open $abiold - $!"; > while (<$OLD_FH>) { > - chomp; > - m/^(\S+)\s(.+)\s(0x[0-9a-f]+)\s(.+)$/; > - $symbols{$4}{'old_type'} = $1; > - $symbols{$4}{'old_loc'} = $2; > - $symbols{$4}{'old_hash'} = $3; > + my ($sym, $type, $loc, $hash, $namespace) = parse_symbol($_, $abiold, $.); > + die "EE: $abiold:$.: duplicate symbol '$sym'\n" if defined($symbols{$sym}{'old_type'}); > + $symbols{$sym}{'old_type'} = $type; > + $symbols{$sym}{'old_loc'} = $loc; > + $symbols{$sym}{'old_hash'} = $hash; > + $symbols{$sym}{'old_namespace'} = $namespace; > $count++; > } > close($OLD_FH); > @@ -158,6 +177,7 @@ $count = 0; > my $moved = 0; > my $changed_type = 0; > my $changed_hash = 0; > +my $changed_namespace = 0; > for my $sym (keys(%symbols)) { > if (!defined($symbols{$sym}{'old_type'}) or !defined($symbols{$sym}{'type'})) { > next; > @@ -172,12 +192,21 @@ for my $sym (keys(%symbols)) { > # Changes to export type are only bad if new type isn't > # EXPORT_SYMBOL. Changing things to GPL are bad. > if ($symbols{$sym}{'type'} ne $symbols{$sym}{'old_type'}) { > - printf(" TYPE : %-40s : %s => %s%s\n", $sym, $symbols{$sym}{'old_type'}. > + printf(" TYPE : %-40s : %s => %s%s\n", $sym, $symbols{$sym}{'old_type'}, > $symbols{$sym}{'type'}, is_ignored($symbols{$sym}{'loc'}, $sym) > ? " (ignored)" : ""); > $changed_type++ if $symbols{$sym}{'type'} ne "EXPORT_SYMBOL" and !is_ignored($symbols{$sym}{'loc'}, $sym); > } > > + # Adding or changing a namespace requires consumers to change their imports. > + # Removing a namespace relaxes that requirement and is compatible. > + if ($symbols{$sym}{'namespace'} ne $symbols{$sym}{'old_namespace'}) { > + printf(" NS : %-40s : %s => %s%s\n", $sym, > + $symbols{$sym}{'old_namespace'} || '(none)', $symbols{$sym}{'namespace'} || '(none)', > + is_ignored($symbols{$sym}{'loc'}, $sym) ? " (ignored)" : ""); > + $changed_namespace++ if $symbols{$sym}{'namespace'} ne '' && !is_ignored($symbols{$sym}{'loc'}, $sym); > + } > + > # Changes to the hash are always bad > if ($symbols{$sym}{'hash'} ne $symbols{$sym}{'old_hash'}) { > printf(" HASH : %-40s : %s => %s%s\n", $sym, $symbols{$sym}{'old_hash'}, > @@ -191,8 +220,9 @@ for my $sym (keys(%symbols)) { > print "WW: $moved symbols changed location\n" if $moved; > print "$EE $changed_type symbols changed export type and weren't ignored\n" if $changed_type; > print "$EE $changed_hash symbols changed hash and weren't ignored\n" if $changed_hash; > +print "$EE $changed_namespace symbols changed namespace and weren't ignored\n" if $changed_namespace; > > -$errors++ if $changed_hash or $changed_type; > +$errors++ if $changed_hash or $changed_type or $changed_namespace; > if ($changed_hash) { > print "II: Module hash change summary...\n"; > for my $mod (sort { $module_syms{$b} <=> $module_syms{$a} } keys %module_syms) { > diff --git a/debian/tests/test_abi_check.py b/debian/tests/test_abi_check.py > new file mode 100644 > index 0000000..928c64a > --- /dev/null > +++ b/debian/tests/test_abi_check.py why python tests and not perl, like the script doing the abi export is using? Could be probably also a simpler array of input, expected tuples to iterate over, but not a must. Also, these tests don't really give us any coverage for the real ABI output? if that diverges (again) we still just miss that? Some actual check and error on unexpected input above would be probably more helpful.