Skip to content

Support Nettle 4 besides version 3 - #844

Open
gsauthof wants to merge 1 commit into
ronf:developfrom
gsauthof:nettle4
Open

gsauthof wants to merge 1 commit into
ronf:developfrom
gsauthof:nettle4

Conversation

@gsauthof

Copy link
Copy Markdown
Contributor

i.e. drop the digest size argument when the nettle library version 4 is loaded due to version 4 API changes

fixes #842

@codecov

codecov Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.57143% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.86%. Comparing base (fab96ac) to head (8aeb0e4).
⚠️ Report is 269 commits behind head on develop.

Files with missing lines Patch % Lines
asyncssh/crypto/umac.py 78.57% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #844      +/-   ##
===========================================
+ Coverage    99.85%   99.86%   +0.01%     
===========================================
  Files           95      102       +7     
  Lines        27784    32702    +4918     
  Branches      2908     3139     +231     
===========================================
+ Hits         27743    32658    +4915     
- Misses          38       40       +2     
- Partials         3        4       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gsauthof

Copy link
Copy Markdown
Contributor Author

Hm, looks like the style of type annotations I copied relies on being evaluated lazily, i.e. requires at least python 3.14, where that lazy evaluation was introduced.

Hence the failing CI checks on python < 3.14.

This also means that the same style inside _build_umac() must also fail on python < 3.14.
Perhaps that code isn't invoked in the CI though because libnettle isn't found or similar.

i.e. drop the digest size argument when the nettle library version 4 is
loaded due to version 4 API changes

fixes ronf#842
@gsauthof

Copy link
Copy Markdown
Contributor Author

This also means that the same style inside _build_umac() must also fail on python < 3.14. Perhaps that code isn't invoked in the CI though because libnettle isn't found or similar.

No, this is wrong.

Apparently, with python < 3.14 these annotations are already lazily evaluated/ignored in a context like in _build_umac() but not at global scope ...

Updated the pull-request with a fix for older python versions.

@ronf

ronf commented Nov 15, 2026

Copy link
Copy Markdown
Owner

I've tried asking the codecov/patch check to run again, but it continues to show the 78% number. I'm rerunning the other tests to see if that helps or not.

@ronf

ronf commented Nov 15, 2026

Copy link
Copy Markdown
Owner

Here are some additional notes from Claude:

The fix itself is correct. The PR (Georg Sauthoff, fixes #842) reads nettle_version_major() when the module loads. On Nettle 4 it leaves out the size argument when calling umacNN_digest, and it turns UMAC off for any version other than 3 or 4. That matches the new Nettle 4 signature quoted in the issue, and the other functions the wrapper uses (set_key, set_nonce, update) didn't change in Nettle 4. All the CI test jobs and the lint job pass.

Coverage: codecov/patch fails. umac.py is at 95.08% on commit 8aeb0e4:

  • Line 50 (_nettle_lib = None for an unsupported version) never runs.
  • Line 146 (the Nettle 4 _digest(self._ctx, result) call) never runs.
  • Line 143 (if _major == 3:) is only partly covered.

CI only has Nettle 3, and a test can't safely call the Nettle 4 form against a Nettle 3 library, so these need # pragma: no cover.

Concerns:

  1. 4 new mypy errors. CI's lint step doesn't run mypy, so it passed anyway. I ran it on the PR branch:

    • Line 48 assigns c_int to an int. The _GetMajor = Callable[[], ctypes.c_int] type alias is wrong, because ctypes returns a plain int. The _GetMajor = None workaround for Python before 3.14 isn't needed either: _major: int = _nettle.nettle_version_major() is enough.
    • Line 61: mypy won't accept two definitions of _Digest chosen by the runtime value of _major.
    • Line 146 then gets "too few arguments" and a wrong argument type error.

    The cleaner fix is to pick the version 3 or version 4 digest call once in _build_umac, so digest() doesn't branch. That fixes the type errors, takes the check off the per-packet path, and leaves one small branch for the coverage pragma.

  2. Very old Nettle builds lose UMAC. Builds that don't export nettle_version_major (I believe it arrived in 3.1) would raise AttributeError. crypto/init.py catches that, so nothing crashes, but UMAC is silently turned off where it worked before. A try/except that falls back to version 3 would avoid this. This is low priority.

I think I'll give Claude a chance to code this (addressing these issues) and see what it comes up with. I'll let you know how it goes.

I'm not really sure longer term how to test both Nettle 3 and 4. I guess I could intentionally leave Nettle 3 libraries for one of my targets (Linux, macOS, Windows) and update the others, so the combined version ends up exercising both code paths. It'd be nice to find a way to test both Nettle 3 and 4 on all of these platforms, though, and perhaps even a version which doesn't report its major number (which we'd treat the same as 3 in terms of the API).

@ronf

ronf commented Nov 15, 2026

Copy link
Copy Markdown
Owner

Here's the Claude version:

Author: Georg Sauthoff <mail@gms.tf>
Date:   Sat Nov 14 17:41:44 2026 -0800

    Support Nettle 4 in UMAC wrapper
    
    Nettle 4 removed the length argument from digest functions, causing
    the UMAC wrapper to pass the digest length as the output buffer
    pointer and crash. Check nettle_version_major() when loading the
    library, pass the length argument only on Nettle 3, and disable UMAC
    support on unrecognized major versions. Libraries too old to export
    the version function are treated as Nettle 3.
    
    Fixes #842
    
    Co-Authored-By: Ron Frederick <ronf@timeheart.net>
    Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

diff --git a/asyncssh/crypto/umac.py b/asyncssh/crypto/umac.py
index 981be41..8f5b270 100644
--- a/asyncssh/crypto/umac.py
+++ b/asyncssh/crypto/umac.py
@@ -31,7 +31,7 @@ if TYPE_CHECKING:
     _SetKey = Callable[[_ByteArray, bytes], None]
     _SetNonce = Callable[[_ByteArray, ctypes.c_size_t, bytes], None]
     _Update = Callable[[_ByteArray, ctypes.c_size_t, bytes], None]
-    _Digest = Callable[[_ByteArray, ctypes.c_size_t, _ByteArray], None]
+    _Digest = Callable[..., None]
     _New = Callable[[bytes, Optional[bytes], Optional[bytes]], object]
 
 
@@ -55,6 +55,9 @@ def _build_umac(size: int) -> '_New':
     _update: _Update = getattr(_nettle, _prefix + 'update')
     _digest: _Digest = getattr(_nettle, _prefix + 'digest')
 
+    # Nettle 4 dropped the length argument from digest functions
+    _digest_len = (ctypes.c_size_t(size // 8),) if _nettle_major == 3 else ()
+
 
     class _UMAC:
         """Wrapper for UMAC cryptographic hash
@@ -113,7 +116,7 @@ def _build_umac(size: int) -> '_New':
             """
 
             result = ctypes.create_string_buffer(self.digest_size)
-            _digest(self._ctx, ctypes.c_size_t(self.digest_size), result)
+            _digest(self._ctx, *_digest_len, result)
             return result.raw
 
         def hexdigest(self) -> str:
@@ -136,4 +139,11 @@ else: # pragma: no cover
 if _nettle_lib: # pragma: no branch
     _nettle = ctypes.cdll.LoadLibrary(_nettle_lib)
 
-    umac32, umac64, umac96, umac128 = map(_build_umac, (32, 64, 96, 128))
+    try:
+        _nettle_major: int = _nettle.nettle_version_major()
+    except AttributeError: # pragma: no cover
+        # Versions of Nettle before 3.1 don't export a version function
+        _nettle_major = 3
+
+    if _nettle_major in (3, 4): # pragma: no branch
+        umac32, umac64, umac96, umac128 = map(_build_umac, (32, 64, 96, 128))

@gsauthof

Copy link
Copy Markdown
Contributor Author

Nettle 3.1.1 was released over 10 years ago. Asyncssh requires Python 3.10 or later which was released 5 years ago and is end of support since this October.
I thus think it's fair and reasonable to require at least nettle version 3.1.1 without writing sloppy code around it.

I don't like the chatbot's regurgitation at all, because it is:

  • adding code for outdated and unsupported nettle versions
  • duplicating code by introducing _digest_len instead of keep using self.digest_size as in the existing code, just in order to avoid writing a straight if-then-else statement later, at the cost of introducing a ternary expression
  • dropping type information form the Callable assignment
  • naming a tuple _digest_len which is empty with Nettle 4, which makes the code harder to review - this is too tricky for zero gain

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UMAC support segfaults with Nettle 4

2 participants