[dimitri-yatsenko]

Summary

Seven core dependencies had no version specifier (numpy, pandas, pyparsing, tqdm, networkx, pydot, packaging), and deepdiff had only a Python 3.15 cap. GitHub's dependency graph omits requirements without a specifier, so these packages were missing from the repository SBOM and from Dependabot alerts. This PR declares a tested minimum for each, and raises two existing minimums that did not work.

Package	Minimum	Reason
numpy	1.22.4	pandas 2.2's numpy minimum on Python < 3.11
pandas	2.2.2	First pandas built against numpy 2. pandas does not cap numpy, so pandas 1.5.3 with numpy 2 installs and then fails on import (numpy.dtype size changed); 2.2.1 still requires numpy < 2
deepdiff	6.6.0	DeepHash(ignore_iterable_order=...) appears in 6.4; before 6.6, deepdiff uses np.float_, which numpy 2 removed
pymysql	1.0.0 (was 0.7.2)	pymysql 0.x rejects the ssl_disabled connection option, so every MySQL connection fails on 0.7.2 through 0.10.1
pyparsing	3.1.1	pp.DelimitedList needs 3.1. 3.1.0 strips the quotes from quoted defaults, so x = "" : varchar(…) is declared with no default (see below)
tqdm	4.66.3	Excludes CVE-2024-34062
networkx	3.0	Tested minimum
pydot	3.0.0	pydot 2.x produces DOT that graphviz rejects for datajoint diagrams (syntax error … near ',')
packaging	22.0	Tested minimum

pixi.lock is refreshed; only the requirement metadata changes, no package versions.

Verification

Python 3.10, created with uv, every package above installed at exactly its minimum, other dependencies at current versions, full suite against MySQL, PostgreSQL, and SeaweedFS:

• With these minimums (numpy 1.22.4): 1234 passed, 14 skipped. The test-only matplotlib was held at 3.8.4, since current matplotlib requires numpy ≥ 1.23.
• Same, with numpy 2.2.6 and current matplotlib: 1234 passed, 14 skipped.
• With pymysql 0.7.2 or 0.10.1: 542 errors, all Connection.__init__() got an unexpected keyword argument 'ssl_disabled'. 1.0.0, 1.0.2, 1.0.3, and 1.1.0 pass.
• With pyparsing==3.1.0: the ~~ job tables and other tables with = "" defaults are declared NOT NULL with no default, and inserts fail with Field 'error_message' doesn't have a default value (MySQL) or NotNullViolation (PostgreSQL).
• With pydot==2.0.0: 11 diagram tests fail; 3.0.0 through 3.0.2 pass.
• DeepHash(..., ignore_iterable_order=False) checked directly on deepdiff 5.0 through 8.0: rejected as an unknown parameter before 6.4; fails with numpy 2 before 6.6.

Users with pyparsing 3.1.0, pydot 2.x, pymysql 0.x, or pandas < 2.2.2 installed get an upgrade from pip with this change.

Related

• ci: Dependabot watches Python dependencies, not only GitHub Actions #1652 adds Dependabot coverage for Python dependencies.
• The conda-forge recipe lists the same requirements without versions and should carry these minimums at the next release.

[MilagrosMarin]

Thanks @dimitri-yatsenko! This is a useful cleanup, and the pyparsing and pydot findings are great catches.

1. pandas 1.5.3 + numpy 2 is allowed but fails on import. pandas 1.5.3 doesn't cap numpy, so pandas>=1.5.3 with numpy>=1.22.4 can resolve to pandas 1.5.3 + numpy 2.x. On Python 3.10, import pandas then fails:

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

The same pandas with numpy 1.26.4 imports fine. The minimum-versions run didn't hit this because numpy was pinned at 1.22.4 there. Suggest raising the bound to pandas>=2.2.2 (the first numpy 2–compatible release) or adding numpy<2 for older pandas.

2. deepdiff still has no lower bound. By the PR's own reasoning (unversioned requirements are left out of the SBOM and Dependabot), it would still be missing. Its only version marker is the <9 cap for Python 3.15.

Smaller:

• I think the numpy reason ("oldest version supported on Python 3.10") is off, since numpy 1.21.x shipped 3.10 wheels. 1.22.4 matches pandas 2.1's numpy minimum, so it may be worth rewording once the pandas bound is settled.
• pymysql>=0.7.2 wasn't in the minimum-versions run. Could be a follow-up.

Happy to re-review once updated!

[dimitri-yatsenko]

Thanks @MilagrosMarin, all four addressed in a98fd31.

1. pandas + numpy 2: Raised to pandas>=2.2.2. Confirmed on Python 3.10: pandas 1.5.3 with numpy 2 fails on import exactly as you describe, 2.2.1 still requires numpy<2, and 2.2.2 imports with numpy 2.2.6.
2. deepdiff: Now deepdiff>=6.6.0. DeepHash(ignore_iterable_order=...) is rejected before 6.4, and before 6.6 deepdiff uses np.float_, which fails with numpy 2. I checked that DeepHash detects changed values and reordering on 5.0 through 8.0.
3. numpy reason: You're right about 3.10 wheels. The table now says 1.22.4 is pandas 2.2's numpy minimum on Python < 3.11.
4. pymysql: Ran it. 0.7.2 and 0.10.1 fail on every MySQL connection (ssl_disabled is not accepted before 1.0), so the minimum is now pymysql>=1.0.0. 1.0.0 through 1.1.0 pass.

Full suite on Python 3.10 with every core dependency at its new minimum: 1234 passed, 14 skipped. Again with numpy 2.2.6: 1234 passed, 14 skipped. The PR description has the details.

[MilagrosMarin]

Thanks @dimitri-yatsenko, all four addressed in a98fd31 — and thanks for running the pymysql minimum rather than leaving it as a follow-up.

✅ pandas>=2.2.2 closes the pandas 1.5.3 + numpy 2 import failure
✅ deepdiff>=6.6.0 gives it a real lower bound, with the 3.15 cap kept
✅ numpy reason reworded
✅ pymysql>=1.0.0, matching the ssl_disabled option the MySQL adapter passes

The inline comments in pyproject.toml make each bound easy to revisit later. LGTM!

[dimitri-yatsenko]

Adding two things on top of a98fd31 (cc4ca9e). @MilagrosMarin, could you re-review?

1. A CI job that keeps these minimums tested. test (minimum dependency versions, py3.10) runs the full suite on Python 3.10 with every runtime dependency pinned at its declared minimum. .github/scripts/minimum_requirements.py derives the pins from [project].dependencies. It fails when a runtime dependency has no minimum, so a bare numpy can't come back. It combines several lines for the same package (the 3.15 deepdiff<9 cap), takes the highest lower bound, normalizes names, and rejects wildcard versions and minimums that a requirement's own other version specifiers exclude. Optional extras are not pinned.

2. Minimums for deepdiff and pymysql raised past known vulnerabilities. Your review settled the oldest versions that work. Because datajoint now runs in secure environments, these two start instead at the first release without a published advisory:

	Oldest that works (a98fd31)	Now	Advisory
deepdiff	6.6.0	8.6.2	CVE-2025-58367 (class pollution in Delta, possible code execution), CVE-2026-33155 (memory exhaustion)
pymysql	1.0.0	1.1.1	CVE-2024-36039 (SQL injection via dict keys)

The comments in pyproject.toml keep your functional reasons and add the CVEs. pixi.lock moves deepdiff from 8.6.1 to 9.1.0 on Python < 3.15.

Verification. On Python 3.10, installed with pip exactly as the job does, at deepdiff==8.6.2 fsspec==2023.1.0 networkx==3.0 numpy==1.22.4 packaging==22.0 pandas==2.2.2 pydantic-settings==2.0.0 pydot==3.0.0 pymysql==1.1.1 pyparsing==3.1.1 tqdm==4.66.3, the full suite passes: 1234 passed, 14 skipped. A pip dry run confirms these are the versions the job installs. actionlint passes, and the branch merges cleanly with #1652 and #1654.

[dimitri-yatsenko]

Done in a69650b, following the review on the advisory floors:

• pymysql: (a), kept at >=1.1.1 for CVE-2024-36039.
• deepdiff: (b), back to >=6.6.0, the oldest version that works. I read both advisories before deciding. CVE-2025-58367 is class pollution through the Delta constructor, leading to pickle deserialization. CVE-2026-33155 is unbounded allocation in _RestrictedUnpickler (pickle_load and Delta). datajoint uses only deepdiff.DeepHash (autopopulate.py:656-711 is the only use in src/), so neither applies. The comment in pyproject.toml says so, and Dependabot will still report the advisories. pixi.lock is back to deepdiff 8.6.1; the only lock change from a98fd31 is the pymysql requirement line.

On Python 3.10, installed with pip as the CI job does, at deepdiff==6.6.0 fsspec==2023.1.0 networkx==3.0 numpy==1.22.4 packaging==22.0 pandas==2.2.2 pydantic-settings==2.0.0 pydot==3.0.0 pymysql==1.1.1 pyparsing==3.1.1 tqdm==4.66.3, the full suite passes: 1234 passed, 14 skipped. Relative to the version you approved, the only floor change is now pymysql 1.0.0 → 1.1.1; the minimum-versions CI job is the other addition.

[MilagrosMarin]

Thanks @dimitri-yatsenko, the additions since a98fd31 look good:

✅ pymysql>=1.1.1 matches the CVE-2024-36039 advisory (fixed in 1.1.1)
✅ Keeping deepdiff>=6.6.0 makes sense: both advisories are in Delta/pickle_load, and DeepHash in autopopulate.py is the only use in src/
✅ The minimum-versions job is a nice guard. Failing on a dependency with no minimum keeps a bare requirement from coming back

Approving. Thanks for adding the CI coverage!

[lum-agilyti]

One thing: this overlaps with PR 1628, which moves pymysql out of the core dependencies into the mysql extra at >=0.7.2. Whichever merges second will need the 1.1.1 floor on the extra. Also, the description's table still lists pymysql at 1.0.0, while the diff is at >=1.1.1.