Repository navigation
Explore results on the template's standard Viewer page - #161
Draft
t0mdavid-m wants to merge 69 commits into
Draft
t0mdavid-m wants to merge 69 commits into
t0mdavid-m wants to merge 69 commits into
Conversation
… left A run over 200 files listed every file on the Run page. The list now shows up to 12: the failed and running ones, the next waiting and the latest finished, and counts the rest in one line. The progress bar estimates the time left from the files already finished. After a run, the first five failed files keep their own error and the others are listed in one expander; a long list of samples has the full list in an expander. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
It jumped too much between files to be useful for now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
A run with several failed files listed each with its own error box. Now each tool that failed gets one line with the count, a table of the files and their errors, and one Tool output section with a file picker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
The per-sample lines with their marks are for following a run. Afterwards the table of failed files and the totals say what happened. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
…everal connections (#578) * Download large imported files as parallel byte ranges A single HTTPS connection to the PRIDE archive gives about 1.3 MB/s from far away, so a 1 GB raw file took over ten minutes. A file of at least 128 MB whose server sends ranges is now fetched in ranges of at least 64 MB over up to eight connections (download_connections), each range resuming and retrying on its own; the connections are shared by all files of an import. The MD5 is computed while the ranges are joined, so the file is not read a second time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGekduWSRSEKBPSAhder * Document download_connections for imports Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGekduWSRSEKBPSAhder * List a PRIDE dataset's files in one request The paged files endpoint returns at most 100 entries, so looking up a dataset of 3426 files took 35 requests and 23 s. files/all returns them in one request (1.6 s for the same dataset); paging stays as fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGekduWSRSEKBPSAhder * Remove the temporary download benchmark Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGekduWSRSEKBPSAhder --------- Co-authored-by: Claude <noreply@anthropic.com>
…m (#577) * Read Thermo .raw files with OpenMS's built-in reader on every platform FileConverter defaults to ThermoRawFileParser on Windows and to the built-in reader on macOS and Linux, and the Docker image had only ThermoRawFileParser on mono. The two write slightly different mzML (extra low peaks, a second precursor entry on supplemental-activation scans), so the same .raw file gave different results per platform. - convert_to_mzml() passes -RawToMzML:reader inprocess whenever the installed FileConverter offers it (thermo_reader_args()). - The full Docker images build OpenMS WITH_THERMO_RAW=ON with the .NET SDK OpenMS pins in tools/ci/install_dotnet_sdk_linux.sh, and check that the reader and its managed assemblies are there. - The Windows desktop app ships the .NET runtime too; ThermoRawFileParser is staged only for an OpenMS without the built-in reader. - The desktop smoke test and the Apptainer test convert a real .raw file with the built-in reader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen6esZDGqZn6bzrBBG * stage-dotnet.sh: strip the CR of jq's output on Windows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen6esZDGqZn6bzrBBG * Tests: conversion race tests stub the Thermo reader lookup Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen6esZDGqZn6bzrBBG * Record the Thermo reader with a converted mzML; fall back to the default reader - A converted .raw file's mzML is reused only when it was made by the reader convert_to_mzml now uses (.<name>.mzML.reader next to it), so mzML of an earlier version's reader is converted again, not mixed in. - When the built-in reader fails (no .NET 8 runtime, e.g. a source install on Windows), the conversion is tried again with FileConverter's default reader, with a warning in the log. - The desktop smoke test checks that the FileConverter .ini and the managed assemblies agree on the built-in reader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen6esZDGqZn6bzrBBG * Retry a failed .raw conversion with the tool's own default reader The retry names FileConverter's default reader, and a value in custom_params now replaces a stored one on the command line, so a reader saved in the workflow's parameters cannot take its place. A file converted after such a fallback is kept on later runs instead of being converted again while .NET is missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen6esZDGqZn6bzrBBG --------- Co-authored-by: Claude <noreply@anthropic.com>
A failed run groups its failed files per tool, also for steps run with allow_failed; a finished run still names the files left out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGepShSG5cNRTqoCXFo1
Both are applied once per cluster by its admin and are no longer part of the template's deployment, so no app's Argo CD project needs a cluster-scoped kind. CI keeps checking its storage stand-in against the production class's name, reclaim policy and mount options. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemTJKJs8ZhnhDz43cQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemTJKJs8ZhnhDz43cQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemTJKJs8ZhnhDz43cQ
Run page: list a long run's failed and running files first
* Draw the first experiment's selector above the view bar experiment_selector() draws an experiment's dataset selector with its buttons where the page calls it, and render_experiments(first_selector=False) then shows the dataset picked there without drawing the selector again. A Viewer can so read top to bottom: what is shown and what can be done with it, how it is shown, the components. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemhZJqShow7AQz5c3q * Screenshots for the pull request Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemhZJqShow7AQz5c3q * Remove pull request screenshots Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemhZJqShow7AQz5c3q * View MS data: one panel style for every plot, full-width peak map Plotly charts get the same rounded panel as Insight components and transparent figure backgrounds, so no grey box sits inside a white page. The peak map takes the whole row unless the 3D plot is shown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemhZJqShow7AQz5c3q * Screenshots for PR description Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemhZJqShow7AQz5c3q * Remove PR screenshots Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemhZJqShow7AQz5c3q * Docs: pass label and buttons to experiment_selector by keyword Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemhZJqShow7AQz5c3q * View bar: a changed ready-made view reaches the stored layout The layout stored for the current view is written again when its rows no longer match the view, so an update that changes a ready-made view shows on the pages that already display it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemhZJqShow7AQz5c3q --------- Co-authored-by: Claude <noreply@anthropic.com>
…581) The installer's license page now names the same copyright holder as LICENSE. Claude-Session: https://claude.ai/code/session_01FkEGenTCt1fet8pkp5v8T2 Co-authored-by: Claude <noreply@anthropic.com>
…ary, template, app Adds a 'Four layers' section to the agent rules in CLAUDE.md (what each layer holds and never holds, a test for each, and the two rules that keep them apart), glossary entries for the visualization library and panel sources, and pointers in the upstream-to-template, build-insight-dashboard and add-visualization skills. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
…ter-config Leave the openms namespace and the storage tier to the cluster
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
…#582) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemrJRTcypYoksP1PDd
Conflicts resolved by ownership: - .github/workflows/build-and-test.yml, docs/kubernetes-deployment.md, content/quickstart.py: app-owned, kept OpenDDA's - .github/workflows/build-desktop-app.yaml: OpenDDA's TOPP_TOOLS; the .NET runtime is now staged on Windows too, since FileConverter uses OpenMS's built-in .raw reader on every platform - .claude/skills/sync-from-template/SKILL.md: template's - content/raw_data_viewer.py: stays deleted in OpenDDA Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGemrJRTcypYoksP1PDd
The Application and the Image Updater now live in the cluster's own configuration, so a push here changes only what the prod overlay contains, not what is deployed from where. Claude-Session: https://claude.ai/code/session_01FkEGemTJKJs8ZhnhDz43cQ Co-authored-by: Claude <noreply@anthropic.com>
show_viewer(ViewerSpec) lists the datasets, applies a pending pick, draws the first experiment's selector with its buttons, the view bar of its kind of dataset and every experiment with its own state. Kinds of dataset (a run, a comparison, a file) carry their catalog, presets, renderers and buttons; an experiment made of several runs is shown as Files of or as Single run | Comparison. View MS data uses it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
openms-insight main (0.7.6.dev0, with SeriesPlot bars and the latest Table and sequence view fixes) as a wheel in dev-wheels/, so the desktop installers can be tested. Revert to a PyPI pin before release. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
…-f170bl # Conflicts: # .github/workflows/build-desktop-app.yaml # src/common/spectrum_view.py
…ode, GO terms computed with the run, readable heatmap labels, numeric heatmap colour bounds Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
…width, plot rows of one height Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Member
Author
|
The desktop build's Windows end-to-end test failed in the rescoring scenario. The app process crashed with a native access violation inside The same scenario passed on Windows one commit earlier (run 43622561636, 2af3933). The commit in between only changes the QC layout and does not touch that table or how its cache is read, so this looks intermittent rather than caused by this PR. I'll rerun the failed job once the macOS job of this run has finished. If it fails again, it needs a closer look in Generated by Claude Code |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
…ialog member_switch() draws the switch outside the standard Viewer, and whole_kind/shown_whole tell the whole of an experiment from its first run when both have one id (run folders). edit_comparison() is a dialog with one tab per comparison step, Start over and Done. The example Results page offers both for an experiment of several runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
…into claude/project-thread-f170bl # Conflicts: # src/Workflow.py
…alog An experiment of several runs is shown as a comparison (every run) or one run on its own, whose abundances, group comparison, PCA and heatmaps then cover its files only (scope.experiment(single=True), a comparison of its own). A run alone offers processing more files for a comparison. Edit comparison is the template's edit_comparison dialog with the four steps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
The workflow now fills a viewer-cache folder in each run with the per-file overview and QC tables, the per-sample counts, the abundance table and the group comparison with the recommended settings, computed through the same functions the Viewer calls. The Viewer then opens on them instead of parsing idXMLs and comparing groups on its first visit. A changed file or setting is computed again; the cache is left out of a run's ZIP. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
The samples of a group are numbered next to each other and joined by a line, and each group gets a line at its mean, so the gap the volcano plot measures is visible. The plot is as tall as the volcano beside it, which leaves no empty space below the volcano. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
A new workspace (online, and the local default one) starts from example-data/workspaces/default: the example experiment (six UPS2 runs in two groups) processed once with the default settings, so Explore results shows quantification, identification and quality control right away. The run's inputs point at example-data/mzML and example-data/db; the viewer caches are left out and are built on the first visit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Picked points ringed in the volcano, its lasso/measure/zoom/home toolbar, selection tools hidden where a plot sends no multi-selection, info button on tables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
…, earlier components and Files left out of the layout manager; Insight dev5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
…ance comparison - Template Viewer collects Insight reruns into one (comparison dialog and Identification/Pathways views no longer loop). - The switch reads Single run | All runs. - Abundance of the picked protein: samples as points, group means as lines. - Help on the GO tables; openms-insight dev build 0.7.6.dev6. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
…red (template) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
… template's library The plots of the comparison steps, the volcano, the PCA and the clustered heatmap are the panels of src/viz, drawn here in the Viewer's cache and state. The clustered heatmap colours groups like the PCA. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
…alization library Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is stacked on #156. It needs OpenMS/streamlit-template#584, which adds the standard Viewer page (
src/layout_manager/viewer.py) and the visualization library (src/viz/). This branch merges that template branch, so it runs on its own. It also mergesmain.What changes
Standard Viewer page
content/viewer.pyno longer builds the page out of the layout manager's pieces. It describes the runs as aViewerSpecwith oneDatasetKindand callsshow_viewer.The template then draws:
Edit comparison opens the template's dialog (
src/layout_manager/comparison.py), one tab per step: groups, filter, missing values, normalize, test.Example workspace with a finished run
A new workspace starts with a processed example run, so the Viewer shows results right away without running the workflow first.
example-data/workspaces/default/topp-workflow/holds the run of the example files (results, logs, settings and experimental design). Its paths point atexample-data/mzML/andexample-data/db/.settings.jsonnames it asdemo_workspaces.new_workspace. Every online workspace and the localdefaultworkspace start from it. Large files are linked, not copied.Everything computed during the run
The Viewer no longer processes anything on its first visit. At the end of a run,
WorkflowTest.prepare_viewer()(src/viewer/precompute.py) fills<run>/viewer-cache/with what the Viewer reads:Each entry is keyed on the file it was computed from (size and modification time), so a rewritten file is computed again (
src/common/disk_cache.py). A failure there never stops a run: the Viewer computes what is missing. Switching between the example views (Identification, Quality control, Pathways, Quantification) shows no spinner and no plot left loading.Plots from the template's visualization library
The plots FLASHApp draws the same way now come from the template's
src/viz/, with OpenDDA's own tables as their source:go_panel.py): category selectors, the number of selected proteins on a line of its own with Selection | All proteins, and the enriched terms and the GO terms of all proteins as paginated Insight tables.comparison_steps.py): where values are missing, the values imputation fills in, the boxes per sample before and after normalization, and the p-value histogram.src/common/postprocessing_plots.pykeeps only the filter curve, which only OpenDDA draws.volcano.py,pca.py,heatmap.py).The abundance profile, the plain heatmap and the comparison table stay in OpenDDA.
Layout manager
The "Earlier components" (run summary, identification quality, protein explorer) and "Files", which is not a visualization, are no longer offered. Their content is in the current components.
Cleaner viewer
src/viewer/teaching.py.polars.exceptions.ComputeError: cannot compare string with numeric type (f64).Depends on
SeriesPlotbox and bar modes, used by the comparison steps). This branch uses its development build fromdev-wheels/. Before a release, that must become a PyPI pin.Screenshots (example workspace, light)
Volcano and abundance per sample:
The screenshots were added in one commit and removed in the next, so they are not part of the diff. Screenshots of the library's group colours in the clustered heatmap are still to come.
Tests
test_gui.py tests/) passes (1328), includingtests/test_disk_cache.py,tests/test_go_panel.py,tests/test_viewer_plots.py(now on the library's data functions) and the abundance profile test intests/test_viewer.py.🤖 Generated with Claude Code
https://claude.ai/code/session_01FkEGen2HSCa8BFE1JW4jxN