Skip to content

[FEATURE] Partition SOF into DSP kernel and userspace source/include trees #11306

Open

Description

@lgirdwood

Partitioning user/kernel code for trust boundary and later safety island. Claude generated.

Is your feature request related to a problem? Please describe.

SOF is moving LL pipelines, DP modules and IPC processing into Zephyr user
threads (CONFIG_SOF_USERSPACE_LL, CONFIG_SOF_USERSPACE_PROXY, the
handler-kernel.c / handler-user.c split, and the syscalls in zephyr/syscall/).
The source tree doesn't show that boundary, though:

  • Kernel-only code (MMIO, IRQs, k_thread_create, k_mem_domain_*, PM,
    clocks, raw dma_*, IDC, LLEXT loading) sits next to code that must be
    user-safe, often in the same file. Examples: ipc-common.c,
    zephyr_domain.c, zephyr_ll.c, lib/dma.c, lib/dai.c.
  • User-facing headers pull in kernel headers. component.h includes
    rtos/idc.h, sof/lib/dai.h (→ zephyr/drivers/dai.h) and
    schedule/schedule.h. audio_stream.h includes sof/lib/dma.h
    (→ zephyr/drivers/dma.h). rtos/alloc.h includes sof/lib/memory.h,
    which includes platform/lib/memory.h. sof/platform.h is included by about 54
    audio/math/ipc files. As a result, every component and every LLEXT module
    (they inherit zephyr_interface include dirs) can see platform, DMA and DAI
    internals.
  • There is no build-time enforcement, so you only find out that user code
    calls a privileged API when it faults on hardware.

Describe the solution you'd like

Partition the sources and headers into DSP kernel and userspace parts,
and have the build enforce the split.

Definitions:

  • Kernel = code that must run in supervisor mode.
  • User = everything else. It must be user-safe: plain C, syscall-backed Zephyr
    APIs, and SOF syscalls.
  • The directory says what the code is allowed to do, not where it runs in a
    particular config. With SOF_USERSPACE_LL=n, user code still runs in
    supervisor mode, and that's fine.

Layout:

src/kernel/                     # new: supervisor-only code, own CMake lib (sof_kernel)
  include/sof/kernel/...        # kernel-private headers -> #include <sof/kernel/...>
  init/ idc/ library_manager/ logging/ platform/ drivers/ arch/ probe/
  base_fw/ (base_fw*.c, intel_uaol.c)  mic_privacy/
  ipc/ (ipc-zephyr.c, ipc4/handler-kernel.c, notification*.c, ipc3/)
  schedule/ lib/ debug/ audio/    # kernel halves of split files
src/{audio,math,module,samples,ipc,schedule,lib,debug}/   # user (what remains)
src/include/                    # user-visible headers only
zephyr/                         # Zephyr module root stays where it is (OS glue = kernel)

Decisions:

  1. Include path. Kernel-private headers are included as
    <sof/kernel/...>. They live in src/kernel/include/, which is a PRIVATE
    include dir of the sof_kernel library. It is not on zephyr_interface, so
    neither user code nor LLEXT modules can include them, and a simple rule
    (no <sof/kernel/ in user sources) is checkable in CI and checkpatch. The host
    ABI dirs src/include/kernel/ and src/include/user/ (Linux driver and tools
    ABI) are not touched.

  2. XTOS is deleted rather than moved: xtos/, the XTOS firmware build,
    XTOS-only platform, driver and arch code, and the old debug/gdb stub.
    Removing the legacy-driver path that Kconfig.xtos exists for
    (dai-legacy.c, host-legacy.c, the DMA domain schedulers and
    ll_schedule_xtos.c) depends on the remaining non-native platforms moving to
    native Zephyr drivers and on the testbench no longer using the legacy
    host/DAI components (see Phase 0b).

  3. zephyr/ stays at the top level. It is the Zephyr module root and OS
    glue, and is treated as kernel.

  4. IPC3 is kernel. There is no user IPC3 thread, and it uses raw DMA
    (ipc3/dai.c, host-page-table.c). It moves to src/kernel/ipc/ipc3/.

  5. lib/notifier.c is deleted and replaced with Zephyr mechanisms. It
    uses k_spin_lock + IDC, so it can't be called from user mode, and its
    callbacks run in the notifier's context. Per event:

    Event Live users after XTOS removal Replacement
    CPU_FREQ / SSP_FREQ none (ids are set in clock_info, but nothing is ever fired on Zephyr) delete
    DMA_COPY, DMA_IRQ, DMA_DOMAIN_CHANGE legacy drivers and DMA domains only (dai-legacy.c, host-legacy.c, dw/dma.c, AMD/MTK legacy DMA, dma_*_chan_domain.c) deleted along with the legacy-driver path. Native drivers use Zephyr dma_config.dma_callback
    DAI_TRIGGER dai-zephyr.c DAI-group sync trigger (irq_local_disable + notifier_event) the last DAI to trigger walks the group member list itself. The atomic section becomes a kernel syscall (e.g. sof_dai_group_trigger()) that holds the IRQ lock
    KPB_CLIENT_EVT google_hotword_detect.c, detect_test.c → kpb.c k_msgq (or k_event + shared payload) owned by kpb and granted to the pipeline memory domain. Both are syscall-backed
    MIC_PRIVACY_STATE_CHANGE mic_privacy_manager IRQ → copier.c k_event_post() from the ISR. Copier checks it with k_event_test() / K_NO_WAIT in its copy path and reads the state through a syscall or a mapped partition. No callback runs in IRQ context

Enforcement: scripts/check_kernel_includes.py, which reads the ninja
dependency log of a build and reports kernel-only headers included by
user-space sources, classified in scripts/kernel_user_partition.txt. It starts
as report-only and becomes enforcing once the headers are moved.

Implementation is a series of PRs. Every commit builds in all CI configs. Directory
moves are git mv plus path fixups only, so history is preserved.


Checklist

Phase 0: guard rails, XTOS and ALSA plugin removal

0a: no platform loses support. Merged in #11312:

  • zephyr: remove the references to the deleted hda-dma.c and lps_wait.c
  • ci: remove the no-op XTOS reproducible build check (test-repro-build.sh
    exits 0 because xtensa-build-all.sh no longer exists)
  • test: cmocka: remove the dead Xtensa (xt-run) unit test build
  • test: cmocka: move the unit test defconfigs from src/arch/xtensa/configs
    to test/cmocka/configs
  • cmake: remove the legacy XTOS firmware build (top-level add_executable(sof),
    xtensa toolchain file, scan-build wrapper, XTOS-only per-platform/driver
    CMake files, Chrome XTOS config overrides)
  • platform: amd: remove Renoir, Vangogh and ACP 6.3
  • drivers: amd: remove the sources that no build uses
  • platform: mediatek: remove the per-SoC XTOS dirs (src/platform/mt81xx,
    src/drivers/mediatek/mt8*, afe-sgen); Zephyr uses src/platform/mtk
  • platform: imx: remove the XTOS DMA, DAI, interrupt and timer drivers
  • drivers: dw: remove the DesignWare DMA, GPIO and SPI drivers (CONFIG_DW
    has no prompt and nothing selects it, so CONFIG_DW_DMA cannot be enabled)
  • drivers: remove the unused Intel HDA headers
  • platform: posix: remove the unused base_fw_platform.c
  • debug: remove the old SOF GDB stub (CONFIG_GDB_DEBUG, cannot build for
    any firmware target); the Zephyr CONFIG_GDBSTUB path is unaffected
  • zephyr: move the legacy XTOS IRQ header into zephyr/include; then remove
    xtos/include
  • installer: remove the XTOS firmware installer
  • docker: sof_builder: drop the XTOS crosstool-ng toolchains
  • tools: plugin: remove the SOF ALSA plugin, its topology and the
    shared-library module build (CONFIG_COMP_MODULE_SHARED_LIBRARY_BUILD)
  • library: always build the host library statically (drop
    CONFIG_LIBRARY_STATIC)
  • scripts: add check_kernel_includes.py and kernel_user_partition.txt;
    ci: run it report-only in the Zephyr workflow. Current baseline for PTL:
    148 user-space sources include 28 kernel-only headers.
  • src/trace: removed in Kernel/user partition: remove the SOF DMA trace #11421. Zephyr builds can't enable the SOF DMA trace,
    so it was only built for the host. The testbench and host library now print
    their traces through a small trace-host.h, with the same levels and output.
    src/trace, dma-trace.h, trace-soflogger.h, dma-copy.c and
    CONFIG_TRACE* are gone. The host ABI in include/ipc/trace.h stays.

Verification for 0a: the non-debug sections of zephyr.elf are identical on all
15 platforms that build on main, and so are all 66 LLEXT modules. The libsof.a
symbol table is unchanged. testbench, tools and all cmocka configurations pass.
The full error sets of the platforms that already fail on main are unchanged.

0b: legacy driver path (needs the AMD and MediaTek maintainers). These platforms
do not build on main today, and CI does not build them:

  • acp_6_0, mt8186, mt8188, mt8195 and mt8365 fail in ipc3/dai.c and
    host-legacy.c.
  • acp_7_x fails because Kconfig symbols are missing in the pinned Zephyr.
  • mt8196 fails because the mediatek,sof-host-dma DT node is missing.

Items:

  • AMD acp_6_0 (Rembrandt): native Zephyr ACP DAI/DMA, and zephyr_ll.c
    instead of ll_schedule_xtos.c + dma_multi_chan_domain.c
  • MTK mt8186, mt8188, mt8195, mt8365: CONFIG_ZEPHYR_NATIVE_DRIVERS=y
    (zephyr_mtk_*), as already done for mt8196
  • testbench: stop building host-legacy.c and dai-legacy.c
  • Delete the legacy-driver path:
    • dai-legacy.c, host-legacy.c, dma-legacy.h
    • legacy AMD/MTK DMA drivers, dummy-dma.c
    • dma_*_chan_domain.c, zephyr_dma_domain.c, ll_schedule_xtos.c
    • rtos/interrupt-legacy.h + src/drivers/interrupt.c
    • Kconfig.xtos, ZEPHYR_NATIVE_DRIVERS

Phase 1: header untangle (no moves)

First part: merged in #11330, header hygiene with no functional change. Kernel-only headers pulled into user-space sources on PTL: zephyr/drivers/dma.h 145 → 27, mailbox.h 141 → 20, rtos/timer.h 139 → 10, rtos/userspace_helper.h 136 → 10, rtos/idc.h 135 → 26. What remains mostly comes through rtos/alloc.h → sof/lib/memory.h (the platform memory map), which the second part addresses.

Second part: merged in #11345. PLATFORM_DCACHE_ALIGN now comes from the Zephyr cache line size and PLATFORM_MAX_CHANNELS is the Kconfig option CONFIG_PLATFORM_MAX_CHANNELS. The cold code attributes move to sof/lib/cold.h, and rtos/alloc.h, rtos/sof.h and ipc/msg.h no longer include the platform memory map or sof/platform.h. On PTL, sof/platform.h now reaches 15 user-space sources instead of 130, and the platform clock headers 20 instead of 130. sof/lib/memory.h is still reached through sof/coherent.h, which needs the cached/uncached address helpers.

Third part: merged in #11364, with no functional change. The cached/uncached alias helpers move to rtos/cache.h (Zephyr cache API on ACE, unchanged elsewhere), so sof/coherent.h no longer includes the memory map. comp_dai_config() moves to sof/audio/component_dai.h. PLATFORM_MAX_STREAMS becomes CONFIG_PLATFORM_MAX_STREAMS. The LL period and user LL lock helpers move to ll_schedule.h, and unused component_ext.h includes are dropped. On PTL, user-space sources including any kernel-only header go from 143 to 45: the memory map 141 → 34, rtos/idc.h 26 → 11, zephyr/drivers/dma.h 27 → 10, the clock headers 20 → 14 and sof/platform.h 15 → 11.

Fourth part: merged in #11386, with no functional change. CONFIG_ACE_VERSION_2_0_OR_LATER replaces the ACE_VERSION > ACE_VERSION_1_5 checks, so the DAI code drops sof/platform.h. The DRC, multiband DRC and SRC sources switch from the memory map to sof/lib/cold.h, unused rtos/ includes are dropped, and the volume peak meter mailbox write moves out of volume.h. On PTL, user-space sources including any kernel-only header go from 45 to 26: the memory map 34 → 19, mailbox.h 20 → 12 and rtos/userspace_helper.h 10 → 2. PLATFORM_PRIMARY_CORE_ID turned out to come from sof/lib/cpu.h, not sof/platform.h.

Phase 2: src/kernel + sof_kernel library, move the pure-kernel dirs
First part: merged in #11394. It adds src/kernel and the sof_kernel library, and moves src/init there. The library is created in zephyr/CMakeLists.txt next to modules_sof, and src/kernel/CMakeLists.txt makes it the current Zephyr library for its subtree, so the moved directories' CMake files don't change. The generated code is unchanged. The link order changes, but symbols and SYS_INIT() order are identical in every image. Boot, playback and playback after runtime suspend were checked on TGL. On the Cortex-M7 builds sof_init() changes size from commit to commit only because it embeds the git hash, and Thumb-2 encodes some hash values more compactly than others.

Phase 3: split mixed source files (handler-kernel.c pattern), move the kernel halves

The kernel half of src/X/foo.c goes to src/kernel/X/foo-kernel.c. The kernel side gets:

  • boot-time init;
  • thread, timer and kernel-object setup;
  • memory domains and access grants;
  • the z_impl_/z_vrfy_ syscall halves.

What runs in the scheduler, IPC and DP threads stays user. Every split is checked function by function against the original file, and the per-function disassembly of every image is compared. The only differences are inlining across the new file boundary, described in each commit.

Phase 3 is complete: #11422, #11424, #11459 and #11461 are merged.

Schedulers: merged in #11424. The shared data structures are in sof/schedule/zephyr_ll.h and sof/schedule/zephyr_dp_schedule.h.

Phase 4: move the kernel headers, turn on enforcement

First part: merged in #11464. The include checker is enforced in CI against a baseline of the user-space sources that still leak, so new kernel-only includes fail the build. Every source in src/ outside src/kernel is now checked as user-space code, which adds the Phase 3 user halves. It also fixes a regression: rtos/timer.h in telemetry.h (20206e3) had brought sof/platform.h back into every component source, taking PTL from 28 to 140 leaking user-space sources. The baseline has 41 sources; removing them finishes the Phase 1 exit criterion and unblocks the header moves.

Second part: draft PR #11466. The IDC message format moves to rtos/idc_msg.h, so the component API no longer includes the IDC driver header or ll_schedule_domain.h. 32 unused kernel-only includes are removed from user-space sources, and four user headers drop kernel includes they did not need. In the CI configurations, rtos/idc.h, sof/ipc/driver.h, sof/llext_manager.h, sof/drivers/dmic.h and sof/lib/mm_heap.h no longer reach any user-space source, and the (source, kernel header) pairs go from 854 to 686. The baseline goes from 41 to 39 sources. Most remaining leaks are kernel API uses, not includes.

  • Remove the remaining leaks and shrink [leak-baseline] in scripts/kernel_user_partition.txt to empty.
  • git mv the kernel headers to src/kernel/include/sof/kernel/... and
    rewrite the kernel includes to <sof/kernel/...>.
  • Platform include dirs become PRIVATE to sof_kernel, and the user-needed subset
    is exported through user headers.
  • Rebuild every sof_llext_build module and the LMDK samples against the
    reduced include set.
  • Install rules install only the user include set.
  • Switch the checker to enforcing in CI (kernel/user partition, Phase 4 part 1: enforce the include checker, fix a telemetry include regression #11464, merged, with a leak baseline), and add a checkpatch rule against
    <sof/kernel/ in user dirs.

Phase 5: remove the notifier and close the user-safety gaps

  • DAI-group trigger: drop the notifier and add a sof_dai_group_trigger() syscall
    for the IRQ-locked section.
  • KPB client events: replace with a k_msgq (or k_event) for the hotword
    detector → kpb path, covering google_hotword_detect.c and detect_test.c.
  • Mic privacy: replace with an ISR k_event_post() and copier polling.
  • Remove the notification_id/notification_mask fields from clock_info.
  • Delete lib/notifier.c, notifier.h, IDC_MSG_NOTIFY, init_system_notify()
    and sof->notify_data.
  • Remaining kernel calls from user-classified code:
    • the kernel heap allocators rmalloc(), rzalloc(), rballoc_align() and rfree(),
      called directly from about 30 user-space sources (mostly src/audio); they only
      work in kernel mode (found by the Phase 3 check, Phase 3 (part 4): move the kernel-only debug stream code to src/kernel/debug #11461)
    • user-safe getters that live in kernel files: sof_sys_user_heap_get(),
      sof_sys_heap_get() (zephyr/lib/alloc.c) and vregion_mem_info()
      (zephyr/lib/vregion.c)
    • idc_send_msg / ipc4_process_on_core from the user IPC thread
    • idc_cmd_user() and the IDC component handlers it calls run in the user-space IPC
      thread but live in src/kernel/idc/idc.c (found in kernel/user partition, Phase 4 part 2: untangle the IDC and component headers, remove unused kernel includes #11466)
    • k_cycle_get_64 / sof_cycle_get_64 in host-zephyr, kpb, probe and
      perf_cnt (check whether this is wall-clock MMIO on intel_adsp)
    • kpb pm_runtime_disable and its EDF task
    • lib_notif_msg_send
    • probe raw dma_* → sof_dma_*, which would let probe move back to user
  • Re-enable the features app/overlays/ptl/ll_userspace_overlay.conf currently
    turns off for user LL: telemetry, CROSS_CORE_STREAM, MIC_PRIVACY, xrun
    notifications, cold-store debug.

Describe alternatives you've considered

  • Path-preserving header move (kernel headers keep their <sof/lib/...>
    path but move to a private include root). This means no include churn, but you
    can't tell from an include line which headers are kernel-only. We chose
    <sof/kernel/...> so the boundary is grep- and checkpatch-able.
  • Renaming the host ABI src/include/kernel/ and src/include/user/ to free
    up the names. Rejected: about 245 include sites, the topology get_abi.sh
    parsing, and the hand-synced Linux copies.
  • Moving legacy XTOS code into src/kernel/legacy/ instead of deleting it.
    Rejected, because it moves code no Zephyr build compiles.
  • Keeping the notifier and making it a syscall. Rejected: callbacks would
    still run in the poster's context across the privilege boundary. Zephyr
    k_event / k_msgq already give syscall-backed, memory-domain-aware
    signalling.

Additional context

  • Related to the userspace LL / DP work: CONFIG_SOF_USERSPACE_LL,
    CONFIG_SOF_USERSPACE_PROXY, zephyr/syscall/, src/ipc/ipc4/handler-{kernel,user}.c.
  • Expect merge conflicts with in-flight PRs during the Phase 2 and Phase 4 moves. Moves
    will be one dir per PR and announced ahead of time.
  • Validation per phase: CI build matrix (Intel cAVS2.5/ACE1.5/2.0/3.0/4.0, i.MX,
    AMD, MTK, native_sim/fuzz, testbench, cmocka, ztest), zephyr.elf section
    diffs, and boot+playback on TGL and PTL/ARL hardware.

Activity

  1. added
    enhancementNew feature or request
    ZephyrIssues only observed with Zephyr integrated
    on Oct 14, 2026
  2. self-assigned this
    on Oct 14, 2026
  3. lgirdwood commented on Oct 14, 2026

    @lgirdwood
    MemberAuthor

    I've updated the Phase 0 checklist in the description after checking each item against main.

    Corrections to the original plan

    • src/trace is still used: the testbench builds it (CONFIG_TRACE defaults to y for host builds). It is now a decision item rather than a deletion.
    • ll_schedule_xtos.c and the DMA domain schedulers are still built for acp_6_0 on Zephyr, and the testbench builds host-legacy.c/dai-legacy.c. Both moved to 0b, the legacy driver path.
    • The include checker is a new script based on the ninja dependency log. The safety-island checker mentioned before is not upstream.
    • The per-SoC MediaTek directories are confirmed unused by the Zephyr build.

    New findings

    • The legacy-driver platforms already fail to build on main, and CI does not build them. Details are in 0b.
    • CONFIG_DW cannot be enabled, so all of src/drivers/dw is dead, including the DMA driver that zephyr/CMakeLists.txt still lists.
    • The daily "XTOS-reproducible" job passes without building anything, because test-repro-build.sh exits 0 when xtensa-build-all.sh is missing.
    • The sof_builder docker image no longer builds on main: its alsa-lib/alsa-utils stage needs a newer autoconf than Ubuntu 22.04 provides. This is unrelated to the XTOS toolchain removal.

    Added to Phase 0: removal of the SOF ALSA plugin, together with the shared-library module build and CONFIG_LIBRARY_STATIC, which only existed for it.

    UUID registry entries of removed code are kept, because the registry requires that distributed mappings never change.

  4. lgirdwood commented on Oct 14, 2026

    @lgirdwood
    MemberAuthor

    Status: Phase 0a is up as draft PR #11312, with 23 commits and one patch per checklist item.

    • Every commit builds on tgl, imx8m, acp_7_0 and qemu_xtensa, plus the host library and cmocka.
    • At the tip, zephyr.elf is section-identical on all 15 platforms that build on main, and all 66 LLEXT modules are unchanged.
    • testbench, tools and all cmocka configs pass.
    • The platforms that already fail on main have unchanged error sets.

    Still open:

    • Decide what to do with src/trace: keep it as host-only trace, or move the testbench to a printf backend.
    • Phase 0b (legacy driver path) needs the AMD and MediaTek native-driver ports, and the testbench needs to stop using host-legacy.c/dai-legacy.c.
    • Phase 1 (header untangle) can start in parallel. The new checker in CI tracks progress.
  5. lgirdwood commented on Oct 21, 2026

    @lgirdwood
    MemberAuthor

    Status: Phase 0 is merged (#11312). The first part of Phase 1 is up as draft PR #11330: 10 header-hygiene patches with no functional change.

    • Per commit: every commit builds on all 15 platforms that build on main, with identical object files and LLEXT modules. The one exception is the file name/line text in two inlined assertions on qemu_xtensa.
    • At the tip: testbench, tools and all cmocka configurations pass. The platforms that already fail on main have unchanged error sets.

    Next in Phase 1: remove the rtos/alloc.h → sof/lib/memory.h dependency. That needs a small user-safe cache alignment header per platform, because PLATFORM_DCACHE_ALIGN differs per platform. Then split the mixed headers (component_ext.h, sof/lib/dma.h, dai-zephyr.h, the scheduler headers).

  6. lgirdwood commented on Oct 23, 2026

    @lgirdwood
    MemberAuthor

    Status: the first Phase 1 part is merged (#11330). The second part is up as draft PR #11345 (6 patches):

    • PLATFORM_DCACHE_ALIGN now comes from the Zephyr cache line size. This changes it from 32 to 128 on i.MX8/8X/8M/8ULP (only the IPC3 buffer allocation uses it there) and from 4 to 32 on qemu_xtensa.
    • PLATFORM_MAX_CHANNELS becomes the Kconfig option CONFIG_PLATFORM_MAX_CHANNELS, with unchanged values.
    • Cold code attributes move to sof/lib/cold.h.
    • rtos/alloc.h, rtos/sof.h and ipc/msg.h stop including the platform memory map and sof/platform.h.

    On PTL, sof/platform.h reaches 15 user-space sources instead of 130.

    Next:

    • the cached/uncached address helpers for sof/coherent.h (they decide is_uncached() semantics on i.MX/AMD/MTK);
    • the remaining platform constants (PLATFORM_MAX_STREAMS, PLATFORM_PRIMARY_CORE_ID, ACE_VERSION);
    • the mixed headers (component_ext.h, sof/lib/dma.h, dai-zephyr.h, scheduler).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

ZephyrIssues only observed with Zephyr integratedenhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions