Skip to content

Add LLM streaming response guardrails skill and prompt - #95

Merged
rmkr-dev merged 2 commits into
mainfrom
add-llm-streaming-guardrails
Nov 12, 2026
Merged

rmkr-dev merged 2 commits into
mainfrom
add-llm-streaming-guardrails

Conversation

@rmkr-dev

Copy link
Copy Markdown
Owner

Adds a Skillbook entry and a Promptbook entry for streaming (SSE) responses in LLM gateways and Spring Boot services.

Skill: llm-streaming-response-guardrails

  • Event contract with a finish or error frame plus [DONE] on every exit path
  • Auth, budget, and target selection before the first byte; no retry or fallback after content is sent
  • Upstream cancellation on client disconnect, including during the time-to-first-token wait
  • Time-to-first-token, inter-chunk idle, and total deadlines with SSE heartbeats
  • ingress-nginx buffering/timeouts and compression on streaming paths only; bounded buffers
  • Pass-through, windowed, or buffered output checks; tool-call fragment assembly
  • Final or estimated usage, stream metrics, shutdown drain, and fake-upstream tests

Prompt: harden-llm-streaming-responses

  • Inventory, relay diff, deadlines, edge settings, usage, and a test list, with placeholders for stack, providers, edge, and output checks

Copilot, Claude Code, and Codex placement notes are included. Catalogs now 57/57; node scripts/site-check.mjs passes locally.

@rmkr-dev
rmkr-dev merged commit cc6069b into main Nov 12, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant