Skip to content

Add LLM structured output guardrails skill and prompt - #93

Merged
rmkr-dev merged 3 commits into
mainfrom
add-llm-structured-output
Nov 10, 2026
Merged

rmkr-dev merged 3 commits into
mainfrom
add-llm-structured-output

Conversation

@rmkr-dev

Copy link
Copy Markdown
Owner

Adds the llm-structured-output-guardrails skill and matching enforce-llm-structured-output prompt.

The LLM entries so far touch structured output only in passing: prompt injection says to validate tool arguments, routing skips targets without json_schema, and the eval gate lists JSON validity as a hard check. None of them says how a service actually gets a schema-valid document or a clean failure. This pair covers that.

  • Native schema mode or strict tools where the target supports them, chosen from capability config. JSON mode and prompt-only targets are named explicitly and get the same checks.
  • Schemas are versioned files in the repo: object root, additionalProperties: false, every property required (nullable instead of optional), enums with an escape value, no free-form unions. Rules a provider's strict subset rejects move into the validator instead of being deleted.
  • Fixed check order: stop reason first (refusal, truncation, filter, unknown stop are typed outcomes), then extract, size cap, strict parse, JSON Schema validation, bind, business rules.
  • Server-side validation even in native mode. networknt json-schema-validator is the default (3.x for Jackson 3 / Boot 4, 2.x for Jackson 2), compiled at startup, no remote $ref, format assertions on when a field depends on format.
  • One repair with {path, keyword} feedback and no values, then fail closed with a problem+json code or a sealed result type. No guessing fixers, no relaxing the schema on retry.
  • Model output is never evaluated. Tool calls need an allowlisted name, server-side validation, and end-user authorization; tenant and user never come from arguments.
  • Counters by route, schema version, mode, outcome, and keyword. No payloads or values in logs, labels, or spans.
  • Offline conformance fixtures on every PR plus a first-attempt conformance rate per schema version in the existing eval gate.

Cross-links to the prompt-injection, routing, PII redaction, eval gate, and cost/latency entries rather than repeating them.

Only skillbook/skills.json and promptbook/prompts.json change. Catalogs are 55 skills and 55 prompts. Local site check and script tests passed.

@rmkr-dev
rmkr-dev merged commit b433dfa into main Nov 10, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant