Repository navigation
system-prompt: the prompt goes after the agent's last system message - #863
wahaha1024 wants to merge 2 commits into
Conversation
An agent does not always use the field an API reserves for a system prompt. Codex-style agents on the Responses API send their prompt as `developer` messages in `input` and leave `instructions` empty; Chat clients may send several `system` messages. The middleware wrote the prompt to `instructions` when it was empty, and to `messages[0]` for Chat, so the agent's own prompt came after it and overrode it: the prompt was in the request and had no effect. The prompt now goes to the last `developer` or `system` message of the run an agent opens with, in Responses' `input` and in Chat's `messages`, which is the one the model reads as the final word. `instructions` is still used when the input carries no such message, and `prepend` still puts the prompt before that message. `anthropic` and `gemini` already wrote last and are unchanged. Cases in cases.json cover a prompt after input's last developer message, after the last of several system messages, prepend, the override-off pass-through, and the fallback to `instructions`.
|
Thanks. The problem is real: with omp on 1. No version bump. 2. Responses looks for the last developer/system item anywhere in
Please use the same rule as Chat: the last developer/system message of the run 3. For 4. Two smaller points:
Please also add cases for a mid-conversation developer message (after On idempotence: magpie's gateway runs What I ran at 6a5b7db (on macOS, Bun 1.4.2, temp HOME):
CI's |
…epend before it Review follow-ups on the first cut. Only the run a conversation opens with counts, in both APIs. `inputSystem` walked back from the end of `input`, so a developer message deep in the conversation (a Codex permission or sandbox update) or a trailing one (omp's `# Juice: 0 !important` scaffolding item) took the prompt. That put it behind the agent's own prompt, and moved it again on the next update, breaking the prompt cache from there on. Responses now uses the same rule as Chat: the last item of the leading developer/system run, stopping at the first item of another role. `prepend` goes before the whole leading run again: at the start of the first system/developer message on Chat, and into `instructions` on Responses, which comes before every input item. It had gone in before the last message of the run, i.e. in the middle of the agent's prompt. `override: false` no longer writes `instructions` when `input` carries a leading developer prompt. The agent's own developer messages would come after it and override it, which is what this change is about; README says so, and says that on omp, whose prompt is a leading developer message, `override: false` adds nothing. A Responses developer item whose content is neither a string nor an array (`null`, say) now falls back to `instructions` instead of dropping the prompt. Version 0.1.1: npm has 0.1.0, and publish.yml only publishes versions npm does not have yet. Cases: mid-conversation and trailing developer items left alone, prepend before two leading messages, prepend into `instructions`, null content falling back. `bun test packages/system-prompt`: 23 pass, 0 fail. `bun scripts/check.mjs`: ok.
|
All four points are in, pushed as 989f3f7. 1. Version. 2. Responses uses the same rule as Chat.
3. 4. The smaller points.
Cases added: mid-conversation developer, trailing developer, prepend before two leading messages, prepend into What I ran here (Linux, Bun 1.4.4):
Also verified against a live magpie gateway with omp as the client: with |
yetone
left a comment
yetone
left a comment
There was a problem hiding this comment.
Re-reviewed at 989f3f7. Points 1 (0.1.1; npm has 0.1.0), 2 (leadingRun for Responses too), the Chat half of 3, and the null-content fallback are in. One thing is still wrong.
Responses prepend ignores override: false. system-prompt.middleware.js:106-109 writes instructions for prepend before the !over check at :110. So with override: false and position: "prepend", a request whose own prompt is a leading developer message still gets yours:
responses, {override:false, position:"prepend"}, input [developer "OMP", user "hi"]
-> instructions: "MINE" (append with override:false: unchanged)
chat, {override:false, position:"prepend"}, [system "A", user "hi"]
-> unchanged
That contradicts the README this PR adds ("With override: false, a request that has one of those is left alone even when instructions is empty"), the Options text ("keeps it unchanged, unless override is true"), and Chat. Please do the !over && last >= 0 return before the prepend branch, and add a case for Responses prepend + override: false with a leading developer message (unchanged).
Smaller: the README's Responses row ("instructions, or the last developer or system message in input") doesn't say when each applies. A request with instructions set and a leading developer run in input (Codex) still gets the prompt appended to instructions, which is ahead of those developer messages. That's main's behaviour and not a blocker, but the row should say "when instructions is empty".
What I ran at 989f3f7 (a separate Mac, Bun 1.4.2, temp HOME):
bun test packages/system-prompt: 23 pass, 0 fail.bun scripts/check.mjs: ok.- Broke it by hand, one rule at a time:
leadingRunturned back into "last developer/system anywhere": 3 red (Chat late system message, Responses mid-conversation, Responses trailing Juice).- Chat prepend to
messages[last]: 1 red. - The Responses prepend branch removed: 1 red.
- A probe of the shapes above, plus string
input, typed{type:"message"}items (handled), anullinput item (falls back toinstructions), and Codexinstructions+ developer. - Not run: the repo-wide
bun test. CI shows no checks on this head.
What happens now
An agent does not always use the field an API reserves for a system prompt:
developermessages ininputand leaveinstructionsempty. The middleware wrote the prompt toinstructions, so it reached the model before the agent's own developer messages, which then overrode it.systemmessages. The middleware wrote tomessages[0], so every latersystemmessage came after the prompt and overrode it.Either way the prompt was in the request and had no effect. Observed with omp on
/v1/responses: a captured request has noinstructionskey at all and a 12,971-characterdevelopermessage atinput[0]; the injected prompt sat ininstructionsahead of it and the model ignored it.What this changes
responses: wheninstructionsis empty, the prompt goes after the lastdeveloperorsystemmessage ininput(string content joined, array content pushed as{type: "input_text"}).instructionsis still written when the input carries no such message.chat: the prompt goes to the lastsystemordevelopermessage of the run the agent opens with (the ones before the first message of another role), not tomessages[0].prependkeeps its meaning in both: the prompt goes before that message.anthropicandgeminialready wrote last and are unchanged.Tests
cases.jsongains six cases: a prompt after the last developer message ininput, after the last of several system messages,prependbefore it, theoverride: falsepass-through, the fallback toinstructions, and the Chat equivalents. Existing cases are unchanged.Verified against a live magpie gateway with omp as the client: after the change, asking the model what its developer message says about the thinking language returns the injected rule, where it previously described the agent's own prompt.