[davidschachterADFA]

Summary

ADFA-6805. A 1.5B on-device model called search_project instead of read_file for "Show me app/build.gradle" and "What strings are defined in the app?" (ADFA-6800 saw it once; a three-run baseline on merged main saw it every time). Two wording changes, one commit each so the effect can be attributed:

• C1 — plugins/AI-Agent-Local/.../prompts/tools.yml: the open_file example is replaced by a read_file example ("Show or read a file (source, build.gradle, strings.xml; a bare name is fine)"). The local prompt had no read_file example at all, only open_file. Same example count and index, so no extra prompt tokens. A new LocalSystemPromptTest case pins the example.
• C2 — plugins/AI-Core/.../prompts/tool_descriptions.yml: read_build_output now reads "Read the log of the last build and whether it passed; to read build.gradle or any other file, use read_file" (was "Read the current build output and status", which a small model took for "show me build.gradle"). This file is shared by every backend, including the cloud ones' native tool definitions. The new text is accurate for them too; it is about 68 characters longer.

Measured on a Pixel 6 Pro (Android 17)

qwen2.5-coder-1.5b-instruct-q4_k_m, "Build Apps Anywhere" test project, a fresh chat per prompt, every write declined, three cold-start runs of the 20 ADFA-6800 prompts per build. The baseline is merged main at 07aab78; the two builds are on 18a9c97 (a Markdown-preview change in between).

Build	Correct first calls per run	"Show me app/build.gradle"	"What strings are defined in the app?"
Baseline	18, 18, 18 of 20	search_project ✗ ×3	search_project ✗ ×3
C1	20, 20, 20 of 20	read_file ×3	read_file ×2, search_project(query="strings.xml") ×1
C1 + C2	20, 20, 20 of 20	read_file ×3	read_file("…/strings.xml") ×3

• The other 18 prompts pick the same tool as the baseline in every run of both builds, including "Open TetrisGame.kt in the editor" (still open_file), "Show me the build output" and "Why did my last build fail?" (still read_build_output).
• Time to first output is unchanged within noise: cold first message 82.5 / 103.7 / 108.6 s (C1), 98.9 / 105.8 / 103.4 s (C1 + C2), 67.0 / 105.3 / 106.9 s (baseline); warm prompts 11–32 s.
• The scorer checks the first call's tool, not its arguments. A search_project call whose query names strings.xml counts as correct, which is how C1 reaches 20 in all three runs.

What this does not fix (found while checking)

I ran the two edit prompts through to the approval dialog, three runs each, on C1 + C2:

• "Change the app name in strings.xml to Block Stacker": in all three runs the model calls edit_file three times with the same guessed old_string (app_name = "Block Stacker", identical to new_string), gets the same rejection ("new_string is identical to old_string…") each time, and the run ends REPEATED after 3 turns. It never calls read_file. The guess is rejected before the approval dialog, so nothing is written, but the edit does not happen.
• "Add a file Util.kt with an empty class Util": the model calls create_file with content="", which is rejected as "Missing required argument(s): content"; on the next turn it answers "I can't assist with that request." and stops.

Neither is caused by this change; both are the same on the baseline's first calls, which the 20-prompt score counts as correct. They need a different fix (handler behaviour, not wording), so I have left them out of this PR.

Review

An independent read-only review of the diff found no correctness problems: nothing pins the old text, examples[1] and the example count are unchanged, open_file keeps its description and its place in the compact set, and the new test fails if the example is removed. The other backends' tools.yml files still carry open_file examples; they are separate prompts and are not changed here.

Not verified

• One phone, one model file; Q4_0 was not re-run. Three runs at temperature 0.15 agree closely, so they show the failures are deterministic, not that the model is reliable.
• The read_build_output wording was not tried against a cloud model on a device.
• The prompt's token count was not compared across builds (the log's prefill: figure depends on what is cached). C1 is the same size by design.

Test plan

• AI-Core and AI-Agent-Local testDebugUnitTest pass
• Same-phone, three-run A/B above
• Independent review
• addons check exits 0

🤖 Generated with Claude Code

https://claude.ai/code/session_01Vh2DAaJxvRXKRYkgiDqwGQ

[davidschachterADFA]

Independent review (code-review skill, high) at 596cb7c

An independent reviewer read the diff and returned 5 findings. I then checked each against the code. None blocks the merge and I made no change.

ID	Finding	Checked	Conclusion
F01	The new read_file example says a bare build.gradle or strings.xml is fine, but those names are often ambiguous	True: ReadFileHandler fails an ambiguous name with "Multiple files named 'X' — specify a path". rules.yml:20 already said bare names are accepted	The model gets a clear error to recover from. Change only if the phone shows it going wrong
F02	The only open_file example was replaced, so the prompt no longer shows how to open a file in the editor	True. open_file is still in COMPACT_BUILT_INS, the tool list and rules.yml:20	No change. Whether "show" now picks the wrong tool is a model question, not settled by code
F03	The read_build_output description now points at read_file in config shared by every backend	True: PromptToolCatalog and ChatViewModel both use it. read_file is always a built-in, so the hint never names a missing tool	About 12 more words. I did not check whether this tool ever shows in an approval dialog
F04	The new test only checks that the example string is present	True. A prompt's effect on the model has no code-level test	Covered by the phone runs, not by the test
F05	README says EXAMPLE_FILE_NAME is a name both read_file and open_file accept	Not a defect: the statement is still true	None

Not device-verified beyond the phone runs already described in the PR.

Posted by Claude Code for David Schachter.