Skip to content

Announce replies, stops and approvals to screen readers - #393

Open
coolestnick wants to merge 1 commit into
useagenthq:mainfrom
coolestnick:announce-agent-replies
Open

coolestnick wants to merge 1 commit into
useagenthq:mainfrom
coolestnick:announce-agent-replies

Conversation

@coolestnick

@coolestnick coolestnick commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Closes #350

What this changes

Screen reader users were told when a turn is running (the running footer has role="status") and when one fails (the thread error banner has role="alert"), but nothing announced a finished reply, a user stop, or a newly waiting approval or question. So they had no way to know the agent was done or waiting on them.

This adds one visually hidden polite live region to the conversation that announces exactly those three events, once each, and nothing else.

Approach

  • components/chat/agent-announcement.ts is a pure function, agentAnnouncement(prev, next), that picks the single thing to announce (or nothing) from a before/after snapshot. It never sees streamed tokens and is unit tested without a DOM. A real failure returns null on purpose so the error banner stays the only failure announcement.
  • components/chat/agent-announcer.tsx is the live region (role="status", aria-live="polite", sr-only). It holds only a short sentence, never the answer text. A keyed <span> remounts per event so two identical sentences in a row (e.g. a second approval) are each read out.
  • components/chat/conversation-announcer.tsx maps the thread's state to that snapshot: a completed turn is a reply, a user cancel is a stop, a real failure stays silent. It keys the announcer on the thread root so switching threads resets it instead of replaying the new thread's last event.
  • components/ai/approval-card.tsx: both the gateway and native approval cards already send their error through the shared ApprovalCard, so a single role="alert" on that error line covers both lanes.

Why role="status" and not role="log"

The issue points at the role="log" region in message-scroller.tsx as a starting point. log is for append-only histories where only new additions are read; here each event is a single current message, so role="status" (implicitly polite and atomic) is the right primitive. The keyed-span remount handles re-announcing an identical message.

Done when

  • A finished reply, a failure, and a new approval or question are each announced once (reply / stop / waiting via the new region; failure via the existing banner).
  • Streaming text is not read token by token.
  • Unit tests cover the function that picks what to announce.

Tests

  • components/chat/agent-announcement.test.ts - the pure rule: every transition, reply vs stop, failure stays silent, priority, baseline on first snapshot.
  • components/chat/agent-announcer.interaction.test.tsx - the live region under happy-dom: polite status, announce once, no re-announce on resolve, keyed re-announce.
  • components/chat/conversation-announcer.interaction.test.tsx - the mapping end to end: completed turn to reply, user cancel to stop, real failure silent, new gateway approval announced.
  • components/ai/approval-card.test.tsx - asserts the error line is a role="alert".

Checks

  • bun run typecheck (repo root) - pass
  • cd frontend && bun run test - 1787 pass, 0 fail
  • cd frontend && bun run lint - clean
  • cd frontend && bun run build - pass
  • bun test scripts/agent-docs.test.ts - pass

How to see it

A /lab section ("Chat - Screen-reader announcements") drives the real announcer: fire each event and see what it speaks, or turn on VoiceOver / NVDA to hear it. Since the region is invisible, the screenshots below show that lab, and the recording shows it being read out.

announcer-dark announcer-light

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

The running footer (role="status") and the thread error banner
(role="alert") already tell screen reader users when a turn is running
or has failed, but nothing announced a finished reply, a user stop, or a
newly waiting approval or question. Closes useagenthq#350.

Add one visually hidden polite live region to the conversation that
speaks a short sentence for each of those events, once. What to say is a
pure function (agent-announcement.ts) picked from a before/after
snapshot, so it never streams tokens and is unit tested without a DOM.
The region (agent-announcer.tsx) re-reads an identical sentence via a
keyed span, and conversation-announcer.tsx maps thread state to it and
keys it on the thread root so switching threads resets it.

Real failures stay silent here so the error banner remains their sole
announcement. Both approval cards already route their error through the
shared ApprovalCard, so one role="alert" there covers both lanes.

A /lab section drives the announcer so it can be heard with VoiceOver or
NVDA.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tell screen reader users when the agent replies or needs them

1 participant