fix: serve robots.txt with Content-Signal

[cblanc]

Summary

• /robots.txt returns a JSON 404 today. Add static/robots.txt (Docusaurus source) and the same file in the committed build/, which is what Express serves and the Dockerfile ships.
• The file grants search, AI input and AI training (Content-Signal: search=yes, ai-input=yes, ai-train=yes) inside the User-agent: * group, with Allow: / and the sitemap.
• build/ is copied by hand rather than re-running docs:build, which would re-hash every asset. Docusaurus copies static/ verbatim, so the next docs build produces the same file.

Test plan

• pnpm build && PORT=8881 node dist/server.js
• curl -si localhost:8881/robots.txt: 200 text/plain, signal line under User-agent: *
• /sitemap.xml and / unchanged
• After the image rolls out: /robots.txt on postcodes.io passes the isitagentready robots, AI-rules and content-signal checks

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[idealbot]

🎉 This issue has been resolved in version 20.0.3 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀