Skip to content

[Other] Tell the agent where a fact came from and how this computer reaches the internet - #4441

Open
xvcezrqpbxtpzblrt wants to merge 1 commit into
Niko1221:mainfrom
xvcezrqpbxtpzblrt:agent/evidence-and-network-027
Open

xvcezrqpbxtpzblrt wants to merge 1 commit into
Niko1221:mainfrom
xvcezrqpbxtpzblrt:agent/evidence-and-network-027

Conversation

@xvcezrqpbxtpzblrt

Copy link
Copy Markdown
Contributor

Problem

The agent had no standing rule about where a fact came from or about the proxy this computer uses, so a search result from a forum could be stated as fact, and a request that failed behind a proxy could be worked around instead of reported.

Result

Three sentences added to the app's system prompt:

  • say where an answer came from: the maker's documentation or source is the reference for what their product does; a blog, forum, video or issue thread is one person talking, so use it only as a hint and mark it unconfirmed; give the address actually opened, never one from memory, with the version or date when the answer depends on it;
  • this computer may reach the internet through a proxy or VPN; never switch a proxy off to make a download or search work; if a request fails because of it, name the proxy and let the person choose a direct connection;
  • write in the language the person writes in, and write a helper's task in that language too, so the helper answers in it.

Validation

node --test tests/functional-fixes.test.mjs → 7 pass.

The prompt is now 2,284 characters (about 600 tokens). The former budget of 1,600 was raised to 2,600 in the same test, with the reason written beside it; it is still a fifth of the Harness's own text.

Limits

  • Wording only: the model can still ignore it. The rule matters most for web search and API-backed models.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant