Skip to content

feat: add fal.ai generative media tool to the assistive model - #3086

Open
azghr wants to merge 5 commits into
webbrain-one:mainfrom
azghr:feat/falai-generative-media
Open

azghr wants to merge 5 commits into
webbrain-one:mainfrom
azghr:feat/falai-generative-media

Conversation

@azghr

@azghr azghr commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a new generate_image agent tool powered by the fal.ai queue API, allowing the assistant to generate images directly instead of routing users to third-party image-generation websites.

What changed

  • Added generate_image tool

    • Implemented in src/{chrome,firefox}/src/agent/fal-media.js.
    • Uses the fal.ai queue API (queue.fal.run/{model}).
    • Polls the generation status every 2 seconds, with a 120-second timeout.
    • Returns the generated image URL to the model once complete.
  • Added fal.ai settings

    • Added Assistive Models → Generative media (fal.ai) settings.
    • Supports fal.ai API key configuration and model selection.
    • Added API key validation and a Test Connection button.
    • The connection test accepts 401, 403, 404, 405, and 422 responses as valid authentication responses.
    • Configuration is persisted in chrome.storage.local under imageGenModel.
  • Updated tool routing

    • Added generate_image to the Act system prompt and planner tool catalog.
    • Image-generation requests can now be handled directly by the tool instead of navigating to external image-generation websites.
  • Added access gating

    • generate_image is available only in the full tier.
    • It is excluded from Ask mode and the compact/mid tool sets.
    • Uses the existing network-spend capability gate in permission-gate.js.
  • Documentation

    • Added setup, supported models, and testing information to docs/providers-and-models.md.
  • Localization

    • Updated all interface locale catalogs for both Chrome and Firefox builds.

Testing

  • Ran node test/run.js.
  • fal.ai media tests pass.
  • Overall result: 2397 passed / 6 failed.
  • The remaining 6 failures are pre-existing on main:
    • 1 licensing-text test
    • 5 sidepanel source-scan expectations
  • These failures are unrelated to this feature.
  • Test Connection was verified against the live fal.ai API.
  • Full end-to-end image generation could not be completed because the fal.ai account used for testing has no remaining balance (403: Exhausted balance).
  • Image generation should work once the account has available credits.

@vercel

vercel Bot commented Sep 24, 2026

Copy link
Copy Markdown

@azghr is attempting to deploy a commit to the esokullu's projects Team on Vercel.

A member of the Team first needs to authorize it.

@esokullu

Copy link
Copy Markdown
Collaborator

Addressed all five review findings in 23e01f6d5:

  • Bound generate_image network permission to the real queue.fal.run egress host.
  • Classified fal.ai provider URLs and error text as untrusted in Chrome and Firefox.
  • Threaded the run abort signal through submit, polling, and result requests, added an abortable total deadline, and best-effort cancels queued jobs with fal.ai's cancel_url.
  • Replaced the 405-accepting key probe with fal.ai's read-only, required-auth workflows endpoint; only a 2xx response reports Connected.
  • Added imageGenModel to config export/import and encrypted profile sync, including merge timestamps and restore behavior in both builds.

Validation:

  • node test/run.js: 2,402 passed; the same 6 unrelated baseline failures remain (all new regression coverage passes).
  • npm run test:security: 60/60 passed.
  • npm run test:build-unpacked: 4/4 passed.
  • Syntax checks and git diff --check: passed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants