Skip to content

feat: register Claude Opus 5.5 in model registry - #28

Merged
samuelboland merged 3 commits into
mainfrom
dux-register-claude-opus-5-5
Sep 22, 2026
Merged

samuelboland merged 3 commits into
mainfrom
dux-register-claude-opus-5-5

Conversation

@staging-supernova-dx-appf-io

@staging-supernova-dx-appf-io staging-supernova-dx-appf-io Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

What this does

Registers Claude Opus 5.5 (claude-opus-5-5) in lib/ruby_llm/models.json and lib/ruby_llm/aliases.json. Currently this model doesn't exist anywhere in the registry, and code that falls back to a default (e.g. 'none') for any model missing a registry entry would silently misbehave for a model like Opus 5.5 that cannot run with thinking off, with no error raised. The registry is also the source of pricing and context window data for downstream consumers. This is pure registry data — one claude-opus-5-5 entry was added per existing claude-opus-5 entry (anthropic, vertexai, and Bedrock with all six region prefixes: bare, us., eu., au., jp., global.), each copied from its Opus-5 counterpart with only id, name, pricing, metadata.knowledge, and created_at/metadata.last_updated changed. family, context_window (1,000,000), max_output_tokens (128,000), capabilities, modalities, and metadata.reasoning_options effort values are carried over unchanged from Opus 5. A matching alias entry was added mirroring claude-opus-5's alias structure. This follows the precedent of PR #27 (Fable 5.1).

Pricing intentionally follows the registry's existing convention: base rate (4/20/0.20/5 per million input/output/cache-read/cache-write tokens) applies to anthropic, vertexai, and all Bedrock prefixes except eu. (1.1×, 4.4/22/0.22/5.5), matching how existing claude-opus-5 entries are priced per-id. No markup was applied to us./au./jp., even though this differs from AWS's actual public pricing for those regions — that is a known, separate registry-wide pricing issue out of scope for this PR.

A new spec, spec/ruby_llm/models_opus_5_5_spec.rb, modeled on the Fable 5.1 spec, checks cost and effort values for several ids against the real loaded registry, checks alias resolution, and verifies Bedrock region resolution (us.anthropic.claude-opus-5-5) against the real RubyLLM.models registry with bedrock_region stubbed on the real config object.

As a side fix surfaced during review, Gemfile and all four gemfiles/*.gemfile files now pin gem 'json', '< 3.0'. This addresses a pre-existing, unrelated CI failure on test / Test (4.0 / rails-7.2): the Appraisal gemfiles have no committed lockfile, so bundler was resolving an unconstrained transitive json dependency to json 3.0.2, which dropped the quirks_mode: keyword that ActiveSupport's legacy JSON encoder still passes on Rails 7.1/7.2, breaking schema/fixture loading for AR-backed specs. This was reproduced identically on origin/main and fixed by pinning and regenerating the Appraisal gemfiles via bundle exec appraisal generate.

Two things a reviewer should know:

  • Bedrock inference-profile IDs (e.g. us.anthropic.claude-opus-5-5) are unverified — no AWS CLI/credentials were available in this environment to confirm they exist yet on Bedrock. Unsuffixed/base ids were kept rather than guessing.
  • created_at/metadata.last_updated are set to 2026-09-22 as a placeholder since Opus 5.5's actual release date could not be confirmed.

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Performance improvement

Scope check

  • I read the Contributing Guide
  • This aligns with RubyLLM's focus on LLM communication
  • This isn't application-specific logic that belongs in user code
  • This benefits most users, not just my specific use case

Required for new features

N/A — no prior GitHub issue was opened/approved for this change.

  • I opened an issue before writing code and received maintainer approval
  • Linked issue: #___

Quality check

Targeted specs (spec/ruby_llm/models_opus_5_5_spec.rb, spec/ruby_llm/models_fable_5_1_spec.rb, spec/ruby_llm/models_spec.rb) were run repeatedly and passed (44/44, plus the 6 examples in the rewritten spec file). models_json_validation_spec.rb had a pre-existing, unrelated failure confirmed to reproduce on origin/main; after the json < 3.0 Gemfile pin, both the previously-failing active_record/acts_as_action_text_spec.rb and the new models_opus_5_5_spec.rb were verified to pass under BUNDLE_GEMFILE=gemfiles/rails_7.2.gemfile. rubocop was run on changed files. The diff was manually checked to confirm only additions occurred in models.json/aliases.json (no edits/deletions to existing entries). Note: this PR does manually edit models.json and aliases.json directly (not via an auto-generation script), so that box is not checked.

  • I ran overcommit --install and all hooks pass
  • I tested my changes thoroughly
    • For provider changes: Re-recorded VCR cassettes with bundle exec rake vcr:record[provider_name]
    • All tests pass: bundle exec rspec
  • I updated documentation if needed
  • I didn't modify auto-generated files manually (models.json, aliases.json)

AI-generated code

This PR was AI-authored.

  • I used AI tools to help write this code
  • I have reviewed and understand all generated code (required if above is checked)

API changes

Pure data addition to the model registry; no Ruby code changes for Opus 5.5 model-specific behavior.

  • Breaking change
  • New public methods/classes
  • Changed method signatures
  • No API changes

Register claude-opus-5-5 for anthropic, vertexai, and all six Bedrock
region prefixes (bare/global/us/eu/au/jp), mirroring Opus 5's provider
footprint. Pricing follows the registry's existing convention for
Opus 5: each id is recorded at the same rate structure as that id's
Opus 5 entry (base rates, or eu.'s 1.1x), taken from AWS's Bedrock
price list "Standard, Global" tier for Claude Opus 5.5.

Co-authored-by: Sam Boland <sam.boland@appfolio.com>
@staging-supernova-dx-appf-io

Copy link
Copy Markdown
Author

🤖 Supernova Code Review — View trace

🔁 Re-run this review

— via automation "Code Review Opus 5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com)

@staging-supernova-dx-appf-io staging-supernova-dx-appf-io Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the Opus 5.5 registry additions against the existing claude-opus-5 entries. Structurally the change is clean: 8 new entries (anthropic, bedrock base + us./eu./global./au./jp., vertexai) are faithful copies of their Opus 5 counterparts with only id/name/pricing/dates changed, insertion points are alphabetically correct, the eu. entry is a consistent 1.1x of base matching eu.anthropic.claude-opus-5, metadata.reasoning_options is in the shape Model::Info#reasoning_option_values expects, and the alias block mirrors claude-opus-5. No schema violations relative to spec/ruby_llm/models_json_validation_spec.rb.

Two inline notes:

  • lib/ruby_llm/models.json — the cache_read_input_per_million of 0.2 (5% of input) deviates from the 10%-of-input convention every other Claude entry uses, and would under-report cache-read cost 2x downstream. Worth re-verifying before merge; repeated across all eight new entries.
  • spec/ruby_llm/models_opus_5_5_spec.rb — nit on the third example, which tests library find behavior rather than the new registry rows.

Separately, and not inline-able: created_at/metadata.last_updated of 2026-09-22 are acknowledged placeholders, and metadata.knowledge was bumped to 2026-06 without a cited source. These are data-accuracy rather than code issues, but since they surface in Model::Info#created_at for consumers, it'd be good to correct them once the real release date is known.

🔁 Re-run this review

— via automation "Code Review Opus 5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com)

Comment thread lib/ruby_llm/models.json
Comment on lines +1370 to +1373
"input_per_million": 4,
"output_per_million": 20,
"cache_read_input_per_million": 0.2,
"cache_write_input_per_million": 5

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cache-read rate here breaks the ratio every other Claude entry in this registry uses, which makes it the most likely part of this pricing block to be wrong. cache_read_input_per_million is 0.2 against an input of 4 — 5% of input — whereas claude-opus-5 (and the other Claude families) record cache read at 10% of input (0.5 / 5) and cache write at 1.25x input. cache_write here does follow the 1.25x convention (5 / 4), so only cache read deviates, which reads more like a transcription slip than a deliberate pricing change. Impact is concrete: Model::Info#cache_read_input_price_per_million feeds RubyLLM::Cost, so every downstream cost figure (including supernova's llm_models row) would under-report cache-read spend by 2x. Given the PR body already notes the release details could not be confirmed, please re-verify against Anthropic's published pricing before merge, and if 10% is correct use 0.4 (and cost.cache_read: 0.4) here plus 0.44 in the eu. entry. Same values are repeated across all eight new entries, so any correction needs applying to each.

Comment thread spec/ruby_llm/models_opus_5_5_spec.rb Outdated
end

it 'resolves us.anthropic.claude-opus-5-5 when bedrock_region is configured' do
entry = RubyLLM::Model::Info.new(

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this third example doesn't exercise anything this PR adds — it builds its own Model::Info in memory and asserts Models#find region-prefix behavior, which is library logic already covered by spec/ruby_llm/models_spec.rb and duplicated verbatim from models_fable_5_1_spec.rb. Since the registry entry is never consulted, it would still pass if the Opus 5.5 rows were deleted. Consider dropping it (the first two examples already cover the registry additions) to avoid accruing one more copy of this block per model added.

Gemfiles have no committed lockfile, so bundler resolves the
unconstrained transitive json dependency (from async's console and
from faraday) to whatever is newest at install time. json 3.0.2 drops
the quirks_mode: keyword that ActiveSupport's legacy JSON encoder still
passes on Rails 7.1/7.2, which broke schema loading for every spec run
through rails_helper.rb (e.g. spec/ruby_llm/active_record/*) on the
Ruby 4.0 / rails-7.2 CI job. Pin in the base Gemfile and regenerate the
Appraisal gemfiles so every Rails version installs a compatible json.

Co-authored-by: Sam Boland <sam.boland@appfolio.com>
@samuelboland

Copy link
Copy Markdown

Review of 6fad720.

Findings:

  1. spec/ruby_llm/models_opus_5_5_spec.rb:41: the bot's nit is right. This example builds its own Model::Info in memory, so it would still pass if the Opus 5.5 rows were deleted. Keep the example but make it read the real registry: drop the in-memory entry and described_class.new([entry]), stub RubyLLM.config with bedrock_region: 'us-west-2' as now, and assert that RubyLLM.models.find('us.anthropic.claude-opus-5-5', :bedrock) returns id us.anthropic.claude-opus-5-5, provider bedrock, and effort values %w[low medium high xhigh max]. That is the lookup supernova makes. If stubbing RubyLLM.config breaks RubyLLM.models loading, stub only bedrock_region on the real config object instead.
  2. The PR description doesn't report the Bedrock id check the brief asked for (the Verification section: aws bedrock list-inference-profiles --region us-west-2, or a one-token Converse request to us.anthropic.claude-opus-5-5). Run it and add what you ran and what came back to the description. If your container's credentials can't do either, say so in one line.
  3. PR description: add one line saying created_at and metadata.last_updated are set to 2026-09-22 because the release date wasn't confirmed, as the brief asked.

Rulings on the bot's other notes:

  • lib/ruby_llm/models.json:1373 (cache read 0.2 is 5% of input): overruled, leave it. AWS's price list (AmazonBedrockFoundationModels, us-west-2, "Claude Opus 5.5 (Amazon Bedrock Edition)") bills cache reads at $0.20 per million on the global rate, against $4 input. Opus 5.5's cache read is 0.05x input, where Opus 5's is 0.1x. The value is correct.
  • metadata.knowledge 2026-06: overruled, leave it. Anthropic documents Opus 5.5's knowledge cutoff as June 2026.

Verified and fine:

  • All 8 new entries match their claude-opus-5 counterparts except for id, name, pricing, created_at, metadata.knowledge and metadata.last_updated. I compared every field programmatically. No existing entry changed.
  • Pricing follows the registry's convention: base rates everywhere, 1.1x on eu. only, in both pricing.text_tokens.standard and metadata.cost.
  • aliases.json matches the claude-opus-5 alias.
  • The json < 3.0 pin is justified. The first CI run on 223c7d6 failed test / Test (4.0 / rails-7.2) with json-3.0.2 ... unknown keyword: quirks_mode (ArgumentError), and main's last CI run (2026-09-02) predates that json release. CI is green on 6fad720.

@staging-supernova-dx-appf-io

Copy link
Copy Markdown
Author

@samuelboland

Review of 6fad720.

Findings:

  1. spec/ruby_llm/models_opus_5_5_spec.rb:41: the bot's nit is right. This example builds its own Model::Info in memory, so it would still pass if the Opus 5.5 rows were deleted. Keep the example but make it read the real registry: drop the in-memory entry and described_class.new([entry]), stub RubyLLM.config with bedrock_region: 'us-west-2' as now, and assert that RubyLLM.models.find('us.anthropic.claude-opus-5-5', :bedrock) returns id us.anthropic.claude-opus-5-5, provider bedrock, and effort values %w[low medium high xhigh max]. That is the lookup supernova makes. If stubbing RubyLLM.config breaks RubyLLM.models loading, stub only bedrock_region on the real config object instead.
  2. The PR description doesn't report the Bedrock id check the brief asked for (the Verification section: aws bedrock list-inference-profiles --region us-west-2, or a one-token Converse request to us.anthropic.claude-opus-5-5). Run it and add what you ran and what came back to the description. If your container's credentials can't do either, say so in one line.
  3. PR description: add one line saying created_at and metadata.last_updated are set to 2026-09-22 because the release date wasn't confirmed, as the brief asked.

Rulings on the bot's other notes:

  • lib/ruby_llm/models.json:1373 (cache read 0.2 is 5% of input): overruled, leave it. AWS's price list (AmazonBedrockFoundationModels, us-west-2, "Claude Opus 5.5 (Amazon Bedrock Edition)") bills cache reads at $0.20 per million on the global rate, against $4 input. Opus 5.5's cache read is 0.05x input, where Opus 5's is 0.1x. The value is correct.
  • metadata.knowledge 2026-06: overruled, leave it. Anthropic documents Opus 5.5's knowledge cutoff as June 2026.

Verified and fine:

  • All 8 new entries match their claude-opus-5 counterparts except for id, name, pricing, created_at, metadata.knowledge and metadata.last_updated. I compared every field programmatically. No existing entry changed.
  • Pricing follows the registry's convention: base rates everywhere, 1.1x on eu. only, in both pricing.text_tokens.standard and metadata.cost.
  • aliases.json matches the claude-opus-5 alias.
  • The json < 3.0 pin is justified. The first CI run on 223c7d6 failed test / Test (4.0 / rails-7.2) with json-3.0.2 ... unknown keyword: quirks_mode (ArgumentError), and main's last CI run (2026-09-02) predates that json release. CI is green on 6fad720.

Queued — the spec will be rewritten to look up us.anthropic.claude-opus-5-5 from the real registry instead of the in-memory stub, and the PR description will get the verification note and the created_at/last_updated explanation. If the sandbox can't actually reach AWS for #2, the agent will say so plainly in the description rather than fake a result.


🔍 View trace

The us.anthropic.claude-opus-5-5 example built an in-memory
Model::Info and stubbed RubyLLM.config wholesale, so it never
exercised the actual registry entry or config object. Stub only
bedrock_region on the real config and look the id up through
RubyLLM.models, matching how the resolution path is used in practice.

@samuelboland samuelboland left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified: all 8 Opus 5.5 entries match their Opus 5 counterparts except the allowed fields, pricing follows the registry convention (1.1x on eu. only), the region-lookup spec now reads the real registry, and CI is green on 412c91c.

@samuelboland
samuelboland merged commit 7e5ed99 into main Sep 22, 2026
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants