feat: register Claude Opus 5.5 in model registry - #28
Conversation
Register claude-opus-5-5 for anthropic, vertexai, and all six Bedrock region prefixes (bare/global/us/eu/au/jp), mirroring Opus 5's provider footprint. Pricing follows the registry's existing convention for Opus 5: each id is recorded at the same rate structure as that id's Opus 5 entry (base rates, or eu.'s 1.1x), taken from AWS's Bedrock price list "Standard, Global" tier for Claude Opus 5.5. Co-authored-by: Sam Boland <sam.boland@appfolio.com>
|
🤖 Supernova Code Review — View trace — via automation "Code Review Opus 5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com) |
There was a problem hiding this comment.
Reviewed the Opus 5.5 registry additions against the existing claude-opus-5 entries. Structurally the change is clean: 8 new entries (anthropic, bedrock base + us./eu./global./au./jp., vertexai) are faithful copies of their Opus 5 counterparts with only id/name/pricing/dates changed, insertion points are alphabetically correct, the eu. entry is a consistent 1.1x of base matching eu.anthropic.claude-opus-5, metadata.reasoning_options is in the shape Model::Info#reasoning_option_values expects, and the alias block mirrors claude-opus-5. No schema violations relative to spec/ruby_llm/models_json_validation_spec.rb.
Two inline notes:
lib/ruby_llm/models.json— thecache_read_input_per_millionof 0.2 (5% of input) deviates from the 10%-of-input convention every other Claude entry uses, and would under-report cache-read cost 2x downstream. Worth re-verifying before merge; repeated across all eight new entries.spec/ruby_llm/models_opus_5_5_spec.rb— nit on the third example, which tests libraryfindbehavior rather than the new registry rows.
Separately, and not inline-able: created_at/metadata.last_updated of 2026-09-22 are acknowledged placeholders, and metadata.knowledge was bumped to 2026-06 without a cited source. These are data-accuracy rather than code issues, but since they surface in Model::Info#created_at for consumers, it'd be good to correct them once the real release date is known.
— via automation "Code Review Opus 5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com)
| "input_per_million": 4, | ||
| "output_per_million": 20, | ||
| "cache_read_input_per_million": 0.2, | ||
| "cache_write_input_per_million": 5 |
There was a problem hiding this comment.
The cache-read rate here breaks the ratio every other Claude entry in this registry uses, which makes it the most likely part of this pricing block to be wrong. cache_read_input_per_million is 0.2 against an input of 4 — 5% of input — whereas claude-opus-5 (and the other Claude families) record cache read at 10% of input (0.5 / 5) and cache write at 1.25x input. cache_write here does follow the 1.25x convention (5 / 4), so only cache read deviates, which reads more like a transcription slip than a deliberate pricing change. Impact is concrete: Model::Info#cache_read_input_price_per_million feeds RubyLLM::Cost, so every downstream cost figure (including supernova's llm_models row) would under-report cache-read spend by 2x. Given the PR body already notes the release details could not be confirmed, please re-verify against Anthropic's published pricing before merge, and if 10% is correct use 0.4 (and cost.cache_read: 0.4) here plus 0.44 in the eu. entry. Same values are repeated across all eight new entries, so any correction needs applying to each.
| end | ||
|
|
||
| it 'resolves us.anthropic.claude-opus-5-5 when bedrock_region is configured' do | ||
| entry = RubyLLM::Model::Info.new( |
There was a problem hiding this comment.
nit: this third example doesn't exercise anything this PR adds — it builds its own Model::Info in memory and asserts Models#find region-prefix behavior, which is library logic already covered by spec/ruby_llm/models_spec.rb and duplicated verbatim from models_fable_5_1_spec.rb. Since the registry entry is never consulted, it would still pass if the Opus 5.5 rows were deleted. Consider dropping it (the first two examples already cover the registry additions) to avoid accruing one more copy of this block per model added.
Gemfiles have no committed lockfile, so bundler resolves the unconstrained transitive json dependency (from async's console and from faraday) to whatever is newest at install time. json 3.0.2 drops the quirks_mode: keyword that ActiveSupport's legacy JSON encoder still passes on Rails 7.1/7.2, which broke schema loading for every spec run through rails_helper.rb (e.g. spec/ruby_llm/active_record/*) on the Ruby 4.0 / rails-7.2 CI job. Pin in the base Gemfile and regenerate the Appraisal gemfiles so every Rails version installs a compatible json. Co-authored-by: Sam Boland <sam.boland@appfolio.com>
|
Review of 6fad720. Findings:
Rulings on the bot's other notes:
Verified and fine:
|
Queued — the spec will be rewritten to look up |
The us.anthropic.claude-opus-5-5 example built an in-memory Model::Info and stubbed RubyLLM.config wholesale, so it never exercised the actual registry entry or config object. Stub only bedrock_region on the real config and look the id up through RubyLLM.models, matching how the resolution path is used in practice.
samuelboland
left a comment
There was a problem hiding this comment.
Verified: all 8 Opus 5.5 entries match their Opus 5 counterparts except the allowed fields, pricing follows the registry convention (1.1x on eu. only), the region-lookup spec now reads the real registry, and CI is green on 412c91c.
What this does
Registers Claude Opus 5.5 (
claude-opus-5-5) inlib/ruby_llm/models.jsonandlib/ruby_llm/aliases.json. Currently this model doesn't exist anywhere in the registry, and code that falls back to a default (e.g.'none') for any model missing a registry entry would silently misbehave for a model like Opus 5.5 that cannot run with thinking off, with no error raised. The registry is also the source of pricing and context window data for downstream consumers. This is pure registry data — oneclaude-opus-5-5entry was added per existingclaude-opus-5entry (anthropic,vertexai, and Bedrock with all six region prefixes: bare,us.,eu.,au.,jp.,global.), each copied from its Opus-5 counterpart with onlyid,name, pricing,metadata.knowledge, andcreated_at/metadata.last_updatedchanged.family,context_window(1,000,000),max_output_tokens(128,000),capabilities,modalities, andmetadata.reasoning_optionseffort values are carried over unchanged from Opus 5. A matching alias entry was added mirroringclaude-opus-5's alias structure. This follows the precedent of PR #27 (Fable 5.1).Pricing intentionally follows the registry's existing convention: base rate (4/20/0.20/5 per million input/output/cache-read/cache-write tokens) applies to
anthropic,vertexai, and all Bedrock prefixes excepteu.(1.1×, 4.4/22/0.22/5.5), matching how existingclaude-opus-5entries are priced per-id. No markup was applied tous./au./jp., even though this differs from AWS's actual public pricing for those regions — that is a known, separate registry-wide pricing issue out of scope for this PR.A new spec,
spec/ruby_llm/models_opus_5_5_spec.rb, modeled on the Fable 5.1 spec, checks cost and effort values for several ids against the real loaded registry, checks alias resolution, and verifies Bedrock region resolution (us.anthropic.claude-opus-5-5) against the realRubyLLM.modelsregistry withbedrock_regionstubbed on the real config object.As a side fix surfaced during review,
Gemfileand all fourgemfiles/*.gemfilefiles now pingem 'json', '< 3.0'. This addresses a pre-existing, unrelated CI failure ontest / Test (4.0 / rails-7.2): the Appraisal gemfiles have no committed lockfile, so bundler was resolving an unconstrained transitivejsondependency tojson3.0.2, which dropped thequirks_mode:keyword that ActiveSupport's legacy JSON encoder still passes on Rails 7.1/7.2, breaking schema/fixture loading for AR-backed specs. This was reproduced identically onorigin/mainand fixed by pinning and regenerating the Appraisal gemfiles viabundle exec appraisal generate.Two things a reviewer should know:
us.anthropic.claude-opus-5-5) are unverified — no AWS CLI/credentials were available in this environment to confirm they exist yet on Bedrock. Unsuffixed/base ids were kept rather than guessing.created_at/metadata.last_updatedare set to2026-09-22as a placeholder since Opus 5.5's actual release date could not be confirmed.Type of change
Scope check
Required for new features
N/A — no prior GitHub issue was opened/approved for this change.
Quality check
Targeted specs (
spec/ruby_llm/models_opus_5_5_spec.rb,spec/ruby_llm/models_fable_5_1_spec.rb,spec/ruby_llm/models_spec.rb) were run repeatedly and passed (44/44, plus the 6 examples in the rewritten spec file).models_json_validation_spec.rbhad a pre-existing, unrelated failure confirmed to reproduce onorigin/main; after thejson < 3.0Gemfile pin, both the previously-failingactive_record/acts_as_action_text_spec.rband the newmodels_opus_5_5_spec.rbwere verified to pass underBUNDLE_GEMFILE=gemfiles/rails_7.2.gemfile.rubocopwas run on changed files. The diff was manually checked to confirm only additions occurred inmodels.json/aliases.json(no edits/deletions to existing entries). Note: this PR does manually editmodels.jsonandaliases.jsondirectly (not via an auto-generation script), so that box is not checked.overcommit --installand all hooks passbundle exec rake vcr:record[provider_name]bundle exec rspecmodels.json,aliases.json)AI-generated code
This PR was AI-authored.
API changes
Pure data addition to the model registry; no Ruby code changes for Opus 5.5 model-specific behavior.