Repository navigation
Challenge 20: Unbounded proofs for the char Searchers of str::pattern - #740
hung1962003 wants to merge 7 commits into
Conversation
Prove next, next_match, next_back, next_match_back, next_reject and
next_reject_back of CharSearcher, MultiCharEqSearcher and the four
wrappers (CharArraySearcher, CharArrayRefSearcher, CharSliceSearcher,
CharPredicateSearcher) for haystacks of symbolic length, without changing
any method body.
The loops of the provided Searcher methods and of
CharSearcher::next_match{,_back} get loop invariants and a precise
loop_modifies through a specialization hook (SearcherLoopContract), so
that only the cursor is havocked. MultiCharEqSearcher::next/next_back get
contracts, proved with proof_for_contract and used with stub_verified in
the loops. The slice and UTF-8 decoding functions are modeled as the
challenge allows, each model checked against upstream.
Requires Kani with model-checking/kani#4947 and model-checking/kani#5071.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- CharSearcher::next / next_back get step contracts, proved with proof_for_contract and used through stub_verified in the provided next_reject / next_reject_back loops, as MultiCharEqSearcher already does. check_char_searcher_next_reject_back now takes 33 s instead of 17 min, and 2 GB of memory instead of 6 GB. - The next_code_point_reverse model assumes the local UTF-8 rule only at the last byte, where the decode starts: that is all its safety needs. - The proof_for_contract harnesses no longer repeat the contract's ensures; kani_step_ok is shared by both searchers. - multi_char_eq_inv uses the wrapping_sub intrinsic (loop invariants call casts and intrinsics only). All 46 harnesses take 78 min instead of 125 min; the slowest takes 3.2 min instead of 17 min. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A replaced step (stub_verified) havocs a modifies target element by element, and each of those writes is checked against the write set of the enclosing loop. Naming char_indices as its 3 words instead of its 24 bytes cuts the peak memory of every MultiCharEq loop proof from about 7.3 GB to about 2 GB (and its time from about 150 s to about 40 s), so they fit the 7 GB macOS CI runner. The loop_modifies names the same bytes as before. All 46 harnesses: 47 min instead of 78 min; peak memory 4.4 GB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The slice equality model compares the at most 4 bytes of a char's encoding without recursion, and asserts that bound, so the two CharSearcher next_match harnesses no longer need kani::unwind. - kani_pred_state is empty by default: Kani checks every write against modifies and loop_modifies, so a pattern that writes state it does not name fails its proofs. Only the stateful test predicate Toggle names its state, in place of four specializations to empty. All 46 harnesses verify (46 min, peak memory 4.2 GB). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
check_from_ptr_contract reaches CharSearcher::next_match only through
panic formatting, where it never runs. Its new loop contract made Kani
apply loop contracts to this harness, and CBMC 6.11 then rejects the
contract-less loop of its built-in strlen ("Check that len is
assignable", fixed by diffblue/cbmc#9164). Stub next_match in this
harness so the loop contract is not compiled into it; in the
instrumented program next_match still has no body, so the harness
proves the same.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The step contracts check every write to the cursor against their write
set, and each check copies the set. Upstream moves the byte iterator
once per byte, and Iter::len goes through the contract of
NonNull::offset_from_unsigned. The decoder models now read the bytes
from the slice the iterator has left and move it once, and a stub of
Iter::len calls the raw pointer offset_from_unsigned, which keeps its UB
checks. This halves the memory of the step proofs.
check_iter_len_model_matches_upstream checks the new stub; the existing
check_next_code_point{,_reverse}_model_matches_upstream check the
decoders.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Share the char-boundary check, the memchr model, the allocation helper and the result check between the two searchers, and merge the decoder harnesses into check_decoder_models_match_upstream. The reverse decoder reads its bytes first and moves the iterator once, without the `?` operator (its ControlFlow temporary is tracked by the write set). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@feliperodri #740 is ready for review. It proves every method of the six char searchers for a symbolic haystack of any length up to 2^48 bytes. The shipped bodies only gain loop contract attributes; no statement changes, and all other verification code is additive. Locally all 46 harnesses verify and every cover is satisfied. The 4 red CI jobs fail only on the expected "Check that … is assignable" in the 24 loop-contract harnesses. They pass with model-checking/kani#4947 (approved) plus the small follow-up model-checking/kani#5071. Neither has run CI yet: their workflow runs have been waiting for approval since 2026-10-24 and 2026-10-27. Once they are merged, I will bump the Kani pin in this PR. |
There was a problem hiding this comment.
🟡 Changes recommended
The pinned Kani toolchain cannot run the affected proofs, and verification also leaves gaps for mutable predicates, array sizes, and larger slices.
4 open issues
What changed in this PR
This PR proposes a Kani solution to Challenge 20 (#277), adding symbolic-length safety proofs for character-based str::pattern searchers without changing their runtime method bodies.
Changes:
- Adds searcher invariants, step and loop contracts, and 46 verification harnesses with supporting models.
- Adds Kani-only iterator accessors for inspecting cursor state.
- Adds a targeted stub to the existing
CStr::from_ptrharness to avoid a CBMC loop-contract limitation.
| File | Description |
|---|---|
library/core/src/str/pattern.rs |
Adds contracts, models, and harnesses for character-searcher safety. |
library/core/src/slice/iter.rs |
Exposes iterator pointer fields through Kani-only helpers. |
library/core/src/ffi/c_str.rs |
Isolates an existing harness from the new searcher loop contracts. |
Review effort: Balanced
| kani::loop_modifies( | ||
| SearcherLoopContract::kani_fwd_cursor(&*self), | ||
| SearcherLoopContract::kani_pred_state(&*self) | ||
| ) |
| default fn kani_pred_state(&self) -> *const [u8] { | ||
| crate::ptr::slice_from_raw_parts(&raw const self.char_eq as *const u8, 0) |
| /// CBMC's object model: with `--object-bits 12` an object has 52 offset | ||
| /// bits. Same bound as the accepted VecDeque solution. It only excludes | ||
| /// allocations no real machine could back either. | ||
| const MAX_ALLOCATION_BYTES: usize = 1 << 48; |
| multi_char_eq_proofs!(char_array, CharArraySearcher, [char; 2], |set| set); | ||
| multi_char_eq_proofs!(char_array_ref, CharArrayRefSearcher, &[char; 2], |set| &set); |


This is a solution to Challenge 20. It proves the six searching methods (
next,next_match,next_back,next_match_back,next_reject,next_reject_back) ofCharSearcher,MultiCharEqSearcher,CharArraySearcher,CharArrayRefSearcher,CharSliceSearcherandCharPredicateSearcher. The proofs are unbounded: the haystack has symbolic length (up to 2^48 bytes) and arbitrary contents, character sets have symbolic size, and no loop over the haystack is unwound. No method body is changed: everything added iscfg(kani)attributes, contracts, hooks and harnesses.Depends on model-checking/kani#4947 and model-checking/kani#5071. Kani's explicit
loop_modifiescannot yet name the MIR temporaries in these loops (e.g. the destination ofself.next()). CI will fail until the Kani pin includes both PRs; I will bumptool_config/kani-version.tomlhere once they merge. Everything else runs with the pinned CBMC 6.11.0.One change outside
str::pattern:ffi::c_str::verify::check_from_ptr_contractreachesCharSearcher::next_matchonly through panic formatting, where it never runs. Its new loop contract made Kani apply loop contracts to that harness, and CBMC 6.11 then rejects the contract-less loop of its built-instrlen(Check that len is assignable, fixed by diffblue/cbmc#9164). The harness now stubsnext_match, so the loop contract is not compiled into it;next_matchstill has no body in its instrumented program, so it proves the same. The stub can go once Kani ships a CBMC with rust-lang#9164.Approach
Type invariants
C.CharSearcher:finger <= finger_back <= haystack.len(), both on char boundaries, andutf8_encoded[..utf8_size]is the UTF-8 encoding ofneedle.MultiCharEqSearcher<C>: theCharIndicescursor covershaystack[front_offset..back]with both ends on char boundaries (kani_ok).Loops. The provided methods
next_match,next_reject,next_match_backandnext_reject_back(trait defaults), plusCharSearcher::next_match{,_back}, get#[safety::loop_invariant]and a precise#[kani::loop_modifies]. The write set names only the cursor, through aSearcherLoopContracthook trait with specialization. Havocking the whole searcher would also havoc its haystack pointer, and CBMC then loses that pointer's value set (the issue in model-checking/kani#4993).Contracts. The steps
next/next_backofCharSearcherandMultiCharEqSearchergetrequires(C)/ensures(C && the step is in bounds and on char boundaries)and amodifiesthat names only the cursor (forCharSearcher,Chere is its fingers part: the safety of a step depends on nothing else). They are proved withproof_for_contractand used throughstub_verifiedinside the provided-method loops. So the loops never read through the havockedslice::Itercursor (a user-side answer to model-checking/kani#4993), and a loop proof does not verify its step again. ThemodifiesofMultiCharEqSearchernames its cursor as the three words it consists of: a replaced step havocs a target element by element, and three writes instead of 24 byte writes keep each loop proof near 2 GB of memory instead of 7 GB.Predicates. A predicate returns any result on every call. Two predicates are covered: one with no state of its own (
fn(char) -> bool) and one that writes its own state on every call (Toggle, like amoveclosure updating a capture). That state is in the contracts'modifiesand the loops'loop_modifies.Assumptions (as allowed by the challenge)
Slice module:
memchr/memrchrreturn any matching index orNone.<[char]>::containsreturns any result, so a character set is never read and may have any size.<slice::Iter<T>>::lenruns the upstream body on the raw pointeroffset_from_unsigned, which keeps its UB checks. It skips the contract of theNonNullwrapper, which costs a copy of the write set per call in the step contracts.check_memchr_models_cover_upstream,check_slice_eq_model_matches_upstreamandcheck_iter_len_model_matches_upstreamcheck the models against upstream.validations.rs:next_code_point{,_reverse}assume the local UTF-8 rule at the position where decoding starts, then run the upstream arithmetic on the bytes the iterator has left. They move the iterator once where upstream moves it per byte, because every move is checked against the step contract's write set. Where upstream reads a byte withunwrap_unchecked, the models assert that it is there.check_decoder_models_match_upstreamshows that the models only add assumptions.The haystack is valid UTF-8. UTF-8 validity is never stated as a quantified property. It is assumed locally, only where a position is consumed, and every valid haystack satisfies it.
Criteria
check_*_into_searcher: a searcher built byinto_searcherfrom any haystack satisfiesC.C. The step contracts'ensures(fornext/next_back) and the harnesses (for the other four methods) check that every returned index lies on a char boundary in the haystack, and that the searcher still satisfiesC.Results
46 harnesses, all
VERIFICATION:- SUCCESSFULwith Kani at the pin plus the two PRs above, and the pinned CBMC 6.11.0. Each harness takes between 0.5 s and 5 min (46 min for all 46, run one after another) and at most 4.3 GB of memory (the twoCharSearcher::next_match{,_back}proofs; every other one stays under 2.3 GB, the step contracts under 1.4 GB), and everykani::coveris satisfied (no vacuous proofs).Mutation testing
Each mutant changes one line so that a method returns, or moves its cursor to, an index that is out of bounds or not on a char boundary. With all ten built together, every harness of a mutated method fails; the unmutated code verifies.
CharSearcher::next:finger += 1check_char_searcher_next(ensures)CharSearcher::next_back:finger_back -= 1check_char_searcher_next_back(ensures)CharSearcher::next_matchreturns(found_char + 1, finger)check_char_searcher_next_match(char boundary)CharSearcher::next_match_back:finger_back = found_char + 1check_char_searcher_next_match_back(bounds, char boundary)MultiCharEqSearcher::nextreturnsReject(i, i + 1)char_array::check_next,char_predicate_state::check_next(ensures)MultiCharEqSearcher::next_backreturnsReject(i, i + 1)char_array::check_next_back(ensures)next_matchreturns(a, b + 1)char_array::check_next_match(bounds, char boundary)next_rejectreturns(a, b + 1)check_char_searcher_next_reject,char_array::check_next_reject(bounds, char boundary)next_match_backreturns(a, b + 1)char_array::check_next_match_back(bounds, char boundary)next_reject_backreturns(a, b + 1)check_char_searcher_next_reject_back,char_array::check_next_reject_back(bounds, char boundary)Resolves #277
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.
🤖 Generated with Claude Code