Collections, In-Memory Indexing, Media Handling Modernization

[hparadiz]

Divergence v3.3.0 Change Log

Divergence v3.3.0 introduces indexed in-memory collections and integrates them with ActiveRecord hydration and persistence, alongside media-processing updates and native PHP dynamic-method errors.

Indexed Collections

• Added Divergence\Data\Collections\Collection, a standalone base for non-Divergence objects and model classes, usable without Divergence's ORM.
• Added Divergence\Models\Collections\RecordCollection which extends that base for Divergence ActiveRecord models.
• Collection operations cover adding, removing, bulk mutation, data retrieval, and database storage.
• Field indexes maintain:
  • distinct field cardinalities;
  • record-to-cardinality mappings;
  • cardinality-to-record mappings;
  • lazily rebuilt ordering data for range comparisons.
• Indexes update automatically when records are added, removed, replaced through array access, or explicitly re-indexed.
• Negative array offsets are supported, and iterator position is corrected when records before the current position are removed.
• getByField(), getAllByField(), getByCriteria(), and getAllByCriteria() provide indexed collection lookup.
• Criteria create missing field indexes just in time.
• Typed criteria expressions include:
  • equal and not equal;
  • greater than and greater than or equal;
  • less than and less than or equal;
  • LIKE and NOT LIKE, including % and _ wildcards;
  • IN and NOT IN;
  • null/not-null and exists/not-exists;
  • comparisons between two indexed fields.
• Criteria groups support nested AND, OR, NOT AND, and NOT OR operations, including single-criterion negation and defined empty-group behavior.
• Indexed date strings and Unix-epoch values are normalized as integers for indexing.
• Floating-point index values use a stable string representation.

Record Identity

• In order to make indexing extremely efficient in the PHP engine we normalize identities against a MAX_INT PHP constant which is a 64 bit integer.
• Added Data\KeyToHashInt for packing primary-key values into integer hash keys.
• Data\Collections\RecordKey as the single collection identity resolver.
• Divergence models use their trusted single-value primary key for collection identity.
• Phantom models and plain objects fall back to spl_object_id() until a model receives a primary key.
• Integer and numeric-string primary keys retain their numeric identity; non-numeric strings use xxh64.
• Duplicate object identity is rejected without allowing the ordered collection and hash index to diverge.
• Phantom records are re-keyed from object identity to primary-key identity after a successful save.

ORM Hydration Changes

• To make best use of the new Collections we implemented a way to enable this functionality on any given model with a new attribute #[InMemoryIndexing(indexes: [...])] for declaring collection indexes directly on a model
• This modifies the hydrator and insantiation layer in the ORM to instead hydrate Collections in the same hydration loop as the rest of the ORM making it most efficient.
• A new class Models\Collections\RecordCollection is used to map collections to their model.
• Record collections set their model class from the first record when no class is supplied and reject records of another class.
• Repeated ORM queries return fresh collections and fresh model instances instead of leaking a prior result set.
• Added record-collection variants of getAllByField() and getAllByCriteria() that preserve the collection class, configured indexes, and record class.
• Added collection-wide isDirty(), save(), and saveWithTransaction() behavior.
• Successful saves propagate generated primary keys through HashKeyIndex and all configured field indexes.
• Subsequent saves update field indexes when indexed model values change.
• Transactional saves snapshot model state so rollback can restore the same object instances, including phantom, primary-key, dirty, data, and mapped-property state through ActiveRecord::restoreState().
• Rollback clears stale post-save hash and field-index entries, then restores the original record order and identities.

ActiveRecord factory and controller creation

• Added a per-model Factory::get() instance registry and routed model getter access through it.
• Added an explicit factory/instantiator path for phantom records.
• ActiveRecord::create() now uses the model factory's phantom instantiator instead of invoking the model constructor directly.
• Record Create and MultiSave endpoints now use the model's static create() API for new records.
• Factory hydration now returns null for false or null single-record database results.
• Prototype initialization was renamed to EventBinder::initPrototype() and phantom state is now passed explicitly to bindRecord().
• Removed the separate PrototypeRegistry; instantiation now creates, initializes, and clones prototypes directly in the instantiator.
• Undefined model factory, static model getter, and request-handler endpoint calls now throw native-style Error objects with PHP-compatible undefined-method messages.
• Updated controller tests to assert Error for dynamic-call failures.

Database Behavior Changes

• MySQL automatic table creation now uses InnoDB instead of MyISAM, enabling transactional rollback for newly generated test and application tables.
• Missing-table recovery records whether a transaction was active before DDL runs.
• If table creation implicitly ends that transaction, missing-table recovery begins a replacement transaction before retrying the original query.
• Updated generated-SQL regression hashes for the InnoDB table definitions.

Media Handling Modernization

• Replaced hard-coded /tmp paths with sys_get_temp_dir() for audio previews, conversions, and remote-media downloads.
• Escaped filesystem paths passed to audio, image, PDF, Ghostscript, ImageMagick, FFmpeg, and FFprobe commands.
• Explicit conversion failures when Ghostscript or PDF conversion does not return image data.
• Partial media records created by Media::createFromFile() are now destroyed when analysis or file handling throws.
• Added base Media::isVariantAvailable() support using readable variant paths.
• Added concrete GdImage|false return behavior for audio, PDF, and video image extraction.
• Modernized video processing from avconv/exiftool to FFmpeg and FFprobe:
  • added a broader video MIME-type-to-extension map;
  • retained a subtype fallback for unknown video/* MIME types;
  • read width, height, duration, streams, and display-matrix rotation from FFprobe JSON;
  • used format duration when stream duration is unavailable;
  • extracted preview frames through FFmpeg;
  • updated H.264 encoding to libx264 with AAC audio;
  • updated WebM encoding to VP9 with Opus audio;
  • normalized 90-, 180-, and 270-degree source rotation during encoding;
  • safely quoted input, temporary-output, and final-output paths;
  • atomically moved completed temporary encodes to their final paths;
  • exposed nullable variant filesystem paths and boolean availability checks.

Code Quality

• Targeting 100% line test coverage along with a series of tests that handle rare edge cases that aren't covered by the line coverage.
• Renamed .php_cs.dist.php to the php-cs-fixer-supported .php-cs-fixer.dist.php filename.