Adsk Contrib - Add color interop ID functions

[doug-walker]

This PR implements the functions described in the ASWF Color Interop Forum Recommendation An ID for Color Interop.

These functions allow an application to perform the two essential tasks when working with interop IDs:

• Searching a config for the color space that implements an ID
• Getting an ID to write to a file format to reference a color space in a config

The functions are:

• Config::findColorSpaceForID(idString)
• Config::generateLocalIDForColorSpace(srcColorSpaceName)
• Config::LocateBuiltinColorSpace(srcConfig, srcColorSpaceName, builtinConfig)

The last function utilizes the color space fingerprinting technique developed for the config merging feature. This PR adds a cache for the fingerprint data in the Config object, similar to the existing Processor caches.

Implements #2170.

Assisted by: Claude Code / Sonnet 5

[zachlewis]

This is great, Doug! I've got a couple of notes / comments / questions inline, and some ideas for a couple of followup PRs I'd like to discuss below -- but I perceive no blockers! This looks pretty good to me for 2.6.0.

That said, there are a couple of outstanding items that I'd like to address over the course of the 2.6.x release cycle (nothing ABI-breaking), in order to get things to a place where OpenImageIO can use what OpenColorIO provides, out of the box.

Better support for context-sensitive color spaces
The fingerprint caching mechanism doesn't handle contexts very gracefully. In particular, "context-sensitive" color spaces are in danger of drifting from their cached fingerprint and eliciting incorrect and explicitly unintended behavior if the live context is edited (e.g. Config.getCurrentContext().setStringVar(...)) so that the definition of the context-sensitive color space changes. On the other hand, using Config.setContext invokes resetCacheIds, blasting all cached fingerprints for all color spaces (for that config instance).

(Personally, given the choice, I think I'd rather incur the cost of recomputing fingerprints on a per-context basis than risk falsely identifying color spaces specifically engineered to take on different identities under specific conditions; but it's not great that we have two similar methods for editing the context that elicit completely different behaviors.)

In a perfect world, context-invariant color spaces are fingerprinted only per config, and context-sensitive color spaces are fingerprinted once per context; and maybe resetting the cache ids would only reset context-sensitive fingerprint caches, and leave invariant color space fingerprint caches alone, provided the config's cacheID hasn't changed. Anyway, I've implemented such a caching mechanism for OIIO that I'd far rather have live here, if you'll have it, along with some utilities for quickly interrogating and classifying color spaces and the transforms that define them, in order to ascertain a color space's context sensitivity, structural complexity, likelihood of representing a "simple" primaries+transfer-function RGB encoding versus... something else, etc. And this will serve both to inform which color spaces incorporate the context in their cached fingerprint keys, and which do not; AND it will double as a means for improving IdentifyBuiltinColorSpace's coverage of viable candidates (which, I feel, should not be rejecting color spaces on the grounds that they're inactive, or use color space transforms in their definitions... there's some low hanging fruit there for sure!)

Goldilocks Zone for viable candidates
But -- context stuff aside -- the real blocker for using this as-is out of the box for OIIO right now is the same thing that blocks us from backporting and using OCIO-2.5's config merging utilities: eagerly fingerprinting many color spaces at once can drastically balloon the time it takes for oiiotool to process its first frame. As Larry put it to me, there are a lot of folks who, for whatever reason, process frame sequences by invoking oiiotool once per frame, in serial -- so if the latest and greatest oiiotool were to take a second longer to process each frame for whatever reason, that version of oiiotool will have caused a 300-frame sequence to take 5 minutes longer to render.

Point being, in my testing, I found with merging, and I am finding with LocateBuiltinColorSpaces, that eagerly fingerprinting as eagerly and comprehensively as we are incurs too much of a startup penalty for larger configs, and configs that rely heavily on LUT3Ds.

Thanks to tip from @brechtvl, I've been using this https://github.com/Joegenco/PixelManager as a kind of representative worst-case-scenario for my benchmarks, because it's kind of a perfect storm of features and qualities and tendencies that IdentifyBuiltinColorSpace and the config merging mechanisms both struggle with -- lots of inactive color spaces, nearly every color space uses a color space transform in its definition, no display color spaces or view transforms, tons of color spaces employing LUT3Ds, a couple of incorrectly defined or misleadingly named color spaces, and a handful of color spaces that are structurally similar to common color encodings...

All in all, it takes ~3.5 seconds for LocateBuiltinColorSpace to realize there's no ocio:adx16_apd_scene color space in that PixelManager config; It takes IdentifyBuiltinColorSpace only 1.6 ms to arrive at the same conclusion. On the other hand, LocateBuiltinColorSpaces did manage to identify the ~21 correctly-defined simple RGB color spaces in the config; IdentifyBulitinColorSpace identified only two color spaces (aces_interchange and cie_xyz_d65_interchange).

Bottom line is, for OIIO, we need something in between what Locate and Identify currently offer:

• IdentifyBuiltinColorSpace's selection criteria is too conservative, and fails to even check many legitimate spaces (it would otherwise be able to match)
• LocateBuiltinColorSpace is too eager, and its selection criteria isn't discerning enough (for using in the "reverse" direction, swapping the builtin and src configs).

The solution I've come up with for OIIO is to split the baby:

• Improve on IdentifyBuiltinColorSpace's heuristics for better selecting cheap + simple + viable candidate spaces, and filtering out expensive / complex / unique spaces
• At the same time, determine which color spaces are definitely context-invariant, which are definitely context-sensitive, and which are potentially context-sensitive (e.g., owing to context variables used in search paths)
• Sort the candidates so the most stable and prototypical color space (i.e., simple, context-invariant, lut-free color spaces) are fingerprinted and matched before the potentially or explicitly context-sensitive spaces are considered.
• Somewhat lazily fingerprint only the likely candidates automatically (e.g., when trying to find a local match for a known interop id's identity
• Ostensibly, the other spaces are only ever fingerprinted lazily, when specifically requested, or when performing an "exhaustive" resolve, covering the remaining color spaces. In practice, if we've exhausted the list of viable candidates trying to find a match for a known interop_id, we'd just use the builtin config's representation of the color space via a cross-config processor.

....anyway, that's a lot of words about stuff that's outside the scope of this PR; but I just wanted to give you a sense of where and how my OIIO jams align with this PR.

Lastly -- despite the word count, I'd like to make it crystal clear that I think this PR is in a good place for 2.6.0; and I don't realistically expect any followup proposals / PRs I may submit to make it to 2.6.0. But my hope is, anything color-related I write for OIIO today will get folded into / made redundant by whatever OCIO offers out-of-the-box.

Alright, I've exhausted my wordcount for the day.

P.S., in case this isn't clear: Locate is fine as designed; it's using it backwards that costs 3.5 s, and there's no forwards-designed tool for that job.

[zachlewis]

I wonder if these assumptions still hold up for the general purpose of locating color spaces, as opposed to determining which color spaces are effectively "is-unique" for the purposes of naively merging into a config. I'm finding that this is overly conservative for my test cases, causing needless misses.

I think the contents of the transforms are probably a better indicator of whether the color space is suitable for fingerprinting. I can follow up in other PR with what I feel is a more robust alternative.

(And, hey, at least you're never in danger of having to invert a LUT3D with color spaces that define both directions!)

[zachlewis]

Why does IdentifyBuiltinColorSpace only look at active color spaces, anyway? What is this really saving us? I could understand prioritizing active over inactive, but I'd far prefer to match to an inactive space than fail to match altogether.

[zachlewis]

Granted, conceptually, IdentifyBuiltinColorSpace and LocateBuiltinColorSpace are inverses; but they don't behave the same way, and the "inverse" behavior one gets when swapping the builtin and src configs arguments for either function is not the same as the "inverse" behavior elicited when swapping the function instead of the arguments.

That they behave as differently as they do is unintuitive to me. In fact, personally, in my head, "Identify" and "Locate" do exactly the opposite of what I'd imagine those functions would do ("Identify" ought to find the builtin color space that matches the given source config color space! "Locate" should locate a color space in the source config that matches the given builtin color space!)

In any case, I think it's helpful to state clearly and succinctly somewhere around here what the exact difference is:

Identify searches active spaces whose transforms pass the blocked filter; Locate fingerprints every eligible space of builtinConfig eagerly and caches it per Config, so the table is sized by whichever config is passed there.

Which is hopefully enough for folks to reason about the pros and cons of either approach.

[zachlewis]

Claude felt we should be aware that if we're still using pybind11 by the time we make it to cg-config-v99999999999999999999.0.0_... this line would cause the python bindings for GenerateLocalIDForColorSpace to raise an IndexError (as opposed to an OCIO Exception).. 🤷

[doug-walker]

Zach, Cuneyt, thank you, those were awesome and super helpful comments! I implemented the easy ones and plan to continue working on it tomorrow.

Zach, I like your detailed comment above and am excited to collaborate on this with you! And I agree that the optimization aspects could wait for 2.6.1. If you think we need anything else in the public API that would help, I'm open to explore adding some small things.

[zachlewis]

Thanks, Doug!

If you think we need anything else in the public API that would help, I'm open to explore adding some small things.

I can't really think of much that we couldn't add in a non-ABI-breaking way later.

One thing that I'd find immediately useful is a way to filter (and maybe order?) the set of color spaces to fingerprint / compare against, and to more selectively control when and how the fingerprint cache is built up. It might not make too much sense from the perspective of using LocateBuiltinColorSpace as advertised, since the builtin configs are very quick to fully fingerprint, but for matching against custom configs (using Locate in "identify mode"), an optional "only" (or "but") ColorSpaceSet argument would be quite helpful.