Serialize complete TensorRT handle initialization across GPU threads

[zsqdx]

Summary

Serialize the complete TensorRT handle initialization across GPU server threads, rather than only cache access and buildSerializedNetwork().

Why

TensorRT 10.16.1 can fail nondeterministically while initializing heterogeneous GPUs, although each GPU initializes successfully alone. Reports include RTX 3090 + 4090 and RTX 4090 + 5090, with Cask/Myelin tactic failures and Invalid device kernel image.

#1225 serialized buildSerializedNetwork(), but builder, config, network, and parser objects were still created concurrently, and one GPU could begin building while the previous GPU was deserializing its engine and destroying its temporary builder objects.

The observed 5090 + 4090 log is particularly useful: the 5090 cold build completed before the 4090 warm build began, so the two buildSerializedNetwork() calls did not overlap. However, both builder/parser lifetimes did overlap.

This moves the process-wide lock to createComputeHandle() so it covers device selection, the complete builder/parser lifetime, engine building, deserialization, and execution-context initialization.

Impact

Only startup is serialized. Inference behavior, engine precision, tactic selection, cache format, and multi-GPU execution remain unchanged.

This is a conservative application-side workaround for what appears to be a TensorRT 10.16 mixed-architecture initialization issue.

Validation

• Windows Release build with TensorRT 10.16.1 and the timing-cache path (USE_CACHE_TENSORRT_PLAN=0)
• katago version
• katago runtests
• git diff --check

Heterogeneous-GPU validation is still requested, particularly:

• RTX 3090 + 4090 and RTX 4090 + 5090
• cold caches, warm caches, and one cold/one warm
• both device orders