Skip to content

Remove the jsonschema runtime dependency from opentelemetry-configuration - #5830

Closed
ocelotl wants to merge 8 commits into
mainfrom
issue_5392_generated_schema_validator
Closed

ocelotl wants to merge 8 commits into
mainfrom
issue_5392_generated_schema_validator

Conversation

@ocelotl

@ocelotl ocelotl commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Description

Addresses section 2 of #5392 (#5392), which asks whether opentelemetry-configuration should keep depending on jsonschema.

Validating a configuration file against schema.json with jsonschema pulls five packages into the resolved set: jsonschema itself plus attrs, referencing, jsonschema-specifications and rpds-py. rpds-py is a compiled extension, so an installed tree cannot stay pure Python, and jsonschema's own floors (attrs >= 22.2.0 among them) can force an upgrade of a package the instrumented application has already pinned.

This compiles the schema into a validator instead of interpreting it at runtime. fastjsonschema turns a JSON Schema into plain Python source, and running it at development time rather than at import time keeps fastjsonschema out of the runtime dependency set too. The committed _generated_validator.py imports decimal and the exception classes vendored beside it, and nothing else. uv.lock loses 307 lines and gains none.

Regenerate after any change to schema.json:

tox -e generate-config-validator

jsonschema stays a test dependency so tests/test_generated_validator.py can ask both validators to judge ten documents and fail when they disagree.

What changes for a user

The wording of a rejection. jsonschema reported the location as a suffix; fastjsonschema puts it in the message:

before: Configuration does not match schema: 'processors' is a required property (at tracer_provider)
after:  Configuration does not match schema: data.tracer_provider must contain ['processors'] properties

The raised type stays ConfigurationError and the Configuration does not match schema: prefix is unchanged. detailed_exceptions=False would normally degrade a missing required property to a bare data must contain naming nothing; the codegen script rewrites those 21 messages and asserts it rewrote every one it found.

Importing opentelemetry.configuration.file._loader no longer raises MissingDependencyError for jsonschema. It still does for yaml.

Gaps and concerns

  • fastjsonschema ignores a keyword it does not implement rather than refusing the schema, so a schema that started using prefixItems, unevaluatedProperties, unevaluatedItems, dependentRequired, dependentSchemas, maxContains or minContains would be under-enforced silently. test_schema_uses_no_keyword_fastjsonschema_ignores scans for that set; the current schema uses none of them.
  • A validator left unregenerated after a schema change fails nothing on behaviour alone, since the verdict comparison covers ten fixed documents. test_recompiling_the_schema_reproduces_the_committed_validator recompiles and compares bytes.
  • _validation_exceptions.py is a verbatim copy of the exceptions module of fastjsonschema 2.22.2, which is BSD-3-Clause. Those terms require a source redistribution to retain the notice, so the codegen script reads the LICENSE out of the installed distribution and writes it into the copy, and refuses to run if that distribution does not ship exactly one. The file therefore cannot carry the Apache header, and opens with the marker scripts/check_license_header.py already uses for generated output. It is excluded from pyright, because its two reported call-site errors cannot be fixed without diverging from the copy the drift test compares against.
  • The codegen script lives at opentelemetry-configuration/codegen/_generate_validator.py. The leading underscore is deliberate: scripts/public_symbols_checker.py treats an underscored path component as private, and this is unpackaged development tooling whose five module-level names are not public API.
  • schema.json is still shipped. The loader no longer reads it, but the codegen script does, and so do the docker tests under tests/opentelemetry-docker-tests/tests/declconfig/, which validate scenario files and keep their own jsonschema dependency.
  • tests/test_exceptions.py:44 still passes package="jsonschema" when constructing a MissingDependencyError directly. It is an example value in a test of that exception type, does not touch the loader, and still passes. Left alone to keep this change tight.
  • The generated module is 2935 lines, so it is excluded from pylint and ruff. It is excluded from neither pyright nor the test suite.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist:

  • I am a human and not a bot
  • I have run tox -e precommit
  • I have run tox -e typecheck
  • I have run the relevant tests and linting tox environments
  • This requires the following PR in contrib:

tox -e py312-test-opentelemetry-configuration gives 456 passed and 107 subtests passed, tox -e lint-opentelemetry-configuration rates 10.00/10, tox -e precommit passes all five hooks including uv-lock, and tox -e typecheck reports no new errors.

fastjsonschema turns a JSON Schema into plain Python source. Running it at
development time instead of at import time keeps both jsonschema and
fastjsonschema out of the runtime dependency set of
opentelemetry-configuration.

The script compiles with detailed_exceptions=False, which keeps the output at
roughly a fifth of the size of the detailed variant, and then repairs the one
message that choice degrades: a missing required property otherwise reports a
bare "data must contain" naming no property. Both the repair and the import
rewrite assert on what they matched, so a change in fastjsonschema's code
generation fails the script rather than producing quietly wrong output.

The generate-config-validator tox environment runs it. The typecheck
environment gains fastjsonschema so pyright can resolve the script's imports.
Both files are the output of 'tox -e generate-config-validator' and are not
edited by hand. _generated_validator.py imports decimal and the exception
classes next to it, and nothing else. _validation_exceptions.py is a copy of
the exceptions module of fastjsonschema 2.22.2, so that the raised types do
not come from a package the installed tree no longer contains.

Three linters are told to leave them alone. ruff would reformat both and the
drift check compares the committed bytes against what the generator produces.
pylint rates 2935 lines of generated code against rules written for
hand-written code. pyright reports two call-site errors in the vendored
exceptions module, which cannot be fixed without diverging from the upstream
copy the drift check compares against.
jsonschema stays a test dependency precisely so these can run. The generated
validator is only useful if it reaches the same verdicts as the reference
implementation, and the only way to know it does is to ask both and compare,
which the first test does over ten documents.

Two gaps are checked that behaviour tests cannot reach. fastjsonschema
ignores a keyword it does not implement rather than refusing the schema, so a
schema that started using one would be under-enforced silently; the schema is
scanned for that set. A validator left unregenerated after a schema change
also fails nothing, since the verdict comparison only covers ten fixed
documents; the schema is recompiled and the result compared against the
committed file.
_validate_schema calls the generated validator instead of jsonschema. The
import guard that raised MissingDependencyError for jsonschema goes away
along with it, as does the lazy read of schema.json: the schema is compiled
into the module rather than parsed at first use.

The wording of a rejection changes. jsonschema reported the location as a
suffix, "(at tracer_provider -> processors)", while fastjsonschema puts it in
the message itself, "data.tracer_provider must contain ['processors']
properties". The ConfigurationError type and the "Configuration does not
match schema: " prefix are unchanged.

The SchemaError branch is gone because there is no longer a schema to parse
at runtime. A malformed schema now fails in the generator instead.
Five packages leave the resolved set: jsonschema itself and attrs,
referencing, jsonschema-specifications and rpds-py behind it. rpds-py is a
compiled extension, so its removal is what lets an installed tree of this
package stay pure Python. jsonschema's own floors, attrs >= 22.2.0 among
them, could also force an upgrade of a package the instrumented application
had already pinned.
lint-license-header-check rejected two files. The codegen script is our own
code and simply lacked the Apache header.

_validation_exceptions.py cannot carry that header, because it is a verbatim
copy of a BSD-3-Clause module and is not ours to relicense. BSD-3-Clause
requires a redistribution in source form to retain the copyright notice, the
conditions and the disclaimer, so the generator now reads the LICENSE file of
the installed fastjsonschema distribution and writes it into the copy, and
fails if that distribution does not ship exactly one. The file opens with the
marker scripts/check_license_header.py already uses to recognise generated
output, which is now registered there.
public-symbols-check reported the script's five module-level names as public
API added by this branch. The script is development tooling under
opentelemetry-configuration/codegen/, which is not packaged, so none of them
are API at all.

scripts/public_symbols_checker.py already treats any path component with a
single leading underscore as private, so renaming the module says that once
and in the place that is true, rather than underscoring each of the five
names and then having to silence protected-access everywhere the tests reach
for them.
@ocelotl
ocelotl requested a review from a team as a code owner October 26, 2026 03:44
@opentelemetry-pr-dashboard

opentelemetry-pr-dashboard Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Pull request dashboard status

Closed · refreshed 2026-10-26 03:50 UTC

Status above doesn't look right?
  • Anything look wrong? Report it with what you expected; it helps us improve the dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant