fix: make kl_divergence and chi_square audits run on DuckDB and Trino

[shinichitazawa]

Description

The built-in kl_divergence audit fails on every engine I tried, and chi_square fails on Trino. Found while testing statistical audits on Trino 483 + Iceberg (follow-up to the docs fix in #6128).

kl_divergence

Problem	Effect
table_a_with_q joins USING (@source_column) against table_b_with_q, which only has @target_column	Different columns: Column 'bytes' is missing from right side of join (Trino) / Column "bytes" does not exist on right side of join (DuckDB)
Both sides carry num_rows	Same column on both sides: AMBIGUOUS_NAME (Trino) / Ambiguous reference to column name "num_rows" (DuckDB)
Final SELECT ... GROUP BY @source_column, @target_column	The threshold is compared per value instead of against the divergence of the whole distribution
Single-argument LOG(x)	log10 on DuckDB; Trino has no single-argument log

The rewrite names the value column x in both distributions, joins them on it, computes SUM((p - q) * LN(p / q)) over the values present in both columns, and compares that single number with @threshold. (p - q) * ln(p / q) equals p ln(p/q) + q ln(q/p), i.e. the same terms the old query tried to add. Values present in only one column are skipped, as before (their terms were NULL and dropped by SUM).

Behaviour change: results now use the natural log (as in the linked definition of the symmetrised divergence). On DuckDB the old query, where it ran, used log10, so the same threshold now means something different.

chi_square

HAVING referenced the chi_square select alias. DuckDB accepts that, Trino does not: Column 'chi_square' cannot be resolved. The statistic is now computed in a statistic CTE and filtered with WHERE. The computation itself is unchanged.

Test Plan

Ran each audit (_non_blocking) on an INCREMENTAL_BY_TIME_RANGE model for one hourly interval (328,846 rows, two categorical columns with 27 distinct values each), with SQLMesh 0.236.3 and this builtin.py swapped in (the file is identical between v0.236.3 and main). The reference KL value was computed separately in Python with the natural log: 0.0373.

Case	Expected	DuckDB 1.5.6	Trino 483
Before the change, both audits	—	SQL errors above	SQL errors above
kl_divergence, same column on both sides	pass	pass	pass
kl_divergence, threshold := 0.0372	fail	1 row failed	1 row failed
kl_divergence, threshold := 0.0374	pass	pass	pass
chi_square, default dependent := true	pass (columns are dependent)	pass	pass
chi_square, dependent := false	fail	1 row failed	1 row failed

Unit tests: tests/core/test_audit.py updates the expected SQL for test_chi_square_audit and adds test_kl_divergence_audit. Result: 43 passed (42 before). One test, test_model_inline_audits, errors both before and after this change in my environment because the BigQuery extra is not installed.

Checklist

• I have run make style and fixed any issues: ran ruff check and ruff format --check on both changed files (clean). mypy was not run.
• I have added tests for my changes (if applicable)
• All existing tests pass (make fast-test): not run. Only tests/core/test_audit.py was run, as described above.
• My commits are signed off (git commit -s) per the DCO

🤖 Generated with Claude Code