[v1.0] Make the authoritative validation gate fail closed on security tools

[codeforester]

Goal

Ensure the manifest-declared authoritative validation command cannot report success after silently skipping required security gates.

Background

base_manifest.yaml declares ./tests/full_validate.sh as authoritative, and the setup instructions require the quality extra. Yet the script requires only python, ruff, and mypy; Bandit is conditional, Node validation is conditional, and pip-audit is never invoked:

base-cli/tests/full_validate.sh

Lines 3 to 34 in 1df8bc4

	# Authoritative local validation entry point for the Base manifest.
	set -euo pipefail
	required_commands=(python ruff mypy)
	for command in "${required_commands[@]}"; do
	command -v "$command" >/dev/null 2>&1 || {
	printf 'Missing validation tool: %s. Install the dev and quality extras first.\n' "$command" >&2
	exit 1
	}
	done
	./tests/validate.sh
	python -m pytest --cov=base_cli --cov-report=term-missing --cov-report=json:coverage.json --cov-fail-under=80
	python -m mypy --strict examples/typed_consumer.py
	python -m mypy --strict lib/python/base_cli
	ruff format --check lib/python/base_cli scripts examples tests
	ruff check lib/python/base_cli scripts examples tests
	python scripts/validate_docs.py
	python scripts/validate_changelog.py
	python scripts/validate_schemas.py
	python scripts/validate_contract_fixtures.py
	if command -v node >/dev/null 2>&1; then
	node scripts/validate_contract_fixtures.mjs
	fi
	python scripts/generate_compatibility_dashboard.py --check
	python scripts/benchmark_runtime.py --check
	python -m compileall -q examples
	python scripts/validate_coverage.py coverage.json
	if command -v bandit >/dev/null 2>&1; then
	bandit -q -r lib/python/base_cli scripts -lll -iii
	fi

The hosted quality job runs Bandit and pip-audit separately:

base-cli/.github/workflows/tests.yml

Lines 73 to 83 in 1df8bc4

	- name: Run static security checks
	run: |
	bandit -q -r lib/python/base_cli scripts -lll -iii
	# The project itself is installed from this checkout and may not be
	# published to PyPI yet (for example, while validating a release PR).
	# Audit every installed third-party package without asking pip-audit
	# to resolve the unpublished project distribution.
	python -m pip freeze \
	| grep -Eiv '^base-cli([[:space:]]|$)' \
	> "$RUNNER_TEMP/base-cli-audit-requirements.txt"
	pip-audit --strict -r "$RUNNER_TEMP/base-cli-audit-requirements.txt"

Therefore the documented local/Base gate can print Full base-cli validation passed. without static security analysis or dependency vulnerability auditing. The docs describe only "available security gates", which hides rather than attests a partial result:

base-cli/docs/testing.md

Lines 3 to 14 in 1df8bc4

	The Base manifest declares `./tests/full_validate.sh` as the authoritative
	test command. It runs the repository baseline checks, Python tests with the
	coverage policy, strict typing, formatting and lint checks, schema and
	contract validation, documentation checks, compatibility-dashboard and
	performance checks, and the available security gates.
	Run it from a clean checkout after installing the development and quality
	extras:
	```bash
	python -m pip install '.[dev,typer,quality]'
	./tests/full_validate.sh

The 2026-09-01 review ran Bandit, pip check, and pip-audit separately; they found no current high-severity static finding, broken dependency, or known third-party vulnerability. This ticket is about false-green orchestration.

Scope

• Define required versus optional validation capabilities explicitly.
• Require and run Bandit and pip-audit in the authoritative security profile.
• Either require Node for cross-language contract validation or emit a machine-readable partial/skip result that cannot be called full success.
• Keep local, Base manifest, and hosted security commands in parity.
• Avoid auditing the editable project as an unresolved PyPI requirement.

Acceptance criteria

• Missing Bandit or pip-audit makes the full/security gate fail with installation guidance.
• A deliberately vulnerable fixture/dependency and a Bandit fixture each make the gate fail.
• The gate audits every resolved third-party dependency while excluding only the editable checkout itself.
• Skipped optional checks are explicitly reported and produce a distinct non-authoritative result.
• CI calls the same versioned security orchestration rather than duplicating shell logic.

Validation

Exercise complete, missing-tool, offline-cache, unpublished-editable, and vulnerability-detected cases.

Project fields

• Status: Backlog
• Priority: P1
• Area: Security
• Initiative: v1.0 Readiness
• Size: S

Ownership

• Assignee: @codeforester