Skip to content

Enforce publication invariants with database constraints #893

Description

@bencap

Summary

Several things the code relies on about published and private records are true only because one code path writes them together. Once RLS hides private rows from the api, those assumptions have to hold in the database, because Python can no longer check rows it can't see. This issue adds CHECK constraints for them, after confirming existing data satisfies them.

Terms are defined in the glossary on #808.

Constraints

Table Constraint Why
scoresets, experiments, experiment_sets CHECK (private = (published_date IS NULL)) Search, statistics, published_variants_materialized_view and the public export filter on published_date; policies filter on private. The constraint makes them the same predicate
scoresets, experiments, experiment_sets CHECK (NOT private OR urn LIKE 'tmp:%') URN generation counts sibling rows it can see. That's safe only if no hidden row holds a published URN
score_calibrations CHECK (NOT ("primary" AND private)) A primary calibration is always public, so finding one never needs to see private rows

The publish handler in routers/score_sets.py already writes private = False and published_date together for the score set, experiment and experiment set, and nothing sets private back to true or clears published_date.

Steps

  1. Run these in production. Each must return nothing:

    SELECT 'scoresets', urn FROM scoresets WHERE private <> (published_date IS NULL) OR (private AND urn NOT LIKE 'tmp:%')
    UNION ALL SELECT 'experiments', urn FROM experiments WHERE private <> (published_date IS NULL) OR (private AND urn NOT LIKE 'tmp:%')
    UNION ALL SELECT 'experiment_sets', urn FROM experiment_sets WHERE private <> (published_date IS NULL) OR (private AND urn NOT LIKE 'tmp:%')
    UNION ALL SELECT 'score_calibrations', id::text FROM score_calibrations WHERE "primary" AND private;
  2. Fix any rows it returns with a data migration, or record why a row is an exception.

  3. Add the constraints in one migration.

Acceptance criteria

  • The query returns nothing in production before the migration deploys.
  • The migration adds all seven constraints.
  • A test asserts publishing a score set still succeeds.
  • A test asserts a private row with a non-tmp: URN is rejected.
  • A test asserts marking a private calibration primary is rejected by the database.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    app: backendTask implementation touches the backendapp: databaseTask implementation requires database changes

    Type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions