Problem
gnomAD links are shared across score sets, because alleles are. On RT (the allele-centric mapping work on feature/bencap/allele-centric-mapping-and-storage), gnomad_allele_links has a partial unique index allowing one live link per allele, and link_gnomad_variants_to_alleles (lib/gnomad.py) retires the old link and inserts the new one. Two gnomAD jobs on score sets with overlapping alleles can both try to insert the live link for the same allele. The second then fails on the unique index, and the job fails for work the first job already did.
Scope
- Make the retire-and-insert for each allele safe under concurrency: insert with
ON CONFLICT DO NOTHING on the live-link index, and treat an existing identical live link as done.
- Take the per-allele writes in a consistent order, sorted by allele id, so concurrent jobs lock rows in the same order.
Acceptance criteria
Problem
gnomAD links are shared across score sets, because alleles are. On RT (the allele-centric mapping work on
feature/bencap/allele-centric-mapping-and-storage),gnomad_allele_linkshas a partial unique index allowing one live link per allele, andlink_gnomad_variants_to_alleles(lib/gnomad.py) retires the old link and inserts the new one. Two gnomAD jobs on score sets with overlapping alleles can both try to insert the live link for the same allele. The second then fails on the unique index, and the job fails for work the first job already did.Scope
ON CONFLICT DO NOTHINGon the live-link index, and treat an existing identical live link as done.Acceptance criteria