You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The ClinVar refresh job uses up to 5 GB of memory and may run past its time limit on large score sets. This issue makes it read only the ClinVar rows it needs, and look up each allele once instead of twelve times.
Problem
refresh_clinvar_controls (worker/jobs/external_services/clinvar.py) is heavy on memory and time. A 2026 ClinVar release takes about 3 GB in memory once parsed (lib/clinvar/constants.py), and one job peaks around 5 GB. The CAID-to-ClinVar allele ID lookup repeats for each of the 12 releases, so large score sets likely run past the 150-minute job limit.
Scope
Stream each release's TSV and keep only the rows for the job's alleles, instead of loading the whole release.
The ClinVar refresh job uses up to 5 GB of memory and may run past its time limit on large score sets. This issue makes it read only the ClinVar rows it needs, and look up each allele once instead of twelve times.
Problem
refresh_clinvar_controls(worker/jobs/external_services/clinvar.py) is heavy on memory and time. A 2026 ClinVar release takes about 3 GB in memory once parsed (lib/clinvar/constants.py), and one job peaks around 5 GB. The CAID-to-ClinVar allele ID lookup repeats for each of the 12 releases, so large score sets likely run past the 150-minute job limit.Scope
Acceptance criteria