Modelling - #4
Merged
Merged
Conversation
…r, NVFlare federation, KT report Person-level knowledge graph built from the published 1000 Genomes HaploGraph (chr22), real ancestry/population/sex labels and per-site proteomics joined on the sample id; PyTorch Geometric hetero-GNN encoder with SVD/Node2Vec/raw starts and negative controls; synthetic proteome with saved ground truth; GraphRAG decoder on NVIDIA NIM with citation validation; NVIDIA FLARE FedAvg over three sites plus site-alone comparison; Docker image built from the repo root, Brev deploy script, config.py/.env.example, unit tests, implementation guide, RESULTS.md and the knowledge-transfer report (PDF/DOCX/LaTeX). Verified by a fresh-clone run on CPU and a rebuilt-image run on an A100.
…ata), Mermaid diagrams in RESULTS.md, neutral team note
…ges, transfer) and a worked decoder example
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical and moderate issues affect compatibility, inference correctness, data locality, and pipeline behavior.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds an end-to-end genomics/proteomics knowledge-graph pipeline with GNN modelling, federated training, inference, deployment tooling, and documentation.
Changes:
- Adds v1/v2 graph construction, multimodal GNN training, embeddings, saliency, and baselines.
- Adds proteomics generation, GraphRAG, Neo4j, EDA, and federated workflows.
- Adds tests, setup scripts, Docker configuration, and technical documentation.
File summaries
| File | Description |
|---|---|
genomics/train_gnn.py |
v1 GNN training |
genomics/train_gnn_v2.py |
Multimodal GNN training |
genomics/tests/test_haplokg.py |
Knowledge-graph tests |
genomics/tests/test_haplokg_proteins.py |
Protein-layer tests |
genomics/tests/conftest.py |
Test configuration |
genomics/setup.sh |
Environment setup |
genomics/run_v2.sh |
v2 pipeline orchestration |
genomics/run_all.sh |
v1 pipeline orchestration |
genomics/requirements.txt |
Dependency pins |
genomics/proteomics_synth_1000g.py |
Synthetic proteomics generation |
genomics/proteome_linear_baseline.py |
Ridge baseline |
genomics/neo4j_load.py |
Neo4j graph loading |
genomics/Makefile |
Pipeline entry points |
genomics/load_env.sh |
Shell environment loading |
genomics/infer.py |
GNN inference and benchmarking |
genomics/haplokg.py |
Core graph construction |
genomics/haplokg_proteins.py |
Protein graph integration |
genomics/graphrag_decoder.py |
GraphRAG retrieval and decoding |
genomics/graph_explore.py |
Graph analysis and visualization |
genomics/fetch_data.sh |
Input data downloads |
genomics/federated/model.py |
Federated model definition |
genomics/federated/local_only.py |
Local-only federated comparison |
genomics/federated/job.py |
Federated orchestration |
genomics/federated/evaluate_global.py |
Federated model evaluation |
genomics/federated/client.py |
Federated client training |
genomics/embeddings.py |
Embedding evaluation |
genomics/eda.py |
Exploratory analysis reports |
genomics/docs/METHODS.md |
Methods documentation |
genomics/docs/DEEP_DIVE.md |
Detailed technical documentation |
genomics/docs/architecture.mmd |
Architecture diagram |
genomics/Dockerfile.dockerignore |
Docker build exclusions |
genomics/Dockerfile |
Container image definition |
genomics/docker-compose.yml |
Neo4j and pipeline services |
genomics/cooccurrence_analysis.py |
Phenotype association analysis |
genomics/config.py |
Runtime configuration |
genomics/build_kg.py |
v1 graph builder |
genomics/build_kg_v2.py |
v2 graph builder |
genomics/brev_deploy.sh |
GPU deployment automation |
genomics/baseline.py |
Logistic-regression baseline |
genomics/.gitignore |
Generated-file exclusions |
genomics/.env.example |
Configuration template |
Review details
Files not reviewed (1)
- genomics/docs/report/build_report.js: Generated file
Suppressed comments (7)
genomics/haplokg_proteins.py:73
- Rows with a missing
log2_intensityare converted toz=0byharmonise(), then retained inmeasured, where they setobserved=1and become MEASURED edges. This makes a missing detection-limit value look like a real median measurement, contrary to the documented “missing = no edge” behavior; drop missing intensities before constructing the measured/abundance tables.
genomics/infer.py:40 - This cache selection is unsafe for
--init node2vec: training falls back to SVD whenpyg-libis unavailable and only writes the SVD cache, but an oldernode2vec{k}_*.npypair can still exist and will be loaded here. In that case inference feeds node2vec embeddings into weights trained with SVD, producing incorrect predictions without an error; persist the actual initializer/seed in the run report or validate the cache against the run before loading it.
genomics/proteomics_synth_1000g.py:43 - The v2 CLI accepts arbitrary
--chromvalues, but its default protein BED is alwaysuniprot_chr22.bed. A non-chr22 v2 run consequently filters to zero proteins and later fails on empty arrays (baseline.max()), while the other v2 entry points also hard-code chr22 protein inputs. Either provide chromosome-matched defaults or reject unsupported chromosomes with a clear parser error.
genomics/requirements.txt:14 - The non-dry decoder imports
requests, but it is not a direct dependency of this environment. A clean setup therefore relies on an unrelated transitive dependency (and can fail when the API-key path is exercised); declare the HTTP client in the pinned requirements.
genomics/setup.sh:13 PYTHONis documented as the way to choose the interpreter, but theuvbranch always creates the environment with Python 3.13 and never usesPY. On a host with uv,PYTHON=python3.12 bash setup.shsilently ignores the requested version; pass the requested interpreter touv venvwhile retaining the 3.13 default.
genomics/train_gnn_v2.py:272ind2.loc[te, "individual_id"]is a Series retaining the original global row indices, whiletrueandpredare new zero-based lists. Pandas aligns dictionary values by index here, so the CSV can contain extra rows and mismatched/NaN IDs; downstream GraphRAG can then look up incorrect phenotype predictions. Convert the IDs to an array, as the v1 exporter does.
genomics/train_gnn_v2.py:282- The saliency score averages signed gradients before ranking, but the generator samples phenotype effects with either sign. A causal cluster with a negative effect therefore contributes a negative score and is systematically omitted from the descending top-k, biasing the reported causal precision. Average the per-case absolute gradients (and export that magnitude) instead.
- Files reviewed: 46/86 changed files
- Comments generated: 8
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+102
to
+103
| print(f"{name:32s} dim={emb.shape[1]:<4d} " + " ".join(f"{k.split('_')[-1]}: sil={result[f'silhouette_{k.split('_')[-1]}']:.3f} " | ||
| f"knn={result[f'knn5_test_balanced_accuracy_{k.split('_')[-1]}']:.3f}" for k in ("silhouette_ancestry", "silhouette_population", "silhouette_sex"))) |
Comment on lines
+50
to
+52
| kg = haplokg.load_kg(args.kg_dir) | ||
| pt = hp.load_protein_tables(args.kg_dir) | ||
| data = torch.load(Path(args.kg_dir) / "hetero_v2.pt", weights_only=False) |
Comment on lines
+66
to
+67
| train_args = (f"--genomics-dir {GENOMICS} --kg-dir {kg_dir} --split-path {split_path} --model-config {cfg_path} " | ||
| f"--local-epochs {args.local_epochs} --seed {args.seed}") |
Comment on lines
+76
to
+77
| data_dir: Path = field(default_factory=lambda: Path(_env("PROGENOME_DATA_DIR", str(GENOMICS / "data")))) | ||
| outputs_dir: Path = field(default_factory=lambda: Path(_env("PROGENOME_OUTPUTS_DIR", str(GENOMICS / "outputs")))) |
Comment on lines
+123
to
+125
| codes = ind.loc[labelled, f"{target}_code"].to_numpy() | ||
| classes = kg["label_maps"][target] | ||
| res = association(carries, codes, len(classes)) |
Comment on lines
+191
to
+193
| check = validate_citations(answer, context) | ||
| result = {"individual": args.individual, "gnn_run": run_dir.name, "llm": meta, "citation_check": check, "answer": answer} | ||
| (out_dir / f"{args.individual}_insight.json").write_text(json.dumps(result, indent=2)) |
| if cl_path.exists() and ind_path.exists(): | ||
| cl_init, ind_init = np.load(cl_path), np.load(ind_path) | ||
| else: | ||
| ind_init, cl_init, _ = svd_embeddings(kg["carries"], k, 42) |
| model.load_state_dict(torch.load(run_dir / "model.pt", map_location=device)) | ||
| model.eval() | ||
| wrapped = Wrapped(model, edge_index_dict, edge_weight).eval() | ||
| inputs = (x_dict["individual"], x_dict["cluster"], x_dict["block"]) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.