fix: bootstrap MP imports and document setup sequence - #1
Open
Irratzo wants to merge 1 commit into
Open
Conversation
Make the Materials Project import scripts work from a fresh checkout by updating the pinned MP stack, adding pymatgen compatibility shims, and creating the SQLite schema before import. Also document the required bootstrap/retrain steps and fix the demo results path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR makes the backend “Materials Project → SQLite → model training” bootstrap path reproducible from a fresh checkout by pinning a compatible MP dependency stack, adding a small import-compat shim, ensuring DB tables exist before inserts/queries, and documenting the required setup sequence.
Changes:
- Update backend dependency pins for the MP stack and add a
pymatgen/emmet-corecompatibility shim used before importingmp_api. - Ensure the SQLite schema is created before import scripts insert/query data, and add clearer failure output when MP client init/import fails.
- Document the bootstrap sequence in
README.mdandQUICKSTART.md, and write demo output relative torun_demo.py.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Documents the new required bootstrap sequence and renumbers setup steps. |
| QUICKSTART.md | Adds bootstrap instructions so a fresh checkout can be initialized correctly. |
| backend/scripts/fetch_mp_data.py | Adds DB initialization and more explicit MP client import/init checks before fetching. |
| backend/scripts/bulk_import_mp.py | Adds pymatgen compatibility shim usage and DB initialization before bulk import. |
| backend/scripts/bulk_fetch_mp.py | Moves MP imports behind the compatibility shim and initializes DB before bulk fetch/insert. |
| backend/run_demo.py | Writes demo results relative to the script location (not the current working directory). |
| backend/requirements.txt | Updates MP/pydantic pins and adds emmet-core pin to stabilize imports. |
| backend/app/utils/pymatgen_compat.py | Introduces module aliasing/patches to bridge pymatgen layout differences for emmet-core. |
| backend/app/database.py | Adds init_db() helper to create schema by importing models and calling create_all(). |
| backend/app/core/materials_project.py | Calls compatibility shim before mp_api import and records init errors for better diagnostics. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
1
to
+3
| import sys; sys.path.insert(0, "backend") | ||
| import asyncio, json, time | ||
| from pathlib import Path |
Comment on lines
+18
to
+30
| from app.utils.pymatgen_compat import ensure_pymatgen_compat | ||
| ensure_pymatgen_compat() | ||
| from mp_api.client import MPRester | ||
| from app.database import SessionLocal | ||
| from app.database import SessionLocal, init_db | ||
| from app.models.material import Material, Property | ||
| from sqlalchemy import insert | ||
|
|
||
| client = MPRester(api_key=api_key) | ||
| try: | ||
| client = MPRester(api_key=api_key) | ||
| except Exception as exc: | ||
| print("ERROR: Materials Project client initialization failed.") | ||
| print(f"Client error: {exc}") | ||
| return |
Comment on lines
+21
to
+32
| from app.utils.pymatgen_compat import ensure_pymatgen_compat | ||
| ensure_pymatgen_compat() | ||
| from mp_api.client import MPRester | ||
| from app.database import SessionLocal, init_db | ||
| from app.models.material import Material, Property | ||
|
|
||
| client = MPRester(api_key=api_key) | ||
| try: | ||
| client = MPRester(api_key=api_key) | ||
| except Exception as exc: | ||
| print("ERROR: Materials Project client initialization failed.") | ||
| print(f"Client error: {exc}") | ||
| return |
Comment on lines
+37
to
+43
| try: | ||
| ensure_pymatgen_compat() | ||
| from mp_api.client import MPRester | ||
| except ModuleNotFoundError as exc: | ||
| print("ERROR: Materials Project client could not be imported.") | ||
| print(f"Import error: {exc}") | ||
| print("") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR makes the Materials Project import path reproducible from a fresh checkout.
What changed:
Notes: