Repository navigation
feat: raise explicit missing-dependency errors when engines are const… - #106
Merged
Merged
Conversation
…ructed Engines import their optional libraries lazily, so a forgotten extra surfaced partway into a run as an error that named neither the engine nor the package to install. BaseEngine now declares REQUIRED_MODULES and INSTALL_EXTRA, and __init__ calls verify_dependencies() before doing any other work. The raised MissingDependenciesError subclasses ImportError, names every missing module at once rather than one per failed run, and carries the matching `pip install lakebench[...]` hint. Availability is resolved with importlib.util.find_spec rather than installed distribution names so that modules supplied by a managed runtime, such as pyspark on Fabric, Synapse, and HDInsight, count as present. FabricDataWarehouse overrides the hook to also check for the Microsoft ODBC Driver 18 for SQL Server, which pip cannot install, and its connection string now references the same constant so the two cannot drift. Tests assert the declared modules of every engine are actually installed by the extra they name. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Engines import their optional libraries lazily, so a forgotten extra surfaced partway into a run as an error that named neither the engine nor the package to install.
BaseEngine now declares REQUIRED_MODULES and INSTALL_EXTRA, and init calls verify_dependencies() before doing any other work. The raised MissingDependenciesError subclasses ImportError, names every missing module at once rather than one per failed run, and carries the matching
pip install lakebench[...]hint.Availability is resolved with importlib.util.find_spec rather than installed distribution names so that modules supplied by a managed runtime, such as pyspark on Fabric, Synapse, and HDInsight, count as present.
FabricDataWarehouse overrides the hook to also check for the Microsoft ODBC Driver 18 for SQL Server, which pip cannot install, and its connection string now references the same constant so the two cannot drift.
Tests assert the declared modules of every engine are actually installed by the extra they name.