Release 0.9.4: Python 3.12/3.13 support, standard logging, initial population seeding, and expanded model coverage (#41)

* Next version with revision will be 0.9.4

* Modified: improved example

* Modified: default search spaces

* Added initial and default individual options.

* Fixed mlflow test

* Improved plots to reduced size

* Updated sphinx gallery examples

* Improved plots with responsiveness

* Added default parameters, pytest and gallery sphinx examples of catboost, lightgbm and other sklearn ml models

* Added attributes with number of n_trials_ and optimization_time_ and a future parameter disable_file_output to reduce overhead when a fast execution is needed

* Added disable_file_output to reduce overhead in experiments

* Added optimization_time_ and n_trials_ to the sphinx gallery examples

* Fix parallelization using joblib instead of multiprocessing

* Added: test of ml models

* Updated and fixed: mlflow integration improved and documented, Phase 1 about params

* Updated documentation conf.py to remove mlflow problems and fix scipy

* Modified: version from func

* Added: default conf files for ml scikit learn models

* Updated README with the new features and new example.

* Fix test

* Added tests for initial params in genetic population and disabling the file output

* Removed to do

* Improved gitignore

* The library now follows the standard Python library logging pattern

* Added verbose option

* Added python 3.12

* Added python 3.13

Author: Caparrini

.github/workflows/CI.yml

@@ -8,7 +8,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [3.9, '3.10', 3.11]
+        python-version: [3.9, '3.10', 3.11, 3.12, 3.13]
         os: [ubuntu-latest]
 
 

.gitignore

@@ -348,4 +348,69 @@ dmypy.json
 *tmpfiles
 *lol
 
-# End of https://www.toptal.com/developers/gitignore/api/pycharm,python,test,macos
+# End of https://www.toptal.com/developers/gitignore/api/pycharm,python,test,macos
+
+### mloptimizer specific ###
+# Experiment output directories (timestamped)
+[0-9]*_*Classifier/
+[0-9]*_*Regressor/
+
+# MLflow tracking databases
+mlflow.db
+mlflow_test.db
+mlruns/
+mloptimizer/test/mlruns/
+docs/mlruns/
+examples/mlruns/
+
+# Pickle files (saved datasets)
+*.pkl
+saved_datasets.pkl
+
+# Virtual environments
+.cvenv/
+
+# Notebooks and experiments
+notebooks/
+experiment/
+
+# Analysis and temporary markdown files
+*_SUMMARY.md
+*_GUIDE.md
+*_ANALYSIS.md
+*_FIX.md
+*_PLAN.md
+PRELIMINARY_*.md
+PAPER_REVISION_ADVICE.md
+
+# Generated documentation
+docs/auto_examples/
+docs/html/
+docs/sg_execution_times.rst
+
+# Example outputs
+examples/Evolution_example/
+examples/Optimizer/
+examples/Search_space_example/
+examples/catboost_info/
+
+# Test outputs
+mloptimizer/test/application/reporting/*.html
+mloptimizer/test/application/reporting/*.png
+mloptimizer/test/catboost_info/
+mloptimizer/test/test_genoptimizer/
+
+# Scripts and temporary files
+scripts/
+notexamples/
+review.txt
+softwx-draft.tex
+test_*.py
+verify_*.py
+demo_*.py
+dockerfile
+
+# Benchmark results
+openml_*.png
+openml_*.csv
+openml_*.tex

README.md

@@ -21,12 +21,19 @@ The genetic algorithm used in mloptimizer provides an efficient and flexible app
 - Default hyperparameter ranges
 - Default score functions for evaluating the performance of the model
 - Reproducibility of results
+- Early stopping to prevent overfitting
+- Population seeding with known good configurations
+- Performance tracking (trials count, optimization time)
+- Zero file output mode for cleaner workflows (enabled by default)
 
 ## Advanced Features
 - Extensible with more machine learning algorithms that comply with the Scikit-Learn API
 - Customizable hyperparameter ranges
 - Customizable score functions
 - Optional mlflow compatibility for tracking the optimization process
+- Generation-level MLflow tracking with nested runs
+- Responsive Plotly visualizations with WebGL acceleration
+- Joblib-based parallelization for better compatibility
 
 ## Installation
 
@@ -67,17 +74,24 @@ X, y = load_iris(return_X_y=True)
 hyperparameter_space = HyperparameterSpaceBuilder.get_default_space(DecisionTreeClassifier)
 
 # 3) Create the optimizer and optimize the classifier
-# - 10 generations starting with a population of 10 individuals, other parameters are set to default
-
-opt = GeneticSearch(estimator_class=DecisionTreeClassifier,
-                    hyperparam_space=hyperparameter_space,
-                    **{"generations": 5, "population_size": 5}
-                    )
-
-# 4) Optimize the classifier, the optimization returns the best estimator found in the optimization process
+opt = GeneticSearch(
+    estimator_class=DecisionTreeClassifier,
+    hyperparam_space=hyperparameter_space,
+    generations=10,
+    population_size=20,
+    early_stopping=True,        # Stop early if no improvement
+    patience=3,                  # Wait 3 generations
+    cv=5,                        # 5-fold cross-validation
+    seed=42                      # Reproducibility
+)
+
+# 4) Optimize (no files created by default)
 opt.fit(X, y)
 
-print(opt.best_estimator_)
+# Access results
+print(f"Best score: {opt.best_estimator_.score(X, y)}")
+print(f"Trials evaluated: {opt.n_trials_}")
+print(f"Time taken: {opt.optimization_time_:.2f}s")
 ```
 Other algorithms can be used, such as `RandomForestClassifier` or `XGBClassifier` which have a 
 default hyperparameter space defined in the library.
@@ -96,8 +110,14 @@ Examples can be found in [examples](https://mloptimizer.readthedocs.io/en/master
 The following dependencies are used in `mloptimizer`:
 
 * [Deap](https://github.com/DEAP/deap) - Genetic Algorithms
-* [XGBoost](https://github.com/dmlc/xgboost) - Gradient boosting classifier
+* [XGBoost](https://github.com/dmlc/xgboost) - Gradient boosting framework
+* [CatBoost](https://catboost.ai/) - Gradient boosting framework
+* [LightGBM](https://lightgbm.readthedocs.io/) - Gradient boosting framework
 * [Scikit-Learn](https://github.com/scikit-learn/scikit-learn) - Machine learning algorithms and utilities
+* [Plotly](https://plotly.com/python/) - Interactive visualizations
+* [Seaborn](https://seaborn.pydata.org/) - Statistical visualizations
+* [joblib](https://joblib.readthedocs.io/) - Parallel processing
+* [tqdm](https://github.com/tqdm/tqdm) - Progress bars
 
 Optional:
 * [Keras](https://keras.io) - Deep learning library

docs/_static/custom.js

@@ -10,4 +10,15 @@ document.addEventListener("DOMContentLoaded", function() {
             sidebar.style.display = 'none';
         }
     }
+
+    // Force plotly figures to recalculate size on page load
+    // This fixes the initial rendering issue with responsive plots
+    setTimeout(function() {
+        if (typeof Plotly !== 'undefined') {
+            const plotlyDivs = document.querySelectorAll('.plotly-graph-div');
+            plotlyDivs.forEach(function(div) {
+                Plotly.Plots.resize(div);
+            });
+        }
+    }, 100); // Small delay to ensure DOM is fully rendered
 });

docs/conf.py

@@ -52,13 +52,14 @@
     'python': ('https://docs.python.org/3', None),
     'numpy': ('https://numpy.org/doc/stable/', None),
     'scikit-learn': ('https://scikit-learn.org/stable/', None),
-    'mlflow': ('https://www.mlflow.org/docs/latest/', None),
+    # MLflow inventory temporarily disabled due to 403 errors
+    # 'mlflow': ('https://www.mlflow.org/docs/latest/', None),
     'xgboost': ('https://xgboost.readthedocs.io/en/latest/', None),
     'lightgbm': ('https://lightgbm.readthedocs.io/en/latest/', None),
     'pandas': ('https://pandas.pydata.org/pandas-docs/stable/', None),
     'matplotlib': ('https://matplotlib.org/stable/', None),
     'seaborn': ('https://seaborn.pydata.org/', None),
-    'scipy': ('https://docs.scipy.org/doc/scipy/reference/', None),
+    'scipy': ('https://docs.scipy.org/doc/scipy/', None),  # Updated: removed '/reference/' from URL
     'deap': ('https://deap.readthedocs.io/en/master/', None),
 }
 

docs/sections/Advanced/index.rst

@@ -9,6 +9,7 @@ The advanced customization options in `mloptimizer` enable fine-tuning of the op
    score_functions
    reproducibility
    parallel
+   logging
 
 
 Overview of Customization Options
@@ -20,4 +21,6 @@ Overview of Customization Options
 
 - **Parallel Processing**: Accelerate optimization by distributing computations across multiple cores. Parallel processing can significantly reduce runtime, especially for complex models or extensive hyperparameter spaces.
 
+- **Logging Configuration**: Configure logging output to monitor optimization progress, save logs to files, or integrate with your existing logging setup. mloptimizer follows the standard Python library logging pattern for maximum flexibility.
+
 Each section provides detailed guidance on implementing these advanced options.

docs/sections/Introduction/overview.rst

@@ -131,6 +131,24 @@ Setting the same `seed` value across multiple runs will produce identical result
 
    On macOS with newer processor architectures (e.g., M1 or M2 chips), users may experience occasional reproducibility issues due to hardware-related differences in random number generation and floating-point calculations. To ensure consistency across runs, we recommend running `mloptimizer` within a Docker container configured for reproducible behavior. This approach helps isolate the environment and improves reproducibility on macOS hardware.
 
+Logging and Verbosity
+---------------------
+
+By default, `mloptimizer` runs silently without logging output. To enable logging, use the ``verbose`` parameter:
+
+.. code-block:: python
+
+    # Silent (default)
+    opt = GeneticSearch(..., verbose=0)
+
+    # Info level - shows optimization lifecycle
+    opt = GeneticSearch(..., verbose=1)
+
+    # Debug level - shows detailed info
+    opt = GeneticSearch(..., verbose=2)
+
+For more advanced logging configuration, see the :doc:`../Advanced/logging` section.
+
 MLflow Integration Example
 ------------------------------
 

docs/sections/MLflow/index.rst

@@ -0,0 +1,37 @@
+MLflow Integration
+==================
+
+MLflow is an open-source platform for managing the machine learning lifecycle, including experiment tracking, model versioning, and deployment. The `mloptimizer` library integrates seamlessly with MLflow to provide comprehensive tracking of genetic algorithm optimization runs, enabling you to monitor evolution progress, compare hyperparameter configurations, and analyze results.
+
+.. toctree::
+   :hidden:
+
+   mlflow_basics
+   mlflow_viewing
+   mlflow_remote
+
+Overview of MLflow Features
+----------------------------
+
+- **Experiment Tracking**: Automatically log all optimization runs with their configurations, metrics, and results. Track generation-level metrics to visualize how fitness evolves across generations.
+
+- **Result Visualization**: Use the MLflow UI to interactively explore runs, compare different optimization strategies, and analyze hyperparameter impact on model performance.
+
+- **Remote Tracking**: Configure MLflow to use remote tracking servers for team collaboration and centralized experiment management. Share optimization results across your organization.
+
+Each section provides detailed guidance on using MLflow with mloptimizer.
+
+Key Benefits
+------------
+
+**Generation-Level Tracking**
+   Every generation's best, average, and worst fitness scores are logged, allowing you to visualize the evolution of your population over time.
+
+**Comprehensive Metadata**
+   Dataset characteristics, optimization configuration, early stopping information, and timing metrics are automatically recorded.
+
+**Flexible Storage**
+   Use local file-based storage for quick experiments or configure remote MLflow servers with database backends for production deployments.
+
+**Seamless Integration**
+   Simply add ``use_mlflow=True`` to your ``GeneticSearch`` configuration - no additional code required.