Skip to content
Open
Show file tree
Hide file tree
Changes from 130 commits
Commits
Show all changes
145 commits
Select commit Hold shift + click to select a range
5789a99
draft MVP
jakubchlapek Feb 10, 2026
dc48b97
covariate support
jakubchlapek Feb 12, 2026
0cc488e
pyfunc series info extension
jakubchlapek Feb 12, 2026
b6ce538
unit tests
jakubchlapek Feb 12, 2026
605ecbe
add mlflow to dependencies
jakubchlapek Feb 12, 2026
3c2a2e2
changed to sqlite
jakubchlapek Feb 12, 2026
1750b11
kwargs pyfunc extension
jakubchlapek Feb 12, 2026
30456f5
removing pyfunc draft support: tbd if to include in future
jakubchlapek Feb 16, 2026
b69de4a
slight refactor by leveraging built-in mflflow validation util methods
jakubchlapek Feb 16, 2026
c488cd7
restructuring module
jakubchlapek Feb 16, 2026
7fe3cf3
autologging refactor w/ mlflow decorator
jakubchlapek Feb 16, 2026
ada47b7
refactoring log_model to leverage Model.log
jakubchlapek Feb 16, 2026
9fbe42a
unit test improvenemnts
jakubchlapek Feb 17, 2026
a0e663f
ForecastingModel subclasses handling for autolog
jakubchlapek Feb 17, 2026
93ca22a
save models with clean=True
jakubchlapek Feb 17, 2026
ac4b0c6
logging update
jakubchlapek Feb 17, 2026
a35a834
unused var, tfmodel.load handles .ckpt internally
jakubchlapek Feb 17, 2026
2695cc9
documentation
jakubchlapek Feb 18, 2026
662b8e3
added autolog logging default/provided metrics for all models
jakubchlapek Feb 18, 2026
f19d6bf
autolog metric unit tests
jakubchlapek Feb 18, 2026
bd932a9
removed redundant tests
jakubchlapek Feb 18, 2026
3b0ecb2
changed callback inject to true by default
jakubchlapek Feb 18, 2026
2d6158d
feat: ensure contiguous tensors in metric updates
jakubchlapek Feb 18, 2026
f684041
example quickstart for mlflow
jakubchlapek Feb 18, 2026
4495115
Merge branch 'master' into feat/mlflow-base
jakubchlapek Feb 18, 2026
894779b
unit test mps fix for torch
jakubchlapek Feb 18, 2026
c9a1301
typehinting fix
jakubchlapek Feb 18, 2026
a65612d
CI hotfix
jakubchlapek Feb 18, 2026
2123742
temporary fix for catboost, to be removed later
jakubchlapek Feb 18, 2026
2b41feb
scrap backtesting in favor of metric patching
jakubchlapek Feb 24, 2026
b119e7b
metric patching unit tests
jakubchlapek Feb 24, 2026
f561438
pytorch autolog
jakubchlapek Feb 25, 2026
df7e097
unit tests for pytorch autolog
jakubchlapek Feb 25, 2026
a7e4646
Merge branch 'master' into feat/mlflow-base
daidahao Mar 5, 2026
99f161c
Revert catboost cap
daidahao Mar 5, 2026
1c7b073
Replace raise_if and raise_if_not
daidahao Mar 5, 2026
e885b64
Use abs path
daidahao Mar 5, 2026
b46cf95
Use direct path for saving model
daidahao Mar 5, 2026
51a47f2
Update #2
daidahao Mar 5, 2026
28c11ed
Use mlflow class utils to replace insepctlib
daidahao Mar 5, 2026
ddae08a
Fix tests
daidahao Mar 5, 2026
91f6660
Get all forecasting models ith inspect
daidahao Mar 5, 2026
ce56906
Rename `log_torch_metrics`
daidahao Mar 5, 2026
3dcceac
Change `save_model` param order
daidahao Mar 5, 2026
1b7fa89
Update `log_model` params
daidahao Mar 5, 2026
b8a4f8d
Remove `log_params` from `log_model()`
daidahao Mar 5, 2026
331c4c3
Update unit tests
daidahao Mar 5, 2026
2396f16
Simplify `_is_torch_model`
daidahao Mar 5, 2026
7646dde
Move `_patched_fit` inside `_autolog()`
daidahao Mar 5, 2026
948ab56
Handle single series in metric patching
daidahao Mar 5, 2026
6ad82f7
Update covariate logging logic
daidahao Mar 5, 2026
abf24f5
Update tests and wait for completion
daidahao Mar 5, 2026
774a222
Remove `PL_AVAILABLE` flag
daidahao Mar 5, 2026
3c9296f
Add post-fitting metric TODO note
daidahao Mar 5, 2026
a7bca2e
chore: rename mlflow jupyter from 26 to 27
mizeller Mar 6, 2026
eb8f8cd
chore: fix jupyter for mlflow; work in progress
mizeller Mar 6, 2026
ab59196
Update .gitignore
mizeller Apr 17, 2026
0194744
chore: local test files; remove later
mizeller Apr 17, 2026
1c1b3d0
fix: use MLFlow>=3.0
mizeller Apr 17, 2026
e4b6823
Update mlflow_test_v2.py
mizeller Apr 20, 2026
4c239a8
Update pyproject.toml
mizeller Apr 20, 2026
3ce5475
Update mlflow_test_v2.py
mizeller Apr 21, 2026
2d1cd65
feat: support backtest/historical forecasts
mizeller Apr 21, 2026
5acd8de
fix: mlflow test script
mizeller Apr 21, 2026
c0b1c6e
fix: remove obsolte file
mizeller Apr 21, 2026
d5977c8
fix: formatting
mizeller Apr 21, 2026
bbe0bf9
fix: deprecate manage_run
mizeller Apr 21, 2026
09ad117
Update mlflow_test_v2.py
mizeller May 13, 2026
5a9f493
feat: metric kwarg support for autolog
jakubchlapek May 26, 2026
9113c41
chore: remove redundant metric check
jakubchlapek Jun 23, 2026
845dcd6
fix: check for active run in fit patch
jakubchlapek Jun 24, 2026
b9a7b62
feat: infer backtest output dim based on metric/backtests kwargs
jakubchlapek Jun 24, 2026
bf497e6
feat: backtest metric testing suite
jakubchlapek Jun 24, 2026
9e4934d
fix: update failing tests due to manage_run deprecation
jakubchlapek Jun 24, 2026
83bf76c
Merge branch 'master' into feat/mlflow-base
jakubchlapek Jun 24, 2026
7b8dbcd
chore: remove temporary test file
jakubchlapek Jun 24, 2026
502db71
feat: update mlflow notebook
jakubchlapek Jun 24, 2026
6dda789
fix: add MLFLOW_AVAILABLE flag
jakubchlapek Jun 24, 2026
c3f1df6
feat: align direct metric logging shape to backtesting (based on metr…
jakubchlapek Jun 25, 2026
8b8157b
feat: add missing autolog shape unit test
jakubchlapek Jun 25, 2026
3deccb1
feat: save per-series metrics to csv and the aggregate to mlflow
jakubchlapek Jun 25, 2026
277153f
feat: autolog per-series csv saving tests
jakubchlapek Jun 25, 2026
f746d7b
feat: add metric shape and per-series csv saving explanation section …
jakubchlapek Jun 25, 2026
a486b25
feat: handle name param for metrics
jakubchlapek Jun 25, 2026
e2c6758
chore: changelog
jakubchlapek Jun 25, 2026
248770f
Merge branch 'master' into feat/mlflow-base
dennisbader Jun 30, 2026
de8e149
docs: documentation clarifications
jakubchlapek Jul 21, 2026
94eaafe
chore: small code review improvements
jakubchlapek Jul 21, 2026
defe8aa
chore: update gitignore
jakubchlapek Jul 21, 2026
b5ba0bc
chore: change log_model=False by default and formatting
jakubchlapek Jul 21, 2026
44ca8da
feat: change on error from logs to raise
jakubchlapek Jul 21, 2026
965d802
chore: raise on failure instead of log
jakubchlapek Jul 21, 2026
eff9014
chore: remove quantile_num from _infer_metric_axes return tuple
jakubchlapek Jul 21, 2026
5e6a718
feat: remove dataset_name from logged metric pattern
jakubchlapek Jul 21, 2026
ce8fd01
fix: align tests to code review changes
jakubchlapek Jul 21, 2026
a93ad19
chore: remove classificaition metrics check
jakubchlapek Jul 21, 2026
554e02f
feat: change model import approach
jakubchlapek Jul 21, 2026
e36b551
chore: remove redundant dependencies
jakubchlapek Jul 21, 2026
9f93440
feat: add new import logic tests
jakubchlapek Jul 21, 2026
d1e80d2
feat: improve metric logging approach and fix import-order issue
jakubchlapek Jul 21, 2026
0d51dc4
feat: new metric logging approach tests
jakubchlapek Jul 21, 2026
ae8a072
feat: change per-series saving logic to append
jakubchlapek Jul 22, 2026
b3f3031
feat: append on save logic tests
jakubchlapek Jul 22, 2026
529c58f
feat: infer forecast horizon from backtest args
jakubchlapek Jul 22, 2026
069c28a
feat: fc_hzn inference unit test
jakubchlapek Jul 22, 2026
2d513a7
fix: handle multi series covariate logging
jakubchlapek Jul 22, 2026
aac668d
feat: covariate tracking unit tests
jakubchlapek Jul 22, 2026
66a0b10
docs: clarify explanation for pytorch autolog approach
jakubchlapek Jul 22, 2026
9570990
feat: remove extra explicit mlflow gorup
jakubchlapek Jul 22, 2026
381cc15
chore: hide log_model mlflow params, expose model and kwargs only
jakubchlapek Jul 22, 2026
9954928
chore: change default pip to core darts
jakubchlapek Jul 22, 2026
16c881f
chore: clean up unreachable code
jakubchlapek Jul 22, 2026
9081dff
feat: align backtesting windows on aggregation to end
jakubchlapek Jul 22, 2026
c1abac2
feat: windows alignment on backtest aggregation tests
jakubchlapek Jul 22, 2026
ce7e0a2
chore: change quantile formatting to :3f
jakubchlapek Jul 22, 2026
6f30c90
chore: align quantile label tests to :3f format
jakubchlapek Jul 22, 2026
fc6ce68
chore: rename quantile/interval/labels labels to axis labels
jakubchlapek Jul 22, 2026
109adc7
chore: align test to new axis naming convention
jakubchlapek Jul 22, 2026
19b2a8b
docs: clean up backtest docstrinjg
jakubchlapek Jul 22, 2026
603eb8e
feat: add agg_func option to autolog() and unit tests
jakubchlapek Jul 22, 2026
69b9ca5
feat: log model creation params as json + tests
jakubchlapek Jul 23, 2026
4c3d014
docs: cleanup
jakubchlapek Jul 23, 2026
c9e56e1
feat: ensure same amount of components for logging
jakubchlapek Jul 23, 2026
644374c
docs: align changelog
jakubchlapek Jul 23, 2026
a3dde9c
feat: align notebook to code review changes
jakubchlapek Jul 23, 2026
10cf6a6
fix: restrict component count only when not comp reduction
jakubchlapek Jul 23, 2026
9c547aa
fix: log model params fix
jakubchlapek Jul 23, 2026
b9e6488
minor updates to docs
dennisbader Jul 30, 2026
de3d9ee
Merge branch 'master' into feat/mlflow-base
dennisbader Jul 30, 2026
cf4aa3e
Merge branch 'master' into feat/mlflow-base
dennisbader Jul 31, 2026
9568b3d
feat: docs cleanup
jakubchlapek Aug 4, 2026
1b3ac1e
docs: active run clarification
jakubchlapek Aug 4, 2026
efb6b9d
feat: log target series info and encoder covs
jakubchlapek Aug 4, 2026
f4840c5
chore: remove redudnant comment
jakubchlapek Aug 4, 2026
e415957
feat: support logging for historical forecasts retrain=True without p…
jakubchlapek Aug 4, 2026
4424a0d
fix: remove the redundant inconsintent axis logic
jakubchlapek Aug 4, 2026
8067aad
chore: simplify block after hfc logic changes in #3165
jakubchlapek Aug 4, 2026
bb44876
chore: extract shared code into helpers from _log_{backtest/metric}
jakubchlapek Aug 5, 2026
805a471
docs: doc clarification
jakubchlapek Aug 5, 2026
c40a106
feat: agg timestep metrics
jakubchlapek Aug 5, 2026
e059938
chore: align notebook to current solution
jakubchlapek Aug 5, 2026
dd1e2c6
chore: simplify changelog entry
jakubchlapek Aug 5, 2026
811e16c
feat: negative x-axis to indicate most recent values
jakubchlapek Aug 5, 2026
f9eb3f8
fix: use 0-based step for predictions
jakubchlapek Aug 5, 2026
6f73130
fix: docs fix for CI
jakubchlapek Aug 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,4 @@ docs_env
.venv
.env
uv.lock
*mlruns/
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ but cannot always guarantee backwards compatibility. Changes that may **break co

**Improved**

- 🚀🚀 Added a custom MLflow model flavor for Darts, available under `darts.utils.mlflow`. It provides an MLflow integration for any Darts `ForecastingModel` (statistical, scikit-learn-like, and PyTorch-based). [#3022](https://github.com/unit8co/darts/pull/3022) by [Jakub Chłapek](https://github.com/jakubchlapek), [Zhihao Dai](https://github.com/daidahao) and [Michel Zeller](https://github.com/mizeller).
- Added `save_model()`, `load_model()`, and `log_model()` to persist and reload Darts models as MLflow models, including model and covariate metadata.
- Added `autolog()` to automatically log model creation parameters, covariate usage, metrics, and the trained model artifact when `log_models=True` (default `False`) on every `fit()`, `backtest()` and `historical_forecasts()` call within an active MLflow run.
- Metric functions from `darts.metrics` are automatically logged when called inside an active run, with keys reflecting the metric output shape (per-component, per-quantile/interval, per-label, and per-timestep results charted across MLflow steps). Multi-series results log the aggregate over series and write the full per-series breakdown to a JSON artifact.
- Added a new notebook for [MLflow quickstart](https://unit8co.github.io/darts/examples/29-MLflow-quickstart.html) with detailed usage examples.
- Note: model serving and deployment (MLflow's `pyfunc` flavor, model signatures, and input examples) are not yet supported.
- Added support for per-timestep (non-aggregated) encoder and decoder variable importances in `TFTExplainer`, exposed as `TimeSeries` via `TFTExplainabilityResult.get_encoder_importance_over_time()` and `get_decoder_importance_over_time()`. [#3170](https://github.com/unit8co/darts/pull/3170) by [exactml](https://github.com/exactml).
- Calling `TFTModel.fit_from_dataset()` on a dataset that does not have future covariates now raises an informative exception. [#3149](https://github.com/unit8co/darts/pull/3149) by [YOON KIWOONG](https://github.com/kiwoongyoon).
- 🔴 Percentage and range-based metrics (`ape`, `mape`, `sape`, `smape`, `wmape`, `ope`, `arre`, `marre`, `coefficient_of_variation`) no longer raise a hard `ValueError` when the denominator is exactly zero. A new `zero_division` parameter controls the behavior: [#3122](https://github.com/unit8co/darts/pull/3122) by [Mahimn](https://github.com/mahimn01).
Expand All @@ -26,6 +32,8 @@ but cannot always guarantee backwards compatibility. Changes that may **break co

**Dependencies**

- Added `mlflow>=3.0` to the optional dependency group, enabling the MLflow integration. [#3022](https://github.com/unit8co/darts/pull/3022) by [Jakub Chłapek](https://github.com/jakubchlapek), [Zhihao Dai](https://github.com/daidahao) and [Michel Zeller](https://github.com/mizeller).

### For developers of the library:

## [0.46.1](https://github.com/unit8co/darts/tree/0.46.1) (2026-07-20)
Expand Down
23 changes: 23 additions & 0 deletions darts/metrics/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,22 @@ def wrapper_classification_support(*args, **kwargs):
return wrapper_classification_support


_metric_callbacks: list[Callable] = []


def register_metric_callback(callback: Callable) -> None:
"""Register a callback to be invoked after every top-level metric call."""
_metric_callbacks.append(callback)


def unregister_metric_callback(callback: Callable) -> None:
"""Remove a previously registered metric callback, if present."""
try:
_metric_callbacks.remove(callback)
except ValueError:
pass


def multi_ts_support(func) -> Callable[..., METRIC_OUTPUT_TYPE]:
"""
This decorator further adapts the metrics that took as input two (or three for scaled metrics with `insample`)
Expand All @@ -169,6 +185,9 @@ def multi_ts_support(func) -> Callable[..., METRIC_OUTPUT_TYPE]:

@wraps(func)
def wrapper_multi_ts_support(*args, **kwargs):
original_args = args
original_kwargs = dict(kwargs)

actual_series = (
kwargs["actual_series"] if "actual_series" in kwargs else args[0]
)
Expand Down Expand Up @@ -312,6 +331,10 @@ def wrapper_multi_ts_support(*args, **kwargs):
elif series_seq_type == SeriesType.SINGLE:
vals = vals[0]

# invoke registered callbacks (e.g. MLflow autologging)
for cb in _metric_callbacks:
cb(func=func, result=vals, args=original_args, kwargs=original_kwargs)

# flatten along series axis if n series == 1
return vals

Expand Down
7 changes: 5 additions & 2 deletions darts/models/forecasting/pl_forecasting_module.py
Original file line number Diff line number Diff line change
Expand Up @@ -450,8 +450,11 @@ def _update_metrics(self, output, target, metrics):
pred = output.squeeze(dim=-1)

# torch metrics require 2D targets of shape (batch size * ocl, num targets)
target = target.reshape(-1, self.n_targets)
pred = pred.reshape(-1, self.n_targets)
# contiguous() is needed because model outputs can be non-contiguous views
# (e.g. NBEATS slices the last dimension), and some torchmetrics implementations
# call .view() internally which requires a contiguous tensor.
target = target.reshape(-1, self.n_targets).contiguous()
pred = pred.reshape(-1, self.n_targets).contiguous()

metrics.update(pred, target)

Expand Down
1 change: 1 addition & 0 deletions darts/tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ def _package_available(*names: str) -> bool:
PLOTLY_AVAILABLE = _package_available("plotly")
IPYTHON_AVAILABLE = _package_available("IPython")
TIREX_AVAILABLE = _package_available("tirex")
MLFLOW_AVAILABLE = _package_available("mlflow")

tfm_kwargs: dict[str, Any] = {
"pl_trainer_kwargs": {
Expand Down
Loading
Loading