Skip to content

[ENH] twe memory and speed improvements#3563

Open
TonyBagnall wants to merge 1 commit into
mainfrom
ajb/twe
Open

[ENH] twe memory and speed improvements#3563
TonyBagnall wants to merge 1 commit into
mainfrom
ajb/twe

Conversation

@TonyBagnall

@TonyBagnall TonyBagnall commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Part of #3261 (memory optimization for elastic distances). Applies the two-row dynamic-programming optimization established for DTW in #3258 (and MSM in #3562) to TWE.

closes #3269

  • Public API and outputs are unchanged. Signatures, defaults (nu=0.001, lmbda=1.0), docstrings, and return values of twe_distance, twe_pairwise_distance, twe_cost_matrix, and twe_alignment_path are all the same.
  • The full cost matrix is intentionally retained. _twe_cost_matrix is untouched and still backs twe_cost_matrix (which by definition returns the whole matrix) and twe_alignment_path (which needs the full matrix to backtrack the alignment via
    compute_min_return_path).
  • _pad_arrs, the pointwise Euclidean cost, the pairwise / multiple-to-multiple drivers, and the bounding-matrix / windowing / Itakura logic are untouched. All callers of _twe_distance (the pairwise functions) already pass padded arrays and are
    unaffected.

Validation

  • Equivalence: matches the previous full-matrix result to floating-point tolerance — max absolute difference ~3.5e-14 over 400 randomized cases (univariate/multivariate, varying nu/lmbda, windows, and unequal lengths), growing to ~1e-9 at n=8000
    (≈1e-13 relative).

Benchmark (univariate, n = m, no window)

n Runtime baseline → opt Peak mem baseline → opt Mem reduction
1,000 15.6 ms → 13.6 ms (1.1×) 9.0 MB → 1.0 MB 9.0×
2,000 67.9 ms → 60.1 ms (1.1×) 36.0 MB → 4.0 MB 9.0×
4,000 0.26 s → 0.21 s (1.3×) 144.1 MB → 16.1 MB 9.0×
8,000 1.89 s → 0.85 s (2.2×) 576.3 MB → 64.3 MB 9.0×

The optimization eliminates the float64 cost matrix (n²·8 bytes); the ~9× reduction reflects that the remaining memory of the public call is the O(n²) boolean bounding matrix (n²·1 byte, window=None).

@TonyBagnall TonyBagnall requested a review from chrisholder as a code owner June 24, 2026 22:20
@aeon-actions-bot aeon-actions-bot Bot added distances Distances package enhancement New feature, improvement request or other non-bug code enhancement labels Jun 24, 2026
@aeon-actions-bot

Copy link
Copy Markdown
Contributor

Thank you for contributing to aeon

I have added the following labels to this PR based on the title: [ enhancement ].
I have added the following labels to this PR based on the changes made: [ distances ]. Feel free to change these if they do not properly represent the PR.

The Checks tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our pre-commit code quality check fails, please run pre-commit locally and push the fixes to your PR branch.

Don't hesitate to ask questions on the aeon Discord channel if you have any.

PR CI actions

These checkboxes will add labels to enable or disable CI functionality for this PR. This may not take effect immediately, and a new commit may be required to run the new configuration.

  • Run pre-commit checks for all files
  • Run mypy typecheck tests
  • Run all pytest tests and configurations
  • Run all notebook example tests
  • Run numba-disabled codecov tests
  • Disable numba cache loading
  • Regenerate expected results for testing
  • Push an empty commit to re-run CI checks

@TonyBagnall

Copy link
Copy Markdown
Contributor Author
image

@TonyBagnall

Copy link
Copy Markdown
Contributor Author
image

@TonyBagnall

Copy link
Copy Markdown
Contributor Author

speed up less here, but memory gain the same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

distances Distances package enhancement New feature, improvement request or other non-bug code enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ENH] Optimize memory complexity for TWE distance

1 participant