Skip to content

Do time arithmetic on numpy arrays instead of pandas Index objects - #159

Merged
kandersolar merged 1 commit into
pvlib:mainfrom
nirabo:perf/numpy-internals
Oct 13, 2026
Merged

kandersolar merged 1 commit into
pvlib:mainfrom
nirabo:perf/numpy-internals

Conversation

@nirabo

@nirabo nirabo commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Extract the datetime fields (year, month, day, dayofyear, julian date, fractional hour) as numpy arrays once, at the top of each algorithm. Arithmetic on pandas Index objects allocates a new Index per operation, which dominated runtime for small inputs; SG2 also indexed pandas Series by a per-timestamp row array to look up its delta-T coefficients.

Outputs are bit-identical (165 output arrays over 20k random timestamps, 4 sites, all algorithms). Speed-up vs main: 2-6x for a single timestamp, 1.0-1.8x for a year at 1-minute resolution (SG2 1.8x).

Extract the datetime fields (year, month, day, dayofyear, julian date,
fractional hour) as numpy arrays once, at the top of each algorithm.
Arithmetic on pandas Index objects allocates a new Index per operation,
which dominated runtime for small inputs; SG2 also indexed pandas Series
by a per-timestamp row array to look up its delta-T coefficients.

Outputs are bit-identical (165 output arrays over 20k random timestamps,
4 sites, all algorithms). Speed-up vs main: 2-6x for a single timestamp,
1.0-1.8x for a year at 1-minute resolution (SG2 1.8x).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013LERsHQC5eP3qvgBsbbory
@nirabo nirabo closed this Sep 15, 2026
@nirabo
nirabo deleted the perf/numpy-internals branch September 15, 2026 05:02
@echedey-ls

Copy link
Copy Markdown
Member

Even if AI, I think it would be worth to review and merge if appropriate?
Not that this will fix any time issues, but faster code also means less energy consumption. I mean, maybe one day it offsets the AI energy for this PR.

@nirabo
nirabo restored the perf/numpy-internals branch September 15, 2026 10:41
@nirabo

nirabo commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

Wanted to add a bit more improvements, but yeah, why not merge time arithmetic first.

@nirabo nirabo reopened this Sep 15, 2026

@kandersolar kandersolar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems fine to me, and an optimization/consistency improvement I was going to do soon anyway. I observe speed ratios (ratio of new to old) in the range 1.1x to 1.3x, except for SG2 which is more like 1.5x and USNO which is more like 8x (!)

@kandersolar
kandersolar merged commit 498c7cc into pvlib:main Oct 13, 2026
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants