Skip to content

update ungrib to handle files larger than 2GB#282

Open
guoqing-noaa wants to merge 1 commit into
wrf-model:developfrom
guoqing-noaa:ungribBeyond2G
Open

update ungrib to handle files larger than 2GB#282
guoqing-noaa wants to merge 1 commit into
wrf-model:developfrom
guoqing-noaa:ungribBeyond2G

Conversation

@guoqing-noaa

Copy link
Copy Markdown
  1. Update byte offset variables (start/newpos in C, IB/JB in Fortran, iseek/lskip/KS in callers) to use 64-bit
  2. Byte count variables (NB/lgrib/no/size/nactual) remain 32-bit, assuming individual GRIB messages are under 2GB
  3. Use long long C type to match Fortran's INTEGER(KIND=8)

@guoqing-noaa

Copy link
Copy Markdown
Author

Tag @mgduda for awareness. Thanks!

@weiwangncar

Copy link
Copy Markdown
Collaborator

@guoqing-noaa Can you let us know which dataset this PR will help? I assume this will have no effect on fields that are less than 2 GB?

@guoqing-noaa

Copy link
Copy Markdown
Author

@guoqing-noaa Can you let us know which dataset this PR will help? I assume this will have no effect on fields that are less than 2 GB?

@weiwangncar Yes, it has no effect on files less than 2GB in size. I did tests using ungrib.exe before and after this PR and I got identical intermediate binary files.

The RRFSv1 grib2 files cover the North American domain and each file is about 9GB in file size.

Also, this WRF forum post has requested the capability to handle files larger than 2GB: https://forum.mmm.ucar.edu/threads/apparent-2gb-grib-file-size-limit.8758/

Thanks!

@weiwangncar

Copy link
Copy Markdown
Collaborator

@guoqing-noaa Thanks for the info, and that's good to know. We will run some checks too.

@weiwangncar

Copy link
Copy Markdown
Collaborator

@jimbresch Would you be interested in taking a look at this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants