Skip to content

add BGC-Argo+ dataset support to ArgoFloat.open_dataset()#623

Draft
RaphaelBajon wants to merge 5 commits into
euroargodev:masterfrom
RaphaelBajon:feature/bgcargo-plus-store
Draft

add BGC-Argo+ dataset support to ArgoFloat.open_dataset()#623
RaphaelBajon wants to merge 5 commits into
euroargodev:masterfrom
RaphaelBajon:feature/bgcargo-plus-store

Conversation

@RaphaelBajon

@RaphaelBajon RaphaelBajon commented May 22, 2026

Copy link
Copy Markdown
Collaborator

Integrate the BGC-Argo+ dataset (https://www.bgc-argo-plus.info) into argopy so that users can load QC-processed, outlier-removed BGC float files directly from the SOEST FTP server via the familiar ArgoFloat API:

    from argopy import ArgoFloat
    ds = ArgoFloat(6903091).open_dataset('BGCArgoPlus')

Changes

  • argopy/stores/float/bgcargo_plus.py (NEW)
  • argopy/stores/float/spec.py (MODIFIED)
  • tests/test_store_bgcargo_plus.py (NEW, generated with AI - Claude Sonnet 4.6, tested and 100% passed locally)
  • argopy/stores/float/argo_float.py (MODIFIED, docstring)
  • docs/advanced-tools/stores/bgcargo_plus.rst (NEW)

@gmaze

gmaze commented May 29, 2026

Copy link
Copy Markdown
Member

Hi @RaphaelBajon !
thanks a lot for your proposed contribution, let me think and check it and I'll get back to you

before that, do you think it is ready for merge and review ?

@gmaze gmaze moved this from Queued to Todo in Argopy Management Dashboard May 29, 2026
@gmaze gmaze added enhancement New feature or request, development argo-BGC About biogeochemical variables labels May 29, 2026
@gmaze gmaze moved this from Todo to In Progress in Argopy Management Dashboard May 29, 2026
Comment thread argopy/stores/float/bgcargo_plus.py Outdated
#: Path template on the FTP server.
#: ``{version}`` can be overridden via :attr:`BGCArgoPlusStore.version`.
BGCARGO_PLUS_PATH_TEMPLATE = (
"/bgc_argo_plus/outliers_removed/{version}/{wmo}_Sprof_BGCArgoPlus.nc"

@gmaze gmaze May 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RaphaelBajon are you sure of the url ?
I guess it should read:

'/bgc_argo_plus/outliers_removed/{version}/Individual_Floats/{wmo}_Sprof_BGCArgoPlus.nc'

@RaphaelBajon RaphaelBajon Jun 2, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @gmaze ,
Yes, you were right. The path depends on the version of dataset I add something hardcoded in bgcargo_plus.py to handle this:
path = BGCARGO_PLUS_PATH_TEMPLATE.format(wmo=wmo, version=version) if version != 'v0.1_2026_04' else BGCARGO_PLUS_PATH_TEMPLATE.format(wmo=wmo, version=version+"/Individual_Floats").

@RaphaelBajon

RaphaelBajon commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator Author

I was not able to compute ds.argo.profiletopoint() loading BGCArgoPlus dataset. I think this was because I was using h5py to open the dataset and not decoding correctly byte-string variables (requirement to use h5py). I was able to fix it by decoding the loaded data (hardcoded) within the BGCArgoPlusStore, but I was wondering if this is the best way to it. What do you think about it @gmaze?

Is there other function already implemented in utils.py that can be useful? I remember this was also linked to the integration of bgc variable within argopy and how to process these data types.

WMO = 6903091
af = ArgoFloat(WMO)
ds_bgcargoplus = af.open_dataset(name='BGCArgoPlus', version='v0.1_2026_04')
ds_point = ds_bgcargoplus.argo.profile2point()
ds_profback = ds_point.argo.point2profile()

with:

argopy.show_versions()

SYSTEM
------
commit: None
python: 3.12.13 | packaged by conda-forge | (main, Mar  5 2026, 17:06:14) [Clang 19.1.7 ]
python-bits: 64
OS: Darwin
OS-release: 25.5.0
machine: arm64
processor: arm
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
libhdf5: 1.14.6
libnetcdf: 4.9.3

INSTALLED VERSIONS: CORE
------------------------
aiohttp     : 3.12.15     
argopy      : 1.4.0       
decorator   : 5.3.1       
erddapy     : 3.2.0       
fsspec      : 2026.4.0    
h5netcdf    : 1.8.1       
netCDF4     : 1.7.4       
packaging   : 26.2        
requests    : 2.34.2      
scipy       : 1.17.1      
toolz       : 1.1.0       
xarray      : 2025.9.0    

INSTALLED VERSIONS: EXT.UTIL
----------------------------
gsw         : -           
pyco2sys    : -           
tqdm        : 4.67.3      

INSTALLED VERSIONS: EXT.FILES
-----------------------------
boto3       : -           
kerchunk    : -           
numcodecs   : -           
s3fs        : -           
zarr        : -           

INSTALLED VERSIONS: EXT.PERF
----------------------------
dask        : -           
distributed : -           
joblib      : -           
numba       : -           
pyarrow     : -           

INSTALLED VERSIONS: EXT.PLOT
----------------------------
IPython     : 9.13.0      
cartopy     : -           
ipykernel   : 7.2.0       
ipywidgets  : 8.1.8       
matplotlib  : -           
pyproj      : -           
seaborn     : -           

INSTALLED VERSIONS: DEV
-----------------------
aiofiles    : -           
black       : -           
bottleneck  : -           
cfgrib      : -           
cftime      : 1.6.5       
codespell   : -           
flake8      : -           
numpy       : 2.4.6       
pandas      : 3.0.3       
pip         : 26.1.1      
pytest      : 9.0.3       
pytest_cov  : -           
pytest_env  : -           
pytest_localftpserver: 1.5.0       
setuptools  : 82.0.1      
sphinx      : -           

INSTALLED VERSIONS: PIP
-----------------------
pytest-reportlog: -  

@gmaze

gmaze commented Jun 10, 2026

Copy link
Copy Markdown
Member

@RaphaelBajon does the .argo extension recognize the dataset ?
if no, there is no chance the profile2point would work
if yes, we identified limits to the applicability of the .argo methods to un-expected datasets (although recognised), you probably fell into this edge case

what do you mean by other utilities ?

from ...utils import check_wmo


def _decode_bytes_dataset(ds: xr.Dataset) -> xr.Dataset:

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gmaze yes the .argo extension recognizes the dataset. I just created another hardcoded function _decode_bytes_dataset while loading the 'BGCArgoPlus' data to be able to go through .argo.profile2point() and back to .argo.point2profile().

I was thinking you would have something already implemented in argopy utils that handles the decode byte-string variables to str function of _decode_bytes_dataset.

You can start a review, all pytests are working on my local machine.

@gmaze

gmaze commented Jun 11, 2026

Copy link
Copy Markdown
Member

@RaphaelBajon , before the review to start, the Argopy team needs to publish a formal framework:

  • for providing access to non-official Argo dataset,
  • with regard to IAg-based contributions.

Setting the path with regard to these 2 issues from the very beginning is very important to us.
And your PR and another have just accelerated this need.
This could take a few weeks, so I thank you for your patience, and I'll get back to you asap.

poke @quai20

@gmaze

gmaze commented Jun 11, 2026

Copy link
Copy Markdown
Member

Moving to draft because of #637 and #638

@gmaze gmaze marked this pull request as draft June 11, 2026 05:08
@gmaze

gmaze commented Jun 25, 2026

Copy link
Copy Markdown
Member

hi @RaphaelBajon !

Can you please check if your PR follows our new AI policy ?
it's here: https://argopy.readthedocs.io/en/latest/contributing.html#policy-regarding-ai-powered-contributions
(same thing here: https://github.com/euroargodev/argopy/blob/master/AI_POLICY.md)

Also, we're still working on #638 , we'll get back to you asap !

g

@RaphaelBajon

RaphaelBajon commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator Author

Hi @gmaze !

I updated the PR and disclosed the use of which AI model in the code for pytests (not specified before).

I guess some changes will be needed if you restructure the float stores and create a new repository for external sources of Argo data.

Signed-off-by: Raphaël Bajon <94925217+RaphaelBajon@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

argo-BGC About biogeochemical variables enhancement New feature or request, development

Projects

Status: Blocked

Development

Successfully merging this pull request may close these issues.

2 participants