Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zarr v3 fill_value #10064

Open
5 tasks done
aldenks opened this issue Feb 20, 2025 · 2 comments
Open
5 tasks done

Zarr v3 fill_value #10064

aldenks opened this issue Feb 20, 2025 · 2 comments
Labels
bug needs triage Issue that has not been reviewed by xarray team member

Comments

@aldenks
Copy link

aldenks commented Feb 20, 2025

What happened?

I'm attempting to set a nan fill value in the encoding of a zarr v3 format array created with xarray and seeing 0.0 in the fill value metadata instead. Not sure if this is user error or a bug.

What did you expect to happen?

see "fill_value": "NaN" in the variable's encoding in the array's zarr.json.

Minimal Complete Verifiable Example

import json

import numpy as np
import xarray as xr

variable_data = np.full((4, 3), np.nan, dtype="float32")

ds = xr.Dataset({"example_var": (("x", "y"), variable_data)})

ds["example_var"].encoding["_FillValue"] = "NaN"
ds["example_var"].encoding["fill_value"] = "NaN"
# Also tried this but got the same 0.0 float value
# xr_dataset["example_var"].encoding["_FillValue"] = np.nan
# xr_dataset["example_var"].encoding["fill_value"] = np.nan

ds.to_zarr("xarray_fill_value_test.zarr", mode="w", zarr_version=3)

with open("xarray_fill_value_test.zarr/zarr.json") as f:
    metadata = json.load(f)
    # prints 0.0 instead of NaN
    print(
        f"{metadata['consolidated_metadata']['metadata']['example_var']['fill_value'] =}"
    )

Note this doesn't appear to be a zarr-python issue, I'm seeing behavior I expect there:

import json

import numpy as np
import zarr

data = np.full((4, 3), np.nan, dtype="float32")

array = zarr.create_array("zarr_fill_value_test.zarr", data=data, fill_value=np.nan)

with open("zarr_fill_value_test.zarr/zarr.json") as f:
    metadata = json.load(f)
    print(f"{metadata['fill_value']=}")  # prints "NaN", good

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.
  • Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Environment

xarray==2025.1.2
zarr==3.0.3
@aldenks aldenks added bug needs triage Issue that has not been reviewed by xarray team member labels Feb 20, 2025
Copy link

welcome bot commented Feb 20, 2025

Thanks for opening your first issue here at xarray! Be sure to follow the issue template!
If you have an idea for a solution, we would really welcome a Pull Request with proposed changes.
See the Contributing Guide for more.
It may take us a while to respond here, but we really value your contribution. Contributors like you help make xarray better.
Thank you!

@aldenks
Copy link
Author

aldenks commented Feb 20, 2025

If this is indeed a bug I'd be happy to work on it if someone would confirm it's not user error and point me in the right direction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug needs triage Issue that has not been reviewed by xarray team member
Projects
None yet
Development

No branches or pull requests

1 participant