[Bug]: Validation error when passing a "0" integer

[rettigl]

Contact Details

[email protected]

What happened?

When a field with type "NX_FLOAT" gets filled by a "0" integer (rather than "0.0" float), the checking routine reports this key as undocumented. Specifically, the value is converted to float here:

pynxtools/src/pynxtools/dataconverter/helpers.py

Line 668 in ef01369

return accepted_types[0](value)

This returns 0, which in turn makes this check fail:

pynxtools/src/pynxtools/dataconverter/validation.py

Line 588 in ef01369

if is_documented(not_visited_key, tree):

and hence the value ends up in undocumented.
This is tested on the multi-format-reader, but should apply to main as well.

Relevant log output

This is output from the Tutorial 3 of the sed package, which uses the mpes reader and pynxtools converter internally:

Input:
sp.binned.attrs["metadata"]["energy_calibration"]["tof"] = 0
sp.save(data_path + "/binned.nxs")

Output:
Using mpes reader to convert the given files:  
• ../sed/config/NXmpes_config.json
Field /ENTRY[entry]/PROCESS_MPES[process]/energy_calibration/original_axis written without documentation.
The output file generated: /mnt/pcshare/users/Laurenz/AreaB/sed/sed/tutorial/datasets/WSe2/binned.nxs.

Input: 
sp.binned.attrs["metadata"]["energy_calibration"]["tof"] = 0.0
sp.save(data_path + "/binned.nxs")

Output:
Using mpes reader to convert the given files:  
• ../sed/config/NXmpes_config.json
The output file generated: /mnt/pcshare/users/Laurenz/AreaB/sed/sed/tutorial/datasets/WSe2/binned.nxs.

[GinzburgLev]

It seems that there are potentially two issue here, which are somewhat separated:

1. if the schema has a field with one type (such as NX_FLOAT) and the .h5 has the same field but filled with 0 in a different format (for example, NX_INT), the conversion function (pynxtools/src/pynxtools/dataconverter/helpers.py/is_valid_data_field) returns 0 (which is supposed to mean the result of conversion), but it is later interpreted as 0==False (convertion is impossible). This is easy to fix by changing function such that it has two outputs: boolean (valid field?) and any (result of conversion).

2. However, for whatever reason the value written in the field is not the converted value but the original one (so, in the example above we still get int(0) written in the field which is supposed to be NX_FLOAT). Do we want to change that? If yes, I can not find where exactly the assignment is happening, but I believe we just need to change value -> is_valid_data_field(value, nxdl_type, ...)[1] there.

[lukaspie]

Second part to be discussed in #523