References should provide direct access to the referenced object

[jmuhlich]

All of the Reference types in the schema are effectively pointers to entities defined elsewhere in the document. It would be nice if the Python API gave direct access to those referenced objects.

Currently:

>>> filter_set.emission_filter_ref[0]
FilterRef(id='Filter:5')

Ideally:

>>> filter_set.emission_filters[0]
Filter(id='Filter:5', lot_number='J34', manufacturer='Ink Inc.', ...)

To avoid creating any circular reference chains, the implementation should probably store weakrefs internally and provide proxy containers that perform the dereferencing upon access. The root OME object can walk its children in post_init and set all of these weakrefs, but this only helps with read-only use cases. Developing a good write API will be challenging.

[jmuhlich]

Should we leave the ..._ref fields visible and add a @property that provides the proxy access (read-only for now), or should the _ref fields be hidden?

[tlambert03]

I don't mind the _ref fields being there ... but if you feel strongly that they should be hidden that's fine with me. by hidden do you just mean to make them private with an underscore, or something more complicated? The underscore prefix might be a reasonable intermediate...

[jmuhlich]

Pydantic doesn't allow fields that start with an underscore for some reason. There are some ongoing issues and PRs around this, not really sure where it stands. I'll leave them visible for now and see how it goes.

[jmuhlich]

After spending some time on this it's proving harder than I thought, and there are some wrinkles. I currently have an API that works like this:

>>> filter_set.emission_filter_ref[0].ref
FilterRef(id='Filter:5')

ref is a property function that raises an Exception if called on an instance that hasn't set its internal weakref field ref_ by the root OME constructor. (Didn't want to return None since that's an allowable value for most references)

This seems OK, but I was planning on adding the "convenient" proxies so you could write filter_set.emission_filters[0]. Then I discovered these Settings subclasses which subclass Reference and add several of their own fields (like ObjectiveSettings). These don't make sense to access through a simple proxy to the referenced object because that ignores all those other fields. So with the current API, image.objective_settings.ref returns the referenced Objective and the objective_settings field still provides access to the settings-specific fields medium, refractive_index, etc.

I'll submit a PR as it stands and probably move on to something else. I'm happy to work with this API for now and see how it feels.