TWiki> IVOA Web>IvoaDataModel>AstroPyVOMivot (2022-07-25, MarkusDemleitner) EditAttach

# Follow-up of the discussions on the design of a AstroPyVO API processing annotated data.

At last Interop (April 2022), we have had a PyVO hands-on session where we have open a discussion on the requirements for a PyVO API processing annotated data.

The minutes of that session are on GitHUB
We all agreed that this new API must have no impact on the current one and that its design must be based on real use-cases.
The expectation for using annotated data is not necessary to allow doing new things. It is rather to facilitate common operations.To move on, we have decided to start from some typical PyVOT codes and to figure out how to do the same things with a code based on annotated data.
The rule is to write realistic pseudo-code.We are proposing now a brief discussion focused on a first sketch based on the proposal presented by Gilles in May 2021 and taking advantage of the experience we got while exercising MIVOT parsing.
Meeting Wiki page:

Note of the meeting held on 2022 July 21

Go on the GitHub Wiki to get more details about the work progress


Attendees - 6
Laurent Michel LM, Renaud Savalle RS, Tom Donaldson TD, Omar Laurino OL, Mark CD MCD, Jesus Salgado JS

Meeting notes

LM: Presentation of a brief introduction on the subject, PyVO-DM
The proposed model facilitate the way to map DMs into VOTables and how to read these VOTables in a efficient way in python using astropy (and PyVO)
In the client side, there are two things to do with this annotation: extract the mapping block adn and process this mapping block
Also, write the python instances that map the data model instances (e.g position, photometry, etc)
The focus of today is design and astropy API compliant with the native API and supports working with models
As a result of the hackathon, pages under https://github.com/ivoa/modelinstanceinvot-code/wiki were created
_Examples created by Vizier (Gilles Landais) described at https://github.com/ivoa/modelinstanceinvot-code/wiki/vizier-proposal__

_as they have experience on the difficulties of mapping dm elements into vizier tables
Gilles propose to iterate on measures on the resources and for each one obtain the values (there is another examople for spectral data that has not been explored further)
There is an interpretation of this pseudocode into pyvo code including a query to the service, requesting dm_mapping. That could allow to created mangoinstances from the response and obtain a modelview using modelviewer
That allows the navigation of the result in terms of mangoObjects.
It can be also iterated on the object parameters to get the values of the labels
Example of the parsing of a PhotometryFilter using the model description
MCD: the sentence containsMangoInstances could be false for, e.g. cubes or time series (the object is fully returned)
OL: how is this compared with the library RAMA?
TD: we need to have client code to navigate the vo/dml
LM: the example showed is based on mango but the idea is to create astropy classes to do the parsing. The process has been divided in three phases 1 parse VO/Table 2 work with the parameters inside classes 3 create classes
OL: Why you did not start with the library RAMA as most of the requirements are already implemented there?
LM: it was started the exercise as RAMA was not compatible with the mapping syntax
OL: yes, but this is a detail that could be implemented as other things are already there
The main target for today is to discuss about the model components vs astropy components at
https://github.com/ivoa/modelinstanceinvot-code/wiki/21_07_22
MCD: if the idea is to create an skycoord instance from an annotation, the first thing is to study the constructor
TD: it is an overloaded constructor and "humans" knowing the location of the columns create the instances using a simple constructor version
There is a affiliated package called SpecUtils that could be explored
We just draft some ideas on the aspect of the API during the last interop. We could check RAMA and I have some ideas on how to handle this in astropy
LM: Table with the correspondance between the model instances and astropy types (in the case of polarization, photometry, etc it would be astropy quantity)
For cases like PhotCal and Flag, there is not a clear correspondance with astropy
Also there is not clear link with magnitudes and error, it does not support data grouping and there is not a way to iterate over the list of available measures
MCD: it could be used an array for iteration
LM: but associated values are difficult to link
The path is to reuse API astropy classes, prevent the use of affiliated classes, extend astropy for quantities, extend to support, missing metadata, errors and data grouping
TD: we should not be just limited to astropy classes as many metadata is not included into the astropy classes and it should be available for clients
I think astropy core team will resist to include all these new classes but what we can do is upgrade the VOTable parser ensuring that we can load all the required metadata
PyVO is the natural place to put these extended classes
LM: so the idea would be to extend astropy adding these classes to pyVO

Community Notes

Previous Work

I note in passing that (for a simpler annotation) I've implemented an API and showed how it could be used in an astropy branch: https://github.com/msdemlei/astropy

And I would agree that most of this needs to go into astropy rather than pyVO, in particular basically everything dealing dealing with the annotation of coordinate metadata and error annotation.

Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r3 - 2022-07-25 - MarkusDemleitner
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback