DataLink-1.1 Next

This topic collects proposals for modifications of the DataLink-1.1 specification in order to improve the next revision of the specification.

The discussion which occurred between 1.0 and 1.1 can still be found here and of course on github.

* discussed during 1.0 -> 1.1 phase, but not integrated in 1.1 eventually :

• service descriptor : URL could benefit go be templated (for example to pick up part of the path from the user or from the table and not only parameters). Proposals have been made but it was delayed to next version until some service implement it. (issue DataLink #27). Probably DataLink 2.0
• DataLink underlying data model : is content_type, content_qualifier, semantiics triplet enough. Should we distinguish conveyed "information" and conveyed "realtionship" (issue DataLink #44). This is a discussion for DataLink 2.0.
• Service descriptor inputParams enhancing distinction between required and optional parameters (issue DataLink #51). No obvious solution has been found at the moment.
• Service Descriptor to be removed from DataLink and pushed to VOTable (issue DataLink #53). This is a major revision of DataLink but also of VOTable. Version 2.0. This may also be completed by introduction of the templating mechanism or even more by the use of PDL inside inputParams.
• content parameter in datalink mime type. (issue DataLink #82). This is apparently to be solved in VOTable before DataLink1.1 becomes a REC
• UCD for ID column in {links} service output . Is not the same as the UCD of ID parameter in any SODA service referring to the ID column in the {links} table. Is that an issue ? apparently not (issue DataLink #89)
• interface version attribute in spec example . Should it be #links-1.1 or #links-1.0 (DataLink #96 - see also #62)

New issues from November 2023 onwards

• inputParams could refer to PARAM as alternative to FIELD (https://github.com/ivoa-std/DataLink/issues/115)

I have a suggestion/request from ESDC to allow service descriptor inputParam entries to refer by ID/ref not only to FIELDs in the results table but also to PARAMs. The existing text in sec 4.3:

For services where the parameter value(s) come from the "results" resource, the value attribute is empty (value="") and the PARAM includes a ref attribute to indicate the FIELD (column) that contains the values.

does not explicitly allow this possibility, but given the relationship between FIELDs and PARAMs in general, it seems almost implied.

Although the same effect could be achieved by rearranging information in the VOTable document (use a constant valued-service descriptor instead of a referenced PARAM), I understand that at least for the Gaia DataLink documents under consideration doing it like this would make for more straightforward implementation.

I would support this change (clarification?) for DataLink 1.1 and I am willing to write a PR, or open discussion on the mailing list, or add a note on the RFC page, on request. But the editors might consider it too late in the DL1.1 process to include this change to the text. @pdowler what do you think?

-- MarkTaylor - 2023-11-20

On second thoughts introducing this could raise backward compatibility issues: if a DataLink 1.0 client encountered a DataLink 1.1 service descriptor with a PARAM reference it would likely fail to understand it. So maybe not such a good idea at a minor revision. But still interested if other people have thoughts about it.

-- MarkTaylor - 2023-11-21

I could probably live with the backwards compatibility woes; I'd be curious how many of these 1.0 clients are around in the first place, and how many of those actually implement the full trickery of 1.0 descriptors correctly. I'd volunteer on doing an impact assessment for pyvo if there's interest in that kind of thing.

But frankly, I'm not so overly happy to add even more variability to a standard that already confuses many with its many options. If all that's itching ESA is the repetition of a PARAM -- which is machine-generated anyway, so repetition is basically free --, I'd say it's not worth it, in particular because we'd be introducing an id/ref pair where we previously had an immediate value. And id/ref is about the most expensive thing we can write into a standard.

-- MarkusDemleitner - 2023-11-21

The reason I think this sounds sensible is that conceptually a PARAM is just a non-varying FIELD, so you'd expect PARAM references to work in the same way that a FIELD references do - so perhaps if the original text had been written more thoughtfully it would have said "FIELD or PARAM" where it now says "FIELD". From that point of view this is more like a clarification than adding a new feature.

But backward compatibility issues would apply at least to existing versions of topcat - topcat currently understands service descriptors that reference FIELDs but not PARAMs. And the same may be true of other clients, PyVO included or not.

-- MarkTaylor - 2023-11-21

I think I understand the need and rationale. This looks like a minor change and tend to follow Mark there. From the client side it just a minor change to apply .

Do you think that an erratum would be more adapted ?

-- FrancoisBonnarel - 2023-11-24