VO-DML V1.0 Proposed Recommendation: Request for Comments

VO-DML describes a standard modelling language, or meta-model for expressing data models in the IVOA.

For historical reference, the development twiki page can be found here.

Latest version of VO-DML V1.0 can be found at:

• IVOA Documents and Standards page: http://www.ivoa.net/documents/VODML/20180307/VO-DML-PR-v1.0.pdf
• Changes to that document will first be made on the Volute repository: https://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/doc/VO-DML-PR-v1.0.pdf and will be uploaded to the documents page when completed.

A basic "VO-DML: Beginner's Guide" has been generated to assist data modelers get acquainted with the procedures involved with generating VO-DML compliant models.

More tools and examples can be found in the tooling page

Reference Interoperable Implementations

The following is sourced from the development page description of implementations.

The normative part of the specification defines an XML schema representation for Data Models. Strictly speaking, a reference implementation supports reading or writing of XML documents written according to the schema, covering the entire schema definition, or provides validators for XML instances.

XML documents describing hypothetical or actual data models using all of the specification features also seem to qualify as Reference Implementations.

In a more general sense, however, implementations may map the meta-model defined by VODML to other meta-models, using the XML schema as an exchange format between custom and standard representations of the meta-model and its instances. In this sense the list below includes software implementations or graphical representations that can be used to read or write data model instances in their standard representations or to create new products from them.

In any case, all the implementations below can be used as a "golden standard reference" for the different implementations of the specification.

DatasetMetadata data model

Current document is located in volute.

The model was defined by following VODML prescriptions and has a standard representation as a VODML XML file exercising all features of the VODML meta-model. The model validates against both the schema and the schematron presented above . A browseable representation of the model is also available. The implementation demonstrates that the VODML meta-model is sufficient to express a complex real life data model, and that the VODML schema can unambiguously represent such a data model in a standardized fashion.

The model was developed by the DM working group with Mark Cresitello-Dittmar (SAO) as editor.

STC2 data model

This effort is still in progress, however model diagrams are hosted on volute. The model is being developed following VODML prescriptions and has a standard representation as a VODML XML file exercising all features of the VODML meta-model. The model validates against both the schema and the schematron presented above, however it makes use of a discouraged feature (attributes of arbitrary multiplicity) which produces a warning in the validation report. A browseable HTML representation of the model is also available. The implementation, although in progress, demonstrates that the VODML meta-model can express highly complex data models, and that the VODML schema can unambiguously represent such a data model in a standardized fashion.

The model was developed by the Arnold Rots (SAO) with contributions from other WG members, and it is currently in the WG discussion phase.

HTML generator

Valid VODML representations can be used to produce, automatically or semi-automatically, a number of documentation products. One example is the HTML generator included in the VODML tools suite.

The tool takes a standard model definition as input and produces a browseable, interactive web page describing the model, with cross-references, figures, and tables of contents. A UML diagram is also automatically generated if the required libraries and applications are installed (some examples have been provided in the above sections for DatasetMetadata and STC2 data models).

The implementation shows that the VODML language is expressive enough to allow the automation of tasks related to the data modeling effort.

The implementation was developed by Gerard Lemson (GaVO and JHU) with contributions from WG members using different UML modelers.

UML translation scripts

Data modeling can sometimes start with a UML modeler tool. Although the XMI format was designed as a standardized exchange format for meta-model descriptions, including the serialization of UML models, the standard is complex and the level of interoperability is far from perfect. In order to allow translation from XMI files produces by widely used UML modelers to standard (and much simpler) VODML model descriptions, a set of translation scripts were included in the aforementioned VODML toolset. This implementation shows that different meta-models can be mapped to VODML and enable interoperability with other standards. It also shows the simplicity of VODML when compared to other similar solutions with a much wider scope.

The implementation was developed by Gerard Lemson (GaVO and JHU).

Jovial software library

The jovial library and toolset is an ongoing effort to provide a supporting toolbox to IVOA data modelers. At its core it provides a full implementation of the VODML meta-model and schema. It is developed in Groovy but it is compatible with Java.

The library defines a Domain Specific Language for expressing data models and instances of data models, and allows to read standard VODML descriptions into Java objects, as well as to serialize Java/Groovy in-memory representations or DSL descriptions to VODML documents. A data model browser GUI is also currently under development.

The implementation shows that software libraries and applications can be easily implemented against the VODML meta-model employing widely used programming languages. One of the applications of the library is to provide a format-agnostic language to express instances of data models that can be serialized in a number of formats, so far VOTable using the ongoing VODML Mapping in VOTable WD.

The library was used to produce reference DatasetMetadata serializations starting from VODML XML descriptions of the participating Data Models (DatasetMetadata, ivoa, STC2).

The implementation was developed by Omar Laurino (SAO).

cadc-vodml validator

I have implemented a very basic VO-DML/XML validator that performs schema validation and ISO schematron validation.

• source code: https://github.com/opencadc/core/tree/master/cadc-vodml
• binaries of the library are available in the jcenter maven repository: org.opencadc:cadc-vodml:[1.0,)
• binaries of the command-line validator: http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/software/ (available in various package formats)

I have created a VO-DML/XML description of the CAOM-2.2 data model (in production use at CADC and MAST) that passes both schema and schematron validation. Comments and issues I encountered when doing this are provided in more detail below. This does import the base ivoa data model for some primitive type definitions.

• source: https://github.com/opencadc/caom2/tree/master/caom2-dm

-- PatrickDowler - 2017-01-20

Interoperability of Reference Implementations

It is important to show that reference implementations are actually interoperable, as interoperability is the key requirement for IVOA specifications.

A number of implementations were developed using different technologies and languages in a complex network of interactions. One interesting demonstration of interoperability is the pipeline that leads from abstract UML models in a modeler desktop application to the serialization of complex data model elements to VOTable. Although the serialization to VOTable, as well as support for UML, are not parts of the normative part of the specification, interoperability should be clearly demonstrated:

* Data Models (e.g. DatasetMetadata, STC2 and ivoa basic types) were defined as UML models following VODML prescriptions using Modelio and MagicDraw, two widely used modeling desktop applications.

* DatasetMetadata includes the ivoa model

* Standardized descriptions of DatasetMetadata and ivoa models were produced by translation scripts from the XMI representation of the Model in the application-specific XMI flavor. XML descriptions refer to each other (DatasetMetadata imports ivoa and STC2).

* The standardized descriptions can be loaded in a Groovy software library to facilitate users in defining instances of the Data Models and serializing them, e.g. to VOTable. The application can also be used to define Models and save them in standardized VODML XML format to be used by other Models or applications.

* The standardized descriptions can also be used to produce human readable documentation in a common format, i.e. HTML documents with clickable cross-references.

Participants in the pipeline used different operating systems, programming languages, and desktop applications.

Implementations Validators

A Reference Implementation for validators was developed by means of eXtensible Style Sheet Language Transformation scripts, schema validation, and schematron.

The script is available on volute as part of a bigger suite of implementations, also described in this list. Some documentation is available.

The validator takes care of verifying that instances are compliant with the schema. Also, it checks that instances comply with rules that are harder or impossible to descibe in a schema, by means of a schematron ruleset.

The validator was developed by Gerard Lemson (GaVO and JHU).

cadc-vodml validator

See also the cadc-vodml implementation above. -- PatrickDowler - 2017-01-20

RFC Review Period: 12-Oct-2016 - 23-Nov-2016

Comments from the IVOA Community during RFC period: 12-Oct-2016 - 23-Nov-2016

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

• Sample comment by WikiName
  • Response (by WikiName)

Comments by Markus Demleitner

First, some general remarks.

It is my personal conviction that the specification could be greatly improved and clarified if all references to VO-UML were removed. Sure, it would be a major change, and you'd lose a quasi-standard graphical representation, but I'd consider that a minor loss when weighed against the gains in readability and conciseness. Also, to me it's more confusing than helpful that both the schema and instance documents are presented as if they were representations of the same thing rather than specification and instance. I think the exposition would be much clearer if you put current "VO-DML/XML" sections first and simply labled them "example" and then had the current "VO-DML/Schema" sections as " Schema Definition".

Reply [GL]:

---

- Use of VO-UML: I think Markus' opinion should be compared to those of others. I personally think it helps to have a graphical representation, maybe the more so since we have done work on UML->VO-DML translators. It also tries to bring about the connection to UML which clearly was a major influence on VO-DML. But maybe it does not belong in a standards document? But if most (serious) readers agree with him we may need to do something about it. This question will be posed to mailing list. Note, other co-authors preferred to keep VO-UML in document as well.

- Schema and instance snippets/examples not well distinguished: Similar useful point of discussion. In my opinion this document is really a (hopefully) human readable documentation/explanation of the normative XML schema+schematron, and those should be read side-by-side with the text. But if this document is confusing clearly updates need to be made.

Actions: I have made an attempt to improve the flow in the volute version of the document and make it distinction between XSD and XML snippets more explicit using different background color.:
First: describe element.
Second: VO-UML
Third: VO-DML/XSD snippet (in separate background color, grey)
Fourth: VO-DML/XML example (in separate background color, beige)
Fifth: describe sub-elements in text.

---

If I'm altogether honest, I think I'd like it best if VO-DML wasn't specified in XML Schema but in VO-DML itself (I seem to remember there is such a model in reality). But disregard this if it's a major complication.

Reply [GL]:

---

Though it may be possible to create a VO-DML-like representation of VO-DML, I don't think this would be sufficient for this specification. Main reason is that VO-DML is NOT a data model but a language for defining data models. VO-DML/XML is the machine readable serialization of VO-DML models. VO-DML has otherwise no formal way of expressing what that serializations should be, unless it was all in text. So the goal of the xml-schema+schematron docs is to formally define the machine executable structure of valid VO-DML/XML documents. I think this is very useful as the schema(tron)-based validator proves. "VO-DML without XML" does not do that. It is not executable.

On a more technical note, since VO-DML is an application specific language, not a data model, certain restrictions that may make sense in the latter context, do not, or can be avoided in the other. For example VO-DML/XML allows xsd:Attribute-s to be contained both by xsd:ObjectType and xsd:DataType, even though for VO-DML data models we forbid this.

Finally, the diagrams in Figs 1-3 are closest representation of VO-DML in VO-DML (VO-UML actually).

---

In a similar vein, I think the document structure is broken as long as sub-elements of VO-DML elements are introduced before the schema, (UML if you must), and example fragments, as those then logically belong to the subsubsection of the last sub-element. For instance, in 4.13, the material about role appears to belong to sub-section 4.13.2 Multiplicity, which they don't. To fix this, I'd present the formal material before discussing the the sub-elements. I don't think comprehensibility would suffer.

Reply [GL]:

---

Current update on volute follows MDs suggestion. Easy enough to turn back.
Action: pose question about this to mailing list.

---

Then, some more specific issues:

Reply [GL] :

---

Where no comment is added I have tried fixing these as much as possible, needs a second pair of eyes to check.
Some get their own comment, some explicitly left OPEN, either requiring a bit more work or discussions on mailing list.

---

(1) Abstract: "VO-DML is restricted... it only" -- I think there's a "since" or something like this missing.

(2) p. 10: "...point at was left unspecified, this..." -- I think the comma should be a full stop.

(3) sect. 3.1 has links into volute; at least as long as volute is not formally part of the VO infrastructure, I maintain these should be replaced by links into the document repository, at least now that this is PR.

(4) sect. 3.3, "...language can and be interpreted as a subset..." -- I guess the "and" should go.

(5) Footnote 21: "...as VO-DML does not allows multiple inheritance,..." -- "allows" must be "allow"

(6) p. 15 "...is shown in Figure 3 and Figure 3...." -- some accident has happened there. I suppose the first should be "Figure 2"?

(7) 4.1.1 has backslashes after VO-DML that should probably be forward slashes.

(8) 4.1.1 I am not sure the regular expression matches what is actually intended. According to F.1.1 of "XML Schema Part 2 (at least in the version of 28 Oct 2004), \w means, essentially, "All characters except punctuation, separator, and other". That's in contrast to what it means in most libraries I'm aware of, and I doubt we actually want the character 1f4a9 in vodml-ids. Hence, though perhaps this is western-bigoted, I think we should agree upon [A-Za-z._-]+ (perhaps adding digits -- also: do we want leading digits and dashes?) Similarly in 4.1.2 and some other places. For reference, XML 1.0 allows for id (which is relevant since vodml-ids should be used for HTML anchors by 4.5.4) (Letter | '_' | ':') (NameChar)*, where NameChar is Letter | Digit | '.' | '-' | '_' | ':' | something; Letter there includes a bunch of non-ASCII letters, and something is a smaller bunch of other, excuse me, weird stuff. But that set is still, I think, a good deal smaller than XML schema's \w (I haven't really checked, though, but let's keep things such that one doesn't need to).

[GL] : I agree with MD and in volute version have updated regexes.
For vodml-id: [a-zA-Z][a-zA-Z0-9_\.]*
For vodml-ref: [a-zA-Z][a-zA-Z0-9_\-]*:[a-zA-Z][a-zA-Z0-9_\.]*
For name: [a-zA-Z][a-zA-Z0-9_]*
Also updated the XSD on volute.

(9) p. 19, footnote 25: I'm not sure how this relates to 4.5 -- doesn't 4.5 define the prefix binding? Incidentally, what is the rationale for requiring prefixes in intra-model references? My instict would be that overall implementation is simpler if these are prefix-less.

[GL] : this footnote was removed, see also comments from Pierre below Re TBDs etc.

(10) p. 25: Is Fig. 7 actually something necessary or even merely helpful in a specification? Ditto Fig. 8.

[GL]: Agreed these figures were not really VO-UML and have been removed. They were too specific for MagicDraw as well.

(11) Sect. 4.5: recursive model imports sound scary, and not only for reasons of managing standards development. You're positive we cannot outlaw them?

[GL]: Don't really know what you mean by "outlawing", but also regarding a comment from Pierre the text has been rewritten. It still intends to say that only those models must be imported for which element are explicitly used in the model. But those models MUST then als be imported. I.e. it is not sufficient to import a model that already imports the other model. That would be recursive (or as Pierre says, transitive). So, recursive/transitive import is NOT supported.

(12) Sect. 4.11: As important as the distinction between DataTypes and ObjectTypes may be, I feel explaining it again here becomes tedious. I'd all be for concentrating that discussion in one place (presumably in the introduction) and move the prose both from here and from the opening paragraphs of 4.6 there (with some streamlining, of course).
[GL]: accepted, still to be changed in the document on volute.

(13) Sect. 4.12: "ObjectType is meant for representing such kind of objects." -- this almost looks like a tautology, if only because object is such a polysemic term. If you actually feel the need to stress this statement again, I suggest to use some other term, "data model elements" perhaps. The following sentence ("Those for which their existence...") I think should go in the spirit of (12).

[GL]: OPEN accepted. spirit of (12) stlll to be changed in dthe ocument on volute.

(14) 4.12.2: "And finally a certain ObjectType can only be contained in one parent ObjectType." and following. I think the discussion should be concentrated in one of 4.12.2 or 4.17, and then one should just reference the other.

[GL]: OPEN agreed, still to be changed in document on volute

(15) 4.19.2, section title: this indicates a range of 0..1, whereas it really is -1 to infinity, no?

[GL]: rewritten, allowed values are indeed -1 - infinity. specifying -1 indicates infinity. If not specified the default value of 1 is intended.

(16) 4.21 SubsettedRole extends Constraint, but Constraint may, by the current documentation, grow some formal way of specifying constraints in a future version. How would that interact with the constraints given by SubsettedRole? Wouldn't it make much more sense to state in 4.20 that future subclasses may add such formally specified additional constraints? Frankly, given the TBD at the end of 4.21 which indicates there's no DM actually using SubsettedElement, I think I'd vote for throwing out SubsettedRole entirely.

[GL]: Added sentence that maybe sub-types will be used for specializations towards OCL for example. There are actually many examples of subsetted role in data models. TBD removed, as i referred only to sample model. We have added an example now.

(17) Sect. 5.2: Isn't that material for the accompanying note?

[GL]: This is section 6.2 in the new version of the document on volute. OPEN possibly, leave open for now

Comments by Pierre Fernique

1) This document implies that all IVOA data models (past and future) will have to be compliant to the VODML specifications, ie provides a VODML/XML valid representation. Concretely, all DMs are/will be built as a "hierarchy" of DMs, using at the bottom of this hierarchy the DM of primitive objects: the "IVOA model" (page 16). Unfortunately, this "IVOA model" is not correctly defined in the document. It is briefly evocated in non normative appendice E, and this appendice points to an external XML document (https://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/models/ivoa/IVOA.vo-dml.xml. - version 15/7/2016 - take care that the internal PDF link is wrong and still point to the old Google site and not GAVO site). This XML document does not correspond to the DM described in the appendice. It misses several primitive object definitions: "duration", "decimal", "UCD" (cited in the PR's appendice), + "AtomicValue", "BooleanValue", and "StringValue" which are displayed in the schema of appendice E.

1. a) It is particularly annoying in the sense that this IVOA basis model defines the notion of "Quantity" which seems to be important for the future VOTable mapping as it is explained page 54 "mapping of scientific measurement values directly to FIELD-S and PARAM-s in a VOTable" but without more details.

1.b) Also, in this "IVOA model" some primitive types are a little bit surprising. Why these primitives types: "complex", "rational", "dateTime", "anyUri", "duration", "decimal", and not others. Why simultaneously "decimal" and "real" ? It seems quite arbitrary and not consistent, and at the least, bad defined (ex; "dateTime": "Represents a moment in time using a date+timestamp"... what is the format, the reference, the syntax - ISO, JD, MJD ?..

So I think that this "IVOA model", basis of all the VO-DML structure has to be clearly standardized, not in an appendice, but directly in the normative part of the document in order to be sure that this part of the work will be consistent and well documented.

Reply [GL]:
--------------------------------------------------------------------
It is correct that the IVOA base model is important and we agree with your suggestion to make it part of the normative specification. In the version of the doc on volute we have therefore moved it from the appendix to the (normative) chapter 5. It has also been synchronzed wth the actual model on volute, which has already been stripped of some of the more arbitrary types. Note, this model is supposed to contain some "pure" primitives, not implementation specific ones. The two quantities were explicitly introduced so that values-with-unit could be simply mapped to VOTable FIELDs or PARAMs, without requiring a separate UNIT mapping.

The IVOA model, in all its relevant representations, should be added to the Documents library on the IVOA site. This is TODO
--------------------------------------------------------------------

2) The requirement 9 described in page 10 - "Support runtime model interpretation" is certainly difficult to implement based on the current "IVOA model" due to the fact that the notion of "coding" is not described in VODML (integer => byte ? short ? int ? long ? / string => ASCII, UTF8 ? / ...). Concretely, a binary serialization or a automatic DB generation from a VO-DML model will be not able to map the objects in their adapted binary coding . I suggest to remove this requirement, or adapt the IVOA model according to this requirement.

Reply [GL]:
--------------------------------------------------------------------
The requirement nr 9 is meant to be implemented using a machine readable representation of the model and as such cannot be removed. It should allow machines to be able to read the model and possibly use it in interpreting annotated representations.

It is not intended to enforce a standardized mapping to runtime models. When someone wants to implement the model directly in code, as for example is done from vo-dml->Javain the XSLT script in https://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/xslt/vo-dml2pojo.xsl impementers must make a choice how to perform this mapping. SOme may chose real->float, others real->double etc. But this is beyond the scope of this particular specification. And even the mapping specification allows flexibility and does not mandate specific mappings.

---

3) The Constraint definition (page 50) uses a non normative syntax directly mapped in the description field (ex: "SkyCoordinate {-90<=self.latitude.value<=90, 0<=self.longitude.value<360}"). It is clear that this part of the VO-DML standard will be ambiguous. I suggest to provide a well defined constraint syntax, or remove this Constraint paragraph.

Reply [GL]:
---------------------------------------------------------------------------
A formal constraint language was seen as beyond the scope of this specification. Nevertheles it was felt useful to have the option to provided an explicit element for expressing constraints beyond the mere description elements. Currently this will have to be read by humans and can be used in their implementations of a model. It also serves a purpose as a super-type for the SubsettedRole type. The example in the sample model tries to be a bit formal (OCL-like), but this is not mandatory. For now it is up to model developers to use a syntax (if any) that they think may resonate with their intended target audience.
---------------------------------------------------------------------------

4) The notion of "app-info" (page 54) is defined as required (MUST), but this notion is not described in the document. So I suggest to describe the notion of "app-info", or remove this sentence.

Reply [GL]:
-----------------------------------------------
I have rewritten this part, using app-info s a possible way to annotate a formalized mapping to XML-schema, not as a MUST.
-----------------------------------------------

5) Some paragraphes are still in "TBD" status (page 19 - note 25 and page 53). Should be defined before REC.

Reply [GL]:
-------------------------------------------------------

I have removed most TBDs, only one is left, referring to the fact that the proper location of the IVOA base model should be found and then the text will be updated accordingly.
-----------------------------------------------

6) Some URL cited in the document are wrong. The text of the anchors is correct, but the internal links point to the old Google site (page 54, page 63, ...)
Reply [GL]: I have tried fixing them all.

7) The term "recursivity" page 27 may be replaced by "transitivity" which seems more appropriate.

Reply [GL]: Agreed, I have updated the text, see also comment on one of MDs comments above.

-- PierreFernique - 2016-10-19

Comment by Arnold Rots

I discovered a somewhat obscure issue last week regarding the multiplicity of attributes.

Currently the document allows two possible value types for attribute multiplicity:
[n] where n is a positive integer literal (1, 2, 3, ...)
[0..1] meaning an optional single value, equivalent to [0],[1] (though [0..1] is the required expression in VODML).

The multiplicity of attributes has been the subject of much (more or less heated) debate in the past and I had been left with the impression that a compromise had been reached requiring the multiplicity to be a specific number (i.e., a positive integer literal), but allowing the attribute to be optional regardless of the value of n.
In other words, both [n] and [0],[n] would be allowed (I use the expression [0],[n] in a logical sense, not intending to pretend that this is how it would be expressed in VODML).
The current text of the document is asymmetric, in that it allows optional for single attributes, but not for arrays of attributes. I think there is value in allowing arrays to be optional, too, while retaining the ability to distinguish between a required and an optional attribute, scalar or vector.

Reply [GL] : Actually it is worse, the text on multiplicity in the section on Attribute is not consistent with the text on attributes under Multiplicity. In the latter we allow, but strongly discourage open ended multiplicities. We do not discuss 0..n, but I propose we allow that and mean it to imply attributes that are either an n-dimensional arry, or null. I wil updatre both the description under Multiplicity (allowing and interpreting 0..n), and under Attribute (referring to the Multiplicity section). This will be added for now on volute, and that copy will asap be uploaded to the IVOA document library.
I will also update the schematron rules dealing with the attribute multiplicity to conform to the updated text.

---

---

Late comments by Patrick Dowler

These comments come from trying to create a valid VO-DML/XML description of the CAOM-2.2 data model. I should note that I wrote it by hand and overall this was successful. I did not resort to the specification document very much except for some basic concepts and examples, so I feel the spec itself could be much much smaller than it is. Here are the things I ran into. I should also note that many of the data models on the volute site are clearly not consistent (have not been kept up to date) with the PR.

Reply [GL] : Indeed, we have only tried keeping the sample models used in the spec up to date as well as the models that Arnold and Mark are actively working on. I'd hope more models will follow soon. As to the length of the specification document itself, not everyone may be familiar with data modeling concepts, hence we tried being explicit and exhaustive. It also has many examples, snippets from the XSD etc. If there are specific suggestions how to shorten it, that might help.

1. multiplicity: several examples have content like 1 and others have minOccurs and maxOccurs; the xsd requires minOccurs and maxOccurs.

Reply [GL] : Will go through document and fix this. I asusme the examples you refer to are XML examples, not UML.

2. multiplicity: in 4.17 the UML composition has multiplicity 0..* is very typical construct. In VO-DML/XML this is maxOccurs -1 (because integer) but special values like that are really counter-intuitive and require all downstream code that might process the model description to do something carefully. Why not just make maxOccurs optional in the xsd? The concept/terminology clearly came from XMLSchema which does allow this and with the obvious meaning.

Reply [GL] : [Note, while commenting on this, the frequent use of star symbol sometimes causes the text on the page to become bold-faced in certain lines. I may not yet have found out how to escape this.] UML editors in general allow selecting from a default list of possible multiplicities, which in general uses '*'. These can be hand edited, for example to produce 0..3, but that is the main reason why the sample UML diagrams use '*'. Indeed -1 is the value in VO-DML/XML. Some history, originally the Multiplicity was an enumeration with values 0..1,1,0..* and 1..*. This is not flexible enough. XMI has lowerValue/upperValue, XMl schema has minOccurs/maxOccurs; the latter terminolgy is used in VO-DML as it is likely more familiar. in VO-DML -1 was chosen so we can model all multiplicites as integers and because XMI (at least the XMI producd by my version of MagicDraw) uses -1. Other tools do indeed use '*'. Interestingly, when in MagicDraw I type '0..-1', it is automatically changed to '0..*', though still saved with upperValue=-1.
XML Schema has 1 as default value for minOccurs and maxOccurs, not 'unbounded'. Clearly the spec can be changed, though it is arguable which value is to be preferred:'*', 'unbounded', or -1 or indeed no-value all have to be dealt with explicitly in code. To me the fact that -1 is an integer is useful. I suggest to have this discussed on the mailing list.

3. enumeration: named constants with numeric values are not valid vs the xsd (eg. caom:CalibrationLevel use an enumeration for constants but values are from ObsCore calibLevel). How is one supposed to define such things?

Reply [GL] : The relation of a model enumeration literal and the possible occurrence of the same literal concept somewhere else should be made clear in the literal's description element. To create a model for all the possible alterative representations is beyond the scope of the current spec. See also next point.

4. enumeration: named constants with arbitrary string values (eg X-ray from VOResource) are not valid literal names. The literal name seems to be intended as an identifier, so how is one supposed to specify the serialised value? I agree that arbitrary strings (eg with whitespace) may be going too far with enumerations... maybe just the syntax needs to be relaxed a little more to allow additional special characters (if the literal name is intended to be the value).

Reply [GL] : We had a discusion about this on the mailing list (TBD find reference). It was agreed that for serializations, say in VOTable, people will likely have to provide an explicit mapping between the model values and the values in the serialization. MArkus made the case (and I agree with him) that in many concrete cases where one is tempted to use an Enumeration a semantic vocabulary should be used instead.

5. primitiveType vs dataType vs objectType clarity: I had a hard time figuring out when to use these three constructs and nothing jumped out at me in the document. Maybe I was just looking for immediate gratification and it is there, but this seems lie it has to be front-and-centre and very clear.

Reply [GL] : Noted, will try to improve text.

6. Schematron validation can only work for an xslt-based pipeline; an xpath-based validator fails because the vo-dml.sch.xml makes use of xslt-specific functions and specifies queryBinding="xslt2". Removing the latter doesn't fix it because of the functions. I am not sure if this is intended or just side effect of the long development path, but at a minimum the document should make this limitation clear somewhere. Note: cadc-vodml validator uses ph-schematron library, which implements both XSLT and XPath validation.

Reply [GL] : Schematron validation was implemented on volute using an XSLT based validator from https://github.com/Schematron/stf/tree/master/iso-schematron-xslt2. Would be good to get the schematron file to work in other cases as well I suppose. Note that we do rather complex validation, for example we check whether types in imported data models exist and have the right 'type' in the remote model document. If all of this can be done using a more generically used implementation it would be interesting to see that.

7. A dataType with an attribute with maxOccurs > 1 causes a schematron warning that this is strongly discouraged (eg. dataType Polygon contains 1..* of dataType
Point and Point is a dataType because Circle also contains 1). I know there has been extensive discussion about multiplicity but it is simply a fact of life that real data models have this kind of thing all the time.

Reply [GL] : Not only Schematron gives a warning, also in the document it is noted that one should try to avoid this construct. Lots of real data models do not have this construct for example because ithey try to conform to first normal form. As we hope that data models can be used to annotate relational tables, say TAP schemas, putting this warning was deemed acceptable (I personally would have preferred to not alow attributes (or references) to have >1 multiplicity).

8. The vo-dml.xsd has default abstract="false" on objectType; XML parsers only inject those defaults when schema validation is enabled so it tends to lead people to write code that assume this is set and then fails when schema validation is turned off... and the IVOA note on XML schema usage recommends clients not do schema validation for compatibility reasons: not a fan XSD defaults. Just make abstract a normal optional attribute.

Reply [GL] : I assume you are referring to the default="false" decaration on the boolean attribute 'abstract' on 'Type'. Is that correct? I can remove it. Is it understood i XSD that missing boolean value implies false?
On a side note, the schema versioning Note in http://wiki.ivoa.net/twiki/bin/view/IVOA/XMLVersRFC seems to argue i favour of schema-based validation.

-- PatrickDowler - 2017-01-20

9. The normal meaning of "collection" implies multiple values, where the meaning in VODML is really "composition". This concept is called Composition in Fig 1. Can the element name in VODML/XML be changed to composition for consistency/clarity?

-- PatrickDowler - 2017-01-24

Notice by Mark Cresitello-Dittmar

I note that the schmatron implementation failed to catch instances where an element has 2 attributes with the same name. This was found while exercising the caom2 vo-dml/xml file through the validation and html generation scripts.

a: Axis datatype with ctype and cunit attributes both have name 'ctype', but different vodml-id assignments.

b: Point datatype with cval1 and cval2 attributes, both with name 'cval1', but different vodml-id assignments.

The schematron should be updated to catch these cases.

More late comments by Markus Demleitner

While implementing what I figure the new mapping spec will be to be I noticed that with it the types in the ivoa: DM that is part of the VO-DML spec become actually relevant outside of the lofty heights of drawing diagrams: VOTable writers will have to infer them for writing LITERALs.

Only then did I realise that ivoa: really defines another type system. Now, DaCHS already translates between a gazillion type systems (among others, postgres, python, TAP, VOTable, FITS, numpy, XSD). Based on this, you could say it doesn't really matter.

But another type system still is another source of bugs, impedance mismatch ("oh boy, what do I translate rational into?"), and annoyance for our users ("What, LITERAL has a dmtype attribute that's using something that has nothing to do with PARAM's @type?"). Is it really, really, really necessary that ivoa: does not just re-use the VOTable type system?

Since a thorough investigation of this matter might slow down the RFC: How difficult would it be to pull the ivoa: DM out of VO-DML and have a document of its own for it? That'd also give us a bit more time to think about Quantity; also, I think our future selves will be grateful if they don't have to re-issue VO-DML itself when they just want to fix ivoa:...

-- MarkusDemleitner - 2017-04-11

Even later comments by Patrick Dowler

Following my presentation at the interop, discussion of enumeration literals continued. I pointed out that one cannot generate or write code or an XSD file from only the vo-dml spec of a model due to the lack of enumeration literals with the actual value: some other input would be necessary. Given the goal of having a normative machine readable model specification, I find the current enumeration feature lacking compared to real usage requirements and I think punting this to serialisation or mapping step leaves a gap.

-- PatrickDowler - 2017-05-17

TCG Review Period: 13/02/2017 - 03/03/2017

Comments from TCG member during the TCG Review Period: 13/02/2017 - 03/03/2017

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

GL: The updates to the document based on the comments below can be follwed in volute (http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/doc/VO-DML-PR-v1.0.docx and http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/doc/VO-DML-PR-v1.0.pdf). When all have been incorporated the document library will be updated).

TCG Chair & Vice Chair ( _Matthew Graham, Pat Dowler )

The top-level model element does not support/allow a version attribute as recomended in the XML Schema Versioning EN. This is of course harmless now but will be somewhat painful when VO-DML-1.1 is introduced.

GL: fixed by Pat.

Applications Working Group ( _Pierre Fernique, Tom Donaldson )

Apps approves. -- TomDonaldson - 2018-05-01

Data Access Layer Working Group ( François Bonnarel, Marco Molinaro )

- We approve this document which is a really usefull work to propose a clean design to define IVOA datamodels to be reused for example in the VO context.

- We have some suggestions and typo-discoveries below, which you can take into account for better understanding we guess.

- *We have one more critical point on ivoa datamodel and we make a non-blocking suggestion below

Reply [GL]: I am going over all suggestiosn and make the fixes. Some questions below where I do not understand the point that is made.

- Page 2 - TYPO - extra "s" in "..use of standards compliant data models..." before Status.

- Page 7 - TYPO - extra stop mark before §1.1

- Page 7 - Appendix D reference: suggest to read it before? Anyway Appendix D has its own issue (see later).

GL: I do not understand what you'd like to see changed.

- Page 6 third goal : the concept of "portable Data Model references" is unclear. Could be replaced by "Data Mode references in pre-existing dataformats". The subsequent paragraph should emphasize that this specific work will be done in another Spec.

GL: in fact it is this specification that specifies the identifiers of data model elements. These are used already in the current spec, for example when referring to elements in imported data models. And indeed they are also used in the mappng specification, but not defined there.

- Page 9 : semantic heterogeneity. This paragraph lack the fact that IVOA made a substantial progress to solve the problem of semantic heterogeneity for quantities by inventing and using the ucd feature (which of course doesn't solve all other aspects of semantic heterogeneity, eg gratia : relative roles)

GL: UCDs are mentioned some paragraphs below. In this cha[ter we are trying to put the IVOA and particularly data modelling effor tin wider context of data integration, in particular use of "global schemas", i.e. more complexly structured than semantic vocabularies. Hence I don't think we need to emphasize here already that the IVOA has other specs in an area that is not quite the focus of this chapter.

- Page 10, last paragraph. I don't think VOTABLE and TAP schema (actuallly vodaservice tablesets) play the same role. The first one is a format used for representation and serializations, while the second one is a structural definition of data representation (in VOTABLE or whatever othre tabular format). What was the intent ?

GL: rewritten this a bit to make clear we're focusing here on the metadata annotations, element sin VOTable, TAP_SCHEMA in TAP.

- Page 12, second paragraph. Usage of word "serialization" here is confusing because for mots of the people serialization of a model is actually creating a set of instances of classes of the model. Here it should be better to speak of a model definition language

GL: ok

- Page 13 - TYPO - missing stop mark after "These files implement all the concepts described in section 4?

- Page 14 - TYPO? - second paragraph of §3.3 "can and be interpreted"?

- Page 16 : typo "based on an dependency hierarchy" "based on a dependency hierarchy"

- Page 21: in "ElementRef" section. A small "VO-DML/XML" example at this place should be interesting for immediate understanding (just after VO-DML/Schema). Examples come in later, but first approach to reading the spec is quite steep in learning curve.

- Page 21 : "A package has separate collections for each of the type classes". Usage of the word "collection" here is confusing for us. Is it "modelling" vocabulary ? A note or explanation could be usefull ?

GL: Thanks, I have changed the text to "A package uses explicit element names for each of the type classes: <objectType>, <datatype>,<primitiveType> and <enumeration>. " Hope that is clearer?

- Page 23 - TYPO - (minimal), there's an extra space in the text value in the example on top.

- Page 25/26 - We don't think IVOA DM WG should be a naming authority, you are using the ivoa.net, which is the full IVOA authority, that's enough, DM should only moderate/evaluate the process.

GL: I don't know exactly what you mean here. Maybe this? We have had lots of discussions in the original tiger team and beyond about the way model names should be treated, especially as prefixes. The agreement was the uniquenes of the modle name for models registered in the IVOA, and that in vodml-ref that exact name should be used as prefix. This in contrast to the explicit xmlns-prefix declarations one has for example in XML/XML schema. Do you have a suggestion for required changes in the text here?

- Page 27 : last paragraph of model import. Why are these features not part part of VO-DML language? I find this very confusing. If they are not in the language why representing them ? Extra note may be needed ?

GL: I assume you are referring to the VO-UML paragraph? We accept that UML tools will (and maybe SHOULD?) be used for data modelling, but are not mandating any particular way in which to use any particular tool. In a sense we are treating VO-UML as a "whiteboard" modelling language, a way to illustrate models. And find it useful as illustrations (particularyy for people familiar with UML) to include these illustrations. But the mandatory/normative part is at the VO-DML/XML level. In particular for modelimport for example the three tools we have used and for which XSLT scripts exists (MagicDra CE12.1, Modelio, Umodel) the pattern is different how to represent model import.

- Page 32 : 4.6.2. SubsettedRole is actually inherited from Constraint which is not said there.

GL: it is said in the actual definition of SubsettedRole, but I have added a few words here already.

- Page 37 - §4.11.1 "They are like columns in a table, simple elements in XML.." why not simple elements or attributes in XML? (really minor)

GL: captured if you wish in the "etc.". Particularly because I personally always aim to use elements when mapping DM attributes to XML. Allows for easier extension to structured data types and common treatment of all attributes, whether simple or complex.

- Page 40 - §4.13 First sentence, the (type "source") parenthetic should be like (call it of type "source") for homogeneity in the sentence.

GL: , done.

- Page 40 - "VO-DML/Schema: should there be a ":" at the end of the first line of text?

GL: I don't think there should be one.

- Page 42 : add "below" for immediate clarity after "but it may be more general, see the definition of the SemanticConcept definition." in 4.14.1

GL: since I give an explicit hyperlink I try to avoid such statements. Particularly because in the (long) cause of writing this things have moved back and forth.

- Page 43 - I would put "Vocabularies in the Virtual Observatory" within quotes.

GL: thanks, I use italics now.

- Page 44 - is there a way to make figure 13 a little larger? It's at the edge of unreadability.

GL: tried it, pease check. problem is the length of the attributes (being URLs)

- Page 46 - TYPO missing "s" in object-s? at the end of the first sentence in §4.17

- Page 46 : 4.17 : "Composition is a special type of Relation that represents the observation that often..." "represents the observation" seems confusing. We would prefer "represents the fact"

GL: thanks

- Page 49 - using -1 (or any negative number) for §4.19 Multiplicity. We think this would be confusing. We know it's difficult to put it into an XML constraint to use only -1, but later (page 51) it is again stated that only -1 is allowed as negative.

GL: agreed, will remove.

- Page 49 - the couple of SHOULD NOT(s) before VO-UML. The explanation is only in the footnotes: maybe it should be moved inside the main text. This is somehow important in my opinion.

GL: ok

- Page 51 - see above for "Can only take values between -1 and infinity"

- Page 51 - TYPO - §4.20 4th line - ... particular_ly_ in ... ?

- Page 51 - TYPO - VO-UML - boolean expression : there's an extra "s"

- Page 53 - TYPO - "form the SSS and 2MASS" -> "from SDSS and 2MASS"

- Page 53 - TYPO - figure 17 caption: TwoMASSSource -> TwoMassSource - and - GenercEllipse -> GenericEllipse

- Page 54 : 4.21.3 "Maybe the super type has not defined a semantic concept for the Role, but the subtype needs that. This attribute allows this assignment. But alse when the Role on the super-type already has a semanticconcept with a topConcept defined on it, the subtype may restrict the values to a narrower concept than that assigned to it on the super-type."

Could be rephrased : "The super type may have defined a semantic concept for the Role or not. This attribute allows either to define the assignement of semantic concept to the subtype in the latter case or to restrict the values to a narrower concept than that assigned to it on the super-type when the role on the supertypê already has a semanticconcept with a topConcept defined on it."

And in any case replace "alse" by "also" on second line.

GL: thanks, improves reading.

- Page 53 : The text says that ivo data model should be used. On the other side this means that it is perfectly possible to create VO-DML consistent data models without using this model.

GL: this depends a bit on the interpretation of SHOULD. I tought that meant to indicate that one needs really good and explicitly stated reasons not to follow the specification. I.e. I would diagree that this implies it is "perfectly possible". Particularly in the context of the IVOA standards process I think such cases SHOULD be treated with great care.

-- FrancoisBonnarel - 2018-05-04

- I'm not really positive on this model and would like this to be rediscussed outside the VO-DML document.

A small independant document could be created for that and reference here.

Two points I don't understand: the list of basic types : where does this list come from. I think something like reusing the VOTABLE datatypes complemented by xtypes would have been better.

GL: The list aims to represent the basic simple datatypes, that moreover occur widely in the IVOA. I strongly disagree that votable datatypes would be useful. Note that data models in vo-dml are explicitly supposed to be serialization agnostic. As explained in the text before 5.1, this allows us to say a certain attribute is a real, without needing to worry whether this should be single or double precision. This we leave to serializations of the data model instances, for example in VOTable.

The quantity type only has the unit attribute. I think most of other VOTABLE FIELDS or PARAMS attributes should be considered here.

GL: Again we have explicitly left this out. Originally we included UCD, but the whole point of a data modelling language is to provide semantics for data structures. It is the annotation with a particular model and for example its attributes that do so, not an extra keyword. Where semantic vocubularies might come in handy in the model, where a data model explicitly leaves some part of the semantics out, it can define explicit attrbutes with <<semanticconcept>>.

This could probably be the source of large discussions. So that's why I would prefer to have this apart.

GL: This has been discussed and the consensus was to not go through a separate effort, This model IS very important to start making models and hence MUST be part of the spec. We do not want to go through a separate standardization effort for it. Future updates of the VO-DML spec as ahole might contain updates to the model. We originally had this text in an Appendix, but moved it into the main text upon request by other commenters.

-- MarcoMoilnaro - 2018-05-04

- So, here we are, with §5 as a whole. I don't like it being [Normative]. Luckily it is just a SHOULD for re-use. I think it swaps the way things should be: from VOTable and other data-typing one should build the datatypes-model base to use in VO-DML, not force it into VO-DML and pretend the existing standards to align forcibly. I also don't understand why use a mathematics based instead of a developers one, given the goal is, at the end, serializing the whole thing. I must say that this section may lead to a lot of confusion, and also represents a model in itself, that should be considered independently of the modeling specification. It also includes some unclear things like the datetime primitive that is roughly put there, the string without the char as opposed to what VOTable does, introduce "quantity" but in really a limited way.

GL: We MUST have a a set of predefined primitive types, as stated in the text. We chose to use an actual data model that should be imported, rather than using an enumerated list of types as for example was used in VO-URP, or how VOTable defines datatypes. Mainly because it is more flexible and extensible. It was decided to make this part of this standard, rather than pushing it off to future work, for neither makes sense without the other.

So, really, this model, with its related appendices, should be moved and replaced by a simple description of the intended means of the general primitive types used within the specification.

- Both FB and MM agree to let this in the current version if it's needed to XMI to VO-DML/XML conversion. But this should be rediscussed rather soon for an upgrade

GL: that is up to the IVOA to decide indeed. Thanks for accepting the doc as is though. As to your "...given the goal is, at the end, serializing the whole thing...", it is NOT intended that serializations MUST directly, 1-1, conform to the VP-DML data model. Hence our mapping document that indicates how serializations in VOTable or TAP can be annotated to give meaning to those data sets. It is there that mappings to primitive types are to be indicated.

Note, the need for primitive types has nothing to do with XMI to VO-DML/XML conversion, which explicitly is not part of the normative standard. It is required to make sense of VO-DML/XML data models.

- Page 58 - paragraph after the bullet list - I suppose the idea could be also that of an Endorsed Note rather than always a REC, unless it created issue with StandardsRegExt.

GL: Maybe?

- Page 62 - TYPO - third line, I think "where appropriate" should go within commas.

- Page 63/64 - text justify (styling)

- Page 64, typo : "that for many first-time users it hard enough to" ---> "...It IS hard" I guess

- Page 65 - bottom: move the table header (or the full B.2) to the next page

- Page 67 - Appendix D: the link goes only to the beginners' guide, maybe the twiki topic has to be re-checked

GL: that wiki page was chosen as endpoint. But indeed that page MUST get links to the data models. Thanks.

- *Generic statement *: A glossary of used UML terms "stereotype", Model, DatatType .. could be usefully added as a new appendix or in appendix A

GL: I think this is out of scope. Instead we provide links to a site with more explanation.

-- FrancoisBonnarel - 2018-05-04 -- MarcoMolinaro - 2018-05-04

Reply [GL]: Thanks a lot for your comprehensive comments, I hope they have been answered to your satisfaction.

Data Model Working Group ( _Mark Cresitello-Dittmar, Laurent Michel )

2018-04-01: Typo-s

• pg 1-2: 'identification mechanism for model which allows these to be referenced in' => 'models which allow'
• pg 2: ' in particular from othe IVOA standard formats' => 'other'
• pg 13: 'the concepts described in section 4 The schma' => 'section 4. The'
• pg 20: 'inside a VO-DML\XML data model' => 'VO-DML/XML'
• pg 26: 'the current model was mad.' => 'made.'
• pg 51: 'interrelationships or the values of attributes may take' => 'values attributes'
• pg 51: 'specialized support, particular in' => 'particularly in'
• pg 51: 'boolean expresions, in some' => 'expression'
• pg 54: 'But alse when the' => 'also'
• pg 58: cite to StandardsRegExt is not resolved
• pg 59: 'only two artefacts have' => 'artifacts'

Reply* *[GL]: I think I fixed all of these now, thanks.

Grid & Web Services Working Group ( Brian Major, Giuliano Taffoni )

• The document html title is currently "IVOA Document Template' and needs to be updated

GL: which HTML document are you referring to?

• Citations/references for terms like UML, UML2 Profile, XML, XSD and XMI can be found near the end of part 3, but should probably be made earlier (upon first reference) instead.

GL: will try to do so.

I approve this document.

Reply [GL]: thanks!

-- BrianMajor - 2018-05-02

Registry Working Group ( _Markus Demleitner, Theresa Dower )

In sect. 5 it says: "The VO-DML/XML document MUST be available online and an accepted version MUST be available in an IVOA registry with a standard IVO Identifier." I'm all for registering DMs, but unfortunately the document gives no rules on how to actually do that. Since fixing these things after the fact is always hard, I'd much prefer if the document specified exactly how to do that. I see three possibilities:

(a) we do a quick update to StandardsRegExt adding a (say) vodml element with prefix, URI, and URL;

(b) this document defines a canonical key name ("vodml-url", say) for a StandardsRegExt standard key and says that the value is the URL of the current version of the VO-DML file, or

(c) we do a quick registry extension in this document.

My personal order of preference: (b), (c), (a). I'd probably strike the requirement on registering the HTML document; that's either accessible from the VO-DML document, or its URL can be part of the VO-DML's registry record.

Reply [GL] : I have no strict opinion about this, leave ot to community to be agreed on, i.e. OPEN for discussion. Note, in the updated document on volute this discussion is now in section 6.

Reply [GL]: I believe the current version has answered these comments, and further comments in email have also been addressed we hope.

Semantics Working Group ( _Mireille Louys, Alberto Accomazzi )

GL: thanks for your careful reading.

Data Curation & Preservation Interest Group ( Francoise Genova )

Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )

Knowledge Discovery in Databases Interest Group ( Kai Polsterer )

Theory Interest Group ( _Carlos Rodrigo )

Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick )

Operations ( _Tom McGlynn, Mark Taylor )

Standards and Processes Committee ( Françoise Genova )

---

TCG Vote : 18/03/2018 - 02/04/2018

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group	Yes	No	Abstain	Comments
TCG	*
Apps	*
DAL	*
DM	*			see DM section above for list of typo-s
G&WS	*
Reg	*			provided we get a DM record into the Registry (which the authors have promised)
Semantics	*			note: vote added by PatrickDowler from approval above
DataCP
KDD
SSIG
Theory
TD
Ops
StdProc