(r15) RegTAP10RFC < IVOA

IVOA Web>WebPreferences>RegTAP10RFC (revision 15) (raw view)~~EditAttach~~
---+ Request for Comment: RegTAP v1.0

This document serves as the RFC center for the Proposed Recommendation entitled "Registry Relational Schema, Version 1.0". The version reviewed during the RFC can be found at http://www.ivoa.net/documents/RegTAP/20140227/index.html.

RFC Review period: March 14, 2014 - April 15, 2014 %BR% TCG Review period: July 28, 2014 - August 28, 2014 %BR% Exec Approved for REC:

To add a comment on the document, please edit this page and add your comment to the list below in the format used for the example (include your !WikiName so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the Resource Registry mailing list, registry@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

---++ Notes on Implementations and Validaters

A provisional validation suite as available at http://docs.g-vo.org/regtapval-2014-06.tar.gz

An implementation of this is available on the TAP services at http://dc.g-vo.org/tap and http://gavo.aip.de/tap (the basic underlying software is identical for those).

Another implementation (out of date with respect to XPath/utypes and some DB fields) is available as a TAP service at http://ia2-vo.oats.inaf.it:8080/registry . Even if not fully compliant, the IA2 team (VObs.it) plans to update it to the REC version and fix some other compliance problems to keep it at least as an independent mirror of the GAVO service. This implementation uses both a programming language (Java) and RDBMS (MySQL) different from the GAVO one.

Implementations are also being worked on at ESAC and STScI; these were reported on in the Registry sessions of the Heidelberg, Waikoloa, and Madrid interops and should become public fairly soon.

On the client side, newer versions of TOPCAT already query against RegTAP. Also [[http://dc.g-vo.org/WIRR][WIRR]] uses RegTAP as a backend.

---++ Comments from the community
   * Sample comment by TWiki.WikiName 
      * Response (by TWiki.WikiName)
---+++ Comments from MarkTaylor

Great job Markus et al. A few minor comments:

   * Sec 1.1: 'The entire standard is now known as "IVOA relational registry schema"' - the title of the document has "IVOA Registry Relational Schema" (words in a different order) - is this an inconsistency?
   * Figure 1 caption: "(tagged with <nop>RegTAP)" - the box in the figure is labelled "Relational Registry" not "RegTAP", so I think this is wrong.
   * Sec 4: The prefix-&gt;namespace table does not appear to be in any discernable order - order it alphabetically either by prefix or by URI?
   * Sec 7.1: The description of =short_name= contains the text "Applications may use to refer to this resource in a compact display." - words missing?
   * Sec 7.1: It would be useful for reference if the the information about field formatting presented in the text of this section could be repeated in the descriptions column of the table. Specifically: 
      * =content_level=, =content_type=, =rights=: note values are hash-separated (this is already done for =waveband=)
      * =creator_seq=: note names are semicolon-separated
   * Sec 7.7: "The table_column table models the content of VOResource's column element" - I might be wrong, but I _think_ that should read "VODataService's column element".
   * Sec 7.7: "hence, this column will contain one of NULL, vs:TAPType, vs:SimpleDataType and vs:VOTableType" - since these will be lowercased by the time they appear in the <nop>RegTAP table, would it be less confusing to mention them in lower case in this text?
   * Sec 7.8: "analoguos" -&gt; "analogous"; "adviced" -&gt; "advised".
   * Sec 7.9: In the =param_description= description, "column's contents" should read "parameter's contents".
   * Sec 7.10: "resoure" - "resource".
   * Sec 8: The first parameter of the =ivo_hashlist_has= UDF is declared with the name "haslist"; I _think_ that should read "hashlist".
   * Appendix A: "suffient" -&gt; "sufficient".
   * Appendix A: This is a long list, and the possibility of typos etc is not negligable. Would it be a good idea to add at the start that in the case of a discrepancy between the xpaths in this list and the XML schemas defined by the relevant standards, one or other of those should be taken as normative?
   * Appendix A: I see a =/capability/testQuery/pos/lat= for SIA, but no corresponding =/capability/testQuery/pos/long= (SSAP has both).
-- IVOA.MarkTaylor - 2014-03-27

Thanks for the useful comments. I hope they're all considered in volute rev. 2497. On the question of precedence of xpaths, I'm now declaring XML schema xpaths as normative and promise Errata in case contradictions go unnoticed. -- IVOA.MarkusDemleitner - 2014-04-01

---+++ Late comments

Some technically late comments:
   * section 4 seems to be missing the slap prefix and namespace from SimpleDALRegExt.
   * while I appreciate (like even) the fact that all the reasonable queries can be performed by natural join, I really don't like multi-column primary and foreign keys. This is especially painful when the extra column to provide the necessary uniqueness is only a smallint that has to be carefully generated while inserting content. Have you considered using the concept of surrogate primary keys? Or at least allowing the implementer to chose the form and datatype of PKs (and hence FKs) so they could implement in that way? For a pure implementation of surrogate PK the ivoid columns would have to be renamed and the implementer would simply be required to provide keys suitable for a natural join. However, just allowing the implementer to chose the datatype for that uniqueness column would enable a variety of simpler techniques to populate it and make the natural joins work correctly. Renaming the ivoid column so it was not used in the natural join would not strictly be needed for correctness, just index efficiency (probably not a concern), so that could be kept as-is... is a known/predicable datatype actually needed?
   * I now always use surrogate primary keys and recently moved to using 128-bit UUIDs, for which there are RDBMS functions (for default) and plenty of libraries with well-known algorithms... sure, 128-bit is complete overkill in the registry, but I'm not sure that smallint is actually enough for simple implementations like an RDBMS sequence or identity column (which I have come to despise for other reasons). With all those long varchar columns, the size of the uniqueness column isn't worth skimping on.
-- IVOA.PatrickDowler - 2014-05-18

We did consider surrogate keys in the very earliest relational registry data models - see RelationalRegistryDM - However, it was decided that ease of writing queries was the primary goal of the design, and therefore it was useful to have the IVORN in as many tables as possible - so for certain queries fewer joins would be necessary. The smallint part of the composite key is generally easily generated on ingestion from the child index in its parent of the main modelled XML element for that table.

-- IVOA.PaulHarrison - 2014-05-23

The slap prefix went in in Volute rev. 2645 -- thanks for catching this.

In Volute rev. 2646, I dropped requirements on the artificial foreign keys. Pat (and others interested in this) -- please review http://volute.googlecode.com/svn/trunk/projects/registry/regtap/RegTAP-fmt.html#primarykeys and the diff (this necessitated changes in several places). Does this fix things as far as you are concerned?

-- IVOA.MarkusDemleitner - 2014-06-03

Yes, making the type for the _index columns implementation specific is sufficient. Thanks.

-- IVOA.PatrickDowler - 2014-07-24

---++ Comments from TCG members during the TCG Review Period:

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

---+++ TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham_ )

---+++ 
---+++ Applications Working Group ( _Pierre Fernique, Tom Donaldson_ )
---+++ 
---+++ Data Access Layer Working Group ( _François Bonnarel, Marco Molinaro_ )
Approved - IVOA.MarcoMolinaro
---+++ 
---+++ Data Model Working Group ( _Jesus Salgado, Omar Laurino_ )
---+++ 
---+++ Grid & Web Services Working Group ( _André Schaaff, Brian Major_ )
---+++ 
---+++ Registry Working Group ( _Markus Demleitner, Pierre Le Sidaner_)
---+++ 
---+++ Semantics Working Group ( _Norman Gray, Mireille Louys_)
---+++ 
---+++ Data Curation & Preservation Interest Group ( _Françoise Genova_ )
---+++ 
---+++ Education Interest Group ( _Massimo Ramella, Sudhanshu Barway_ )
---+++ 
---+++ Knowledge Discovery in Databases Interest Group ( _George Djorgovski_ )
---+++ 
---+++ Theory Interest Group ( _Franck Le Petit, Rick Wagner_ )
---+++ 
---+++ Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick_ )
---+++ 
---+++ Standards and Processes Committee ( _Françoise Genova_ )

<!--  * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup -->
Topic revision: r15 - 2014-08-29 - MarcoMolinaro
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Time Domain
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema