TAP-1.1 Next

This topic collects proposals for modifications of the TAP-1.1 specification in order to improve the next revision of the specification.

Errata to the TAP-1.1 Recommendation can be found on the devoted TAP-1_1-Errata page.

Proposed Features

Modify here to give your input, than possibly alert the community (email, slack, ...).

Feature: User-managed tables

The goal is to enable users to create table, create index, drop table, and load data (only adding rows). Stretch goals: setting permissions (private, groups via GMS, or public), some other table maintenance ops like renaming tables and columns and modifying some table (tap_schema) metadata. When digging into the details, there are lots of things that could be done where it seems like effort >> value, so tha hard part is drawing trhe line of what is in the stds and what is not.

The goal in CANFAR was to support projects that were creating astronomical catalogues so mostly bulk loading of content and that's what the CANFAR youcat service does now. The features could also support users managing their own tables in a more persistent fashion than we have with TAP upload: create a server-side table once, do various queries/joins with other tables over a period of time, and maybe drop it later. I'm sure there are other useful things one could do.

PatrickDowler gave a presentation in College Park (https://wiki.ivoa.net/internal/IVOA/InterOpNov2018DAL/tap-youcat.pdf)

The REST API docs for our youcat service are here: https://ws-cadc.canfar.net/youcat/ but will be added below in a more simplified form.

Feature: number of rows in TAP_SCHEMA.tables

As VODataService now declares the approximate number of rows in a table (and several TAP services already do that), let's have an nrows column in TAP_SCHEMA.tables. Its metadata could be:

• type: bigint
• description: The approximate size of the table in rows

Feature: debug/plan endpoint

To give users a chance to figure out why a query is slow (or perhaps for other diagnostic tasks), standardise a way to retrieve information useful for debugging. What is returned does not have to be machine readable, but it should usually include the query into the database's native SQL and an associated query plan.

Me, I'd say that sort of functionality probably not dramatically important for sync jobs. Users who want to read query plans can deal with async. I'd hence say this information should be retrievable on a child resource "plan" of a job's async resource, i.e., as a child of phase. We should require that, when people have a plan endpoint in the first place, it will return something sensible while a job is PENDING.

New constraint: Require dataType

Somewhat regrettably, VODataService does not require dataType. In TAP_SCHEMA, on the other hand, we require datatype to be non-NULL. For consistency, and in particular to let validators catch the condition, we should extend this non-NULL condition to the tables endpoint. Strictly speaking, this is a breaking change. Pragmatically, that's just uncovering bugs. So, I'd say in sect. 2.5 (VOSI-tables) we should amend the sentence "The content is equivalent to the metadata from the TAP_SCHEMA described in section 4" with "; this means, in particular, that while the dataType child of the column element is optional by VODataService, it MUST be given in the tables endpoints of TAP services. This reflects the non-NULL constraint on the datatype column in TAP_SCHEMA.columns."

New Constraint: VOUnits syntax

Where we talk about units in TAP_SCHEMA, we should say they SHOULD conform to VOUnits. If other people feel we can get away with MUST, I'd probably support that, too.

-- MarkusDemleitner - 2023-06-06"