(r15) ImageDM < IVOA

IVOA Web>IvoaDataModel>ImageDM (revision 15) (raw view)~~EditAttach~~

---++ Image Data Model Effort

The Image Data Model, currently under development, will describe multdimensional image and image cube datasets and instances. It will serve as the basis for the description of Image datasets in archives and VO queries, as well as the basis for Image data access protocols such as SIAV2 and related access methods such as siav2.accessData.

---+++ ImageDM-ObsCore Convergence

It was agreed in the fall 2013 interop to do whatever is necessary to bring the Image data model into convergence with ObsCore (e.g., ObsTAP), in particular making data models such as ImageDM and SpectralDM extensions of ObsCore, all sharing a common core data model. We are fast-tracking an updated "stable" version of both the ImageDM and SIAV2 drafts that will make a first cut at this convergence, sufficient to support ongoing development for the Cube initiative, while further development of the working drafts goes forward. During prototyping the stable version of ImageDM will be used for prototype development while the full standard evolves (i.e., there will be two versions of ImageDM until eventually the stable version is retired and replaced by the final standard). An initial analysis of what is required has been performed by a subset of the ImageDM author group (FB, ML, MCD, DT) and the results are summarized here:

   * [[%ATTACHURL%/WD-ImageDM-20131117.pdf]["Stable" version of ImageDM as of 17 Nov]]
   * [[%ATTACHURL%/imagedm-obscore.xls][ImageDM spreadsheet as of 17 Nov (full data model, ObsCore/Char2 integration)]] 
      * [[%ATTACHURL%/imagedm-obscore.pdf][PDF version of data model spreadsheet]]

   * [[%ATTACHURL%/ImageDM-ObsCore.txt][Initial analysis of ImageDM-ObsCore convergence (19-25 Oct)]] 
      * [[%ATTACHURL%/siav2-obstap.xls][Spreadsheet summarizing the ImageDM and ObsCore differences (DT)]]
      * [[%ATTACHURL%/siav2-obstap-FB.xls][ImageDM and ObsCore Utypes vs Char2 (FB)]]
   * [[%ATTACHURL%/ImageDMObscoreSTC-Utypes-ML8Nov.xls][Char Utypes updated (ML,FB 8 Nov - *new*)]]
The consensus at this point appears to be that we finalize a set of Characterisation V2 Utypes, use these for both ImageDM and SpectralDM, and eventually update the older ObsCore/ObsTAP document to comply, however what is eventually done will depend upon a more detailed analysis still to be performed. This will take a while, but all we need for ongoing Cube development is the updated ImageDM and SIAV2 drafts plus a summary of the revised Utypes planned for ObsCore.

Further updates will be posted here as this effort progresses. Discussion should be carried out on the DM mailing list.

---+++ Discussion Points:

This effort carries on from the above draft (Nov. 2013).

The following is a list of topics to be discussed regarding the draft and will guide the future evolution of the document. It is a dynamic list, topics will be added as needed. Discussion should be carried out on the DM mailing list. Here we will summarize the topics, provide links to the relevant threads and note the resolution. It would be greatly appreciated if we can keep discussion topics within the relevant thread.

<strong><span style="color: dodgerblue;"> <span style="color: mediumblue;"><span style="color: darkgreen;">1. Relation to ObsCore</span></span></span></strong>

Carries on from the Nov. draft, formalizing the relation to the ObsCore 1.0. This is a fairly broad topic, and will undoubtedly spawn other threads. The goal of this topic is to explicitely illustrate the Observation model used by the Image and Spectral models and how that relates to the existing ObsCore model. The result of this effort may lead to a recommendation to update that standard, but should not require it.

Threads:
   * [[http://www.ivoa.net/pipermail/dm/2013-November/004803.html][Revised ImageDM-ObsCore architecture]]
Links:
   * [[http://www.ivoa.net/pipermail/dm/attachments/20131113/c9ef7581/attachment-0001.png][UML diagram (draft)]]
   * [[https://volute.googlecode.com/svn/trunk/projects/dm/ImageDM/doc/ImageDM-ObsCore-Extension-initial.png][UML diagram (20131117)]] - Should accurately reflect the current relation to baseline further work.
<span style="color: darkgreen;"> *2. Observation Relation to Dataset* </span>

This topic spawned from "Relation to ObsCore". This topic is very "ObsCore" centric, but since the Observation/Dataset relation is central to the Image and Spectral Model structure, it has significant ramifications on their design.

It is clear that there is a relationship between "Observation" and a more generic "Dataset". This "Dataset" would contain elements such as the dataProductType, and dataProductSubtype, presumably others. This object has not been formally defined.<br /><br />In ObsCore, there is an implied relationship for Observation as an Extension of Dataset in the location of these attributes. So, I have always interpreted that Observation "is" a Dataset. This is reflected in my choice of the name "ObservationDataset" in the left hand package of my diagram. It implies that it is a Dataset extended for Observation purposes. Recent discussion brings this relationship into question, with assertions that an Observation can be associated with 0 or more Datasets. "

The goal of this thread is to gain concensus on what this relationship currently is, and how it should be represented in the Observation model used by the Image/Spectral models.

Threads:
   * [[http://www.ivoa.net/pipermail/dm/2013-November/004831.html][Observation relation to Dataset]]
   * [[http://www.ivoa.net/pipermail/dm/2013-October/004789.html][ObsCore - Dataset]] earlier brief thread
<strong><span style="color: darkgreen;">3. Mapping </span></strong>

Major dopic for discussion of the Mapping object, its definition and purpose.

Threads:
   * [[http://www.ivoa.net/pipermail/dm/2013-November/004852.html][ImageDM - Mapping object]]
<span style="color: darkgreen;"> *4* <strong>. Provenance</strong></span>

<span style="color: darkgreen;"> </span>

Hopefully a very minor thread. There is an ambiguity in the ObsCore document, where the Utypes imply a "Provenance" container class in the model which is not reflected in the UML diagrams.

The goal of this thread is merely to decide if this is a mistake in the diagram which should be 'fixed' or not.

Thread:
   * [[http://www.ivoa.net/pipermail/dm/2013-November/004832.html][ImageDM: Provenance]]
   * [[http://www.ivoa.net/pipermail/dm/2013-November/004837.html][ - same thread.. got split]]
Resolution:

The concensus seems to be that there is no logical head for Provenance at this time, and the model is not defined enough at this point to presume structure, so we will not include a Provenance class at this time.

<strong><span style="color: darkgreen;">5. Provenance: new elements in ImageDM. </span></strong>

The image model adds new elements under the Provenance umbarella (Bandpass, DataSource, ObsData). There is question regarding where/how these should be included.

Threads:

<strong><span style="color: darkgreen;">6. Derived object.<br /></span></strong>

Minor topic regarding the need and placement of the Derived class.

Threads:
   * [[http://www.ivoa.net/pipermail/dm/2013-November/004853.html][ImageDM - Derived object]]

---+++ 
---+++ Current Drafts

Latest version of the draft:

   * [[%ATTACHURL%/WD-ImageDM-20131117.pdf][Image DM v1.0-20131117]]
   * [[%ATTACHURL%/imagedm-obscore.pdf][Image data model spreadsheet - 20130812]]
Previous versions:

   * [[%ATTACHURL%/WD-ImageDM-20130812.pdf][Image DM v1.0-20130812 (draft for Sep2013 interop)]]
Some related documents can be found here:

   * [[http://wiki.ivoa.net/twiki/bin/view/IVOA/SiaInterface][DAL Image Access discussions and documents]]
In particular, there is a new draft of the [[%PUBURLPATH%/%WEB%/SiaInterface/WD-SIAP-2.0-20130812.pdf][SIAV2 image access specification]] which is sync-ed up with the current version of the ImageDM.

The VAO Cube whitepaper relevant use cases and analysis addressing the problem of modeling and accessing multidimensional image data in the VO:

   * [[http://wiki.ivoa.net/internal/IVOA/SiaInterface/CubeDataInVO.pdf][VAO Cube whitepaper]]
This includes a section on the ImageDM, mainly addressing issues pertaining to the image cube use cases.

===

<br /> <!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
png	DMArch-ImageFull.png	r1	manage	107.5 K	2013-11-11 - 04:10	DougTody	ImageDM UML Class Diagram
txt	ImageDM-ObsCore.txt	r1	manage	16.2 K	2013-10-28 - 13:36	DougTody	Initial analysis of ImageDM-ObsCore Convergence
xls	ImageDMObscoreSTC-Utypes-ML8Nov.xls	r1	manage	22.0 K	2013-11-16 - 06:02	DougTody	Char and ObsCore Utypes updated for Char2 (ML, FB)
pdf	ImageDMObscoreSuggestionForDMElementsbasedOnSTC-19Nov.pdf	r1	manage	128.1 K	2013-11-20 - 10:10	MireilleLouys
ods	ImageDMObscoreSuggestionForDMElementsbased_OnSTC-19Nov.ods	r1	manage	25.2 K	2013-11-20 - 10:01	MireilleLouys	column C and blue cells as suggestions for attribute name updates
pdf	WD-ImageDM-20130505.pdf	r4 r3 r2 r1	manage	1365.2 K	2013-05-17 - 08:25	DougTody	Image Data Model draft (still under development)
pdf	WD-ImageDM-20130812.pdf	r1	manage	1224.0 K	2013-08-16 - 19:56	DougTody	ImageDM draft as of 12Aug2013 (first fairly complete version)
pdf	WD-ImageDM-20131110.pdf	r1	manage	1335.8 K	2013-11-11 - 04:12	DougTody	ImageDM draft - architecture section updated to reflect Image/ObsCore integration
docx	WD-ImageDM-20131115.docx	r1	manage	334.3 K	2013-11-16 - 06:08	DougTody	ImageDM-Stable (15Nov draft)
pdf	WD-ImageDM-20131115.pdf	r1	manage	1278.8 K	2013-11-16 - 06:08	DougTody	ImageDM-Stable (15Nov draft)
docx	WD-ImageDM-20131117.docx	r1	manage	451.2 K	2013-11-18 - 04:33	DougTody	ImageDM-Stable (17Nov draft)
pdf	WD-ImageDM-20131117.pdf	r1	manage	1349.9 K	2013-11-18 - 04:31	DougTody	ImageDM-Stable (17Nov update)
pdf	imagedm-obscore.pdf	r2 r1	manage	97.4 K	2013-11-18 - 04:35	DougTody	PDF version of imagedm-obscore spreadsheet
xls	imagedm-obscore.xls	r2 r1	manage	143.0 K	2013-11-18 - 04:34	DougTody	ImageDM Utypes integrated with ObsCore, Char2 (17Nov)
xls	siav2-keywords.xls	r1	manage	136.0 K	2013-08-16 - 19:58	DougTody	Image (and SIAV2) data model spreadsheet
xls	siav2-obstap-FB.xls	r1	manage	35.5 K	2013-10-28 - 13:42	DougTody	ImageDM and ObsCore Utypes vs Char2 (FB)
xls	siav2-obstap.xls	r1	manage	140.5 K	2013-10-28 - 13:38	DougTody	Spreadsheet contrasting ImageDM, ObsCore diffs (DT)