VOSpace20140918 < IVOA

IVOA Web>TWikiUsers>DaveMorris>VOSpace20140918 (2014-10-06, DaveMorris) (raw view)
Comments and suggestions on the [[http://www.ivoa.net/documents/VOSpace/20140805/WD-VOSpace-2.1-20140805.html][VOSpace-2.1 working draft]].

From the mailing list [[http://mail.ivoa.net/pipermail/grid/2014-August/002637.html][announcement]] :

   * Summary of [[http://volute.googlecode.com/svn/trunk/projects/grid/vospace/doc/VOSpace.html#sec6][recent changes]]
   * Line by line [[https://code.google.com/p/volute/source/diff?spec=svn2701&r=2690&format=side&path=/trunk/projects/grid/vospace/doc/VOSpace.html&old_path=/trunk/projects/grid/vospace/doc/VOSpace.html&old=2654][diff]]

---++ General
---+++ HTML anchors

The HTML anchors use =name= rather than =id=, is this intentional ?
<verbatim>
    <a name="0.1_sec1">1. Introduction</a>
</verbatim>

rather than
<verbatim>
    <a id="0.1_sec1">1. Introduction</a>
</verbatim>

-- IVOA.BrianMajor - 05 Oct 2014 - No, this is not intentional. =id= is a better choice for the anchors. I will address this on conversion to a more easily editable document format (xhtml / ivotex).

---++ Section 1.1
---+++ Create node path

In the create node operation described in section 1.1, the path encoded in the URL of the =HTTP PUT= request
<verbatim>
    http://nvo.caltech.edu/vospace/myData/table123
</verbatim>

does not match the path in the message content
<verbatim>
    vos://nvo.caltech!vospace/mytable1
</verbatim>

In addition, the URL of the =HTTP PUT= request does not point to a location within the ={service}/nodes= tree defined in the specification

If the endpoint for the VOSpace service is
<verbatim>
    http://nvo.caltech.edu/vospace/
</verbatim>

then the target URL for the =HTTP PUT= request should be
<verbatim>
    http://nvo.caltech.edu/vospace/nodes/myData/table123
</verbatim>

and the =uri= in the node XML should either be blank, or it should match the path in the =HTTP PUT= target URL
<verbatim>
    vos://nvo.caltech!vospace/myData/table123
</verbatim>

---+++ Create node text

There are a number of places where the description of the create operation could be clarified.

In the original VOSpace-1.x SOAP service, the create message contained the target URI as part of the message body.

The VOSpace-2.x REST service create message is essentially the same XML message content, but the request is sent to a location inside the ={service}/nodes= tree with target node path embeded in the URL.
<verbatim>
    {service}/nodes/{path}
</verbatim>

This means the node path is encoded in two different places within the same message.

   * The current specification does not *clearly* state how the intended node path should be included in the target URL of the =HTTP PUT= request.
   * The current specification does not clearly state whether or not the node =uri= should be included in the content of the create node message.
   * The current specification does not clearly state whether or not the service should return an error if the path in the =uri= in the message content does not match the path in the target URL of the =HTTP PUT= request

---+++ Push transfer endpoint

In the description of the data transfer in section 1.1, the endpoint URL for the =HTTP POST= _appears_ to point to a location relative to the target node
<verbatim>
    http://nvo.caltech.edu/vospace/myData/table123/transfers
</verbatim>

In VOSpace-2.x transfer operations should be initiated by a =HTTP POST= to a separate UWS ={service}/transfers= endpoint, outside the ={service}/nodes= tree.

If the endpoint for the VOSpace service is
<verbatim>
    http://nvo.caltech.edu/vospace/
</verbatim>

then the endpoint URL for initiating a transfer should be
<verbatim>
    http://nvo.caltech.edu/vospace/transfers
</verbatim>

---+++ Push transfer target

The path of the =target= in the body of the =HTTP POST= message should match the path of the node created in the first step of the example
<verbatim>
    <transfer xmlns="http://www.ivoa.net/xml/VOSpace/v2.1">
        ....
        <target>vos://nvo.caltech!vospace/myData/table123</target>
        ....
    </transfer>
</verbatim>

---+++ Push transfer protocol

The content for the transfer request given in the example uses a non-standard protocol URI
<verbatim>
    <transfer ...>
        ....
        <protocol uri="ivo://ivoa.net/vospace/core#http-put"/>  
        ....
    </transfer>
</verbatim>

The standard protocol URI for =HTTP PUT= transfers listed in section 3.5.3 of the same document is
<verbatim>
    ivo://ivoa.net/vospace/core#httpput
</verbatim>

---+++ Push transfer redirect

The description of the data transfer in section 1.1, misses out a step in the sequence of requests.

Following the initial =HTTP POST= to the ={service}/transfers= endpoint, the current text implies that the server responds with the =HTTP PUT= destination directly in the response.

_"The service will reply with the URL that the user will HTTP PUT the data file"_

This omits the details of the 303 redirect response
<verbatim>
    HTTP/1.1 303 See Other
    ....
    Location: http://nvo.caltech.edu/vospace/transfers/581
</verbatim>

Which directs the client to a representation of the UWS job, which contains the =HTTP PUT= destination that the client should send the data to
<verbatim>
    <uws:job ....>
        <uws:jobId>581</uws:jobId>
        ....
        <uws:jobInfo>
            <vos:transfer>
                ....
                <vos:protocol uri="ivo://ivoa.net/vospace/core#http-put">
                    <vos:endpoint>http://d21.caltech.edu/temp/d558</vos:endpoint>
                </vos:protocol>
                ....
            </vos:transfer>
        <uws:jobInfo>
        ....
    </uws:job>
</verbatim>

In addition, the example URL for the =HTTP PUT= destination,
<verbatim>
    http://nvo.caltech.edu/bvospace/myData/table123/transfers/147516ab
</verbatim>

_appears_ to point to a location relative to the target node.

It would be clearer if the example used a URL that did not overlap with the VOSpace service endpoint in any way
<verbatim>
    http://d21.caltech.edu/temp/d558
</verbatim>

---+++ Pull transfer endpoint

The endpoint URL for the =HTTP POST= should point to the global ={service}/transfers= endpoint, not to what _appears to be_ a location relative to the target node within the node tree.

If the endpoint for the VOSpace service is
<verbatim>
    http://nvo.caltech.edu/vospace/
</verbatim>

then the target URL to initate a transfer should be
<verbatim>
    http://nvo.caltech.edu/vospace/transfers
</verbatim>

---+++ Pull transfer target

The path of the =target= in the body of the =HTTP POST= message should match the path of the node created in the first step of the example.
<verbatim>
    <transfer xmlns="http://www.ivoa.net/xml/VOSpace/v2.1">
        ....
        <target>vos://nvo.caltech!vospace/myData/table123</target>
        ....
    </transfer>
</verbatim>

---+++ Pull transfer redirect

The current text implies that the server responds directly with the =HTTP GET= source URL that the client should use to fetch the data from.

This omits the details of the 303 redirect response
<verbatim>
    HTTP/1.1 303 See Other
    ....
    Location: http://nvo.caltech.edu/vospace/transfers/581
</verbatim>

which directs the client to a representation of the UWS job, which contains the =HTTP GET= location that client can fetch the data from.

In addition, the example URL for the =HTTP GET= location to transfer the data
<verbatim>
    http://nvo.caltech.edu/bvospace/myData/table123/transfers/147516ab
</verbatim>

_appears_ to point to a location relative to the target node within the ={service}/node= tree.

It would be clearer if the example used a URL that did not overlap with the VOSpace service endpoint in any way.
<verbatim>
    http://d22.caltech.edu/temp/e612
</verbatim>

---+++ Push to VOSpace sequence

In section 5.4.1.4 Example: pushToVoSpace (missing section header) The text in the example suggests that the client should submit the transfer job, then poll the job details, waiting for the job to COMPLETE and then using the URL in 'transferDetails' result to fetch the transfer details, which contain the endpoint URL to send the data to.

I don't think that is the correct sequence.

Based on the way the original SOAP service worked, I think the correct pushToVoSpace sequence should be :

   1 <span style="background-color: transparent;">The client submits the job, and follows the redirect to the job details, but there is no need to poll the job status, because the server is now waiting for the client to send the data.</span>
   1 <span style="background-color: transparent;">The job status SHOULD should be PENDING to indicate that the server is waiting for some data.</span>
   1 <span style="background-color: transparent;">The job status MUST contain the full details of the vos:transfer in the uws:jobInfo section, including target URL(s) for each of the vos:protocol elements that the server is offering.</span>
   1 <span style="background-color: transparent;">It is up to the client to pick one of the target URL(s) and transfer the data (pushing the data to the VOSpace).</span>
   1 <span style="background-color: transparent;">During the transfer, the server SHOULD update the job status to RUNNING.</span>
   1 <span style="background-color: transparent;">The server MAY decide to reject additional transfers once a transfer has succeed.</span>
   1 <span style="background-color: transparent;">The server MAY get update the job status to FAILED if no data has been sent after a time limit has expired.</span>
   1 <span style="background-color: transparent;">If a transfers fails, the client MAY try again if the server offered more than one vos:protocol target URL.</span>
   1 <span style="background-color: transparent;">The client SHOULD set the job status to FAILED if it is unable to transfer the data.</span>
   1 <span style="background-color: transparent;">The server MUST update the job status to FAILED if the data that arrives is not valid (see </span><span style="background-color: transparent;">http://mail.ivoa.net/pipermail/grid/2014-September/002649.html</span><span style="background-color: transparent;">).</span>
   1 <span style="background-color: transparent;">The server MUST update the job status to COMPLETED once a transfer has succeed.</span>
   1 <span style="background-color: transparent;">After the data has been sent the client SHOULD check that the final job status is set COMPLETED to verify that the server has accepted the data as valid.</span>
If the above sequence is correct then I expect there are similar issues with the descriptions and examples for the other transfer operations.

---+++ Optional .auto

The list of changes state that .auto is being made optional. However many sections of the document still contain the original text describing how .auto applies, without describing how it can be made optional. If it is optional, then the specification needs to describe a mechanism for a client to discover whether a particular service supports it or not, and how this effects the behaviour in each case where it is applicable.

However, I think making it optional causes too many side effects and inconsistent behaviours between different spaces. As the original sponsor of the .auto functionality I'm happy for it to be removed completely.

---+++ And .null

If we are going to remove .auto, then .null should probably go too. There are better ways of specifying a null value for a direction.

---+++ Neutral URL examples

All our documents should probably use the reserved 'example' domain names for URLs. https://tools.ietf.org/html/rfc2606

   * http://server.example.com/vospace
   * http://server.example.com/vospace/nodes
   * http://server.example.com/vospace/nodes/container
   * http://server.example.com/vospace/nodes/container/file
   * http://server.example.com/vospace/nodes/container/table ....

   * http://server.example.com/vospace/transfers/810
   * http://server.example.com/vospace/transfers/810/status ....

and perhaps use separate server names for uploads and downloads

   * http://other.example.com/upload/312 ....

   * http://download.example.com/public/598 ....
Topic revision: r6 - 2014-10-06 - DaveMorris
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Time Domain
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema