TWiki> IVOA Web>IvoaRepMin>IvoaExecMeetingFM84 (2019-05-27, EnriqueSolano) EditAttach
IVOA Executive Committee Meeting (FM84)

FM84 - May 12 2019 - 4-6pm local time - Salle du Conseil

( note Wednesday meeting has its own Agenda page FM84S)

Access to the Observatory from side entrance at

--> 77 avenue Denfert Rochereau <--

Logistic

Hosted by CXC Data Systems

Sunday, May 12, 2019 10:00 am | 3 hours | (UTC-05:00) Eastern Time (US & Canada)
Meeting number: 731 446 649
Password: fm84_paris
https://sao.webex.com/sao/j.php?MTID=m6efaa072de8c87437ad863c4963b41a1
Join by phone
1-877-668-4493 Call-in toll-free number (US/Canada)
1-650-479-3208 Call-in toll number (US/Canada)
Access code: 731 446 649

Draft Agenda FM84

  1. Roll Call and Agenda
  2. Minutes of TM83
  3. Review of Exec Actions
  4. Project Reports [Significant Events Only]
  5. Overall TCG Status
  6. Approval of new IVOA Recommendation(s) [Standing Item]
  7. Overall CSP Status
  8. Review terms up for expiration in May
    • Data Model : Mark Cresitello-Dittmar - Ch (no ext.), Laurent Michel (no ext.)
    • GWS: Brian Major - Ch (no ext.), Giuliano Taffoni - VCh (no ext.)
    • Semantics: Mireille Louys - Ch (no ext.)
    • Theory: Carlos Rodrigo (1 yr. ext.)
    • Ops: Tom McGlynn - Ch (no ext.), Mark Taylor (no ext.)
    • Time Domain : D. Morris - VCh (1 yr. ext.)
  9. Requested item - Common Archive Observation Model - David Schade
  10. Relationship of IVOA to IAU
  11. Future Interops
  12. AOB
  13. Date of next Exec meeting
  14. Review of New Action Items

Reports from the Projects

Argentina-NOVA

Together with the La Plata Observatory, Nova is begunning to digitize old spectral photographic plates. The goal is to put these thousands of (raw two-dimensional) spectra public through the repository of the La Plata National University. As a second stage, spectra will be extracted and wavelength calibrateed to publish in the NOVA database.

ArVO

(Contact maintained May 2019 - attendance at Interop meeting)

AstroGrid

(Contact maintained May 2019 - UK attendance at Interop meeting)

Australia-VO

Nov 2018 - May 2019 activities of the Australian ASVO:

  • 6 ASVO Node and community technical meetings
  • An ASVO Node retreat (Feb 26-27 2019) to plan a unified approach to interoperability, authorisation and authentication
  • Funding support to ASVO Nodes by Australia Astronomy Limited (partial to full funding depending on Node operations)
  • Australian community user survey Nov-Dec 2018 to assist with ASVO interoperability vision
  • The SkyMapper ASVO released DR2 of the Southern Sky Survey. It is initially available only to Australian-based users (due to funding requirements), but will soon be more broadly available.

BRAVO

(Contact maintained May 2019 )

China-VO

China-VO Highlights in FY2018 (InterOpMay2018 to InterOpMay2019)

Communities and Involvement

  • Nov. 2018, the China-VO and Astroinformatics 2018 was held successfully in Jingdezhen, more than 170 persons attended the event.
  • Dec. 2018, the Proposal of Informatization Working Committee (IWCC) of Chinese Astronomical Society was approved.
  • Apr. 2019, at the ACAMAR 5 (Australia-China Consortium for Astrophysical Research), Chenzhou gave a talk about IVOA.
  • Under the name of EduIG and IAU DAEPO WG
  • IAU GA30 Vienna
  • WG renew application
  • IAUS 358 Astronomy for Equity, Diversity and Inclusion — a roadmap to action within the framework of the IAU 100th Anniversary
  • Data-driven EPO session @ CODATA Beijing 2019
  • Data-driven EPO session @ Scientific Data Conference 2019

Data Releases and Open Access

  • Jul. 2018, LAMOST DR4 released globally, which was included by VizieR in Sep. 2019
  • May. 2018, AST3 ( Three Antarctic Survey Telescopes) DR1 released
  • Apr. 2018, GAIA DR2 was mirrored in China by China-VO
System Development
  • May 2019, China-VO Paperdata, a journal paper data repository, upgraded with VOSpace and DOI implementations, and linked with China-VO Registry. (http://paperdata.china-vo.org/)
  • May 2019, China-VO WWT 2.0 released with HiPS implementation and enhanced VR and Microsoft Kinect support. Dozens of HiPS datasets, including Chang’e-II 7m lunar global data, are accessible from the new version. (http://wwt.china-vo.org)
Contests and Trainings
  • Feb. 2018, LAMOST dataset was used in the Alibaba AI contest.
  • Mar. 2019, PSP dataset was used in the Futurelab AI contest.
  • Aug. 2018, WWT Teacher training 2018 in Tsingtao.
  • Jul. 2018, The 2nd East Asian Workshops on Astrostatistics & R were held in Nanjing and Guiyang. A total of 129 participants from 28 astronomical research institutes of China, Japan, Vietnam and Mongolia attended to the workshop.
Awards and Honors
  • In Oct. 2018, Chinese Astronomical Data Center (CAsDC) passed WDS&RDA CoreTrustSeal Certification.
  • Jun. 2018, Chenzhou Cui was elected as the vice president of IAU Commission B2.
  • Dec. 2018, "WWT Planetarium based on the big data of Virtual Observatory" submitted by China-VO, won the FDSTMC (Foundation for the Development of Science and Technology Museums in China ) Nomination Award.
  • Apr. 2019, LAMOST Data Release System was awarded as Best-practice of Chinese Academy of Sciences e-Science project.

ChiVO

ChiVO, as the official Chilean node of the International Virtual Observatory Alliance (IVOA) has been recently connected (since January 2019) to the academic network of REUNA (the Chilean NREN) at 10 Gbps, so this will benefit the Quality of Service offered by ChiVO’s Datacenter.

The datacenter has the objective of providing storage and processing capacities to the local and foreign astronomers, including mirroring astronomical data generated in Chile. Most of ChiVO services are currently hosted in this data center, and specifically part of the ALMA data is currently replicated here and accessed through VO services, like SIA and SCS.

The ALMA-VO Data Repositoryservice offers ALMA data access through IVOA standard web-services/VO-apps or through a web-page. Currently ChiVO has indexed 28,236 FITS with a size of 4.5 TB corresponding to cycles 0, 1, 2 and 3 of ALMA. SIA and SCS services are available on these FITS. In addition we have already stored 2079 files of the raw data of ALMA, the ASDM, with a total weight of 41 TB.

CVO

Standard Implementations: The CADC has implemented and tested each iteration of PR-TAP-1.1 with complete integration of authentication. We also implement the same authentication patterns in other services (VO and custom) to further verify that the choices are implementable and work in a production environment.

Prototype Developments for VO: CADC has implemented an extended TAP service named YouCat that supports users being able to create, load, and drop tables. Users can also update metadata for their existing tables, create indices, and control permissions for table visibility and maintainence. We intend to bring this work forward as an update to VOSI-tables.

Code to the Data: Under the CANFAR umbrella, CADC has developed a science platform named ARCADE so users can run code inside the data centre (near the data). The initial use cases are to support ALMA data processing and analysis by providing the platform and the many required versions of CASA in an interactive development. The basic system is working, based on separate docker containers for each software component; we are currently moving ARCADE to a kubernetes deployment to support scalability (many users and single users making use of resources that exceed a single machine).

Euro_VO

Euro-VO partners have been active in the past period in particular with the overlap of the ASTERICS project (ended April 2019) and the ESCAPE project (started February 2019).

The ASTERICS project held a number of events in the last 6 months of the project:

There were 3 extra associated events connected to the Technology forum: The ESCAPE project began with a kick-off meeting, and is currently in the preliminary 6 month “ramp-up” phase. CDS - A major upgrade of the main CDS data server is in progress, the CDS “All-Sky-Data” project (~1.4 Pb) which will include the CDS HiPS node.

Contributions at ADASS2018 (http://adass2018.umd.edu/) include tutorials and presentations focused on VO

ESAVO

  • ESASky
    • Solar System Objects, new design, internationalisation
    • version 3.1.1 released just before the interop
  • pyESASky
    • Integration of ESASky into the Jupyter Notebook
    • Able to handle TAP, VOTable input and HiPS from users
  • Gaia Archive
    • Very high use of the TAP module, in particular through astroquery.gaia
    • Discussions on the way to serialise and access Gaia Time Series and Spectra and data models for DR3 -> ESDC promotion of IVOA DMs and protocols
    • Access to other TAPs through new Gaia Archive User Interface about to be released in version 2.6
  • Euclid Archive
    • Continue of the preparation of the Euclid Archive, applying VO standards
    • TAP+, HiPS, SIAP...
    • Needs of "move code to the data" paradigm (to be discussed this interop)
  • New TAP services for XMM-Newton and European HST Archives and SCS for Gaia Archive
  • New GUI for Euro-VO Registry development on-going
  • Visibility (ObjVisSAP) and planned observations (ObsLocTAP) protocols definition progressing
    • Workshop at ESAC (Fall 2018)showed large interest from data providers.
    • Prototype implementations for visibility protocols from Chandra and Gaia
    • Implementation for INTEGRAL ongoing

France VO - Action Spécifique Observatoires Virtuels France (ASOV)

As usual since its creation in 2004, VO France annual meeting gathered astronomers and ingeneers involved in astronomical data sharing. The meeting was held in Paris, 3-4 April. It gathered 30 people from Besançon, Bordeaux, Grenoble, Montpellier, Nice, Strasbourg and Toulouse. The meeting web page (mostly in French), with links to the slides, is here http://www.france-ov.org/twiki/bin/view/ASOVFrance/ReunionASOV2019.

The back-to-back Semi-Hack-a-Thon meeting organised 4-5 April to enable technical exchanges between people involved in data services also gathered 30 participants. The meeting web page is here: http://www.france-ov.org/twiki/bin/view/JOURNEESSpecifiques/Semi-Hack-a-Thon5. These meetings have been organised annually since 2015 with the support of the ASOV.

The ASOV also supports travel of people working in French labs to attend IVOA meetings and collaboration meetings organised in France. This year, it also provided financial support to the organisation of the Paris IVOA meeting.

It can be noted that there are 50 participants from French labs on the 126 participants who registered to attend the Paris Interop meeting, whereas the French astronomical Society helds its annual meeting on the same week in Nice.

The history and impact of OV France will be summarized in a talk presented during the Interop in the plenary session held Monday, May 13th, 14:00-15:30. The history and impact of OV France will be summarized in a talk presented during the Interop in the plenary session held Monday, May 13th, 14:00-15:30. ---

GAVO

The GAVO Data Centers in Heidelberg and Potsdam keep maintaining and developing services. New services and data collections since the College Park interop include an ObsCore publication of the APPLAUSE plate scans, a large collection of photometric time series in the galactic plane from Bochum, and a map from HD numbers to Gaia objects. Our obscore service is now liked with datalink services on the underlying source tables. Significant efforts also went into keeping our Registry services (RegTAP and GlotTS) up to date with RegTAP developments and "security appliances" going wild.

On the software development side, numerous minor fixes and improvements went into our publication suite DaCHS, with release 1.3 planned for shortly after the Paris introp. We have also added SKOS support to the vocabulary management software on ivoa.net. In SPLAT-VO a test implementation of TIMESYS was added, as well as other improvements in TimeSeries, DataLink and SAMP support. Besides that, it's now possible to read spectra in SDSS format, which was not possible before.

In standards, we've thought a lot about what "Capabilities" actually are and wrote a Note about the practical consequences of our results ("caproles"). Towards better support of time series in the VO, we have published a Note and then worked on VOTable 1.4's TIMESYS specification. We have worked on several vocabularies (refframe, timescale, refposition) and helped publish several others. RegTAP 1.1 received some late changes to accomodate for the evolving plans on authentication. We have also worked on a proper formal grammar of ADQL 2.1, written in PEG.

We continued our contributions to the Provenance DM and worked on implementing and improving the agreed compromise model. Two reference implementations were created with real scientific data collections: one for the MuseWISE data processing framework (featuring a relational model, provSAP and an HTML middleware), and the other for the APPLAUSE plate archive.

Updates on what GAVO is doing are posted on a roughly monthly basis on https://blog.g-vo.org.

HVO

(contact maintained - May 2019)

HVO reported to be in a dormant phase, but there is work on the SkyQuery cross-match tool jointly with the JHU group.

JVO

(contact maintained - May 2019)

RVO

(Contact maintained - May 2019.)

Activities noted at the World Data System Asia-Oceania Conference 2019 (May 7-8) http://www.wds-china.org/meeting201905.html where a presentation has been made about IVO data repositories and an RVO scientific application.

SA^3

(Contact maintained - May 2019)

SVO

Vo-India

(contact maintained May 2019 - many activities related to support of IVOA web pages)

Vobs.it

Nov 2018 - May 2019 Report:

Funding and projects:

  1. Funding for VObs.it has been secured by INAF for the 2019 fiscal year.
  2. VObs.it staff have started participation in the EU ESCAPE project. The VObs.it role in the project is the integration of astronomy VO data and services into the European Open Science Cloud (EOSC).
  3. VObs.it staff participated in the EU ASTERICS project:
  • A meeting on Authentication and Authorisation was organised (Jan 2019) in Trieste
  • A VO-related presentation (on VO Forum and Training activities for new-generation infrastructures) was made at the Integrating Event of the ASTERICS project.
  • A VO-related event (presentation of Virtual Reality contents for EPO) was made as conclusion of ASTERICS project.
Collaboration on VO-related activities has started between INAF and ASI: this involves the two data centres (IA2 and SSDC respectively) and VObs.it (IVOA-related) activities.

Initial steps have been taken to migrate to INAF-OATs the part of the IVOA web currently hosted at IUCAA.

Staff:

  1. Marco Molinaro (IVOA DAL Chair) and Giulia Iafrate (IVOA Doc Coordinator) were appointed to permanent INAF positions.
  2. Two new INAF fixed-term staff were selected to work full-time within VObs.it and have taken up duty at INAF-OATs (Trieste). They are Chaitra (formerly at CDS, Strasbourg) and Kalyani Pedamkar.

Ukraine_VO

- Joint Digitized Archive of the UkrVO astronegatives (2 abstracts which were presented during the Astroplate2019 meeting)

- Development of the CoLiTec software for Solar system small bodies, Earth satelllites, variables stars (all the info is avaiable through
http://www.neoastrosoft.com/category/news/?lang=en_us)

- development of several machine learning technigue for the astroinformatics of the large-scale structures of the Universe and for processing the big data extragalactic surveys

- participation with separate chapters in writing the book on astro- and geo- informatics altogether with foreign colleagues (Elsevier, 2019)

USVOA/SAO/CXC

Science Policy: G. Fabbiano coordinated USVOA white paper effort for the US 2020 Decadal: Review. Six white papers were developed discussing exploration (including multi-wavelength archival exploration) as a major source of scientific discovery, and submitted to the various science topic sub-committees. These white papers discuss important unanticipated discoveries, and advocate support for data management, software and archive interoperability. A collation of the white papers can be seen at https://arxiv.org/abs/1903.06634

Chandra Source Catalog 2 (CSC2): All data for the second version of the Chandra Source: Catalog (CSC 2.0) have been processed. Final release of CSC 2.0 is scheduled for summer 2019. IVOA service access to catalog data is planned. CSC2 data visualization is implemented using WWT.

Chandra HIPS: Developed and Implemented a pipeline using HIPSGen on Public Chandra Archive data (1999-Mar2018); Coordinated with ESA to include Chandra HIPS in Integration version of ESASky for review and test; After round of testing we will re-generate the HIPS files (found one issue so far) with a End of May release date expected. Presentation planned for Interop.

New Multi-messenger VO services: Implemented ObjVisSAP v0.4 & ObsLocTAP v0.4 for Chandra: New Protocols for standardizing Observation Visibility and Planning information; Coordinated with Jan-Uwe Ness and group at ESA for early implementation of working draft standard. Released service and currently working 1st round of feedback before announcing more widely. Presentation planned for Interop.

RofR: Implementing upgrades based on discussions in College Park MD; Work on automating registry validation notification reports/emails from the RofR website; Harvested and manually replaced registry resources for the RofR. Planning status presentation at Interop.


DM: Significant effort in the Coordinates, Transforms, and Measurements Data models (Formally one model called STC) to bring to Standards WD process level. All 3 models will be at a mature Working Draft (WD) level going into the May Interop.

Data Curation and Preservation: developed a plan for the transition of the Chandra Data Archive to DOIs as persistent identifiers for Chandra data products (from single observations to aggregated datasets), Chandra Source Catalog(s) and data collection associated to publications. This plan covers the rule for metadata population, relational identifiers, update triggers and landing pages. Raffaele will give a status report presentation at InterOp.

USVOA/NAVO

  • Science White Papers: Preparing to submit state of the profession white papers, which will advocate for the VO (due July 1).
  • The paper "Deep Learning for Multi-Messenger Astrophysics: A Gateway for Discovery in the Big Data Era”, submitted by invitation as an Expert Recommendation in Nature Reviews Physics, advocated the value of the VO in data discovery. (Bruce Berriman co-author)
  • Hosted two sessions at the 233rd AAS Meeting in Seattle, WA, Jan 2019:
    • Workshop Using Python to Search NASA's Astrophysics Archives
    • Special Session: Astrophysics Archives in the 2020s.
  • Plan to hold a NAVO Python tutorial at 235th AAS Meeting in Honolulu, HI, January 2020.
  • AstroData2020s Science Workshop at IPAC (December 2018) hosted at IPAC attended by NAVO staff and explicitly called out the value of the VO.
  • Staff attended the Common Archive Observation Model (CAOM) workshop held at CADC in Victoria, Canada.
  • Continue to work with the Astropy community in developing standardized Python interfaces. Our initial effort has been to provide a simple, intuitive tool to access the VO registry.- Continue to offer new services and data sets, including: HEASARC updated Xamin to use NAVO registry to discover remote sources dynamically. This new version of Xamin incorporates DataLink internally in providing access to HEASARC data products.
  • IRSA released NEOWISE -R catalog with 95 billion row tables.
  • With the recent release of PanSTARRS and impending release of ZTF data, NAVO archives are providing extremely large optical data sets presaging the flood of data anticipated with LSST. These provide datasets for the VO to work with in defining and adopting the standards needed to promote interoperable science on data of these scales.
  • Support for HiPS being built into Montage image mosaic engine (presentation at this meeting).

  • MAST is working with IPAC to conduct timing measurements for VO queries (both inside and outside of NAVO)
    • Purpose is to build a baseline understanding of our current performance, and to identify obvious areas for improvement.
    • Created a Python package to support timing the queries (<a target="_blank" href="https://github.com/NASA-NAVO/servicemon">https://github.com/NASA-NAVO/servicemon</a>
    • So far have done ~100,000 queries on 10 difference resources. Plan to do wider variety of queries on more resources over time.
  • As part of a tech. stack transition, MAST is starting to explore using TAP as a more fundamental part of the infrastructure. Parts of TAP could be used to support both internal and external queries to the more complex mission databases, and the TAP schema framework could be used as the central place to store DB metadata.
  • Deployed a TAP service for data release 2 of the Pan-STARRS catalog.
  • Updating Registry software and records as part of an IVOA-wide registry validation effort.
  • Updating RegTAP as reference implementation for version 1.1 . It supports:
    • Alternate identifiers for people and data
    • Security methods for authenticated services

USVOA/LSST

  • We are taking a “VO-first” approach to providing LSST data access Web services.
  • We have taken the OpenCADC TAP server code and are adapting it to work with the LSST Qserv parallel database.
    • We are currently translating a subset of ADQL to the Qserv SQL dialect
    • This service is running in front of an O(80B row) test dataset in our data access center prototype.
    • Work is continuing on extending the range of ADQL that is supported.
    • We are contributing work upstream as appropriate and look forward to an ongoing collaboration.
  • We will soon be using this server to run an ObsTAP service.
  • We are testing this service with PyVO and astroquery.utils.tap and are/will be providing feedback, including PRs.
  • We have developed a Python SODA service implementation that works with the LSST Python science pipeline code base to perform its image cutout operations.
  • We are in the midst of deploying integrated single-sign-on for all the components of the LSST Science Platform (LSP)
    • Based on OAuth2 technology
    • A single login provides access to:
      • The LSP Portal Aspect web GUI, based on Firefly;
      • The LSP Notebook Aspect, based on JupyterLab;
      • Data services, currently including SODA and shortly to include the CADC-based TAP service; and
      • A user workspace via WebDAV (and eventually via VOSpace as well).
    • We are working on easing the integration of the authorization tokens from our SSO system into Python API calls to our data services, i.e., via PyVO and astroquery.utils.tap.
      • We look forward to detailed discussions and work in this area at the hack sessions in Paris.
    • We are interested in discussing standardization of A&A interactions with community tools such as TOPCAT and Aladin Desktop.
  • We will be looking at adopting community VOSpace implementations, including CADC’s new POSIX-based server
  • We have developed a Firefly-based Web front end to TAP queries.
    • We have tested this with our TAP server as well as with numerous community services (IRSA, NED, CADC, MAST, Gaia, GAVO, …)
    • We are making heavy use of TAP_SCHEMA metadata to allow this portal to display data as specified by the publisher.
    • Special capabilities are included for supporting ObsTAP image metadata queries.
    • We are starting work on supporting DataLink-based behavior.
  • We have worked with STScI, JCMT, and Mark C-D on the STC transforms data model to ensure that it's compatible with current implementations.

Report from the TCG

Report from the Working Groups

*
Attachments Attachments
Topic attachments
I Attachment History Action Size Date Who Comment
PDFpdf actions-for-fm84-20190512.pdf r1 manage 62.9 K 2019-05-12 - 12:32 FrancescaCivano actions for FM84
Edit | Attach | Watch | Print version | History: r30 < r29 < r28 < r27 < r26 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r30 - 2019-05-27 - EnriqueSolano
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback