TWiki> IVOA Web>IvoaTCG>2024ARoadmap (2024-10-30, MarkusDemleitner) EditAttach

IVOA Roadmap for 2024A

This outlines the roadmap for development activities by the various IVOA working and interest groups between the May and November 2024 Interops.


Applications WG

Standards and Related:

  • Get VOTable 1.5 approved and start planning for 1.6 (support for Parquet)
  • Evolution of MOC adding frequency axis to MOC
  • Follow integration of VO-Standards and comon applications into science platform
  • Apps support for P3T prototyping
  • Migrate HiPS and SAMP documents to Github
  • Clean up applications WG wiki pages.
Python
  • Support open development in PyVO (release 1.6)
  • Continue the support for MIVOT integration in PyVO
  • Maintenance, new features? Organize meetings with developers, mainteners, users
  • Maintain votable parsing in Astropy core (add support for VOTable 1.5)

Data Access Layer WG

Primary

  • TAP

    • 1.2 - User managed persistent tables + Experiment with Open API spec (context: P3T)

  • DALI

    • Polygon definition clarification

    • PR-1.2 by Malta

  • ADQL

    • WD-2.2 by Malta (PEG grammar only)

  • SODA

    • return extended metadata

    • New transformation types

    • WD-1.1 by Malta

  • SIA

    • Issue errata for minor issues already noted in GitHub

  • SSA

    • Issue errata for minor issues already noted in GitHub

  • DAP (Data Access Protocol)

    • SIAP-2.0 into DAP

    • Gregory D.-F. become the 2nd Editor with Pat D.

    • Proceed with renaming, expansion of supported data product types

    • Miscellaneous changes

    • WD-1.0 by Malta

  • User Defined Functions catalogue

    • Ready to be endorsed

Secondary

  • Conesearch

    • Modernize required UCDs (UCD 1.0-> UCD 1+)

    • RESPONSEFORMAT, MAXREC

    • Allow services to reject unknown parameters (work to be covered by P3T)

    • WD-1.1 by Malta (PR pending validators and implementations)

  • LineTAP

    • Work to continue

    • Status about client and server implementations

    • Raise PRec once implementations ready

  • ObjObsSAP

    • Work to continue

    • WD-1.0 by Malta

  • ADQL

    • 2.3 - INTERVAL support, MOC support, Array support, string functions, OFFSET

Data Model WG

This still needs to be integrated with the 2023B roadmap and prioritized. These items are primarly our take-away from the last interop.

  • Reconcile STC (Region), FoV (Shape), DALI (dtypes)
    • this came up multiple times during the interop and shouldn't be too difficult to make a plan
    • STC (Region) -- separate shape from Frame => STC-S serialization
      • Note: we don't necessarily have to update STC, just recast STC-S to refer to the replacement models.
  • VOEvent: cast in VODML
    • NOTE: The TDIG should bring this to us..
    • Separate model from serialization *
  • Update DM Twiki Page:
    • seriously overdue..
      • general content
      • model diagrams: I'd like to replace the simple model dependency diagram with the bigger picture diagrams showing which models serve which workflows. *
  • Transform Model
    • Document implementations.. take to RFC *
  • MANGO Model
    • submit review comments as Issues
    • continued development of the model itself. *
  • HEIG
    • work with group to formulate usage threads leveraging Cube
      • optimally something in the Spectral domain.. implemented by Firefly or similar *
  • Standards which combine Model + Usage ( ObsCore, EPNTap, ObsLocTap )
    • can we separate these pieces? EPN = Datamodel; EPNTap = DAL
    • VODML compliant representation of ObsCore? *
  • ProposalDM:
    • where does this fit into the DM ecosystem? The model is built on a single user-base.. so really needs broader review
    • what are the goals for this semester?
      • PH mentioned in his presentation that he dislikes "digging for existing concepts" in the other models, and prefers more modularity.. we should follow-up with him to recommend candidates for separate modeling. (eg: Target) *
  • VODML: can this be expanded to describe/define views?
    • from the Joint session.. if we can define vo-dml concepts which formally define one model as a view of others, then it could generate whatever supporting mechanism is needed to programmatically generate instances of one from the other. This would be HUGE! *
  • CAOM: integrate into the landscape
    • iterate with Pat on the details. *
  • Request for a document (IVOA Note) describing the suggested/recommended FITS serialization for data products
    • email to DM list from Kelle Cruz regarding light curves.
    • could be home-grown recommendations, or OGIP or derived from a study similar to the Units work.

Grid and Web Services WG

  • Execution Broker (previously known as Execution Planner)
Consolidated Working Draft at

https://githuhttps://github.com/ivoa-std/ExecutionBroker

Prototype in development by Dave Morris, expected for Malta, following the P3T new convention

  • UWS
Working on a new version of UWS, to work in coordination with a new TAP server implementation, in line with P3T effort, using <a data-wikiword="OpenAPI" href="https://swagger.io/specification/" target="_blank" title="OpenAPI"> OpenAPI </a>

  • VOSpace

New draft 2.2 was delayed as per P3T new efforts. This version will include - Simplified transfer negotiation - Improved recursive operations - Simplified quotas

  • SSO

New draft 2.1 or 3.0 is being evaluated. SaraBertocco has the lead on this activity

Registry WG

  • CONTINUOUS WORK
    • Curation and Validation: Following up the Registry spring cleaning talk at Sydney, we’re going to discuss about plans to setup a hackathon to correct Registry issues
    • On-boarding Effort: Prepare a workshop (possibly 2-3 days just before an Interop meeting) to discuss and disseminate best practices for registry publishing
    • Continue involvement with DCP IG, esp. on the note “Data Origin and the VO” https://www.ivoa.net/documents/DataOrigin/
    • Continue discussing whether/how does cloud information belong in the registry and how to register information about multiple locations for finding data e.g., on-prem or on S3.

Semantics WG

  • <nop>ObsFacility: propose draft vocabulary + publish IVOA note(s) see 2022 presentation
  • <nop>EPNcore: Prepare lists of terms in vocabularies (directly RDF/XML). See EPNTAP REC and EPNTAPV20RFC
  • IHDEA: Coordination on helio reference frames and work on VEP.
  • UCDList RFM 1.6 : simplify process + UCD as a IVOA vocabulary
  • Linked-data exploration : onto-portal prototype (UAT, DCAT, RDF Graph on small subset of Registry/epncore/datalink resources)
  • Product-type vocabulary: still some work to do before we converge…

Data Curation Preservation IG

Education IG

Knowledge Discovery IG

Artificial Intelligence and Large Language Models in the VO:
- Coordinating an IVOA, inter-group "tiger team" that will scope the current and future potential impact of AI on VO.
- Investigating best practices of building AI-ready datasets for developing Astronomical LLM and foundation models.
- Collecting use cases and requirements for the integration of LLMs in the VO context.

ML-proofing existing and science platforms:
- Investigating where existing astronomy science platforms conpatible with ML methods.
- Investigate whether science platforms can access tabular and non-tabular data through VO interfaces.
- Coordinating science platforms for bringing ML/AI methods to data.

Operations IG

Ongoing activity:

  • Monitor VO health:
    • Coordinate "weather reports" (service compliance)
  • Address non-compliancy:
    • Identify common/persistent errors
    • Support/encourage service operators to improve service compliance
    • Work with WGs on standards-related issues
  • Validation:
    • Ensure validators are available (IvoaValidatorsSummary)
    • Encourage use of validators:
      • on deployed services
      • during standard development
  • Communication:
    • Provide forum for discussion of operational issues
    • Host/invite "site reports"
    • Encourage feedback communication between ops services and data providers
Specific focus:
  • Work with DAL on standards issues:
    • ConeSearch: UCD1+
    • DALI: Sexagesimal markup standards
    • SSA: A couple of Errata suggested
  • Push for service metadata improvements:
    • VOUnits, UCDs
  • Upcoming themes:
    • Operations in the cloud
    • Authentication as an operational necessity

Radio IG

  • Work on remaining issues in the ObsCore Extension For Radio Data now that it has reached PR status (together with DM WG)
    • Help out with working out how to combine multiple ObsCore extension, like those proposed for time domain and high energy
  • Coordinate work on implementations
    • Need at least one other implementation not based on DaCHs?
  • Continue work on Pulsar Radio data Note in collaboration with TDIG
  • Restart work on version 2 of the implementation review note once more ObsCore extension implementations exist

Solar System IG

The next roadblock to knock down for planetary data inside and outside of IVOA: Standard Vocabularies; Standard Vocabularies; Standard Vocabularies...

The goal should be to coordinate with the related activities in planetary archives, repositories, and journals to avoid having to deal with the translation and "cross-walk" problems of other disciplines by establishing permanent reference services now.

Because planetary data in particular, and solar system more generally, has an abundance of both small data sets and unique, effectively unreproducable data sets (from remote sensing missions) that take half a career or longer to obtain, citation is a high priority for the community. The chair will be actively participating in:
  • DOI Note for IVOA
  • Discussions with authors and journal publishers regarding best practices for citing data.

Time Domain IG

The roadmap for this semester is focused on time-domain alert distribution for large and small surveys, and how some aspects of VOEvent can be relevant. In particular, some questions to consider:

* What issues or problems in alert distribution in large or small surveys persist today that the IVOA can help solving with existing standards? What new standards are needed?

* What aspects of VOEvent (either in terms of data model, serialization, or transport protocol) are appropriate for large/small surveys or for brokers/small users?

* How do we ensure that existing infrastructures and future ones can talk to each other in the context of alerts?

* What is the most effective way to extend ObsCore in time-domain aspects, so that it is generally useful and appropriate across all wavelengths?

VOEvent 2.1 is still in use for transport of alerts in some instances (for example, in GCN). However, it does not meet all the requirements in terms of transport protocol and serialization for something of the scale or scope of Rubin (or GCN). However, we see a path forward where some aspects of the data model can be used, and a potential VOEvent 3.0 that separates data model from serialization aspects seems viable. This will require input from stake holders. Such standard should start by defining what the scope should be VOEvent. Perhaps we do not need to solve ALL the problems.

We will start a discussion to think of a standard that is suitable.

Other points:

* Extension to ObsCore for time-domain properties. Ongoing discussion regarding the quantities that need to be defined.

* Time-domain plenary session in Malta (finally, not a parallel session!) Stay tuned!

Standard and Processes

<!--
* Set ALLOWTOPICRENAME = TWikiAdminGroup
-->

Edit | Attach | Watch | Print version | History: r20 < r19 < r18 < r17 < r16 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r20 - 2024-10-30 - MarkusDemleitner
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback