Automatic builds of VO documents



Notes



Number of participants: 24

Presentation by James Dempsey on auto builds of the ADQL standard

Running Questions



TomD: how to decide between Travis vs Github Actions?
Can do it in one go. Easier to get artifacts (PDF) inserted back.

TimJ: Can this be parameterized so that ADQL is mentioned once at the top? 
James: agree ADQL appears too many times

Peter Teuben: In my own Github clone, can I run the equivalent?
James: Easiest to do locally is run "make".
TimJ: try hard so that CI only runs "make".
Markus (in chat): The targets in ivoatex's makefile have been stable since day 1.

Christine Banek: I use Github Actions for builds, and Github Releases for releases, which kicks off Travis.

Marco: Investigated a place to store the PDF but it was not easy. Got stuck because the only way to move the PDF out of the build involved a container, did not know what to do at that point.

Ada Nebot: 1) Would be great to have the PDF go to the same place as the rest of the files. At the moment one has to click on the artifact and there can be more than one.
James: would not want the PDF to go into the repository. But having it go into the PR conversation would be good.
Ada: 2) if I'm working on a fork?
James: automatic build is just on a PR at this stage, not diving into the fork.

TimJ: At Rubin we have an entire infrastructure for deploying documents from GitHub to one of our websites. For example https://dmtn-148.lsst.io — it shows you master and all the branch versions. Set up by Jonathan Sick. When you tag it it uses the highest one, probably uses S3 to store it. Hundreds of documents stored in that system. Hosting cost is negligible. Would there be interest in asking Jonathan about how to do it?
Markus: We're probably too small to worry about keeping all the versions. A lot of moving parts (even though the cost is nothing).
TimJ: Jonathan spends very little time maintaining this system.
Markus: we could have a place on IVOA site; I don't think we need documents hanging around for anything that is not HEAD on any branch. 
TimJ: Click on the Change Version button. We only build HEAD of each branch and replace previous versions.
Markus: two solutions out there. Agree on a preliminary PR for ivoatex. We could have a target CI that takes all the parameters from the Makefile and kicks off the CI.
James: by design, they don't want you to pull things in from the files

Marco: would like to have a nightly build.
TimJ: that can be the build you do every time master is changed, because we know anytime there's been a change to master.

TomD: maybe we can look at James' PR about having this capability for VOTable. Do we have to do the merge first?
James: Have to merge to master first, can't avoid that experimentation.

Markus: Would be nice to have builds for branches because people don't like to read diffs. If you want to look at the effects of this change, point to the PDF for the branch.
TimJ / James: it builds each time you push a change.

Ada: the PDF is there but it is not straightforward to point people to it. Need to know something about Github.
TomD: the Rubin solution that TimJ showed looks very elegant.
Ole Streicher: on Github you can have bots to put something as a comment. We could write such a bot that puts the PDF as an additional comment. Astropy has quite a lot of bots like this.

Now looking at TomD screen-share of PR #13 to VOTable

No way to test the build.yaml commands on your local system.
Christine: it will run on  your branch if you change build.yaml after the initial merge.
James: be sure to run "make" first on your local system.
Markus: run "make" then "make biblio" then run "make forcetex". BibTeX runs are explicit; is not run on every make because it seemed a waste of cycles. "latexmk" will probably fix all these problems. If you have .png files that are made from PDF files, for example, those are not run by just "make forcetex".

Merged the PR. Need a new PR. Need to rebase an existing one.
New PR opened.

A lot of dependencies (TeX etc.) are installed every time. Rubin pulls a container to save time. Maybe we can even use the LSST Docker container for TeX. Put a "docker run" or "docker pull". Can have another repo that builds the container. Rubin has their equivalent of ivoatex; whenever it is updated it builds the container.

The build finishes and the PDF is successfully displayed!

Markus: would love it if someone in the know about using the containers can contribute this to the PR for ivoatex.
TimJ: This is our travis file that builds the container https://github.com/lsst/lsst-texmf/blob/master/.travis.yml
Some people check their PDFs using CI and can really annoy Travis by pushing changes every minute and causing Travis to kill the previous jobs.