Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update GitHub Actions workflows, and make necessary tweaks to support them #279

Merged
merged 11 commits into from
Aug 25, 2022
Merged
133 changes: 47 additions & 86 deletions .github/workflows/R-CMD-check.yaml
Original file line number Diff line number Diff line change
@@ -1,18 +1,14 @@
# NOTE: This workflow is overkill for most R packages
# check-standard.yaml is likely a better choice
# usethis::use_github_action("check-standard") will install it.
# Workflow derived from https://github.com/r-lib/actions/tree/v2/examples
# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
#
# For help debugging build failures open an issue on the RStudio community with the 'github-actions' tag.
# https://community.rstudio.com/new-topic?category=Package%20development&tags=github-actions
# NOTE: This workflow is overkill for most R packages and
# check-standard.yaml is likely a better choice.
# usethis::use_github_action("check-standard") will install it.
on:
push:
branches:
- main
- master
branches: [main, master]
pull_request:
branches:
- main
- master
branches: [main, master]

name: R-CMD-check

Expand All @@ -27,92 +23,74 @@ jobs:
matrix:
config:
- {os: macOS-latest, r: 'release'}

- {os: windows-latest, r: 'release'}
- {os: windows-latest, r: '3.6', rspm: "https://packagemanager.rstudio.com/cran/latest"}
- {os: ubuntu-18.04, r: 'devel', rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest", http-user-agent: "R/4.1.0 (ubuntu-18.04) R (4.1.0 x86_64-pc-linux-gnu x86_64 linux-gnu) on GitHub Actions" }
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub Actions is no longer supporting Ubuntu 18.04, so we switched to 20.04.

Note that r-lib/actions actually just switched to ubuntu-latest as the default, but I think for cpp11 it is worth trying to continue to test on "older" Ubuntu versions like 20.04 r-lib/actions#612

- {os: ubuntu-18.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest", custom: 'no-cpp11test'}
- {os: ubuntu-18.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest", custom: 'gcc 4.8'}
- {os: ubuntu-18.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest"}
- {os: ubuntu-18.04, r: 'oldrel', rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest"}
- {os: ubuntu-18.04, r: '3.6', rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest"}
- {os: ubuntu-18.04, r: '3.5', rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest"}
- {os: ubuntu-18.04, r: '3.4', rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest"}
# Use 3.6 to trigger usage of RTools35
- {os: windows-latest, r: '3.6'}

# Use older ubuntu to maximise backward compatibility
- {os: ubuntu-20.04, r: 'devel', http-user-agent: 'release'}
- {os: ubuntu-20.04, r: 'release'}
- {os: ubuntu-20.04, r: 'release', custom: 'no-cpp11test'}
- {os: ubuntu-20.04, r: 'release', custom: 'gcc 4.8'}
- {os: ubuntu-20.04, r: 'oldrel-1'}
- {os: ubuntu-20.04, r: 'oldrel-2'}
- {os: ubuntu-20.04, r: 'oldrel-3'}
- {os: ubuntu-20.04, r: 'oldrel-4'}

env:
RSPM: ${{ matrix.config.rspm }}
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
R_KEEP_PKG_SOURCE: yes

steps:
- uses: actions/checkout@v2

- uses: r-lib/actions/setup-r@v1
id: install-r
- uses: r-lib/actions/setup-pandoc@v2

- uses: r-lib/actions/setup-r@v2
with:
r-version: ${{ matrix.config.r }}
http-user-agent: ${{ matrix.config.http-user-agent }}

- uses: r-lib/actions/setup-pandoc@v1

- name: Install pak and query dependencies
run: |
install.packages("pak", repos = "https://r-lib.github.io/p/pak/dev/")
saveRDS(pak::pkg_deps("local::.", dependencies = TRUE), ".github/r-depends.rds")
shell: Rscript {0}

- name: Restore R package cache
uses: actions/cache@v2
with:
path: |
${{ env.R_LIBS_USER }}/*
!${{ env.R_LIBS_USER }}/pak
key: ${{ matrix.config.os }}-${{ steps.install-r.outputs.installed-r-version }}-1-${{ hashFiles('.github/r-depends.rds') }}
restore-keys: ${{ matrix.config.os }}-${{ steps.install-r.outputs.installed-r-version }}-1-

- name: Install system dependencies
if: runner.os == 'Linux'
run: |
pak::local_system_requirements(execute = TRUE)
pak::pkg_system_requirements("rcmdcheck", execute = TRUE)
shell: Rscript {0}
use-public-rspm: true

- name : Install compiler version
# We check on this old compiler specifically to support CentOS 7,
# which uses this gcc version. RStudio products support CentOS 7 through
# June 2024.
# https://github.com/r-lib/cpp11/pull/78
# https://www.rstudio.com/about/platform-support/
# Ubuntu 20.04 technically dropped support for gcc 4.8, so we have to
# add old archives back in manually to install it
# https://github.com/r-lib/cpp11/pull/279
if: matrix.config.custom == 'gcc 4.8'
run: |
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial main" | sudo tee -a /etc/apt/sources.list
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial universe" | sudo tee -a /etc/apt/sources.list
sudo apt update
Comment on lines +57 to +69
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have retained support for testing gcc 4.8, but it wasn't easy. The move to Ubuntu 20.04 means that sudo apt-get install -y g++-4.8 doesn't work out of the box anymore.

The way to fix this is to add older archives into the sources.list manually. This was proposed on a number of stack overflow questions, like these
https://stackoverflow.com/questions/68805002/i-need-to-install-gcc4-9-on-ubuntu-20-04-matlab-mex
https://askubuntu.com/questions/1036108/install-gcc-4-9-at-ubuntu-18-04

Corresponds to this commit 6a2994a

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling out the fact that RStudio are doing this to support CentOS7, I wonder if it would make sense to move to using docker for CentOS 7 explicitly, in order to better replicate the environment you are trying to support, IIRC you would have to use Docker for everything then, so maybe not worth it, or you could run this in a separate workflow...

Copy link
Member Author

@DavisVaughan DavisVaughan Aug 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened #281 to consider this

Arrow uses a docker image in this way, so we could probably do the same

sudo apt-get install -y g++-4.8
mkdir ~/.R/
echo $'CXX1X=g++-4.8\nCXX11=g++-4.8' >> ~/.R/Makevars
echo $'CXX1X=g++-4.8\nCXX11=g++-4.8\nCC=gcc-4.8' >> ~/.R/Makevars
Copy link
Member Author

@DavisVaughan DavisVaughan Aug 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The addition of CC is fairly tricky. If you don't add this, the FAQ vignette fails to build with errors about -fdiagnostics-color=always being a flag that isn't supported. Note that the FAQ vignette has {cpp11} knitr chunks that it compiles, and while it was compiling those chunks it was indeed setting the -fdiagnostics-color=always flag.

Some research turned up that this flag was only supported in gcc >=4.9, so gcc is right that it shouldn't be there.

It turns out this flag was being added by pkgbuild during pkgbuild::build().
https://github.com/r-lib/pkgbuild/blob/80762db02d9273997ed6234f654633359269cc9e/R/compiler-flags.R#L51-L58
https://github.com/r-lib/pkgbuild/blob/80762db02d9273997ed6234f654633359269cc9e/R/build.R#L54

It gets added if it looks like the compiler supports colored diagnostics, i.e. the has_compiler_colored_diagnostics() check there. This helper works by attempting to compile a simple C program with -fdiagnostics-color=always set in the Makevars https://github.com/r-lib/pkgbuild/blob/80762db02d9273997ed6234f654633359269cc9e/R/compiler-flags.R#L76. If the compilation fails, we assume there is no color support. The problem is that it attempts this on a C program which uses a compiler that does support color, but the C++ compiler we set through CXX11 does not.

So my fix for this is to just force the C compiler to also be gcc 4.8 so this helper returns FALSE.

Reported to pkgbuild here r-lib/pkgbuild#141

Corresponds to this commit f59c833

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, that sounds right, I wonder if pkgbuild should test both the CC and CXX points.

Alternatively perhaps it is better practice to use the same toolchain for C and C++ anyway (even though there is no C code being compiled here)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if pkgbuild should test both the CC and CXX points.

Gabor and I are talking about exactly that in r-lib/pkgbuild#141


- name: install macOS system dependecies
- name: Install macOS system dependencies
if: runner.os == 'macOS'
run: brew install --cask xquartz

- name: Install dependencies
run: |
pak::local_install_dev_deps(upgrade = TRUE)
pak::pkg_install("rcmdcheck")
shell: Rscript {0}
- uses: r-lib/actions/setup-r-dependencies@v2
with:
extra-packages: any::rcmdcheck
needs: check

- name: Install cpp11 and cpp11test
- name: Install cpp11test
if: matrix.config.custom != 'no-cpp11test'
run: |
options(warn = 2)
install.packages(".", repos = NULL, type = "source")
pak::local_install_dev_deps("cpp11test")
pak::local_install_dev_deps("cpp11test", dependencies = TRUE)
install.packages("cpp11test", repos = NULL, INSTALL_opts = "--install-tests", type = "source")
shell: Rscript {0}

- name: Session info
run: |
options(width = 100)
pkgs <- installed.packages()[, "Package"]
sessioninfo::session_info(pkgs, include_base = TRUE)
shell: Rscript {0}

- name: Check
env:
_R_CHECK_CRAN_INCOMING_: false
run: rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning", check_dir = "check")
shell: Rscript {0}
- uses: r-lib/actions/check-r-package@v2
with:
upload-snapshots: true

- name: Run cpp11test tests
if: matrix.config.custom != 'no-cpp11test'
Expand All @@ -122,20 +100,3 @@ jobs:
library(cpp11test)
test_check("cpp11test", reporter = "progress")
shell: Rscript {0}

- name: Show testthat output
if: always()
run: find check -name 'testthat.Rout*' -exec cat '{}' \; || true
shell: bash

- name: Upload check results
if: failure()
uses: actions/upload-artifact@main
with:
name: ${{ matrix.config.os }}-r${{ matrix.config.r }}-results
path: check

- name: Don't use tar from old Rtools to store the cache
if: ${{ runner.os == 'Windows' && startsWith(steps.install-r.outputs.installed-r-version, '3.6' ) }}
shell: bash
run: echo "C:/Program Files/Git/usr/bin" >> $GITHUB_PATH
2 changes: 1 addition & 1 deletion .github/workflows/format.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ name: format_check

jobs:
format_check:
runs-on: ubuntu-18.04
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2

Expand Down
35 changes: 24 additions & 11 deletions .github/workflows/pkgdown.yaml
Original file line number Diff line number Diff line change
@@ -1,35 +1,48 @@
# Workflow derived from https://github.com/r-lib/actions/tree/master/examples
# Workflow derived from https://github.com/r-lib/actions/tree/v2/examples
# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
on:
push:
branches: [main, master]
tags: ['*']
pull_request:
branches: [main, master]
release:
types: [published]
workflow_dispatch:

name: pkgdown

jobs:
pkgdown:
runs-on: ubuntu-latest
# Only restrict concurrency for non-PR jobs
concurrency:
group: pkgdown-${{ github.event_name != 'pull_request' || github.run_id }}
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v2

- uses: r-lib/actions/setup-pandoc@v1
- uses: r-lib/actions/setup-pandoc@v2

- uses: r-lib/actions/setup-r@v1
- uses: r-lib/actions/setup-r@v2
with:
use-public-rspm: true

- uses: r-lib/actions/setup-r-dependencies@v1
- uses: r-lib/actions/setup-r-dependencies@v2
with:
extra-packages: local::. local::cpp11test pkgdown
extra-packages: any::pkgdown, local::., local::cpp11test
Comment on lines -26 to +33
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retained installation of cpp11test here

needs: website

- name: Deploy package
- name: Build site
env:
CPP11_EVAL: true
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In particular, made sure to retain this!

run: |
git config --local user.name "$GITHUB_ACTOR"
git config --local user.email "[email protected]"
Rscript -e 'pkgdown::deploy_to_branch(new_process = FALSE)'
run: pkgdown::build_site_github_pages(new_process = FALSE, install = FALSE)
shell: Rscript {0}

- name: Deploy to GitHub pages 🚀
if: github.event_name != 'pull_request'
uses: JamesIves/[email protected]
with:
clean: false
branch: gh-pages
folder: docs
62 changes: 43 additions & 19 deletions .github/workflows/pr-commands.yaml
Original file line number Diff line number Diff line change
@@ -1,55 +1,79 @@
# Workflow derived from https://github.com/r-lib/actions/tree/v2/examples
# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
on:
issue_comment:
types: [created]

name: Commands

jobs:
document:
if: startsWith(github.event.comment.body, '/document')
if: ${{ github.event.issue.pull_request && (github.event.comment.author_association == 'MEMBER' || github.event.comment.author_association == 'OWNER') && startsWith(github.event.comment.body, '/document') }}
name: document
runs-on: macOS-latest
runs-on: ubuntu-latest
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v2
- uses: r-lib/actions/pr-fetch@v1

- uses: r-lib/actions/pr-fetch@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- uses: r-lib/actions/setup-r@v1
- name: Install dependencies
run: Rscript -e 'install.packages(c("remotes", "roxygen2"))' -e 'remotes::install_deps(dependencies = TRUE)'

- uses: r-lib/actions/setup-r@v2
with:
use-public-rspm: true

- uses: r-lib/actions/setup-r-dependencies@v2
with:
extra-packages: any::roxygen2
needs: pr-document

- name: Document
run: Rscript -e 'roxygen2::roxygenise()'
run: roxygen2::roxygenise()
shell: Rscript {0}

- name: commit
run: |
git config --local user.email "[email protected]"
git config --local user.name "GitHub Actions"
git config --local user.name "$GITHUB_ACTOR"
git config --local user.email "[email protected]"
git add man/\* NAMESPACE
git commit -m 'Document'
- uses: r-lib/actions/pr-push@v1

- uses: r-lib/actions/pr-push@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}

style:
if: startsWith(github.event.comment.body, '/style')
if: ${{ github.event.issue.pull_request && (github.event.comment.author_association == 'MEMBER' || github.event.comment.author_association == 'OWNER') && startsWith(github.event.comment.body, '/style') }}
name: style
runs-on: macOS-latest
runs-on: ubuntu-latest
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v2
- uses: r-lib/actions/pr-fetch@v1

- uses: r-lib/actions/pr-fetch@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- uses: r-lib/actions/setup-r@v1

- uses: r-lib/actions/setup-r@v2

- name: Install dependencies
run: Rscript -e 'install.packages("styler")'
run: install.packages("styler")
shell: Rscript {0}

- name: Style
run: Rscript -e 'styler::style_pkg()'
run: styler::style_pkg()
shell: Rscript {0}

- name: commit
run: |
git config --local user.email "[email protected]"
git config --local user.name "GitHub Actions"
git config --local user.name "$GITHUB_ACTOR"
git config --local user.email "[email protected]"
git add \*.R
git commit -m 'Style'
- uses: r-lib/actions/pr-push@v1

- uses: r-lib/actions/pr-push@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
Loading