Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Update debian scripts for debhelper-compat 12 #9792

Closed
wants to merge 5 commits into from
Closed

Update debian scripts for debhelper-compat 12 #9792

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
20ec716
Update debian scripts for debhelper-compat 12
callahad Apr 12, 2021
5348d6e
Update docker/Dockerfile-dhvirtualenv
callahad Apr 12, 2021
22020dc
Re-order comment to avoid log spam
callahad Apr 12, 2021
0a56d58
Merge remote-tracking branch 'origin/develop' into debhelper-12
callahad Apr 12, 2021
bd9a08b
Make /etc/default/matrix-synapse optional
callahad Apr 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/9792.misc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Update Debian build scripts to support debhelper-compat level 12.
5 changes: 2 additions & 3 deletions debian/build_virtualenv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,15 +33,14 @@ esac
# Use --builtin-venv to use the better `venv` module from CPython 3.4+ rather
# than the 2/3 compatible `virtualenv`.

# Pin pip to 20.3.4 to fix breakage in 21.0 on py3.5 (xenial)

dh_virtualenv \
--install-suffix "matrix-synapse" \
--builtin-venv \
--python "$SNAKE" \
--upgrade-pip-to="20.3.4" \
--upgrade-pip \
--preinstall="lxml" \
--preinstall="mock" \
--preinstall="wheel" \
--extra-pip-arg="--no-cache-dir" \
--extra-pip-arg="--compile" \
--extras="all,systemd,test"
Expand Down
1 change: 1 addition & 0 deletions debian/changelog
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
matrix-synapse-py3 (1.31.0+nmu1) UNRELEASED; urgency=medium

* Skip tests when DEB_BUILD_OPTIONS contains "nocheck".
* Update Debian build scripts for debhelper-compat level 12.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs moving to the right version, since the world has moved on in the last couple of months


-- Dan Callahan <[email protected]> Mon, 12 Apr 2021 13:07:36 +0000

Expand Down
2 changes: 1 addition & 1 deletion debian/compat
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
9
12
10 changes: 4 additions & 6 deletions debian/control
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,10 @@ Section: contrib/python
Priority: extra
Maintainer: Synapse Packaging team <[email protected]>
# keep this list in sync with the build dependencies in docker/Dockerfile-dhvirtualenv.
# TODO: Remove the dependency on dh-systemd after dropping support for Ubuntu xenial
# On all other supported releases, it's merely a transitional package which
# does nothing but depends on debhelper (> 9.20160709)
Build-Depends:
debhelper (>= 9.20160709) | dh-systemd,
dh-virtualenv (>= 1.1),
debhelper,
dh-virtualenv (>= 1.2.2),
dh-python,
libsystemd-dev,
libpq-dev,
lsb-release,
Expand All @@ -22,7 +20,7 @@ Standards-Version: 3.9.8
Homepage: https://github.com/matrix-org/synapse

Package: matrix-synapse-py3
Architecture: amd64
Architecture: any
Provides: matrix-synapse
Conflicts:
matrix-synapse (<< 0.34.0.1-0matrix2),
Expand Down
31 changes: 0 additions & 31 deletions debian/matrix-synapse-py3.preinst
View file
Open in desktop

This file was deleted.

3 changes: 2 additions & 1 deletion debian/matrix-synapse-py3.triggers
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# Register interest in Python interpreter changes and
# don't make the Python package dependent on the virtualenv package
# processing (noawait)
interest-noawait /usr/bin/python3.5
interest-noawait /usr/bin/python3.6
interest-noawait /usr/bin/python3.7
interest-noawait /usr/bin/python3.8
interest-noawait /usr/bin/python3.9
Comment on lines +6 to +7
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hum; the fact we forgot to do this sooner isn't good. I wonder if we should add a comment somewhere?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(or build this dynamically?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say we could generate this based on our Trove classifiers, but... 😉


# Also provide a symbolic trigger for all dh-virtualenv packages
interest dh-virtualenv-interpreter-update
2 changes: 0 additions & 2 deletions debian/matrix-synapse.default
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion debian/matrix-synapse.service
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Description=Synapse Matrix homeserver
Type=notify
User=matrix-synapse
WorkingDirectory=/var/lib/matrix-synapse
EnvironmentFile=/etc/default/matrix-synapse
EnvironmentFile=-/etc/default/matrix-synapse
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the logic behind removing /etc/default/matrix-synapse?

it feels odd to remove the file but leave this in place.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still suggest people use it in our docs, so we might as well still honor it if present. Plus, existing installations may depend on its behavior.

Systemd upstream considers EnvironmentFile as hack that would ideally go away, but there's no clear replacement other than directly editing the Synapse config or overriding the systemd unit file locally.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly I feel like if we're going to leave the EnvironmentFile directive in place, we should probably also leave an example file there too. In my experience people get scared when they discover a config file doesn't exist at all.

I don't feel too strongly about it though.

ExecStartPre=/opt/venvs/matrix-synapse/bin/python -m synapse.app.homeserver --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/ --generate-keys
ExecStart=/opt/venvs/matrix-synapse/bin/python -m synapse.app.homeserver --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/
ExecReload=/bin/kill -HUP $MAINPID
Expand Down
16 changes: 7 additions & 9 deletions debian/rules
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,12 @@
# assume we only have one package
PACKAGE_NAME:=`dh_listpackages`

override_dh_systemd_enable:
dh_systemd_enable --name=matrix-synapse
override_dh_installsystemd:
dh_installsystemd --name=matrix-synapse

override_dh_installinit:
dh_installinit --name=matrix-synapse

# we don't really want to strip the symbols from our object files.
# we don't really want to strip or compress the symbols from our object files.
override_dh_strip:
override_dh_dwz:

# dh_shlibdeps calls dpkg-shlibdeps, which finds all the binary files
# (executables and shared libs) in the package, and looks for the shared
Expand Down Expand Up @@ -51,7 +49,7 @@ override_dh_shlibdeps:
override_dh_virtualenv:
./debian/build_virtualenv

# We are restricted to compat level 9 (because xenial), so have to
# enable the systemd bits manually.
# TODO: Experiment with --buildsystem=dh_virtualenv
# https://dh-virtualenv.readthedocs.io/en/latest/usage.html#experimental-buildsystem-support
%:
dh $@ --with python-virtualenv --with systemd
dh $@ --with python-virtualenv --buildsystem=pybuild
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note to my future self: previously we used the python_distutils build system; switching to pybuild matches the recommendation at https://wiki.debian.org/Python/LibraryStyleGuide. I don't think there was a good reason why we used python_distutils, other than that it was the default.

2 changes: 0 additions & 2 deletions debian/test/.gitignore
View file
Open in desktop

This file was deleted.

23 changes: 0 additions & 23 deletions debian/test/provision.sh
View file
Open in desktop

This file was deleted.

13 changes: 0 additions & 13 deletions debian/test/stretch/Vagrantfile
View file
Open in desktop

This file was deleted.

10 changes: 0 additions & 10 deletions debian/test/xenial/Vagrantfile
View file
Open in desktop

This file was deleted.

54 changes: 22 additions & 32 deletions docker/Dockerfile-dhvirtualenv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,32 +14,29 @@ ARG distro=""

###
### Stage 0: build a dh-virtualenv
### Based on https://github.com/spotify/dh-virtualenv/blob/master/Dockerfile
###

FROM ${distro} as builder

RUN apt-get update -qq -o Acquire::Languages=none
RUN env DEBIAN_FRONTEND=noninteractive apt-get install \
-yqq --no-install-recommends \
build-essential \
ca-certificates \
devscripts \
equivs \
wget
RUN apt-get update -qq -o Acquire::Languages=none \
&& env DEBIAN_FRONTEND=noninteractive apt-get install \
-yqq --no-install-recommends -o Dpkg::Options::=--force-unsafe-io \
build-essential debhelper devscripts equivs lsb-release \
python3-setuptools \
python3-sphinx python3-mock dh-exec dh-python python3-sphinx-rtd-theme \
&& if test "$(lsb_release -cs)" = "bionic" ; then \
apt-get install -yqq --no-install-recommends -o Dpkg::Options::=--force-unsafe-io \
-t bionic-backports debhelper; fi \
&& apt-get clean && rm -rf "/var/lib/apt/lists"/*

# fetch and unpack the package
# TODO: Upgrade to 1.2.2 once xenial is dropped
RUN mkdir /dh-virtualenv
RUN wget -q -O /dh-virtualenv.tar.gz https://github.com/spotify/dh-virtualenv/archive/ac6e1b1.tar.gz
RUN tar -xv --strip-components=1 -C /dh-virtualenv -f /dh-virtualenv.tar.gz
# fetch and unpack dh-virtualenv 1.2.2 (commit hash used to ensure integrity)
ADD https://github.com/spotify/dh-virtualenv/archive/96963cff2e569bf0a15933f773a8bea7a762e2c4.tar.gz /dh-virtualenv.tar.gz

# install its build deps. We do another apt-cache-update here, because we might
# be using a stale cache from docker build.
RUN apt-get update -qq -o Acquire::Languages=none \
RUN mkdir /dh-virtualenv \
&& cd /dh-virtualenv \
&& env DEBIAN_FRONTEND=noninteractive mk-build-deps -ri -t "apt-get -y --no-install-recommends"

# build it
RUN cd /dh-virtualenv && dpkg-buildpackage -us -uc -b
&& tar --strip-components=1 -xvf /dh-virtualenv.tar.gz \
&& dpkg-buildpackage -us -uc -b

###
### Stage 1
Expand All @@ -55,17 +52,18 @@ ENV distro ${distro}
# http://bugs.python.org/issue19846
ENV LANG C.UTF-8

COPY --from=builder /dh-virtualenv_*.deb /

# Install the build dependencies
#
# NB: keep this list in sync with the list of build-deps in debian/control
# TODO: it would be nice to do that automatically.
# TODO: Remove the dh-systemd stanza after dropping support for Ubuntu xenial
# it's a transitional package on all other, more recent releases
RUN apt-get update -qq -o Acquire::Languages=none \
&& env DEBIAN_FRONTEND=noninteractive apt-get install \
-yqq --no-install-recommends -o Dpkg::Options::=--force-unsafe-io \
build-essential \
debhelper \
dh-python \
devscripts \
libsystemd-dev \
lsb-release \
Expand All @@ -77,16 +75,8 @@ RUN apt-get update -qq -o Acquire::Languages=none \
sqlite3 \
libpq-dev \
xmlsec1 \
&& ( env DEBIAN_FRONTEND=noninteractive apt-get install \
-yqq --no-install-recommends -o Dpkg::Options::=--force-unsafe-io \
dh-systemd || true )

COPY --from=builder /dh-virtualenv_1.2~dev-1_all.deb /

# install dhvirtualenv. Update the apt cache again first, in case we got a
# cached cache from docker the first time.
RUN apt-get update -qq -o Acquire::Languages=none \
&& apt-get install -yq /dh-virtualenv_1.2~dev-1_all.deb
/dh-virtualenv_*.deb \
&& apt-get clean && rm -rf "/var/lib/apt/lists"/*

WORKDIR /synapse/source
ENTRYPOINT ["bash","/synapse/source/docker/build_debian.sh"]
2 changes: 1 addition & 1 deletion docs/systemd-with-workers/system/[email protected]
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Type=notify
NotifyAccess=main
User=matrix-synapse
WorkingDirectory=/var/lib/matrix-synapse
EnvironmentFile=/etc/default/matrix-synapse
EnvironmentFile=-/etc/default/matrix-synapse
ExecStart=/opt/venvs/matrix-synapse/bin/python -m synapse.app.generic_worker --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/ --config-path=/etc/matrix-synapse/workers/%i.yaml
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
Expand Down
2 changes: 1 addition & 1 deletion docs/systemd-with-workers/system/matrix-synapse.service
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Type=notify
NotifyAccess=main
User=matrix-synapse
WorkingDirectory=/var/lib/matrix-synapse
EnvironmentFile=/etc/default/matrix-synapse
EnvironmentFile=-/etc/default/matrix-synapse
ExecStartPre=/opt/venvs/matrix-synapse/bin/python -m synapse.app.homeserver --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/ --generate-keys
ExecStart=/opt/venvs/matrix-synapse/bin/python -m synapse.app.homeserver --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/
ExecReload=/bin/kill -HUP $MAINPID
Expand Down