Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[linux] test_disk_partitions failing in build sandbox #1912

Closed
andir opened this issue Jan 31, 2021 · 2 comments
Closed

[linux] test_disk_partitions failing in build sandbox #1912

andir opened this issue Jan 31, 2021 · 2 comments

Comments

@andir
Copy link

andir commented Jan 31, 2021

Summary

  • OS: Linux, NixOS
  • Architecture: 64bit
  • Psutil version: 5.8.0
  • Python version: 3.8
  • Type: tests

Description

When executing the tests in a build sandbox where /proc/mounts, /proc/filesystems & /proc/self/mounts look like pasted below I am running into the following test failure:

         def test_disk_partitions(self):
                 self.assertIsInstance(nt.device, str)
             def check_ntuple(nt):
                 self.assertIsInstance(nt.mountpoint, str)
                 self.assertIsInstance(nt.fstype, str)
                 self.assertIsInstance(nt.opts, str)
                 self.assertIsInstance(nt.maxfile, (int, type(None)))
                 self.assertIsInstance(nt.maxpath, (int, type(None)))
                     self.assertGreater(nt.maxfile, 0)
                 if nt.maxfile is not None and not GITHUB_ACTIONS:
                 if nt.maxpath is not None:
                     self.assertGreater(nt.maxpath, 0)
    
             # all = False
             ls = psutil.disk_partitions(all=False)
     >       self.assertTrue(ls, msg=ls)
     E       AssertionError: [] is not true : []
$ cat /proc/mounts
overlay / overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /bin/sh overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
tmpfs /build tmpfs rw,relatime 0 0
devtmpfs /dev/full devtmpfs rw,relatime,size=6888252k,nr_inodes=861031,mode=755 0 0
devtmpfs /dev/null devtmpfs rw,relatime,size=6888252k,nr_inodes=861031,mode=755 0 0
devtmpfs /dev/random devtmpfs rw,relatime,size=6888252k,nr_inodes=861031,mode=755 0 0
devtmpfs /dev/tty devtmpfs rw,relatime,size=6888252k,nr_inodes=861031,mode=755 0 0
devtmpfs /dev/zero devtmpfs rw,relatime,size=6888252k,nr_inodes=861031,mode=755 0 0
devtmpfs /dev/urandom devtmpfs rw,relatime,size=6888252k,nr_inodes=861031,mode=755 0 0
overlay /nix/store/0cbzck6xac05rpg8wnz943vmikl1ws1d-libidn2-2.3.0 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/1pms77gm9l4rw8k180wmklagcwsya605-gnumake-4.3 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/1s7h0ll18n2988ax3hhg2pjyni5jrf36-ed-1.16 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/40sgch0d26mqjmrfvg6iq9vmkj2m5b7g-patch-2.7.6 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/48j0w68fyd6lpwwfkjghphxvysgjl84x-libunistring-0.9.10 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/6qyl2yyqjrciypyb8728llsm2mqrhqsp-gnused-4.8 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/782n0xrdgdl1a32nvpdfnrx06xxa1zlp-gzip-1.10 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/7l68hmbqhmxib2wlmwyd5ib5czfyb25g-bzip2-1.0.6.0.1 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/87ipxqn2ah5ba282cnip9z4mm3dbczjc-coreutils-8.32 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/87l6ag9v5pa96axxg756hj4pfcajsmbh-findutils-4.7.0 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/9ikwdf54fsq002g4d36vp6r6qvwk5n3a-xz-5.2.5-bin overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/c5jgwpwxc1nxl9gzd9m4wlalrw7sr0z7-bzip2-1.0.6.0.1-bin overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/cfvww6slpwnvz19hf169ickmjda4l6jb-patchelf-0.12 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/ck3kak4rp8j3649bwy0vk114qpy53q7g-diffutils-3.7 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/f2l18n620bw7cmvrs1vh35kry3nrkfy0-xz-5.2.5 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/gmky7n98rw8fv6h227zc6085bh0bcz8x-stdenv-linux overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/ipd8n0w9pwhfg1hsna9gv1z46y3j76d2-gnugrep-3.6 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/kl6lr3czkbnr6m5crcy8ffwfzbj8a22i-bash-4.4-p23 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/sylh15pml1cphys55sy0dr28cp8c42d0-attr-2.4.48 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/v3aw5ip513vsh6r36fs07niv0b3f7f8l-pcre-8.44 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/vbr3p4y2azrgwq2s7h2hbip8x06w50z5-glibc-2.32-24 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/vkbdjkwqf7qxhxwd86cik7nfdppj04n2-gnutar-1.32 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/vnf5x5ggba946dpan66r1npqj4slrinw-gawk-5.1.0 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/vy7phrnzkbjvsf9j5bkg2f3z8mf0fabv-acl-2.2.53 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/xsqxamd86mh21dk0b775acaqgdijgf6p-gcc-10.2.0-lib overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
none /proc proc rw,relatime 0 0
overlay /nix/store/zi90rxslsm4mlr46l2xws1rm94g7pk8p-busybox-1.31.1-x86_64-unknown-linux-musl overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
none /dev/shm tmpfs rw,relatime,size=3477052k 0 0
none /dev/pts devpts rw,relatime,mode=620,ptmxmode=000 0 0
$ cat /proc/filesystems
nodev   sysfs
nodev   tmpfs
nodev   bdev
nodev   proc
nodev   cgroup
nodev   cgroup2
nodev   cpuset
nodev   devtmpfs
nodev   binfmt_misc
nodev   configfs
nodev   debugfs
nodev   tracefs
nodev   securityfs
nodev   sockfs
nodev   bpf
nodev   pipefs
nodev   ramfs
nodev   hugetlbfs
nodev   devpts
        ext3
        ext2
        ext4
        squashfs
nodev   autofs
        fuseblk
nodev   fusectl
nodev   fuse
nodev   overlay
nodev   mqueue
nodev   oprofilefs
$ cat /etc/mtab || :
cat: /etc/mtab: No such file or directory
$ cat /proc/self/mounts
overlay / overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /bin/sh overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
tmpfs /build tmpfs rw,relatime 0 0
devtmpfs /dev/full devtmpfs rw,relatime,size=6888252k,nr_inodes=861031,mode=755 0 0
devtmpfs /dev/null devtmpfs rw,relatime,size=6888252k,nr_inodes=861031,mode=755 0 0
devtmpfs /dev/random devtmpfs rw,relatime,size=6888252k,nr_inodes=861031,mode=755 0 0
devtmpfs /dev/tty devtmpfs rw,relatime,size=6888252k,nr_inodes=861031,mode=755 0 0
devtmpfs /dev/urandom devtmpfs rw,relatime,size=6888252k,nr_inodes=861031,mode=755 0 0
devtmpfs /dev/zero devtmpfs rw,relatime,size=6888252k,nr_inodes=861031,mode=755 0 0
overlay /nix/store/0cbzck6xac05rpg8wnz943vmikl1ws1d-libidn2-2.3.0 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/1pms77gm9l4rw8k180wmklagcwsya605-gnumake-4.3 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/1s7h0ll18n2988ax3hhg2pjyni5jrf36-ed-1.16 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/40sgch0d26mqjmrfvg6iq9vmkj2m5b7g-patch-2.7.6 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/48j0w68fyd6lpwwfkjghphxvysgjl84x-libunistring-0.9.10 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/6qyl2yyqjrciypyb8728llsm2mqrhqsp-gnused-4.8 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/782n0xrdgdl1a32nvpdfnrx06xxa1zlp-gzip-1.10 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/7l68hmbqhmxib2wlmwyd5ib5czfyb25g-bzip2-1.0.6.0.1 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/87ipxqn2ah5ba282cnip9z4mm3dbczjc-coreutils-8.32 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/87l6ag9v5pa96axxg756hj4pfcajsmbh-findutils-4.7.0 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/9ikwdf54fsq002g4d36vp6r6qvwk5n3a-xz-5.2.5-bin overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/c5jgwpwxc1nxl9gzd9m4wlalrw7sr0z7-bzip2-1.0.6.0.1-bin overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/cfvww6slpwnvz19hf169ickmjda4l6jb-patchelf-0.12 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/f2l18n620bw7cmvrs1vh35kry3nrkfy0-xz-5.2.5 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/ck3kak4rp8j3649bwy0vk114qpy53q7g-diffutils-3.7 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/ipd8n0w9pwhfg1hsna9gv1z46y3j76d2-gnugrep-3.6 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/gmky7n98rw8fv6h227zc6085bh0bcz8x-stdenv-linux overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/kl6lr3czkbnr6m5crcy8ffwfzbj8a22i-bash-4.4-p23 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/sylh15pml1cphys55sy0dr28cp8c42d0-attr-2.4.48 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/v3aw5ip513vsh6r36fs07niv0b3f7f8l-pcre-8.44 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/vbr3p4y2azrgwq2s7h2hbip8x06w50z5-glibc-2.32-24 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/vkbdjkwqf7qxhxwd86cik7nfdppj04n2-gnutar-1.32 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/vnf5x5ggba946dpan66r1npqj4slrinw-gawk-5.1.0 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/vy7phrnzkbjvsf9j5bkg2f3z8mf0fabv-acl-2.2.53 overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/xsqxamd86mh21dk0b775acaqgdijgf6p-gcc-10.2.0-lib overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
overlay /nix/store/zi90rxslsm4mlr46l2xws1rm94g7pk8p-busybox-1.31.1-x86_64-unknown-linux-musl overlay rw,relatime,lowerdir=/nix/store-ro:/nix/store,upperdir=/nix/store-rw,workdir=/nix/store-wrk 0 0
none /proc proc rw,relatime 0 0
none /dev/shm tmpfs rw,relatime,size=3477052k 0 0
none /dev/pts devpts rw,relatime,mode=620,ptmxmode=000 0 0

There are clearly mounted filesystems yet the test is failing with an empty list. Is it expecting some specific partition type to exit on all systems that isn't present here?

@andir andir added the bug label Jan 31, 2021
andir added a commit to andir/nixpkgs that referenced this issue Jan 31, 2021
On systems where swap is available and fully used tests fail like this:

>    def test_swap_memory(self):
>         mem = psutil.swap_memory()
>         self.assertEqual(
>             mem._fields, ('total', 'used', 'free', 'percent', 'sin', 'sout'))
>
>         assert mem.total >= 0, mem
>         assert mem.used >= 0, mem
>         if mem.total > 0:
>             # likely a system with no swap partition
> >           assert mem.free > 0, mem
> E           AssertionError: sswap(total=1048567808, used=1048567808, free=0, percent=100.0, sin=50741248, sout=1316823040)
> E           assert 0 > 0
> E            +  where 0 = sswap(total=1048567808, used=1048567808, free=0, percent=100.0, sin=50741248, sout=1316823040).free

Upstream bug report: giampaolo/psutil#1911

Systems where /proc/mounts doesn't expose any mount points to build the
tests fail like this:

>     def test_disk_partitions(self):
>             self.assertIsInstance(nt.device, str)
>         def check_ntuple(nt):
>             self.assertIsInstance(nt.mountpoint, str)
>             self.assertIsInstance(nt.fstype, str)
>             self.assertIsInstance(nt.opts, str)
>             self.assertIsInstance(nt.maxfile, (int, type(None)))
>             self.assertIsInstance(nt.maxpath, (int, type(None)))
>                 self.assertGreater(nt.maxfile, 0)
>             if nt.maxfile is not None and not GITHUB_ACTIONS:
>             if nt.maxpath is not None:
>                 self.assertGreater(nt.maxpath, 0)
>
>         # all = False
>         ls = psutil.disk_partitions(all=False)
> >       self.assertTrue(ls, msg=ls)
> E       AssertionError: [] is not true : []

Upstream bug report: giampaolo/psutil#1912
andir added a commit to andir/nixpkgs that referenced this issue Jan 31, 2021
Tests are very flaky and depend on the exact execution environment. It
is weird that it has gone well so far but started causing issues across
many of my builders.

Also started rewording some of the comments in the expression so
hopefully the next person looking at this has an easier time to
understand it.

On systems where swap is available and fully used tests fail like this:

>    def test_swap_memory(self):
>         mem = psutil.swap_memory()
>         self.assertEqual(
>             mem._fields, ('total', 'used', 'free', 'percent', 'sin', 'sout'))
>
>         assert mem.total >= 0, mem
>         assert mem.used >= 0, mem
>         if mem.total > 0:
>             # likely a system with no swap partition
> >           assert mem.free > 0, mem
> E           AssertionError: sswap(total=1048567808, used=1048567808, free=0, percent=100.0, sin=50741248, sout=1316823040)
> E           assert 0 > 0
> E            +  where 0 = sswap(total=1048567808, used=1048567808, free=0, percent=100.0, sin=50741248, sout=1316823040).free

Upstream bug report: giampaolo/psutil#1911

Systems where /proc/mounts doesn't expose any mount points to build the
tests fail like this:

>     def test_disk_partitions(self):
>             self.assertIsInstance(nt.device, str)
>         def check_ntuple(nt):
>             self.assertIsInstance(nt.mountpoint, str)
>             self.assertIsInstance(nt.fstype, str)
>             self.assertIsInstance(nt.opts, str)
>             self.assertIsInstance(nt.maxfile, (int, type(None)))
>             self.assertIsInstance(nt.maxpath, (int, type(None)))
>                 self.assertGreater(nt.maxfile, 0)
>             if nt.maxfile is not None and not GITHUB_ACTIONS:
>             if nt.maxpath is not None:
>                 self.assertGreater(nt.maxpath, 0)
>
>         # all = False
>         ls = psutil.disk_partitions(all=False)
> >       self.assertTrue(ls, msg=ls)
> E       AssertionError: [] is not true : []

Upstream bug report: giampaolo/psutil#1912
@giampaolo
Copy link
Owner

Unit tests are basically only run by me and github actions CI only, so I'm not surprised failures occur on different distros / setups. Nothing new here. =)

@risicle
Copy link

risicle commented Nov 3, 2021

How are packagers supposed to know they've built a properly functioning version?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants