Garbage Collection Segv in 5.21.10+

[p5pRT]

Migrated from rt.perl.org#125702 (status was 'resolved')

Searchable as RT125702$

[p5pRT]

From @exodist

This is a bug report for perl from exodist7@​gmail.com,
generated with the help of perlbug 1.40 running under perl 5.22.0.

---

I have attached the latest Test-Stream and Test-Simple dists. I have done
this because I have a workaround I intend to release, so using the cpan
versions to reproduce will not be viable. I tried to boil it down to a
minimal reproducxtion case, but failed.

If you install the attached Test-Stream, then run the tests for the
attached Test-Simple, 3 tests fail. The 3 failing tests have one thing in
common, they run 'exit 0' inside an END block. In all 3 cases you can
replace the 'exit 0' with '$? = 0' to prevent the segv and make the tests
pass.

The problem showed up from this commit​:
Test-More/test-more@d19ff4f

Adding a reference to $self back into that codeblock causes the segv to go
away. (Example​: return $self)

This issue only occurs on newer perls, 5.21.10+ or so based on these
results​: http​://matrix.cpantesters.org/?dist=Test-Simple+1.302007_003

Perhaps a bisect to reveal the perl core commit that causes it to break
will help? I am afraid my ability to debug these kinds of things in
perl-core is limited.

---

---

Flags​:
  category=core
  severity=medium

---

Site configuration information for perl 5.22.0​:

Configured by exodist at Mon Jul 27 19​:49​:55 PDT 2015.

Summary of my perl5 (revision 5 version 22 subversion 0) configuration​:

  Platform​:
  osname=linux, osvers=3.13.0-44-generic,
archname=x86_64-linux-thread-multi
  uname='linux work 3.13.0-44-generic #73-ubuntu smp tue dec 16 00​:22​:43
utc 2014 x86_64 x86_64 x86_64 gnulinux '
  config_args='-de -Dprefix=/home/exodist/perl5/perlbrew/perls/newtest
-Dusethreads
-Aeval​:scriptdir=/home/exodist/perl5/perlbrew/perls/newtest/bin'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O2',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
  ccversion='', gccversion='4.8.4', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678,
doublekind=3
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16,
longdblkind=3
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.19'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib
-fstack-protector'

---

@​INC for perl 5.22.0​:

/home/exodist/perl5/perlbrew/perls/newtest/lib/site_perl/5.22.0/x86_64-linux-thread-multi
  /home/exodist/perl5/perlbrew/perls/newtest/lib/site_perl/5.22.0

/home/exodist/perl5/perlbrew/perls/newtest/lib/5.22.0/x86_64-linux-thread-multi
  /home/exodist/perl5/perlbrew/perls/newtest/lib/5.22.0
  .

---

Environment for perl 5.22.0​:
  HOME=/home/exodist
  LANG=en_US.UTF-8
  LANGUAGE=en_US
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)

PATH=/home/exodist/perl5/perlbrew/bin​:/home/exodist/perl5/perlbrew/perls/newtest/bin​:/home/exodist/bin​:/home/exodist/bin​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/sbin​:/bin​:/usr/games​:/usr/local/games
  PERLBREW_BASHRC_VERSION=0.66
  PERLBREW_HOME=/home/exodist/.perlbrew
  PERLBREW_MANPATH=/home/exodist/perl5/perlbrew/perls/newtest/man

PERLBREW_PATH=/home/exodist/perl5/perlbrew/bin​:/home/exodist/perl5/perlbrew/perls/newtest/bin
  PERLBREW_PERL=newtest
  PERLBREW_ROOT=/home/exodist/perl5/perlbrew
  PERLBREW_VERSION=0.66
  PERL_BADLANG (unset)
  SHELL=/usr/bin/zsh

[p5pRT]

From @exodist

Test-Simple-1.302007_003.tar.gz

[p5pRT]

From @exodist

Test-Stream-1.302009.tar.gz

[p5pRT]

From @iabyn

On Mon, Jul 27, 2015 at 08​:45​:14PM -0700, Chad Granum wrote​:

# New Ticket Created by Chad Granum
# Please include the string​: [perl #125702]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=125702 >

This is a bug report for perl from exodist7@​gmail.com,
generated with the help of perlbug 1.40 running under perl 5.22.0.

-----------------------------------------------------------------

I have attached the latest Test-Stream and Test-Simple dists. I have done
this because I have a workaround I intend to release, so using the cpan
versions to reproduce will not be viable. I tried to boil it down to a
minimal reproducxtion case, but failed.

If you install the attached Test-Stream, then run the tests for the
attached Test-Simple, 3 tests fail. The 3 failing tests have one thing in
common, they run 'exit 0' inside an END block. In all 3 cases you can
replace the 'exit 0' with '$? = 0' to prevent the segv and make the tests
pass.

The problem showed up from this commit​:
Test-More/test-more@d19ff4f

Adding a reference to $self back into that codeblock causes the segv to go
away. (Example​: return $self)

This issue only occurs on newer perls, 5.21.10+ or so based on these
results​: http​://matrix.cpantesters.org/?dist=Test-Simple+1.302007_003

Perhaps a bisect to reveal the perl core commit that causes it to break
will help? I am afraid my ability to debug these kinds of things in
perl-core is limited.

It bisects to this​:

  commit 96d7c88
  Author​: Father Chrysostomos <sprout@​cpan.org>
  AuthorDate​: Wed Sep 17 21​:16​:58 2014 -0800
  Commit​: Father Chrysostomos <sprout@​cpan.org>
  CommitDate​: Sun Nov 2 18​:23​:43 2014 -0800

  [perl #57512] Warnings for implicitly closed handles
 
  If the implicit close() fails, warn about it, mentioning $! in the
  message. This is a default warning in the io category.
 
  We do this in two spots, sv_clear and gp_free. While sv_clear would
  be sufficient to get the warning emitted, the warning won’t contain
  the name of the handle when called from there, because lone IO thing-
  ies are nameless. Doing it also when a GV’s glob pointer is freed--as
  long as the IO thingy in there has a reference count of 1--allows the
  name to be included in the message, because we still have the glob,
  which is where the name is stored.

Looking at a stack backtrace from valgrind,

  ==26423== Invalid read of size 8
  ==26423== at 0x54F7FC​: Perl_ckwarn_d (util.c​:1978)
  ==26423== by 0x48109C​: Perl_gp_free (gv.c​:2553)
  ==26423== by 0x5E21FC​: Perl_sv_clear (sv.c​:6542)
  ==26423== by 0x5E427E​: Perl_sv_free2 (sv.c​:6884)
  ==26423== by 0x4D563B​: S_SvREFCNT_dec_NN (inline.h​:177)
  ==26423== by 0x4D66ED​: Perl_cv_undef_flags (pad.c​:450)
  ==26423== by 0x4D5AB1​: Perl_cv_undef (pad.c​:301)
  ==26423== by 0x5E17D7​: Perl_sv_clear (sv.c​:6461)
  ==26423== by 0x5E427E​: Perl_sv_free2 (sv.c​:6884)
  ==26423== by 0x636EA7​: S_SvREFCNT_dec (inline.h​:166)
  ==26423== by 0x63D471​: Perl_leave_scope (scope.c​:973)
  ==26423== by 0x46C0BD​: S_my_exit_jump (perl.c​:5043)
  ==26423== Address 0x40 is not stack'd, malloc'd or (recently) free'd

it's implicitly closing a filehandle, and its checking to see whether
warnings are enabled​:

  util.c​:1978​: if (isLEXWARN_off)
  util.c​:1979​: return PL_dowarn & G_WARN_ON;

and

  warnings.h​:117​:#define isLEXWARN_off (PL_curcop->cop_warnings == pWARN_STD)

So presumably at this point PL_curcop is NULL.

I don't know whether the fix is simply to change isLEXWARN_off to

  (PL_curcop && PL_curcop->cop_warnings == pWARN_STD)

or whether PL_curcop being NULL at that point represents some deeper
malady.

I haven't tried to reduce the test case yet.

--
Never do today what you can put off till tomorrow.

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @iabyn

On Tue, Jul 28, 2015 at 10​:02​:36AM +0100, Dave Mitchell wrote​:

On Mon, Jul 27, 2015 at 08​:45​:14PM -0700, Chad Granum wrote​:

# New Ticket Created by Chad Granum
# Please include the string​: [perl #125702]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=125702 >

This is a bug report for perl from exodist7@​gmail.com,
generated with the help of perlbug 1.40 running under perl 5.22.0.

-----------------------------------------------------------------

I have attached the latest Test-Stream and Test-Simple dists. I have done
this because I have a workaround I intend to release, so using the cpan
versions to reproduce will not be viable. I tried to boil it down to a
minimal reproducxtion case, but failed.

If you install the attached Test-Stream, then run the tests for the
attached Test-Simple, 3 tests fail. The 3 failing tests have one thing in
common, they run 'exit 0' inside an END block. In all 3 cases you can
replace the 'exit 0' with '$? = 0' to prevent the segv and make the tests
pass.

The problem showed up from this commit​:
Test-More/test-more@d19ff4f

Adding a reference to $self back into that codeblock causes the segv to go
away. (Example​: return $self)

This issue only occurs on newer perls, 5.21.10+ or so based on these
results​: http​://matrix.cpantesters.org/?dist=Test-Simple+1.302007_003

Perhaps a bisect to reveal the perl core commit that causes it to break
will help? I am afraid my ability to debug these kinds of things in
perl-core is limited.

It bisects to this​:

commit 96d7c88819733eaaba892177a967d9e898b2b924
Author&#8203;:     Father Chrysostomos \<sprout@&#8203;cpan\.org>
AuthorDate&#8203;: Wed Sep 17 21&#8203;:16&#8203;:58 2014 \-0800
Commit&#8203;:     Father Chrysostomos \<sprout@&#8203;cpan\.org>
CommitDate&#8203;: Sun Nov 2 18&#8203;:23&#8203;:43 2014 \-0800

\[perl \#57512\] Warnings for implicitly closed handles

If the implicit close\(\) fails\, warn about it\, mentioning $\! in the
message\.  This is a default warning in the io category\.

We do this in two spots\, sv\_clear and gp\_free\.  While sv\_clear would
be sufficient to get the warning emitted\, the warning won’t contain
the name of the handle when called from there\, because lone IO thing\-
ies are nameless\.  Doing it also when a GV’s glob pointer is freed\-\-as
long as the IO thingy in there has a reference count of 1\-\-allows the
name to be included in the message\, because we still have the glob\,
which is where the name is stored\.

Looking at a stack backtrace from valgrind,

==26423== Invalid read of size 8
==26423==    at 0x54F7FC​: Perl\_ckwarn\_d \(util\.c​:1978\)
==26423==    by 0x48109C​: Perl\_gp\_free \(gv\.c​:2553\)
==26423==    by 0x5E21FC​: Perl\_sv\_clear \(sv\.c​:6542\)
==26423==    by 0x5E427E​: Perl\_sv\_free2 \(sv\.c​:6884\)
==26423==    by 0x4D563B​: S\_SvREFCNT\_dec\_NN \(inline\.h​:177\)
==26423==    by 0x4D66ED​: Perl\_cv\_undef\_flags \(pad\.c​:450\)
==26423==    by 0x4D5AB1​: Perl\_cv\_undef \(pad\.c​:301\)
==26423==    by 0x5E17D7​: Perl\_sv\_clear \(sv\.c​:6461\)
==26423==    by 0x5E427E​: Perl\_sv\_free2 \(sv\.c​:6884\)
==26423==    by 0x636EA7​: S\_SvREFCNT\_dec \(inline\.h​:166\)
==26423==    by 0x63D471​: Perl\_leave\_scope \(scope\.c​:973\)
==26423==    by 0x46C0BD​: S\_my\_exit\_jump \(perl\.c​:5043\)
==26423==  Address 0x40 is not stack'd\, malloc'd or \(recently\) free'd

it's implicitly closing a filehandle, and its checking to see whether
warnings are enabled​:

util\.c​:1978​:    if \(isLEXWARN\_off\)
util\.c​:1979​:        return PL\_dowarn & G\_WARN\_ON;

and

warnings\.h​:117​:\#define isLEXWARN\_off    \(PL\_curcop\->cop\_warnings == pWARN\_STD\)

So presumably at this point PL_curcop is NULL.

I don't know whether the fix is simply to change isLEXWARN_off to

\(PL\_curcop && PL\_curcop\->cop\_warnings == pWARN\_STD\)

or whether PL_curcop being NULL at that point represents some deeper
malady.

I haven't tried to reduce the test case yet.

In the meantime it appears to have been fixed by v5.25.2-109-ga2637ca;
so closing.

--
Red sky at night - gerroff my land!
Red sky at morning - gerroff my land!
  -- old farmers' sayings #14

[p5pRT]

@iabyn - Status changed from 'open' to 'resolved'