pandoc -h segmentation fault on windows 7

[stepht]

segv for pandoc -h occurs on both pandoc 2.1.1 and 2.0.6 on windows 7
note that pandoc seems to work fine only -h (--help) is affected

[mb21]

Sounds like #4021 (maybe we should reopen that issue?)

@stepht can you post exact output? and also try pandoc --version?

[stepht]

maybe but pandoc --version is fine.

[jgm]

I'm going to try building with -fexternal-interpreter since some say that helps with this problem. +++ stepht [Jan 20 18 17:27 ]:

…

maybe but pandoc --version is fine. — You are receiving this because you are subscribed to this thread. Reply to this email directly, [1]view it on GitHub, or [2]mute the thread. References 1. #4283 (comment) 2. https://github.com/notifications/unsubscribe-auth/AAAL5DyL89ocQFNA-CZTvbTZIfW6qS5jks5tMiHlgaJpZM4Rlf2s

[jgm]

@stepht Could you try the most recent build, available here?
https://ci.appveyor.com/project/jgm/pandoc/build/artifacts
I used a flag which has been described as a workaround for what might be our problem.
I'd also be curious whether this flag affects performance or functionality at all.

[stepht]

I have just tried the build but the exact same problem persists with -h. sorry

[mb21]

@stepht can you post exact output please?

[agusmba]

FWIW, latest appveyor build is working (only tested the -h option) on my Win 7 box.

[stepht]

Sorry for the delay. I investigated a bit.
First on a conemu64 pandoc -h (as given in the latest build in this thread) goes to completion
without segfault the last two llines being -v and -h.

On mintty (both cyg32 and cyg64) pandoc -h segfaults but gives the same full output as inside conemu64:

% stephan@armen (/c/xopt/pt/pandoc-windows-i386) %
% ./pandoc.exe -h | tail -n 10
--fail-if-warnings
--log=FILE
--bash-completion
--list-input-formats
--list-output-formats
--list-extensions[=FORMAT]
--list-highlight-languages
--list-highlight-styles
-v --version
-h --help
Segmentation fault

Finally on normal cmd console, pandoc -h gives the full output, than hangs about two seconds
and then gives a dialog box with:
pandoc.exe stopped working. windows is looking for a solution
a button gives the possibility to debug.
(the dialog is acctually in spanish as the O.S is a spanish w7 (es_ES so to speak)

I also used the Listsdlls program from sysinternals to check dll loading
but did not see anything suspicious... the wow64 dlls are used correctly
cheers hth

[agusmba]

Ok, reading @stepht I should add that I tested it from conemu also (and it worked).
I'll try to repeat stepht's tests on my win7 box tomorrow (at work) to get a possible confirmation.

EDIT (after testing pandoc -h today):

Combination	Results
conemu + git-bash	OK
conemu + cmd(32)	OK
cmd(32)	full output + stopped working...
cmd(64)	full output + stopped working...
git-bash	full output + segfault
vscode terminal + git-bash	full output + segfault

My box is also a spanish w7 like stepht 's

Pandoc under test is from appveyor four days ago (2.0.1 works fine)

[ataulien]

Just reporting that I have got the exact same issue on my Win7-Machine. So far I have tested Pandoc versions 2.1.1 (segfaults) and 2.0.1.1 (works).

This also causes pypandoc to fail to install, since it sees a non-zero exit code when getting the list of supported formats.

[agusmba]

@ataulien could you clarify what pandoc call is segfaulting for you? So far (AFAIK) we had only identified pandoc -h

[ataulien]

It seems to be this invocation of pandoc, which is causing issues: https://github.com/bebraw/pypandoc/blob/master/pypandoc/__init__.py#L347

 p = subprocess.Popen(
        [__pandoc_path, '--list-output-formats'],  # <--------- This is passed to the pandoc-process
        stdin=subprocess.PIPE,
        stderr=subprocess.PIPE,
        stdout=subprocess.PIPE)

    comm = p.communicate()
    out = comm[0].decode().splitlines(False)
    if p.returncode != 0:      # <------ returncode is not 0, this makes it check for an older version, which obviously fails and causes the setup to break
        # try the old version and see if that returns something
        return get_pandoc_formats_pre_1_18()

As seen from the sourcecode, the --list-output-formats seems to cause the segfault. I could reproduce this on the commandline by calling pandoc --list-output-formats directly.

Pandoc also crashes if you give it a garbage argument, like pandoc --whatever, after printing the usual "unknown argument, try --help".

However, using Pandoc like pandoc somefile.md --output test.docx does not result in a crash.

[agusmba]

Ok, after a quick test, it seems that all pandoc (v > 2.0.1) invocations with --list-* argument also segfaults in Windows 7

[jgm]

It would be helpful if each person reporting could list more systematically: (a) which commands give segfaults (just --list-*, also --help, --version? what about conversions, does it matter if you use -s?) (b) is this consistent or does it vary, e.g. across machines or with different versions of pandoc?

[mb21]

Trying to find the common denominator here. So far we have two reports of pandoc --help and relates commands segfaulting on Windows 7 6.1.7601 Service Pack 1 Build 7601 (@RobertZenz and @robinerd are you both 64bit?).

Is it something with that build or can you guys reproduce on other versions of Windows 7?

[robinerd]

@mb21 I'm on a 64-bit system. I'm unfortunately not able to test other windows versions as I'm on a work computer.

[robinerd]

I can also confirm that all --list-* parameters are crashing (given one at a time). Their usual output is still printed before the crash.

EDIT: Running from cmd, getting the windows dialogue saying Pandoc.exe has stopped working. Same commands in a Cygwin bash give Segmentation fault instead.

[robinerd]

Additional observations (don't know if relevant):
Running cmd as admin doesn't change the behaviour.
Tried to cd into pandoc.exe's folder first, same behaviour.
There's approximately 1 second of delay between the last printed output and the crash, applies both for cmd->dialogue and cygwin->segfault (could be operating system related though)

[mb21]

If you can build pandoc from source on the affected system, it would also be interesting to see whether that would fix it (then the problem would only be in our provided windows binary).

[robinerd]

@mb21 I'm trying that now. Ran into some download issues with local proxies at first but it's compiling now. Will let you know the results soon!

[robinerd]

After running stack install --test I got the below output. Can't find any pandoc.exe produced. Please guide me if there are any logs or more details I can provide you with :)

...
lifted-base-0.2.3.11: download
x509-1.7.2: copy/register
hxt-9.3.1.16: build
hslua-module-text-0.1.2.1: configure
unordered-containers-0.2.8.0: copy/register
tasty-hunit-0.9.2: download
attoparsec-0.13.2.0: copy/register
tasty-quickcheck-0.9.1: download
Progress: 87/118Segmentation fault/access violation in generated code

[mb21]

can you try stack install (so you don't have to build the tests)?

[robinerd]

Compiled successfully with stack install, and the problem seems to be gone!
Tested with -h and --list-output-formats, from cmd. Output is produced, and no crash!
Let me know if you need additional support 👍

[mb21]

That's great to know! The problem is then somewhere in our appveyor build... not sure we can change those to work for Windows 7 without breaking others things though... @jgm?

Either way, building from source seems to be a workaround for those of you on Windows 7.

[jgm]

Let's see if this last change helps, using an earlier version of VS.

[RobertZenz]

Just for the log, no change for me with 2.1.3, still crashing.

[VladimirAlexiev]

2.1.3 also crashes for me: cygwin, win7 64bit, mintty or cmd (works fine in conemu).
In cmd it gives this APPCRASH (in mintty just says Segmentation fault):

 Problem Event Name:	APPCRASH
  Application Name:	pandoc.exe
  Application Version:	0.0.0.0
  Application Timestamp:	00000000
  Fault Module Name:	pandoc.exe
  Fault Module Version:	0.0.0.0
  Fault Module Timestamp:	00000000
  Exception Code:	c0000005
  Exception Offset:	02a485a3
  OS Version:	6.1.7601.2.1.0.256.48
  Locale ID:	1033
  Additional Information 1:	97a8
  Additional Information 2:	97a8436993a2b3cc908d142fe2ee7ba2
  Additional Information 3:	5a07
  Additional Information 4:	5a07d4cda277a8856bbc6177ae279d97

It crashes on commands that spit out internal info: --list-input-formats --list-extensions --bash-completion --help (but not --version)

[RobertZenz]

Good news, the 2.2 x64 build works for me again under Windows 7 x64. However, the i386 build still fails in the same way.

[jgm]

That's useful information. Perhaps everyone who is having this problem can fix it by using the 64 bit build? RobertZenz <[email protected]> writes:

…

Good news, the 2.2 x64 build works for me again under Windows 7 x64. However, the i386 build still fails in the same way. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #4283 (comment)

[yomannnn]

Hi, I can confirm that on my Win7 machine the crash disappeared after installing "pandoc-2.2-windows-x86_64.msi". Many thanks for the fix.

[agusmba]

v2.2 64bit is working for me also.
Thanks!

[GoLangsam]

pandoc 2.2.1 i386 build on 32bit Win7 (up-to-date):
pandoc -h and all kinds of listings issue pandoc.exe has stopped working.

[jgm]

@GoLangsam - are you on a 32 bit or 64 bit machine?
Many of the others above who had this issue with the 32-bit build were able to use the 64-bit build without problems.

[GoLangsam]

@jgm - good ol' 32bit it is ☹️

[nehaljwani]

Faced the same issue on pandoc 2.2.3.2 on Windows, 32bit.
(Not so?) Surprisingly, when I download stack.exe (Windows, 32bit) and then run .\stack.exe --help, I see the same crash.

[nehaljwani]

Could be related to https://ghc.haskell.org/trac/ghc/ticket/15154#no1

[jgm]

Nehal J Wani <[email protected]> writes:

Could be related to https://ghc.haskell.org/trac/ghc/ticket/15154#no1

Possibly so! (The clue about stack.exe --help was very useful, thanks.) Note that we build the Windows binary with ghc 8.2.2. (stackage lts-11) I was not able to get the build working with ghc 8.4.3 (stack lts-12), but it would be worth trying again if it could fix this issue. I've added some lts-12 builds to the appveyor matrix to start testing...

[jgm]

Could someone try the 32-bit build here to see if the issue is still there?
https://ci.appveyor.com/project/jgm/pandoc/build/1.0.2784/job/vu57xw797lwkv1hr/artifacts

I've made some improvements to the build process. We're still using ghc 8.2.2, though, for the 32-bit builds, because of segfaults building with 8.4.3.

[jgm]

I'd really like to figure out whether this issue has been resolved. Can any of you who previously got segfaults with the 32-bit windows build try out the package linked in my last comment?
@nehaljwani @GoLangsam @RobertZenz @agusmba @robinerd @stepht

[agusmba]

Hi @jgm, I didn't have problems with the 32-bit windows build, since I was using the 64bit one, but I just tried the 32-bit windows build you provided (zip) on my machine, and got a segfault from cmd.

I also fired up a Win7-32bit VM to test it on a 32bit system and it also segfaults.

Sorry.

Hopefully someone else will have better results.

Edited: Just to make sure the failure was not related to using the zip pack, I also tested the latest appveyor 64bit version (zip) on and it works.

[mb21]

@agusmba uh, segfaults on which commands? -h, --version and/or --list-output-formats or even when doing conversions?

[agusmba]

@mb21 sorry I didn't specify, but I was referring only to the -h or --list-output-formats that were giving us problems. --version works, and conversions too.

[nehaljwani]

Sorry, I was traveling. I tried the new binary. Still segfaults on --help just like before.

[jgm]

OK, thanks for testing. It may be that we need to build on ghc 8.4.3, but currently this is blocked by commercialhaskell/stackage#3385 This is supposed to be fixed in 8.6. We could try building with 8.0, or building with cabal, to see if we get better results.

[jgm]

OK, I've now got pandoc building on appveyor with ghc 8.6.1.1. Can one of you try the latest 32-bit windows binary to see if the segfault issue has been fixed?
https://ci.appveyor.com/project/jgm/pandoc/build/job/0o6vdkvg18yqsmyk/artifacts

[nehaljwani]

Just checked, doesn't seem to segfault now. All good!

[jgm]

Excellent! That makes sense; the segfault issue is mentioned in the ghc 8.6 changelog, so I expected this would fix it.

[nehaljwani]

@jgm Would you be uploading the latest artifacts in the release section for 2.3 or those will be part of the next release?

[jgm]

It will be in the next release. Nehal J Wani <[email protected]> writes:

…

@jgm Would you be uploading the latest artifacts in the release section for 2.3 or those will be part of the next release? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #4283 (comment)