Google Play crash report mystery thread 1.7 #11493

hrydgard · 2018-10-27T10:55:55Z

It's time for the crash report thread again, preparing for 1.7.1!

Starting off with a golden oldie, this one is not actually new but deserves to be looked at:
(one phone that it happens on is Samsung Galaxy Grand Prime Pro (j2y18lte), Android 7.1)

  #00  pc 000000000035b4a0  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN20DepalShaderCacheGLESC1EPN4Draw11DrawContextE+51)
  #01  pc 000000000035d2c3  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN8GPU_GLESC1EP15GraphicsContextPN4Draw11DrawContextE+62)
  #02  pc 00000000003c5b95  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z8GPU_InitP15GraphicsContextPN4Draw11DrawContextE+60)
  #03  pc 00000000003502c1  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z14PSP_InitUpdatePNSt6__ndk112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEE+216)
  #04  pc 00000000004275cd  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN9EmuScreen8bootGameERKNSt6__ndk112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE+44)
  #05  pc 000000000042b36f  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN9EmuScreen6updateEv+50)
  #06  pc 00000000004baee9  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN13ScreenManager6updateEv+40)
  #07  pc 0000000000425c47  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z12NativeUpdatev+146)
  #08  pc 00000000004203a3  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z20UpdateRunLoopAndroidP7_JNIEnv+10)
  #09  pc 00000000004216f5  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so
  #10  pc 00000000002b7325  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZNSt6__ndk114__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEPFvvEEEEEEPvSA_+24)
  #11  pc 0000000000047d03  /system/lib/libc.so (_ZL15__pthread_startPv+22)
  #12  pc 000000000001a00d  /system/lib/libc.so (__start_thread+6)

The text was updated successfully, but these errors were encountered:

ghost · 2018-10-27T13:27:54Z

My ppsspp version still 1.6.3 :(

hrydgard · 2018-10-27T15:22:34Z

That is expected, as I wrote on the website it's a rollout over the week so I can catch bad bugs before everyone gets them. Patience.

unknownbrackets · 2018-10-27T15:37:18Z

Do we know if that crash is a segfault or what?

I guess it must mean draw is null if it's a segfault. Maybe stop during startup issue? I can't really see how it gets there null...

-[Unknown]

ghost · 2018-10-27T15:43:30Z

@hrydgard Opps... I forgot sorry 😅

unknownbrackets · 2018-10-27T17:30:07Z

I suppose the segfault means that while EmuScreen was looping and called PSP_InitUpdate(), another thread (or an event before bootGame was called?) deleted draw.

Which basically means ShutdownFromRenderThread was called, either on a separate thread, or in a way that failed to prevent EmuScreen::update() from being called. But we call EmuThreadJoin() before each version of that, even on display reinit.

It would've crashed earlier if the draw alloc failed...

-[Unknown]

hrydgard · 2018-10-28T09:36:03Z

I'm not sure that first one was even real (filtering is being weird) but this one is:

  #00  pc 000000000038a1d4  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN22FramebufferManagerGLES17DrawActiveTextureEffffffffffii+631)
  #01  pc 000000000038b81b  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN22FramebufferManagerGLES15BlitFramebufferEP18VirtualFramebufferiiS1_iiiii+1226)
  #02  pc 00000000003b9fb9  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN24FramebufferManagerCommon26DownloadFramebufferForClutEjj+392)
  #03  pc 00000000003db37b  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN18TextureCacheCommon8LoadClutEjj+498)
  #04  pc 00000000003eff79  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN9GPUCommon15ReapplyGfxStateEv+284)
  #05  pc 00000000003ee537  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN9GPUCommon14BeginHostFrameEv+10)
  #06  pc 0000000000383c3d  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN8GPU_GLES14BeginHostFrameEv+6)
  #07  pc 000000000044c8c9  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN9EmuScreen6renderEv+216)
  #08  pc 00000000004deff1  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN13ScreenManager6renderEv+84)
  #09  pc 0000000000445a27  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z12NativeRenderP15GraphicsContext+810)
  #10  pc 00000000004409cf  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z20UpdateRunLoopAndroidP7_JNIEnv+22)

And this:

  #02  pc 00000000004ceed1  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN15GLRenderManager27CopyFramebufferToMemorySyncEP14GLRFramebufferiiiiiN4Draw10DataFormatEPhi+180)
  #03  pc 00000000004cac21  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN4Draw13OpenGLContext27CopyFramebufferToMemorySyncEPNS_11FramebufferEiiiiiNS_10DataFormatEPvi+92)
  #04  pc 00000000003b9def  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN24FramebufferManagerCommon20PackFramebufferSync_EP18VirtualFramebufferiiii+166)
  #05  pc 00000000003b4f93  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN24FramebufferManagerCommon23ReadFramebufferToMemoryEP18VirtualFramebufferbiiii+146)
  #06  pc 00000000003b6221  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN24FramebufferManagerCommon19CopyDisplayToOutputEv+624)
  #07  pc 0000000000383dad  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN8GPU_GLES19CopyDisplayToOutputEv+52)
  #08  pc 00000000002ce4ff  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z13__DisplayFlipi+1582)
  #09  pc 00000000002cd21f  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z14hleEnterVblankyi+366)
  #10  pc 00000000002674cf  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN10CoreTiming7AdvanceEv+366)
  #11  pc 000000000030c933  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z18__KernelReSchedulePKc+30)```

unknownbrackets · 2018-10-28T15:35:50Z

Hm. We seem to be missing a null check for FindDownloadTempBuffer, but that can't be it. For DrawActiveTexture to crash, I guess we ran out of push buffer or render_ became invalid? OOM?

-[Unknown]

hrydgard · 2018-10-28T15:50:04Z

Yeah, OOM can probably explain a lot of the stranger ones...

hrydgard · 2018-10-28T21:16:50Z

  #01  pc 00000000004452ff  /data/app/org.ppsspp.ppsspp-2/lib/arm/libppsspp_jni.so (_Z18NativeInitGraphicsP15GraphicsContext+1086)
  #02  pc 0000000000441c83  /data/app/org.ppsspp.ppsspp-2/lib/arm/libppsspp_jni.so
  #03  pc 000000000026e489  /data/app/org.ppsspp.ppsspp-2/lib/arm/libppsspp_jni.so (_ZNSt6__ndk114__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEPFvvEEEEEEPvSA_+24)

  #08  pc 0000000000a0f965  /data/app/org.ppsspp.ppsspp-2/lib/arm/libppsspp_jni.so (__cxa_pure_virtual+8)
  #09  pc 0000000000381029  /data/app/org.ppsspp.ppsspp-2/lib/arm/libppsspp_jni.so (_ZN20DepalShaderCacheGLESC1EPN4Draw11DrawContextE+60)

Failure during creation?

  #00  pc 00000000004c98de  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN4Draw16RefCountedObject7ReleaseEv+27)
  #01  pc 00000000004cca01  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN4Draw14OpenGLPipelineD2Ev+52)
  #02  pc 00000000004ccaad  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN4Draw14OpenGLPipelineD0Ev+4)
  #03  pc 00000000004cb1d5  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN4Draw13OpenGLContext22CreateGraphicsPipelineERKNS_12PipelineDescE+292)
  #04  pc 0000000000445311  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z18NativeInitGraphicsP15GraphicsContext+1104)
  #05  pc 0000000000441c83  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so
  #06  pc 000000000026e489  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZNSt6__ndk114__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEPFvvEEEEEEPvSA_+24)

Hit an assert:

  #03  pc 00000000005a84a8  /data/app/org.ppsspp.ppsspp-kjnPFXC4QqtPKDFqg4-Y0A==/lib/arm64/libppsspp_jni.so (_Z16AndroidAssertLogPKcS0_iS0_S0_z+164)
  #04  pc 00000000004f1498  /data/app/org.ppsspp.ppsspp-kjnPFXC4QqtPKDFqg4-Y0A==/lib/arm64/libppsspp_jni.so (_ZN12DenseHashMapI9FShaderIDP20VulkanFragmentShaderLS2_0EE4GrowEi+516)
  #05  pc 00000000004effec  /data/app/org.ppsspp.ppsspp-kjnPFXC4QqtPKDFqg4-Y0A==/lib/arm64/libppsspp_jni.so (_ZN12DenseHashMapI9FShaderIDP20VulkanFragmentShaderLS2_0EE6InsertERKS0_S2_+72)
  #06  pc 00000000004f0a54  /data/app/org.ppsspp.ppsspp-kjnPFXC4QqtPKDFqg4-Y0A==/lib/arm64/libppsspp_jni.so (_ZN19ShaderManagerVulkan9LoadCacheEP7__sFILE+488)
  #07  pc 00000000004ea2d0  /data/app/org.ppsspp.ppsspp-kjnPFXC4QqtPKDFqg4-Y0A==/lib/arm64/libppsspp_jni.so (_ZN10GPU_Vulkan9LoadCacheENSt6__ndk112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE+160)
  #08  pc 00000000004eb7d0  /data/app/org.ppsspp.ppsspp-kjnPFXC4QqtPKDFqg4-Y0A==/lib/arm64/libppsspp_jni.so

unknownbrackets · 2018-10-28T21:56:13Z

Hm, that assert is concerning - _assert_msg_(SYSTEM, oldCount == count_, "DenseHashMap: count should not change in Grow()");. Could it mean threaded access to the densehashmap?

For the refcount one, I can't repro by forcing a compile/link failure, but we probably want a null check in render_->DeleteProgram(program_); - though I doubt that's the cause.

DepalShaderCacheGLES pure virtual - more signals that draw is being freed.

NativeInitGraphics crash - hmm. Maybe some related condition where g_draw became invalid?

-[Unknown]

unknownbrackets · 2018-10-30T03:43:53Z

Okay, so I think the hash map thing is happening due to a DeviceLost() or similar while loading cache.

The simplest thing might be to wait for GPU_IsReady() in GPU_Shutdown(), although this might delay things...

-[Unknown]

hrydgard · 2018-10-30T08:09:40Z

A crash during cache loading isn't really such a terrible case, no user data will be lost at least... Not gonna let this stop 1.7.1

hrydgard · 2018-10-31T14:57:26Z

Turns out I was wrong in that the fix will make 1.7.1 :) I don't see any other really surprising big crashers, so I'm going to release 1.7.1 tomorrow and start at a pretty high rollout, before we go to 100%.

ghost · 2018-10-31T15:53:05Z

Sir @hrydgard v1.7.1 is the final version and officialy release tomorrow in PlayStore?
or there's still a minor update like v1.7.2?

p.s sorry for my bad english and asking too much and thank you for your hard work to make ppsspp a better emulator god bless and happy halloween

hrydgard · 2018-10-31T17:33:27Z

Depends if we find more critical issues, but hopefully 1.7.1 will be the final official one in the 1.7 series.

ghost · 2018-11-03T13:28:11Z

Found a graphics glitch

in Burnout Legend using v1.7.1 didn't happen this issue in v1.7.0

hrydgard · 2018-11-03T13:51:19Z

That is wacky, but it has no business being in this thread. Please create a new issue report.

ghost · 2018-11-03T14:14:43Z

Seems ok now in the latest build 1.7.1-28
Do I still need to submit issue?

unknownbrackets · 2018-11-19T05:37:24Z

How are the Play crashes looking now? Should we close this?

-[Unknown]

hrydgard · 2018-11-19T08:03:18Z

Well, it's not perfect but I don't think it's worse than before...

hrydgard · 2018-11-21T17:56:21Z

Actually, this one is still happening, and I don't understand how:

backtrace:
  #00  pc 00000000000178c8  /system/lib/libc.so (__memcpy_base_aligned+172)
  #01  pc 00000000004d2b55  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN13GLQueueRunner18CopyReadbackBufferEiiN4Draw10DataFormatES1_iPh+48)
  #02  pc 00000000004cf625  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN15GLRenderManager27CopyFramebufferToMemorySyncEP14GLRFramebufferiiiiiN4Draw10DataFormatEPhi+216)
  #03  pc 00000000004cb351  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN4Draw13OpenGLContext27CopyFramebufferToMemorySyncEPNS_11FramebufferEiiiiiNS_10DataFormatEPvi+92)
  #04  pc 00000000003ba175  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN24FramebufferManagerCommon20PackFramebufferSync_EP18VirtualFramebufferiiii+428)
  #05  pc 00000000003b52a1  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN24FramebufferManagerCommon23ReadFramebufferToMemoryEP18VirtualFramebufferbiiii+288)
  #06  pc 00000000003b64a1  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN24FramebufferManagerCommon19CopyDisplayToOutputEv+624)
  #07  pc 0000000000383fad  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN8GPU_GLES19CopyDisplayToOutputEv+52)
  #08  pc 00000000002ce6ef  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z13__DisplayFlipi+1582)
  #09  pc 00000000002cd40f  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z14hleEnterVblankyi+366)
  #10  pc 000000000026764f  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN10CoreTiming7AdvanceEv+366)
  #11  pc 000000000030cb23  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z18__KernelReSchedulePKc+30)
  #12  pc 0000000000004420  <unknown>

Mainly on small/weak phones though, could be out of memory when it tries to alloc the temp buffer?

hrydgard · 2018-11-24T15:40:54Z

Quietly pushed 1.7.4 to a subset on Play, and this thing is STILL happening. I don't get it.

  #00  pc 00000000000174b8  /system/lib/libc.so (memcpy+96)
  #01  pc 00000000004d2be5  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN13GLQueueRunner18CopyReadbackBufferEiiN4Draw10DataFormatES1_iPh+48)
  #02  pc 00000000004cf6a5  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN15GLRenderManager27CopyFramebufferToMemorySyncEP14GLRFramebufferiiiiiN4Draw10DataFormatEPhi+216)
  #03  pc 00000000004cb3d1  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN4Draw13OpenGLContext27CopyFramebufferToMemorySyncEPNS_11FramebufferEiiiiiNS_10DataFormatEPvi+92)
  #04  pc 00000000003ba175  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN24FramebufferManagerCommon20PackFramebufferSync_EP18VirtualFramebufferiiii+428)
  #05  pc 00000000003b52a1  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN24FramebufferManagerCommon23ReadFramebufferToMemoryEP18VirtualFramebufferbiiii+288)
  #06  pc 00000000003b64a1  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN24FramebufferManagerCommon19CopyDisplayToOutputEv+624)
  #07  pc 0000000000383fad  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN8GPU_GLES19CopyDisplayToOutputEv+52)
  #08  pc 00000000002ce6af  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z13__DisplayFlipi+1582)
  #09  pc 00000000002cd3cf  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z14hleEnterVblankyi+366)
  #10  pc 000000000026760f  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_ZN10CoreTiming7AdvanceEv+366)
  #11  pc 000000000030cae3  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (_Z18__KernelReSchedulePKc+30)
  #12  pc 00000000000748f0  <anonymous>

unknownbrackets · 2018-11-24T16:17:48Z

My best guess is: what if the frame sync is off, and it never runs PerformReadback. We just assume readbackBuffer_ is allocated at that point.

I don't see how the destination can be wrong, or the stride, or the height. So that's my only idea left...

-[Unknown]

hrydgard · 2018-11-24T17:14:31Z

Yup, that's got to be it. Checked in a fix for that too....

hrydgard · 2018-11-24T17:19:57Z

Pretty weird one:

  #00  pc 000000000066444c  /data/app/org.ppsspp.ppsspp-oDgxNlw1GINOIpodaiGXRA==/lib/arm64/libppsspp_jni.so (Draw::OpenGLContext::CreateGraphicsPipeline(Draw::PipelineDesc const&)+204)
  #01  pc 00000000005ad368  /data/app/org.ppsspp.ppsspp-oDgxNlw1GINOIpodaiGXRA==/lib/arm64/libppsspp_jni.so (NativeInitGraphics(GraphicsContext*)+1260)
  #02  pc 00000000005a8a5c  /data/app/org.ppsspp.ppsspp-oDgxNlw1GINOIpodaiGXRA==/lib/arm64/libppsspp_jni.so
  #03  pc 00000000003483b8  /data/app/org.ppsspp.ppsspp-oDgxNlw1GINOIpodaiGXRA==/lib/arm64/libppsspp_jni.so (_ZNSt6__ndk114__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEPFvvEEEEEEPvSA_+44)
  #04  pc 0000000000067dc0  /system/lib64/libc.so (__pthread_start(void*)+36)
  #05  pc 000000000001ec58  /system/lib64/libc.so (__start_thread+68)

hrydgard · 2018-11-24T17:30:22Z

But if the frame sync is so off that this happens, we have another serious bug somewhere..

unknownbrackets · 2018-11-24T19:06:55Z

Perhaps it's just during shutdown?

Not sure about the pipeline, maybe an allocation failure...

-[Unknown]

hrydgard · 2018-11-24T21:21:36Z

Ah yeah, could be. My checked-in workaround should be good enough if it is.

hrydgard · 2019-01-06T23:33:27Z

I'm declaring this mostly done and closing. Until the 1.8 one...

hrydgard added this to the v1.7.1 milestone Oct 27, 2018

hrydgard mentioned this issue Oct 28, 2018

Improve sanity checks for framebuffer readbacks #11502

Merged

unknownbrackets mentioned this issue Oct 31, 2018

Wait for orderly finish of cache before shutdown #11513

Merged

hrydgard modified the milestones: v1.7.1, v1.7.2, v1.8.0 Nov 1, 2018

hrydgard closed this as completed Nov 19, 2018

hrydgard reopened this Nov 21, 2018

hrydgard added a commit that referenced this issue Nov 21, 2018

Attempt to fix the last very common issue of #11493 by being paranoid

0fd5d10

hrydgard added a commit that referenced this issue Nov 21, 2018

Attempt to fix the last very common issue of #11493 by being paranoid

442b570

hrydgard closed this as completed Jan 6, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Google Play crash report mystery thread 1.7 #11493

Google Play crash report mystery thread 1.7 #11493

hrydgard commented Oct 27, 2018 •

edited

Loading

ghost commented Oct 27, 2018

hrydgard commented Oct 27, 2018

unknownbrackets commented Oct 27, 2018 •

edited

Loading

ghost commented Oct 27, 2018

unknownbrackets commented Oct 27, 2018

hrydgard commented Oct 28, 2018

unknownbrackets commented Oct 28, 2018

hrydgard commented Oct 28, 2018

hrydgard commented Oct 28, 2018 •

edited

Loading

unknownbrackets commented Oct 28, 2018

unknownbrackets commented Oct 30, 2018

hrydgard commented Oct 30, 2018 •

edited

Loading

hrydgard commented Oct 31, 2018

ghost commented Oct 31, 2018

hrydgard commented Oct 31, 2018

ghost commented Nov 3, 2018

hrydgard commented Nov 3, 2018

ghost commented Nov 3, 2018

unknownbrackets commented Nov 19, 2018

hrydgard commented Nov 19, 2018

hrydgard commented Nov 21, 2018 •

edited

Loading

hrydgard commented Nov 24, 2018

unknownbrackets commented Nov 24, 2018

hrydgard commented Nov 24, 2018

hrydgard commented Nov 24, 2018

hrydgard commented Nov 24, 2018

unknownbrackets commented Nov 24, 2018

hrydgard commented Nov 24, 2018

hrydgard commented Jan 6, 2019

Google Play crash report mystery thread 1.7 #11493

Google Play crash report mystery thread 1.7 #11493

Comments

hrydgard commented Oct 27, 2018 • edited Loading

ghost commented Oct 27, 2018

hrydgard commented Oct 27, 2018

unknownbrackets commented Oct 27, 2018 • edited Loading

ghost commented Oct 27, 2018

unknownbrackets commented Oct 27, 2018

hrydgard commented Oct 28, 2018

unknownbrackets commented Oct 28, 2018

hrydgard commented Oct 28, 2018

hrydgard commented Oct 28, 2018 • edited Loading

unknownbrackets commented Oct 28, 2018

unknownbrackets commented Oct 30, 2018

hrydgard commented Oct 30, 2018 • edited Loading

hrydgard commented Oct 31, 2018

ghost commented Oct 31, 2018

hrydgard commented Oct 31, 2018

ghost commented Nov 3, 2018

hrydgard commented Nov 3, 2018

ghost commented Nov 3, 2018

unknownbrackets commented Nov 19, 2018

hrydgard commented Nov 19, 2018

hrydgard commented Nov 21, 2018 • edited Loading

hrydgard commented Nov 24, 2018

unknownbrackets commented Nov 24, 2018

hrydgard commented Nov 24, 2018

hrydgard commented Nov 24, 2018

hrydgard commented Nov 24, 2018

unknownbrackets commented Nov 24, 2018

hrydgard commented Nov 24, 2018

hrydgard commented Jan 6, 2019

hrydgard commented Oct 27, 2018 •

edited

Loading

unknownbrackets commented Oct 27, 2018 •

edited

Loading

hrydgard commented Oct 28, 2018 •

edited

Loading

hrydgard commented Oct 30, 2018 •

edited

Loading

hrydgard commented Nov 21, 2018 •

edited

Loading