Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabling MQTT in WLED 0.14.0 resets every couple minutes #3435

Closed
1 task done
YvesLev opened this issue Oct 10, 2023 · 21 comments
Closed
1 task done

Enabling MQTT in WLED 0.14.0 resets every couple minutes #3435

YvesLev opened this issue Oct 10, 2023 · 21 comments
Labels
bug cannot reproduce Developers are not able reproduce. Might be fixed already, or report is missing important details external Not part of WLED itself - an external plugin/remote etc.

Comments

@YvesLev
Copy link

YvesLev commented Oct 10, 2023

What happened?

Installed latest version of WLED 0.14.0-b6 (Tried AR and non AR) from web installer on a ESP32 Dev board (ESP32-WROOM-32D)
(Also tried on another ESP32 dev board (32D) and also on a ESP32 mini board).

Was working great until I enable MQTT using my Mosquito server IP. (User/Pass empty as I use mosquito anonymously).

With MQTT Enabled, MQTT seemed to work fine for not very long. (Using MQTT Explorer to monitor).
Then, no msg were sent from WLED and after few minutes (1-2), WLED resets itself (Back to default preset)...

It keeps doing this in a loop.

To Reproduce Bug

1- Install WLED 0.14.0-b6 on a ESP32 board.
2- Make a preset with ColorTwinkle effect for example. Leave default preset to 0 (Solid).
3- Enable MQTT (No user/pass, anonymous)
4- After reboot of WLED, choose Preset 1.

Expected Behavior

1- After about 2 minutes (could be from 1 to 5 minutes), it should reset by itself and revert back to preset 0 (Solid).
2- Will keep doing this indefinitely.

Install Method

Binary from WLED.me

What version of WLED?

WLED 0.14.0-b6 Build 2310010

Which microcontroller/board are you seeing the problem on?

ESP32

Relevant log/trace output

[11:44:53]---DEBUG INFO---
[11:44:53]Runtime: 180082
[11:44:53]Unix time: 1696866294,082
[11:44:53]Free heap: 201312
[11:44:53]Wifi state: 3
[11:44:53]State time: 0
[11:44:53]NTP last sync: 999000000
[11:44:53]Client IP: 192.168.2.62
[11:44:53]Loops/sec: 2050
[11:44:53]Loop time[ms]: 2/4
[11:44:53]UM time[ms]: 0/1
[11:44:53]Strip time[ms]: 0/2
[11:44:53]Segments: 1 -> 86B
[11:44:53]Modes: 4*187=748B
[11:44:53]Data: 4*187=748B
[11:44:53]Map: 2*0=0B
[11:44:53]Buffer: 3*144=432B
[11:44:56]Reconnecting MQTT
[11:44:56]Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.
[11:44:56]Core 0 register dump:
[11:44:56]PC      : 0x4011ff0b  PS      : 0x00060630  A0      : 0x801200c3  A1      : 0x3ffb5f50  
[11:44:56]A2      : 0x00000000  A3      : 0x3ffdd620  A4      : 0x00000000  A5      : 0x3ffdd41e  
[11:44:56]A6      : 0x00000014  A7      : 0x3ffb5500  A8      : 0x80085530  A9      : 0x3ffb5f30  
[11:44:56]A10     : 0x3ffd7f90  A11     : 0x0000166f  A12     : 0xf28e6cde  A13     : 0x00000b38  
[11:44:56]A14     : 0x00000020  A15     : 0x00000000  SAR     : 0x00000010  EXCCAUSE: 0x0000001c  
[11:44:56]EXCVADDR: 0x00000000  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff  
[11:44:56]
[11:44:56]ELF file SHA256: 0000000000000000
[11:44:56]
[11:44:56]Backtrace: 0x4011ff08:0x3ffb5f50 0x401200c0:0x3ffb5f70 0x40120104:0x3ffb5f90 0x40156145:0x3ffb5fc0 0x4015c30f:0x3ffb5fe0 0x40160ad2:0x3ffb6010 0x4014b023:0x3ffb6030 0x4008b5d2:0x3ffb6060
[11:44:56]
[11:44:56]Rebooting...
[11:44:56]ets Jul 29 2019 12:21:46
[11:44:56]
[11:44:56]rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[11:44:56]configsip: 0, SPIWP:0xee
[11:44:56]clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[11:44:56]mode:DOUT, clock div:2
[11:44:56]load:0x3fff0030,len:1184
[11:44:56]load:0x40078000,len:13132
[11:44:56]load:0x40080400,len:3036
[11:44:56]entry 0x400805e4
[11:44:57]
[11:44:57]---WLED 0.14.0-b6 2310010 INIT---
[11:44:57]esp32 v3.3.6-16-gcc5440f6a2
[11:44:57]arduino-esp32 v1.0.x
[11:44:57]
[11:44:57]CPU:   ESP32-D0WDQ5 rev.3, 2 core(s), 240MHz.
[11:44:57]FLASH: 4MB, Mode 3 (DOUT), speed 40MHz.
[11:44:57]heap 292388
[11:44:57]PIN ALLOC: Pin 1 successfully allocated by 0x89 (137)
[11:44:57]Registering usermods ...
[11:44:57]heap 292388
[11:44:57]Reading config
[11:44:57]Reading settings from /wsec.json...
[11:44:57]JSON buffer locked. (3)
[11:44:57]JSON buffer released. (3)
[11:44:57]JSON buffer locked. (1)
[11:44:57]Reading settings from /cfg.json...
[11:44:57]Removing all.
[11:44:57]PIN ALLOC: Pin 26 successfully allocated by 0x82 (130)
[11:44:57]Successfully inited strip 0 (len 144) with type 22 and pins 26,186 (itype 21)
[11:44:57]PIN ALLOC: Pin 0 successfully allocated by 0x85 (133)
[11:44:57]Starting usermod config.
[11:44:57]JSON buffer released. (1)
[11:44:57]Initializing strip
[11:44:57]Loading custom palettes
[11:44:57]Loading custom ledmaps
[11:44:57]heap 287932
[11:44:57]Usermods setup
[11:44:57]heap 287932
[11:44:57]initServer
[11:44:57]heap 283928
[11:44:57]lastReconnectAttempt == 0
[11:44:57]Access point disabled (init).
[11:44:57]Connecting to PuttPutt...
[11:44:57]Loop took 128ms.
[11:44:57]Usermods took 0ms.
[11:44:57]Strip took 2ms.
[11:45:00]Loop took 718ms.
[11:45:00]Usermods took 0ms.
[11:45:00]Strip took 1ms.
[11:45:00]
[11:45:00]Connected! IP address: 192.168.2.62
[11:45:00]Init STA interfaces
[11:45:00]mDNS started
[11:45:00]Reconnecting MQTT
[11:45:00]Reconnecting MQTT
[11:45:00]Loop took 35ms.
[11:45:00]Usermods took 0ms.
[11:45:00]Strip took 0ms.
[11:45:00]MQTT ready
[11:45:00]Publish MQTT
[11:45:00]MQTT ready
[11:45:00]Publish MQTT
[11:45:16]WS client connected.
[11:45:16]JSON buffer locked. (12)

Anything else?

I'm using an ASUS Router RT-AC68U with ASUSWRT-MERLIN

Code of Conduct

  • I agree to follow this project's Code of Conduct
@YvesLev YvesLev added the bug label Oct 10, 2023
@blazoncek
Copy link
Collaborator

As there were no changes in MQTT in several months and it appears that MQTT is working on my installs I would suspect something odd in your configuration (broker or client) or messages destined for WLED device.
Please capture MQTT packets (using Wireshark, MQTT Explorer or something similar) and post the message which is followed by a crash.

It would also be very helpful if you can use PIO to build & flash your ESP and then use exception decoder to disassemble backtrace.

@YvesLev
Copy link
Author

YvesLev commented Oct 10, 2023

Thank you very much for your help.

I'm still newbie in this coding, pio things.
What you ask, wireshark, pio exception decoder thing is a challenge for me but I'm might look at that eventually.

I thought maybe the problem could come from the fact I use MQTT anonymous without any user/pass, which, I believe, is not that common these days.

I've been using Mosquito broker windows service since 2018-19 and I use HA, Tasmota, Z2M, esphome, etc all using MQTT for all that time. I doubt my mqtt config is the culprit but who knows.

I'm going to ask a friend to try WLED on an ESP32 wiht MQTT enabled. He has the MQTT HA Addon installed with user/pass.

Also I will try to flash older version of WLED...

Thanks again.

@blazoncek
Copy link
Collaborator

I am not using authenticated MQTT myself. I also use Mosquitto, though it is hosted on Linux machine.
As I said most likely cause may be particular message received.

Wireshark and MQTT explorer are simple tools with plenty of tutorial videos on-line so it should not be too difficult to obtain packets from them.

@blazoncek blazoncek added the waiting for feedback addition information needed to better understand the issue label Oct 10, 2023
@YvesLev
Copy link
Author

YvesLev commented Oct 10, 2023

Hello again,

I tried yesterday night the versions 0.14.0 b4 and also b1 and enabling MQTT creates the self-reseting problem with these versions.

Then I tried the 0.13.3 release with MQTT enabled and it has been working very well for many hours now.
(Unfortunately this version does not support 2D led panel... which I need)

@blazoncek
Copy link
Collaborator

Please provide MQTT packets destined for WLED.

@YvesLev
Copy link
Author

YvesLev commented Oct 10, 2023

When I figure out how to achieve that, I'll post it.

@blazoncek
Copy link
Collaborator

MQTT explorer or Mosquitto client.

@YvesLev
Copy link
Author

YvesLev commented Oct 10, 2023

I use MQTT Explorer.
There are not much msg from/to WLED since it appears WLED publishes only on changes.
(And I just leave WLED running as his without making changes).
And there are no msg sent to WLED for now.

So I'm not sure what you need here.

If I change the brightness, with b6, sometime I'll see the change in MQTT, sometime not.
When I stop seeing WLED changes in MQTT explorer, it is not very long that it resets itself.
Sometime after reset, changes to brightness, status will start to appear again in MQTT explorer, sometime not...

But if I do not touch WLED, there is nothing to see in MQTT explorer and wled will still self reset every couple of minutes.

@blazoncek
Copy link
Collaborator

I cannot reproduce, sorry.
Screenshot 2023-10-10 at 19 27 29

@blazoncek blazoncek added the cannot reproduce Developers are not able reproduce. Might be fixed already, or report is missing important details label Oct 10, 2023
@YvesLev
Copy link
Author

YvesLev commented Oct 10, 2023

13:39:20 - Preset 0, effect solid, Brightness at 128
13:41:02 - I changed to preset 1 which has a brightness 50
13:42:25 - Msg after a self reset.

Eventually, MQTT publishing almost stops working even after self resets.
If I play with the only preset and keep changing the brightness, I might see a value change in MQTT but very seldomly.

Then after a reboot (from info screen reboot button), MQTT publishing comes back very responsive to brightness changes for ex.
For about a minute. Then it freezes once for few sec and comes back sending brightness changes for about 30 sec to 1 min then it stops sending changes to mqtt for good and the self reset follows a minute or two after.
MQTT does not necessarily resume after self reset.

image

image

@YvesLev
Copy link
Author

YvesLev commented Oct 10, 2023

My MQTT config in WLED for information.

image

@blazoncek
Copy link
Collaborator

The only thing I can now think of is some sort of low-level problem with your MQTT set up (not WLED as that looks correct).
Screenshot 2023-10-10 at 22 48 13
I added two more devices and left the MQTT Explorer run for 3 hours. All going strong no reboot or missing messages.

@YvesLev
Copy link
Author

YvesLev commented Oct 11, 2023

My Mosquitto broker (Windows service) was 2.0.14 and I just upgraded to the latest 2.0.18.

Now the ESP32 with wled b6 and MQTT enabled seems to hold with no self resets for over 10 minutes now...
I let it run overnight and post tomorrow.

Thank you for your help and patience.

@blazoncek
Copy link
Collaborator

Please close the issue then.

@YvesLev
Copy link
Author

YvesLev commented Oct 11, 2023

So the ESP32 ran all night with MQTT enabled with no self resets.
Appears that the upgrade of Mosquitto broker from 2.0.14 to 2.0.18 did the trick...

Thanks again.

Closing...

@YvesLev YvesLev closed this as completed Oct 11, 2023
@YvesLev
Copy link
Author

YvesLev commented Oct 17, 2023

I'm going to re-open this issue even if it appears I'm the only one having this problem.

So after updating my Mosquito broker, I thought it solved the problem but the day after, I tried the latest WLED on an ESP32 and the self resetting problem was there.

So I guess the rebooting of my NUC, hosting the Mosquitto Service and also the HA Virtualbox, corrected the problem just for a little while...

Actually, when the self resetting seems to be gone, if I cycle power to the esp, the problem comes back. I don't need to wait long after a reboot of the NUC. (I even wonder if the reboot is related)

As I said earlier, the same ESP32 with a previous release (0.13.x) and MQTT enabled is working perfectly fine.

I will still do some tests and report back.

@YvesLev YvesLev reopened this Oct 17, 2023
@YvesLev YvesLev changed the title Enabling MQTT in WLED 0.14.0-b6 resets every couple minutes Enabling MQTT in WLED 0.14.0 resets every couple minutes Oct 18, 2023
@YvesLev
Copy link
Author

YvesLev commented Oct 25, 2023

I compiled the 0.14.0 source with Debug option.
Upload and monitor...
I'm not sure what to do with this additional information yet...

`Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC : 0x4015ae93 PS : 0x00060830 A0 : 0x8015af87 A1 : 0x3ffb5bd0
A2 : 0x00000000 A3 : 0x3ffdd0b4 A4 : 0x00000000 A5 : 0x00000001
A6 : 0x00060021 A7 : 0x00000000 A8 : 0x80085730 A9 : 0x3ffb5bb0
A10 : 0x3ffdd900 A11 : 0x0000165b A12 : 0x9735dff4 A13 : 0x00000b38
A14 : 0x00000020 A15 : 0x00000000 SAR : 0x00000010 EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000000 LBEG : 0x4000c2e0 LEND : 0x4000c2f6 LCOUNT : 0xffffffff

ELF file SHA256: 0000000000000000

Backtrace: 0x4015ae90:0x3ffb5bd0 0x4015af84:0x3ffb5bf0 0x4015afce:0x3ffb5c10 0x40195285:0x3ffb5c40 0x4019b44f:0x3ffb5c60 0x4019fc12:0x3ffb5c90 0x4018a0eb:0x3ffb5cb0 0x4008b7e6:0x3ffb5ce0
#0 0x4015ae90:0x3ffb5bd0 in AsyncClient::_lwip_fin(tcp_pcb*, signed char) at .pio\libdeps\WLED_ESP32_BH1750\AsyncTCP\src/AsyncTCP.cpp:1153
#1 0x4015af84:0x3ffb5bf0 in AsyncClient::_s_lwip_fin(void*, tcp_pcb*, signed char) at .pio\libdeps\WLED_ESP32_BH1750\AsyncTCP\src/AsyncTCP.cpp:1153
#2 0x4015afce:0x3ffb5c10 in _tcp_recv(void*, tcp_pcb*, pbuf*, signed char) at .pio\libdeps\WLED_ESP32_BH1750\AsyncTCP\src/AsyncTCP.cpp:1153
#3 0x40195285:0x3ffb5c40 in tcp_input at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp_in.c:475 (discriminator 1)
#4 0x4019b44f:0x3ffb5c60 in ip4_input at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/ipv4/ip4.c:750
#5 0x4019fc12:0x3ffb5c90 in ethernet_input at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/netif/ethernet.c:184
#6 0x4018a0eb:0x3ffb5cb0 in tcpip_thread at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/tcpip.c:483
#7 0x4008b7e6:0x3ffb5ce0 in vPortTaskWrapper at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

Rebooting...
ets Jul 29 2019 12:21:46

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DOUT, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1084
load:0x40078000,len:11220
load:0x40080400,len:5360
entry 0x4008067c`

@blazoncek
Copy link
Collaborator

Unfortunately backtrace shows no WLED code.
You will need to open an issue with library provider (AsyncTCP).

@blazoncek blazoncek added external Not part of WLED itself - an external plugin/remote etc. and removed waiting for feedback addition information needed to better understand the issue labels Oct 26, 2023
@YvesLev
Copy link
Author

YvesLev commented Oct 27, 2023

I tried to tweak a bit my wifi config in the router after reading some tips...
But it didn't solve the problem with WLED when MQTT is enabled...

So I deactivated Mosquito windows service and setup the Mosquito addon in Home Assistant.
(Had to reconfigure all my devices but it's ok...)

Up to now, the problem seems gone... Very strange...

@blazoncek
Copy link
Collaborator

blazoncek commented Feb 24, 2024

MQTT reconnect only happens when library detects disconnect from MQTT broker. There is nothing fancy in WLED.

...
  if (mqtt == nullptr) {
    mqtt = new AsyncMqttClient();
    mqtt->onMessage(onMqttMessage);
    mqtt->onConnect(onMqttConnect);
  }
  if (mqtt->connected()) return true;

  DEBUG_PRINTLN(F("Reconnecting MQTT"));
...

I know it may sound corny but please try another broker. It happened many times in the past that Windows versions of Mosquitto caused WLED reset.

@hajeth
Copy link

hajeth commented Feb 24, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug cannot reproduce Developers are not able reproduce. Might be fixed already, or report is missing important details external Not part of WLED itself - an external plugin/remote etc.
Projects
None yet
Development

No branches or pull requests

3 participants