Excessive memory consumption on project search with large files present

[notpeter]

tldr: Project search in directories with big files will cause Zed to crash after consuming all your memory.

Play stupid games, win stupid prizes.

Steps to reproduce:

mkdir /tmp/uspto
cd /tmp/uspto
wget https://bulkdata.uspto.gov/data/patent/grant/redbook/fulltext/2024/ipg241119.zip
unzip ipg241119.zip
rm ipg241119.zip
for i in {1..30}; do cp ipg241119.xml ipg241119-$i.xml; done
azed .

Quit/Save work in any apps you care about, because bad things can happen under memory pressure.
Then: cmd-shift-f and search for <

First time crashed (with 40 files).
Second time succeeded (with 40 files).
Upped the count above to 50 files then I got the OOM prompt, but macOS killed it for me 45 seconds later automagically.

Watch out though, I later ran it with 50 and caused a KP.

Environment

Zed: v0.163.0 (Zed Nightly 3c57a40)
OS: macOS 14.7.0
Memory: 64 GiB
Architecture: aarch64

Zed.log

2024-11-20T23:41:10.003248-05:00 [ERROR] error parsing text: failed to parse
2024-11-20T23:41:18.409071-05:00 [ERROR] error parsing text: failed to parse
2024-11-20T23:41:18.414654-05:00 [ERROR] error parsing text: failed to parse
2024-11-20T23:41:18.437404-05:00 [ERROR] error parsing text: failed to parse
2024-11-20T23:41:18.445419-05:00 [ERROR] error parsing text: failed to parse
...crash happened here....

Zed.log (round 2)

This time there was a hang in the log (not sure why there wasn't the first time).

2024-11-20T23:53:36.359019-05:00 [ERROR] Suspected hang on main thread:
zed[1325643a57976c4c]::reliability::monitor_main_thread_hangs::handle_backtrace_signal::handle_sigusr2
__simple_esappend
core[f827f14b5e761a5d]::ptr::drop_in_place::<alloc[47bc6d386d7ae45f]::boxed::Box<dyn for<'a, 'b, 'c> core[f827f14b5e761a5d]::ops::function::Fn<(&'a dyn core[f827f14b5e761a5d]::any::Any, gpui[436f44c7a696e313]::window::DispatchPhase, &'b mut gpui[436f44c7a696e313]::window::WindowContext<'c>), Output = ()>>>
core[f827f14b5e761a5d]::ptr::drop_in_place::<[gpui[436f44c7a696e313]::key_dispatch::DispatchActionListener]>
core[f827f14b5e761a5d]::ptr::drop_in_place::<gpui[436f44c7a696e313]::key_dispatch::DispatchNode>
<gpui[436f44c7a696e313]::window::WindowContext>::draw
<gpui[436f44c7a696e313]::window::Window>::new::{closure#4}::{closure#1}
<gpui[436f44c7a696e313]::window::Window>::new::{closure#4}
gpui[436f44c7a696e313]::platform::mac::window::step
<gpui[436f44c7a696e313]::platform::mac::platform::MacPlatform as gpui[436f44c7a696e313]::platform::Platform>::run
<gpui[436f44c7a696e313]::app::App>::run::<zed[1325643a57976c4c]::main::{closure#5}>
zed[1325643a57976c4c]::main
std[4f7d7c3ef984657a]::sys::backtrace::__rust_begin_short_backtrace::<fn(), ()>
std[4f7d7c3ef984657a]::rt::lang_start::<()>::{closure#0}
std::rt::lang_start_internal::hdd117cb81a316264
_main

[SomeoneToIgnore]

This seems to be a combination of XML language server issues (leaks reported) + Zed starting a language server for each excerpt (also reported).

So technically a duplicate, but a good repro case for both issues.

[notpeter]

I don't think this is a language server thing.
It happens when searching in Plain Text too.
Amazingly Zed beach-balled, ballooned to over 100GB of ram, finished the search, rendered the multi-buffer and dropped to 8GB of ram. When I closed the multi-buffer it dropped to 450MB.
Bravo Zed. Truly stunning.

While this is a bit exaggerated (40GB of XML == 120GB of memory) it's not that crazy for folks to have a directory with data files in a folder that they are also editing in Zed. In practice it means if you project search for the wrong needle with a large datasets lying around you can KP your mac (I did).

mkdir /tmp/uspto-txt
cd /tmp/uspto-txt
wget https://bulkdata.uspto.gov/data/patent/grant/redbook/fulltext/2024/ipg241119.zip
unzip ipg241119.zip
rm ipg241119.zip
for i in {1..50}; do cp ipg241119.xml ipg241119-$i.txt; done
rm ipg241119.xml
zed .

Seriously. Save your work before trying this.
And when macOS suggests you kill Zed, do it.
Failure to so may result in system freezing.

[RemcoSmitsDev]

This might be related:

• Editor freezes while scrolling through project search results while it's still streaming #20436