High cpu usage

[silence-coding]

Version
[[package]]
name = "tokio"
version = "0.2.13"

Platform
Linux OMP-01 4.18.0-147.5.1.2.h340.eulerosv2r9.x86_64 ................ x86_64 x86_64 x86_64 GNU/Linux

Description

When the process exits, the process does not exit completely, and the CPU usage is high.

TOP

GDB

#0  0x00007f6aa3f06a3c in pthread_cond_wait () from /usr/lib64/libpthread.so.0
#1  0x000055eee924c30c in tokio::park:: thread::Inner::park::h42034565b64c37e5 ()
#2  0x000055eee924244c in tokio::runtime::enter::Enter::block_on::h7bd58150fb51adbb ()
#3  0x000055eee923e31c in tokio::runtime::blocking::pool::BlockingPool::shutdown::h67b3bf4e3e702ec6 ()
#4  0x000055eee923e464 in _$LT$tokio..runtime..blocking..pool..BlockingPool$u20$as$u20$core..ops..drop..Drop$GT$::drop::hd75c0dfcb74ce4d8 ()
#5  0x000055eee8eb17be in core::ptr::drop_in_place::h2390ec3e5ea0f92f ()
#6  0x000055eee8f117c3 in hi::main::h974ca01c377a4991 ()
#7  0x000055eee886ce33 in std::rt::lang_start::_$u7b$$u7b$closure$u7d$$u7d$::hde50d02ef64e1ece ()
#8  0x000055eee929c7c8 in {{closure}} () at src/libstd/rt.rs:52
#9  do_call<closure-0,i32> () at src/libstd/panicking.rs:297
#10 try<i32,closure-0> () at src/libstd/panicking.rs:274
#11 catch_unwind<closure-0,i32> () at src/libstd/panic.rs:394
#12 std::rt::lang_start_internal::h7aa2f6739b31fe58 () at src/libstd/rt.rs:51
#13 0x000055eee8f19812 in main ()

[Darksonn]

The typical reason for the runtime not being able to exit is if there are tasks spawned with spawn_blocking still running, since these are uncancellable, so the runtime will simply wait for them. It can also happen if an ordinary task is blocking the thread. You may want to try using shutdown_timeout.

Regarding the backtrace, I assume that's not the only thread? And could you try it with debug symbols turned on? Note that you can turn on debug symbols while still using release mode.

[silence-coding]

only thread

(gdb) info threads
Id Target Id Frame

• 1 process 270534 "cc" 0x00007f6aa3f06a3c in pthread_cond_wait () from /usr/lib64/libpthread.so.0

[silence-coding]

。。。。，use top -H -p ，there's more than one thread here

(gdb) bt
#0  0x000055eee90b350c in hashbrown::map::HashMap$LT$K$C$V$C$S$GT$::remove::h07f1bbea6c26227c ()
#1  0x000055eee90ae836 in _$LT$hbx..listen..manager..ListenerToken$u20$as$u20$core..ops..drop..Drop$GT$::drop::h607d355d50acee84 ()
#2  0x000055eee8ce9ade in core::ptr::drop_in_place::h9ec6328dcf12b526 ()
#3  0x000055eee8cc7a44 in core::ptr::drop_in_place::h1aeb9ee292cfa4be ()
#4  0x000055eee8d66c7a in tokio::task::core::Core$LT$T$GT$::transition_to_consumed::he3ea2a049faf3a78 ()
#5  0x000055eee85b65f8 in tokio::task::harness::Harness$LT$T$C$S$GT$::do_cancel::hb9a7b03a1c74c851 ()
#6  0x000055eee92367bf in tokio::runtime::thread_pool::worker::GenerationGuard::run::h132b5d9748a652f3 ()
#7  0x000055eee922dfd1 in std::thread::local::LocalKey$LT$T$GT$::with::hc53944dca76bd295 ()
#8  0x000055eee9235ee7 in tokio::runtime::thread_pool::worker::Worker::run::h66bdcf9450037f90 ()
#9  0x000055eee924ecba in tokio::task::core::Core$LT$T$GT$::poll::h37557b3d79ec35ad ()
#10 0x000055eee925490a in tokio::task::harness::Harness$LT$T$C$S$GT$::poll::h62f0931a6dbf0caa ()
#11 0x000055eee923e9fb in tokio::runtime::blocking::pool::Inner::run::h2ddfe0c35845e7d5 ()

the fd is RawFd

the code

impl<'a> Drop for ListenerToken<'a> {
    fn drop(&mut self) {
        let fd = self.fd;
       self.fd = -1;
            if let Some(key) = {
                let mut lock = self.cm.fds.lock().unwrap();
                lock.used.remove(&fd)
            } {
                info!("{} released", key);
            }
        }
    }
}

[udoprog]

@silence-coding if you're seeing high cpu usage it would be useful if you could generate a flamegraph (https://github.com/flamegraph-rs/flamegraph) so we can see where the time is being spent.

Do you think you could do that for us?

[silence-coding]

@udoprog I don't know how to reproduce it.

[udoprog]

@silence-coding All right. Seems like you figured out it's probably not caused by Tokio though by stepping through the assembly. Sweet work!

I guess it means this issue can be closed here?

[silence-coding]

All right