Exception in add route through fboss_route.py #1

andrianyablonskyy · 2015-04-07T08:48:29Z

There is an error during add route through the fboss script.

=================================================

Traceback (most recent call last):
File "fboss_route.py", line 18, in
from neteng.fboss.ctrl import FbossCtrl
ImportError: No module named neteng.fboss.ctrl

=================================================

Steps to reproduce:

Start wedge_agent from sudo (# ./wedge_agent --config sample1.json)
Change directory to './fboss/agent/tools'
Start fboss_route.py (# python fboss_route.py add prefix=1.1.1.0/24 nexthop=10.1.1.1)

oridb · 2015-04-07T17:50:24Z

It looks like there are three problems, and two of them are our bugs:

First, we're not generating the Python bindings for the Thrift interface.
Second, the import path for the thrift interface is wrong.

Finally, running the Python scripts will need to modify the PYTHONPATH to include the generated
thrift bindings.

I'll look at getting that fixed soon.

ghost · 2015-08-04T18:03:06Z

Thank you for reporting this issue and appreciate your patience. We've notified the core team for an update on this issue. We're looking for a response within the next 30 days or the issue may be closed.

Updated getdeps.sh to checkout specific versions of OpenNSL and iproute2

Summary: Currently, HwMacsecTest.installRemoveKeys always fails. It seems to be happening as far as we have records on Conveyer: https://www.internalfb.com/svc/conveyors/fboss/qsfp_service/bundles/1164?node_name=fboss_qsfp_test_elbert_2x16Q_6x8DD_credo-0.4.3_credo-0.4.3&run_id=76759253 This happens in destructor right when the test is finished: ``` E1022 04:04:59.163550 3563869 SaiApiError.h:67] [macsec] Failed to remove sai object : MacsecSaiId(6 59): FAILURE terminate called after throwing an instance of 'facebook::fboss::SaiApiError' what(): [macsec] Failed to remove sai object : MacsecSaiId(659): FAILURE ``` ``` @ 0000000003f80571 facebook::fboss::RefMap<facebook::fboss::RefMapUMap, facebook::fboss::SaiAttribute<_sai_macsec_attr_t, (_sai_macsec_attr_t)0, int, void, void>, facebook::fboss::SaiObject<facebook::fboss::SaiMacsecTraits> >::makeShared<facebook::fboss::SaiObject<facebook::fboss::SaiMacsecTraits> >(facebook::fboss::SaiAttribute<_sai_macsec_attr_t, (_sai_macsec_attr_t)0, int, void, void> const&, facebook::fboss::SaiObject<facebook::fboss::SaiMacsecTraits>&&)::{lambda(facebook::fboss::SaiObject<facebook::fboss::SaiMacsecTraits>*)#1}::operator()(facebook::fboss::SaiObject<facebook::fboss::SaiMacsecTraits>*) const @ 0000000003f805d7 std::_Sp_counted_deleter<facebook::fboss::SaiObject<facebook::fboss::SaiMacsecTraits>*, facebook::fboss::RefMap<facebook::fboss::RefMapUMap, facebook::fboss::SaiAttribute<_sai_macsec_attr_t, (_sai_macsec_attr_t)0, int, void, void>, facebook::fboss::SaiObject<facebook::fboss::SaiMacsecTraits> >::makeShared<facebook::fboss::SaiObject<facebook::fboss::SaiMacsecTraits> >(facebook::fboss::SaiAttribute<_sai_macsec_attr_t, (_sai_macsec_attr_t)0, int, void, void> const&, facebook::fboss::SaiObject<facebook::fboss::SaiMacsecTraits>&&)::{lambda(facebook::fboss::SaiObject<facebook::fboss::SaiMacsecTraits>*)#1}, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() @ 0000000003f7a410 void folly::f14::detail::F14Table<folly::f14::detail::ValueContainerPolicy<_sai_macsec_direction_t, std::unique_ptr<facebook::fboss::SaiMacsecHandle, std::default_delete<facebook::fboss::SaiMacsecHandle> >, void, void, void> >::clearImpl<true>() @ 0000000003f5cd14 facebook::fboss::SaiMacsecManager::~SaiMacsecManager() @ 0000000003fc6fca facebook::fboss::SaiManagerTable::reset(bool) @ 0000000003fc6036 facebook::fboss::SaiManagerTable::~SaiManagerTable() @ 00000000042df2c5 facebook::fboss::SaiSwitch::~SaiSwitch() @ 00000000042df489 facebook::fboss::SaiSwitch::~SaiSwitch() @ 0000000003d4d8f7 facebook::fboss::SaiPlatform::~SaiPlatform() @ 0000000003d36d44 facebook::fboss::SaiElbert8DDPhyPlatform::~SaiElbert8DDPhyPlatform() @ 0000000003d36da9 facebook::fboss::SaiElbert8DDPhyPlatform::~SaiElbert8DDPhyPlatform() @ 0000000003c4b837 facebook::fboss::SaiPhyManager::PlatformInfo::~PlatformInfo() ``` A crash during shut down doesn't affect real world functionality but makes every test fail. This is happening because we create the ACL table when setting up Macsec on port but never delete it. So when we try to delete MacSec, it says MacSec still have objects using it and fails. third-party/tp2/CredoB52SAI/0.26/CredoB52SAI-0.2.6/src/credo/credo_macsec.cpp: ``` sai_status_t CredoMacsec::PreDeconfigure() { if (m_macsec_object_list.size() > 0) { LOG_ERROR("Can't remove %s %d, another macsec object is still in use\n", GetObjectTypeName(), GetResourceId()); foreach(CredoSAIObject* object, m_macsec_object_list) { LOG_ERROR("%s %d is in use\n", object->GetObjectTypeName(), object->GetResourceId()); } return SAI_STATUS_OBJECT_IN_USE; } return SAI_STATUS_SUCCESS; } ``` 2 strange issues during debugging: 1. This code is returning SAI_STATUS_OBJECT_IN_USE. It could have been helpful to know what's going on. Why're we getting SAI_STATUS_FAILURE and printing a generic "FAILURE" message? Some code in the stack must have swallowed the error code and thrown its own generic error. As far as I can tell, Credo SDK is propagating it all the way up. This bug is most likely on our side. 2. This code is helpfully printing out a full list of all objects in use stopping us from removal. Could have been super helpful. Why isn't it there? ``` E1022 04:02:44.107355 3564148 SaiApiError.h:67] [unspecified] Failed to set debug log for api: INVALID PARAMETER ``` https://www.internalfb.com/code/fbsource/[dced1bdd0568c01615fbf64a9ded908d620eefcd]/fbcode/fboss/agent/hw/sai/api/SaiApiTable.cpp?lines=180-191 These are not fixed in this diff. We probably should look into these later to make debugging less of a shot in the dark. Adding the ACL table removal results in another destructor bug, explained in comments in the fix for future engineers. A future improvement with less explanation needed is for ACL table group resources to be managed in an RAII style object in the table handle so upon delete we just know if we have any ACL table groups to delete and act accordingly. Maybe tackle that in a future diff if time permits. Reviewed By: jasmeetbagga, rajank7 Differential Revision: D31889840 fbshipit-source-id: 72e39019a3dd9bcebc8224fdb4187b8105214dd1

Summary: Also fixes an undefined behavior in our code: ``` #0 0xca4b378 in facebook::fboss::phy::BaldEagle::fwSerdesParamsEyes(facebook::fboss::MdioController<facebook::fboss::YampMdio>::FullyLockedMdio&, facebook::fboss::phy::BaldEagle::LaneMode, facebook::fboss::LaneID)::$_46::operator()(int) const (/tmp/setup_link_test_bin-6.5.21+0xca4b378) #1 0xca3d48c in facebook::fboss::phy::BaldEagle::fwSerdesParamsEyes(facebook::fboss::MdioController<facebook::fboss::YampMdio>::FullyLockedMdio&, facebook::fboss::phy::BaldEagle::LaneMode, facebook::fboss::LaneID) (/tmp/setup_link_test_bin-6.5.21+0xca3d48c) #2 0xca3a8b1 in facebook::fboss::phy::BaldEagle::getPortInfo(facebook::fboss::Port const*, std::vector<facebook::fboss::LaneID, std::allocator<facebook::fboss::LaneID> > const&, std::vector<facebook::fboss::LaneID, std::allocator<facebook::fboss::LaneID> > const&) (/tmp/setup_link_test_bin-6.5.21+0xca3a8b1) #3 0x72b0706 in facebook::fboss::YampPhyInterfaceHandler::getPortInfo(int, facebook::fboss::Port const*, facebook::fboss::phy::PhyPortConfig) (/tmp/setup_link_test_bin-6.5.21+0x72b0706) #4 0x71c15d6 in std::_Function_handler<void (), facebook::fboss::YampPort::updateStats()::$_10>::_M_invoke(std::_Any_data const&) (/tmp/setup_link_test_bin-6.5.21+0x71c15d6) #5 0x71a0578 in facebook::fboss::Pim::runOnPimThreadAfterDelay(std::function<void ()>, unsigned int)::'lambda'()::operator()() const::'lambda'()::operator()() const (/tmp/setup_link_test_bin-6.5.21+0x71a0578) #6 0x1073e68e in folly::TimeoutManager::CobTimeouts::CobTimeout::timeoutExpired() (/tmp/setup_link_test_bin-6.5.21+0x1073e68e) #7 0x107182ef in folly::AsyncTimeout::libeventCallback(int, short, void*) (/tmp/setup_link_test_bin-6.5.21+0x107182ef) #8 0x14b144f7 in event_process_active (/tmp/setup_link_test_bin-6.5.21+0x14b144f7) #9 0x14b14717 in event_base_loop (/tmp/setup_link_test_bin-6.5.21+0x14b14717) #10 0x1071e2b3 in folly::EventBase::loopBody(int, bool) (/tmp/setup_link_test_bin-6.5.21+0x1071e2b3) #11 0x1071d780 in folly::EventBase::loop() (/tmp/setup_link_test_bin-6.5.21+0x1071d780) #12 0x107216fd in folly::EventBase::loopForever() (/tmp/setup_link_test_bin-6.5.21+0x107216fd) #13 0x7fc324218660 in execute_native_thread_routine (/usr/local/fbcode/platform009/lib/libstdc++.so.6+0xd9660) #14 0x7fc3240da20b in start_thread (/usr/local/fbcode/platform009/lib/libpthread.so.0+0x920b) #15 0x7fc323feb16e in clone (/usr/local/fbcode/platform009/lib/libc.so.6+0x11816e) UndefinedBehaviorSanitizer: float-cast-overflow fboss/lib/phy/facebook/credo/yamp/BaldEagle.cpp:1455:22 in ``` Reviewed By: birdsoup Differential Revision: D32162784 fbshipit-source-id: f12a5d5a4f1dd2e9a2b7cef89fb6c268e2d4e587

Summary: On Elbert, sometimes when we fail a test, we trigger ThreadSanitizer error due to thread leakage. This is messing up the logs for finding the real error. Example: https://www.internalfb.com/intern/sandcastle/job/18014399092681044/insights ``` WARNING: ThreadSanitizer: thread leak (pid=1196365) Thread T114 (tid=1197183, finished) created by main thread at: #0 pthread_create <null> (setup_qsfp_test_bin-credo-0.7.2+0xb4e184d) #1 CredoDebugCreateInterface(CredoContext*, CredoDebugInterface**) <null> (setup_qsfp_test_bin-credo-0.7.2+0xb2ff2c7) #2 facebook::fboss::SaiApiTable::queryApis(_sai_service_method_table_t*, std::set<_sai_api_t, std::less<_sai_api_t>, std::allocator<_sai_api_t> > const&) <null> (setup_qsfp_test_bin-credo-0.7.2+0x7a2b979) #3 facebook::fboss::SaiElbert8DDPhyPlatform::preHwInitialized() <null> (setup_qsfp_test_bin-credo-0.7.2+0x3fe8e50) #4 facebook::fboss::ElbertPhyManager::initExternalPhyMap() <null> (setup_qsfp_test_bin-credo-0.7.2+0x3f52e01) #5 facebook::fboss::WedgeManager::initExternalPhyMap() <null> (setup_qsfp_test_bin-credo-0.7.2+0x3ede87d) #6 facebook::fboss::ElbertManager::initExternalPhyMap() <null> (setup_qsfp_test_bin-credo-0.7.2+0x3f01cbb) #7 facebook::fboss::QsfpServiceHandler::init() <null> (setup_qsfp_test_bin-credo-0.7.2+0x3eb01bc) #8 facebook::fboss::setupThriftServer(std::unique_ptr<facebook::fboss::WedgeManager, std::default_delete<facebook::fboss::WedgeManager> >) <null> (setup_qsfp_test_bin-credo-0.7.2+0x3ead2b6) #9 facebook::fboss::HwQsfpEnsemble::init() <null> (setup_qsfp_test_bin-credo-0.7.2+0x3eabd50) #10 facebook::fboss::HwTest::SetUp() <null> (setup_qsfp_test_bin-credo-0.7.2+0x3e95920) #11 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) <null> (setup_qsfp_test_bin-credo-0.7.2+0xabcdb8f) #12 __libc_start_main <null> (libc.so.6+0x25dc4) ThreadSanitizer: thread leak (/tmp/setup_qsfp_test_bin-credo-0.7.2+0xb4e184d) in pthread_create ================== ThreadSanitizer: reported 1 warnings ``` Calling the SAI destructor from our destructor should clean that up no matter how the program ends. (exception, gtest abort, normal exit, etc.) Also, when we fail in Setup(), the error is condition never satisfied. It's not clear which condition is failing. Reviewed By: jasmeetbagga Differential Revision: D34468555 fbshipit-source-id: dd0adf4579c404cf7e98e18d1b4f903bc16b5aa6

Summary: This happened during an on-diff link test: ``` ThreadSanitizer:DEADLYSIGNAL ==3804651==ERROR: ThreadSanitizer: SEGV on unknown address 0x0000003a0deb (pc 0x56253d46afc2 bp 0x7f5fceda87e0 sp 0x7f5fceda87d0 T3804675) ==3804651==The signal is caused by a READ memory access. #0 facebook::fboss::fsdb::FsdbPubSubManager::publishState(facebook::fboss::fsdb::OperDelta&&) <null> (setup_qsfp_test_bin-credo-0.7.2+0x4b04fc2) #1 facebook::fboss::fsdb::FsdbComponentSyncer::publishDelta(facebook::fboss::fsdb::OperDelta&&, bool) <null> (setup_qsfp_test_bin-credo-0.7.2+0x4afb231) #2 facebook::fboss::fsdb::FsdbStateComponentSyncer<facebook::fboss::cfg::QsfpServiceConfig>::publisherStateChanged(facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)::'lambda'()::operator()() const <null> (setup_qsfp_test_bin-credo-0.7.2+0x4af553e) #3 void folly::detail::function::FunctionTraits<void ()>::callSmall<facebook::fboss::fsdb::FsdbStateComponentSyncer<facebook::fboss::cfg::QsfpServiceConfig>::publisherStateChanged(facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)::'lambda'()>(folly::detail::function::Data&) <null> (setup_qsfp_test_bin-credo-0.7.2+0x4af5389) #4 void folly::detail::function::FunctionTraits<void ()>::callBig<folly::EventBase::runInEventBaseThreadAndWait(folly::Function<void ()>)::$_4>(folly::detail::function::Data&) <null> (setup_qsfp_test_bin-credo-0.7.2+0xc1dda4a) #5 bool folly::AtomicNotificationQueue<folly::Function<void ()> >::drive<folly::EventBase::FuncRunner&>(folly::EventBase::FuncRunner&) <null> (setup_qsfp_test_bin-credo-0.7.2+0xc1dcd22) #6 folly::EventBaseAtomicNotificationQueue<folly::Function<void ()>, folly::EventBase::FuncRunner>::execute() <null> (setup_qsfp_test_bin-credo-0.7.2+0xc1d722c) #7 non-virtual thunk to folly::EventBaseAtomicNotificationQueue<folly::Function<void ()>, folly::EventBase::FuncRunner>::handlerReady(unsigned short) <null> (setup_qsfp_test_bin-credo-0.7.2+0xc1dedfd) #8 folly::EventHandler::libeventCallback(int, short, void*) <null> (setup_qsfp_test_bin-credo-0.7.2+0xc1e287c) #9 event_base_loop <null> (setup_qsfp_test_bin-credo-0.7.2+0xcd43578) #10 (anonymous namespace)::EventBaseBackend::eb_event_base_loop(int) <null> (setup_qsfp_test_bin-credo-0.7.2+0xc1d95aa) #11 folly::EventBase::loopMain(int, bool) <null> (setup_qsfp_test_bin-credo-0.7.2+0xc1d6127) #12 folly::EventBase::loopForever() <null> (setup_qsfp_test_bin-credo-0.7.2+0xc1d9fa4) #13 folly::run(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&) <null> (setup_qsfp_test_bin-credo-0.7.2+0xc140edf) #14 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > > >::_M_run() <null> (setup_qsfp_test_bin-credo-0.7.2+0xc14190b) #15 execute_native_thread_routine <null> (libstdc++.so.6+0xdf2e4) #16 __tsan_thread_start_func <null> (setup_qsfp_test_bin-credo-0.7.2+0xd17efae) #17 start_thread <null> (libc.so.6+0x979be) #18 clone <null> (libc.so.6+0x127b9f) ``` Publishers are currently protected by publisherMutex_. But here we have publishState()->publishImpl(). We lock in publishImpl(). But publishState() does this without lock: ``` publishImpl(statePathPublisher_.get(), std::move(pubUnit)); ``` If the other thread is calling createStateDeltaPublisher(), this pointer could be half written. We obtain this garbage pointer. Then obtain the lock while trying to use this garbage. Also, removeStateDeltaPublisher() is completely unprotected. It seems to be used only in tests. But adding locks there for completeness. Reviewed By: jasmeetbagga Differential Revision: D38337566 fbshipit-source-id: 08c2f83bfaae649baa591549a53fa1a94d22e313

Summary: Using command from test plan: ``` ================== WARNING: ThreadSanitizer: data race on vptr (ctor/dtor vs virtual call) (pid=2900615) Read of size 8 at 0x7b5400005000 by thread T7: #0 folly::AsyncTimeout::libeventCallback(int, short, void*) folly/io/async/AsyncTimeout.cpp:174 (libfolly_io_async_async_base.so+0x46551) #1 event_process_active /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/event.c:390:5 (libevent-1.4.so.2+0x85e8) #2 event_base_loop /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/event.c:532:4 (libevent-1.4.so.2+0x85e8) #3 folly::EventBase::loopMain(int, bool) folly/io/async/EventBase.cpp:400 (libfolly_io_async_async_base.so+0x55033) #4 folly::EventBase::loopBody(int, bool) folly/io/async/EventBase.cpp:326 (libfolly_io_async_async_base.so+0x54655) #5 folly::EventBase::loop() folly/io/async/EventBase.cpp:305 (libfolly_io_async_async_base.so+0x544cd) #6 folly::EventBase::loopForever() folly/io/async/EventBase.cpp:545 (libfolly_io_async_async_base.so+0x58632) #7 folly::run(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&) folly/io/async/ScopedEventBaseThread.cpp:40 (libfolly_io_async_scoped_event_base_thread.so+0xc592) #8 void std::__invoke_impl<void, void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> >(std::__invoke_other, void (*&&)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*&&, folly::EventBase*&&, folly::Baton<true, std::atomic>*&&, folly::Range<char const*>&&) third-party-buck/platform009/build/libgcc/include/c++/9.x/bits/invoke.h:60 (libfolly_io_async_scoped_event_base_thread.so+0x18545) #9 std::__invoke_result<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> >::type std::__invoke<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> >(void (*&&)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*&&, folly::EventBase*&&, folly::Baton<true, std::atomic>*&&, folly::Range<char const*>&&) third-party-buck/platform009/build/libgcc/include/c++/9.x/bits/invoke.h:95 (libfolly_io_async_scoped_event_base_thread.so+0x18267) #10 void std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > >::_M_invoke<0ul, 1ul, 2ul, 3ul, 4ul>(std::_Index_tuple<0ul, 1ul, 2ul, 3ul, 4ul>) third-party-buck/platform009/build/libgcc/include/c++/9.x/thread:244 (libfolly_io_async_scoped_event_base_thread.so+0x18193) #11 std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > >::operator()() third-party-buck/platform009/build/libgcc/include/c++/9.x/thread:251 (libfolly_io_async_scoped_event_base_thread.so+0x180c8) #12 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > > >::_M_run() third-party-buck/platform009/build/libgcc/include/c++/9.x/thread:195 (libfolly_io_async_scoped_event_base_thread.so+0x1784f) #13 execute_native_thread_routine /home/engshare/third-party2/libgcc/9.x/src/gcc-9.x/x86_64-facebook-linux/libstdc++-v3/src/c++11/../../../.././libstdc++-v3/src/c++11/thread.cc:80:18 (libstdc++.so.6+0xd9660) Previous write of size 8 at 0x7b5400005000 by main thread (mutexes: write M118355245544776416): #0 facebook::fboss::fsdb::FsdbSubscriber<facebook::fboss::fsdb::OperState>::FsdbSubscriber(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, folly::EventBase*, folly::EventBase*, std::function<void (facebook::fboss::fsdb::OperState&&)>, bool, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>) fboss/fsdb/client/FsdbSubscriber.h:44 (libfboss_fsdb_client_fsdb_pub_sub.so+0x16eb06) #1 facebook::fboss::fsdb::FsdbStateSubscriber::FsdbStateSubscriber(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, folly::EventBase*, folly::EventBase*, std::function<void (facebook::fboss::fsdb::OperState&&)>, bool, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>) fboss/fsdb/client/FsdbStateSubscriber.h:14 (libfboss_fsdb_client_fsdb_pub_sub.so+0x16e7d4) #2 std::_MakeUniq<facebook::fboss::fsdb::FsdbStateSubscriber>::__single_object std::make_unique<facebook::fboss::fsdb::FsdbStateSubscriber, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, folly::EventBase*, folly::EventBase*, std::function<void (facebook::fboss::fsdb::OperState&&)>&, bool&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, folly::EventBase*&&, folly::EventBase*&&, std::function<void (facebook::fboss::fsdb::OperState&&)>&, bool&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>&) third-party-buck/platform009/build/libgcc/include/c++/9.x/bits/unique_ptr.h:857 (libfboss_fsdb_client_fsdb_pub_sub.so+0x16d101) #3 _ZZN8facebook5fboss4fsdb17FsdbPubSubManager19addSubscriptionImplINS1_19FsdbStateSubscriberEEEvRKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaISB_EESt8functionIFvNS1_16FsdbStreamClient5StateESI_EENT_19FsdbSubUnitUpdateCbEbRKSB_iENKUlTyRSL_E_clISt13unordered_mapISB_St10unique_ptrISH_St14default_deleteISH_EESt4hashISB_ESt8equal_toISB_ESaISt4pairISN_SW_EEEEEDaSP_ fboss/fsdb/client/FsdbPubSubManager.cpp:243 (libfboss_fsdb_client_fsdb_pub_sub.so+0x16c8cd) #4 _ZN5folly16SynchronizedBaseINS_12SynchronizedISt13unordered_mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10unique_ptrIN8facebook5fboss4fsdb16FsdbStreamClientESt14default_deleteISD_EESt4hashIS8_ESt8equal_toIS8_ESaISt4pairIKS8_SG_EEENS_15SharedMutexImplILb0EvSt6atomicNS_24SharedMutexPolicyDefaultEEEEELNS_6detail22SynchronizedMutexLevelE2EE9withWLockIZNSC_17FsdbPubSubManager19addSubscriptionImplINSC_19FsdbStateSubscriberEEEvRKSt6vectorIS8_SaIS8_EESt8functionIFvNSD_5StateES18_EENT_19FsdbSubUnitUpdateCbEbRSM_iEUlTyRS1B_E_EEDaOS1B_ folly/Synchronized.h:305 (libfboss_fsdb_client_fsdb_pub_sub.so+0x16c6a7) #5 void facebook::fboss::fsdb::FsdbPubSubManager::addSubscriptionImpl<facebook::fboss::fsdb::FsdbStateSubscriber>(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, facebook::fboss::fsdb::FsdbStateSubscriber::FsdbSubUnitUpdateCb, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) fboss/fsdb/client/FsdbPubSubManager.cpp:240 (libfboss_fsdb_client_fsdb_pub_sub.so+0x139f75) #6 facebook::fboss::fsdb::FsdbPubSubManager::addStatePathSubscription(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, std::function<void (facebook::fboss::fsdb::OperState&&)>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) fboss/fsdb/client/FsdbPubSubManager.cpp:190 (libfboss_fsdb_client_fsdb_pub_sub.so+0x139d50) #7 facebook::fboss::TestSubscription<thriftpath::ChildThriftPath<facebook::fboss::cfg::QsfpServiceConfig, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::QsfpServiceData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > > >::TestSubscription(facebook::fboss::fsdb::FsdbPubSubManager*, thriftpath::ChildThriftPath<facebook::fboss::cfg::QsfpServiceConfig, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::QsfpServiceData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > >) fboss/qsfp_service/fsdb/facebook/test/QsfpFsdbSyncerTest.cpp:69 (fsdb-syncer-test+0x104e5d) #8 facebook::fboss::TestSubscription<thriftpath::ChildThriftPath<facebook::fboss::cfg::QsfpServiceConfig, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::QsfpServiceData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > > > facebook::fboss::QsfpFsdbTcvrMgrTest::subscribe<thriftpath::ChildThriftPath<facebook::fboss::cfg::QsfpServiceConfig, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::QsfpServiceData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > > >(thriftpath::ChildThriftPath<facebook::fboss::cfg::QsfpServiceConfig, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::QsfpServiceData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > >) fboss/qsfp_service/fsdb/facebook/test/QsfpFsdbSyncerTest.cpp:158 (fsdb-syncer-test+0xe3417) #9 facebook::fboss::QsfpFsdbTcvrMgrTest_testTcvrMgrConfigUpdate_Test::TestBody() fboss/qsfp_service/fsdb/facebook/test/QsfpFsdbSyncerTest.cpp:195 (fsdb-syncer-test+0xe2cd0) #10 testing::Test::Run() /home/engshare/third-party2/googletest/20201023/src/googletest/googletest/src/gtest.cc:2680:50 (libgtest.so.1.10.0+0x4d6a5) #11 testing::Test::Run() /home/engshare/third-party2/googletest/20201023/src/googletest/googletest/src/gtest.cc:2670:6 (libgtest.so.1.10.0+0x4d6a5) #12 main common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x218a) Location is heap block of size 592 at 0x7b5400005000 allocated by main thread: #0 operator new(unsigned long) <null> (fsdb-syncer-test+0x110b2c) #1 std::_MakeUniq<facebook::fboss::fsdb::FsdbStateSubscriber>::__single_object std::make_unique<facebook::fboss::fsdb::FsdbStateSubscriber, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, folly::EventBase*, folly::EventBase*, std::function<void (facebook::fboss::fsdb::OperState&&)>&, bool&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, folly::EventBase*&&, folly::EventBase*&&, std::function<void (facebook::fboss::fsdb::OperState&&)>&, bool&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>&) third-party-buck/platform009/build/libgcc/include/c++/9.x/bits/unique_ptr.h:857 (libfboss_fsdb_client_fsdb_pub_sub.so+0x16cf9f) #2 _ZZN8facebook5fboss4fsdb17FsdbPubSubManager19addSubscriptionImplINS1_19FsdbStateSubscriberEEEvRKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaISB_EESt8functionIFvNS1_16FsdbStreamClient5StateESI_EENT_19FsdbSubUnitUpdateCbEbRKSB_iENKUlTyRSL_E_clISt13unordered_mapISB_St10unique_ptrISH_St14default_deleteISH_EESt4hashISB_ESt8equal_toISB_ESaISt4pairISN_SW_EEEEEDaSP_ fboss/fsdb/client/FsdbPubSubManager.cpp:243 (libfboss_fsdb_client_fsdb_pub_sub.so+0x16c8cd) #3 _ZN5folly16SynchronizedBaseINS_12SynchronizedISt13unordered_mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10unique_ptrIN8facebook5fboss4fsdb16FsdbStreamClientESt14default_deleteISD_EESt4hashIS8_ESt8equal_toIS8_ESaISt4pairIKS8_SG_EEENS_15SharedMutexImplILb0EvSt6atomicNS_24SharedMutexPolicyDefaultEEEEELNS_6detail22SynchronizedMutexLevelE2EE9withWLockIZNSC_17FsdbPubSubManager19addSubscriptionImplINSC_19FsdbStateSubscriberEEEvRKSt6vectorIS8_SaIS8_EESt8functionIFvNSD_5StateES18_EENT_19FsdbSubUnitUpdateCbEbRSM_iEUlTyRS1B_E_EEDaOS1B_ folly/Synchronized.h:305 (libfboss_fsdb_client_fsdb_pub_sub.so+0x16c6a7) #4 void facebook::fboss::fsdb::FsdbPubSubManager::addSubscriptionImpl<facebook::fboss::fsdb::FsdbStateSubscriber>(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, facebook::fboss::fsdb::FsdbStateSubscriber::FsdbSubUnitUpdateCb, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) fboss/fsdb/client/FsdbPubSubManager.cpp:240 (libfboss_fsdb_client_fsdb_pub_sub.so+0x139f75) #5 facebook::fboss::fsdb::FsdbPubSubManager::addStatePathSubscription(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, std::function<void (facebook::fboss::fsdb::OperState&&)>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) fboss/fsdb/client/FsdbPubSubManager.cpp:190 (libfboss_fsdb_client_fsdb_pub_sub.so+0x139d50) #6 facebook::fboss::TestSubscription<thriftpath::ChildThriftPath<facebook::fboss::cfg::QsfpServiceConfig, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::QsfpServiceData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > > >::TestSubscription(facebook::fboss::fsdb::FsdbPubSubManager*, thriftpath::ChildThriftPath<facebook::fboss::cfg::QsfpServiceConfig, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::QsfpServiceData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > >) fboss/qsfp_service/fsdb/facebook/test/QsfpFsdbSyncerTest.cpp:69 (fsdb-syncer-test+0x104e5d) #7 facebook::fboss::TestSubscription<thriftpath::ChildThriftPath<facebook::fboss::cfg::QsfpServiceConfig, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::QsfpServiceData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > > > facebook::fboss::QsfpFsdbTcvrMgrTest::subscribe<thriftpath::ChildThriftPath<facebook::fboss::cfg::QsfpServiceConfig, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::QsfpServiceData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > > >(thriftpath::ChildThriftPath<facebook::fboss::cfg::QsfpServiceConfig, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::QsfpServiceData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > >) fboss/qsfp_service/fsdb/facebook/test/QsfpFsdbSyncerTest.cpp:158 (fsdb-syncer-test+0xe3417) #8 facebook::fboss::QsfpFsdbTcvrMgrTest_testTcvrMgrConfigUpdate_Test::TestBody() fboss/qsfp_service/fsdb/facebook/test/QsfpFsdbSyncerTest.cpp:195 (fsdb-syncer-test+0xe2cd0) #9 testing::Test::Run() /home/engshare/third-party2/googletest/20201023/src/googletest/googletest/src/gtest.cc:2680:50 (libgtest.so.1.10.0+0x4d6a5) #10 testing::Test::Run() /home/engshare/third-party2/googletest/20201023/src/googletest/googletest/src/gtest.cc:2670:6 (libgtest.so.1.10.0+0x4d6a5) #11 main common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x218a) Mutex M118355245544776416 is already destroyed. Thread T7 'FsdbReconnectTh' (tid=2901715, running) created by main thread at: #0 pthread_create <null> (fsdb-syncer-test+0x1744ed) #1 __gthread_create /home/engshare/third-party2/libgcc/9.x/src/gcc-9.x/x86_64-facebook-linux/libstdc++-v3/include/x86_64-facebook-linux/bits/gthr-default.h:663:35 (libstdc++.so.6+0xd993c) #2 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) /home/engshare/third-party2/libgcc/9.x/src/gcc-9.x/x86_64-facebook-linux/libstdc++-v3/src/c++11/../../../.././libstdc++-v3/src/c++11/thread.cc:135:37 (libstdc++.so.6+0xd993c) #3 folly::ScopedEventBaseThread::ScopedEventBaseThread(folly::EventBase::Options, folly::EventBaseManager*, folly::Range<char const*>) folly/io/async/ScopedEventBaseThread.cpp:68 (libfolly_io_async_scoped_event_base_thread.so+0xc060) #4 folly::ScopedEventBaseThread::ScopedEventBaseThread(folly::EventBaseManager*, folly::Range<char const*>) folly/io/async/ScopedEventBaseThread.cpp:60 (libfolly_io_async_scoped_event_base_thread.so+0xb4b8) #5 folly::ScopedEventBaseThread::ScopedEventBaseThread(folly::Range<char const*>) folly/io/async/ScopedEventBaseThread.cpp:53 (libfolly_io_async_scoped_event_base_thread.so+0xb672) #6 facebook::fboss::fsdb::FsdbPubSubManager::FsdbPubSubManager(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) fboss/fsdb/client/FsdbPubSubManager.h:139 (libfboss_fsdb_client_fsdb_pub_sub.so+0x1371f2) #7 std::_MakeUniq<facebook::fboss::fsdb::FsdbPubSubManager>::__single_object std::make_unique<facebook::fboss::fsdb::FsdbPubSubManager, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) third-party-buck/platform009/build/libgcc/include/c++/9.x/bits/unique_ptr.h:857 (libfboss_fsdb_client_fsdb_syncer.so+0x10247) #8 facebook::fboss::fsdb::FsdbSyncManager::FsdbSyncManager(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) fboss/fsdb/client/FsdbSyncManager.cpp:12 (libfboss_fsdb_client_fsdb_syncer.so+0x100be) #9 facebook::fboss::QsfpFsdbSyncManager::QsfpFsdbSyncManager() fboss/qsfp_service/fsdb/QsfpFsdbSyncManager.cpp:17 (libfboss_qsfp_service_fsdb_fsdb-syncer.so+0x33191) #10 std::_MakeUniq<facebook::fboss::QsfpFsdbSyncManager>::__single_object std::make_unique<facebook::fboss::QsfpFsdbSyncManager>() third-party-buck/platform009/build/libgcc/include/c++/9.x/bits/unique_ptr.h:857 (libfboss_qsfp_service_platforms_wedge_wedge-platform.so+0x1ec45f) #11 facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode) fboss/qsfp_service/platforms/wedge/WedgeManager.cpp:63 (libfboss_qsfp_service_platforms_wedge_wedge-platform.so+0x1ebcab) #12 facebook::fboss::MockWedgeManager::MockWedgeManager(int, int) fboss/qsfp_service/platforms/wedge/tests/MockWedgeManager.h:25 (libfboss_qsfp_service_test_transceiver_manager_test_helper.so+0x5408f) #13 std::_MakeUniq<facebook::fboss::MockWedgeManager>::__single_object std::make_unique<facebook::fboss::MockWedgeManager, int const&>(int const&) third-party-buck/platform009/build/libgcc/include/c++/9.x/bits/unique_ptr.h:857 (libfboss_qsfp_service_test_transceiver_manager_test_helper.so+0x42edb) #14 facebook::fboss::TransceiverManagerTestHelper::resetTransceiverManager() fboss/qsfp_service/test/TransceiverManagerTestHelper.cpp:35 (libfboss_qsfp_service_test_transceiver_manager_test_helper.so+0x43138) #15 facebook::fboss::QsfpFsdbTcvrMgrTest_testTcvrMgrConfigUpdate_Test::TestBody() fboss/qsfp_service/fsdb/facebook/test/QsfpFsdbSyncerTest.cpp:191 (fsdb-syncer-test+0xe2c53) #16 testing::Test::Run() /home/engshare/third-party2/googletest/20201023/src/googletest/googletest/src/gtest.cc:2680:50 (libgtest.so.1.10.0+0x4d6a5) #17 testing::Test::Run() /home/engshare/third-party2/googletest/20201023/src/googletest/googletest/src/gtest.cc:2670:6 (libgtest.so.1.10.0+0x4d6a5) #18 main common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x218a) ThreadSanitizer: data race on vptr (ctor/dtor vs virtual call) folly/io/async/AsyncTimeout.cpp:174 in folly::AsyncTimeout::libeventCallback(int, short, void*) ================== ``` What's going on here? In FsdbStreamClient constructor, we immediately schedule the periodic reconnect on the reconnect thread. The periodic call back is implemented in folly::AsyncTimeout like this: ``` auto timeout = reinterpret_cast<AsyncTimeout*>(arg); ... timeout->timeoutExpired(); ``` We're arg being passed in as a void*. timeoutExpired() is a virtual function we overrided. If this gets called after the base class constructor FsdbStreamClient has finished but before child constructor classes like FsdbDeltaSubscriber has started, the vtable will point at the base class so virtual method calls have incorrect behavior. See https://github.com/google/sanitizers/wiki/ThreadSanitizerPopularDataRaces#data-race-on-vptr-during-construction IMHO, this is probably benign. Because the wrong vtable doesn't actually trigger wrong behavior if nobody override timeoutExpired() in child classes or call virtual methods from the base class implementation. But let's fix this to shut up the warning and prevent future problems. FsdbStreamClient is not an AsyncTimeout. It needed an AsyncTimeout. So making it a member instead of a parent is probably a better fit. That avoids this hard problem altogether. So we don't need to add start() and stop() from everywhere. Reviewed By: jasmeetbagga Differential Revision: D37692128 fbshipit-source-id: fadf86ff7850d37cae49c651cca29c1d2b819b2b

…members Summary: ``` ❯ buck test mode/dev-tsan //fboss/facebook/mka_service/mka_module/tests:MKAModuleTest -- --stress-runs 100 --regex MKAModuleTest.WarmBootConfigLoad ... Summary Pass: 78 Fail: 1 ✗ fboss/facebook/mka_service/mka_module/tests:MKAModuleTest - MKAModuleTest.WarmBootConfigLoad Fatal: 21 ☢ fboss/facebook/mka_service/mka_module/tests:MKAModuleTest - MKAModuleTest.WarmBootConfigLoad ☢ fboss/facebook/mka_service/mka_module/tests:MKAModuleTest - MKAModuleTest.WarmBootConfigLoad ☢ fboss/facebook/mka_service/mka_module/tests:MKAModuleTest - MKAModuleTest.WarmBootConfigLoad ☢ fboss/facebook/mka_service/mka_module/tests:MKAModuleTest - MKAModuleTest.WarmBootConfigLoad ☢ fboss/facebook/mka_service/mka_module/tests:MKAModuleTest - MKAModuleTest.WarmBootConfigLoad ☢ fboss/facebook/mka_service/mka_module/tests:MKAModuleTest - MKAModuleTest.WarmBootConfigLoad ☢ fboss/facebook/mka_service/mka_module/tests:MKAModuleTest - MKAModuleTest.WarmBootConfigLoad ☢ fboss/facebook/mka_service/mka_module/tests:MKAModuleTest - MKAModuleTest.WarmBootConfigLoad ☢ fboss/facebook/mka_service/mka_module/tests:MKAModuleTest - MKAModuleTest.WarmBootConfigLoad ☢ fboss/facebook/mka_service/mka_module/tests:MKAModuleTest - MKAModuleTest.WarmBootConfigLoad ...and 11 more not shown... ``` Problem 1: ``` ThreadSanitizer: data race third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/basic_string.h:926 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::length() const ================== ================== WARNING: ThreadSanitizer: data race (pid=1286130) Read of size 8 at 0x7b84000a3548 by thread T50 (mutexes: write M341564816177216360, read M730281896454009712): #0 memcmp <null> (MKAModuleTest+0x26ebb7) #1 std::char_traits<char>::compare(char const*, char const*, unsigned long) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/char_traits.h:361 (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x8931c) #2 __gnu_cxx::__enable_if<__is_char<char>::__value, bool>::__type std::operator==<char>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/basic_string.h:6236 (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x89286) #3 std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/stl_function.h:356 (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x89141) #4 std::__detail::_Hashtable_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, facebook::fboss::PacketStreamService::ClientInfo>, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Hashtable_traits<true, false, true> >::_M_equals(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long, std::__detail::_Hash_node_value<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, facebook::fboss::PacketStreamService::ClientInfo>, true> const&) const third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/hashtable_policy.h:1616 (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x88f93) #5 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, facebook::fboss::PacketStreamService::ClientInfo>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, facebook::fboss::PacketStreamService::ClientInfo> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_before_node(unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long) const third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/hashtable.h:1821 (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x88a98) #6 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, facebook::fboss::PacketStreamService::ClientInfo>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, facebook::fboss::PacketStreamService::ClientInfo> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_node(unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long) const third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/hashtable.h:795 (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x89ed5) #7 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, facebook::fboss::PacketStreamService::ClientInfo>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, facebook::fboss::PacketStreamService::ClientInfo> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::find(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/hashtable.h:1591 (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x8d553) #8 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, facebook::fboss::PacketStreamService::ClientInfo, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, facebook::fboss::PacketStreamService::ClientInfo> > >::find(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/unordered_map.h:880 (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x8cf85) #9 auto facebook::fboss::PacketStreamService::send(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, facebook::fboss::TPacket&&)::$_3::operator()<std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, facebook::fboss::PacketStreamService::ClientInfo, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, facebook::fboss::PacketStreamService::ClientInfo> > > const>(std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, facebook::fboss::PacketStreamService::ClientInfo, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, facebook::fboss::PacketStreamService::ClientInfo> > > const&) const fboss/agent/thrift_packet_stream/PacketStreamService.cpp:71 (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x8c334) #10 auto folly::SynchronizedBase<folly::Synchronized<std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, facebook::fboss::PacketStreamService::ClientInfo, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, facebook::fboss::PacketStreamService::ClientInfo> > >, folly::SharedMutexImpl<false, void, std::atomic, folly::SharedMutexPolicyDefault> >, (folly::detail::SynchronizedMutexLevel)2>::withRLock<facebook::fboss::PacketStreamService::send(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, facebook::fboss::TPacket&&)::$_3>(facebook::fboss::PacketStreamService::send(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, facebook::fboss::TPacket&&)::$_3&&) const folly/Synchronized.h:342 (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x8c254) #11 facebook::fboss::PacketStreamService::send(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, facebook::fboss::TPacket&&) fboss/agent/thrift_packet_stream/PacketStreamService.cpp:70 (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x8c1cb) #12 facebook::fboss::BidirectionalPacketStream::send(facebook::fboss::TPacket&&) fboss/agent/thrift_packet_stream/BidirectionalPacketStream.cpp:265 (libfboss_agent_thrift_packet_stream_bidirectional_packet_stream.so+0xb92b0) #13 facebook::fboss::AsyncThriftPacketTransport::send(std::unique_ptr<folly::IOBuf, std::default_delete<folly::IOBuf> > const&) fboss/agent/thrift_packet_stream/AsyncThriftPacketTransport.cpp:19 (libfboss_agent_thrift_packet_stream_bidirectional_packet_stream.so+0x6e131) #14 auto facebook::fboss::mka::MKASession::timeoutExpired(folly::Synchronized<std::unique_ptr<facebook::fboss::mka::MKAParticipant, std::default_delete<facebook::fboss::mka::MKAParticipant> >, folly::SharedMutexImpl<false, void, std::atomic, folly::SharedMutexPolicyDefault> >&)::$_8::operator()<std::unique_ptr<facebook::fboss::mka::MKAParticipant, std::default_delete<facebook::fboss::mka::MKAParticipant> > >(std::unique_ptr<facebook::fboss::mka::MKAParticipant, std::default_delete<facebook::fboss::mka::MKAParticipant> >&) const fboss/facebook/mka_service/mka_module/MKASession.cpp:627 (libfboss_facebook_mka_service_mka_module_mka_module.so+0x15e1c3) #15 auto folly::SynchronizedBase<folly::Synchronized<std::unique_ptr<facebook::fboss::mka::MKAParticipant, std::default_delete<facebook::fboss::mka::MKAParticipant> >, folly::SharedMutexImpl<false, void, std::atomic, folly::SharedMutexPolicyDefault> >, (folly::detail::SynchronizedMutexLevel)2>::withWLock<facebook::fboss::mka::MKASession::timeoutExpired(folly::Synchronized<std::unique_ptr<facebook::fboss::mka::MKAParticipant, std::default_delete<facebook::fboss::mka::MKAParticipant> >, folly::SharedMutexImpl<false, void, std::atomic, folly::SharedMutexPolicyDefault> >&)::$_8>(facebook::fboss::mka::MKASession::timeoutExpired(folly::Synchronized<std::unique_ptr<facebook::fboss::mka::MKAParticipant, std::default_delete<facebook::fboss::mka::MKAParticipant> >, folly::SharedMutexImpl<false, void, std::atomic, folly::SharedMutexPolicyDefault> >&)::$_8&&) folly/Synchronized.h:305 (libfboss_facebook_mka_service_mka_module_mka_module.so+0x11c014) #16 facebook::fboss::mka::MKASession::timeoutExpired(folly::Synchronized<std::unique_ptr<facebook::fboss::mka::MKAParticipant, std::default_delete<facebook::fboss::mka::MKAParticipant> >, folly::SharedMutexImpl<false, void, std::atomic, folly::SharedMutexPolicyDefault> >&) fboss/facebook/mka_service/mka_module/MKASession.cpp:619 (libfboss_facebook_mka_service_mka_module_mka_module.so+0x11bf1a) #17 facebook::fboss::mka::MKASession::timeoutExpired() fboss/facebook/mka_service/mka_module/MKASession.cpp:659 (libfboss_facebook_mka_service_mka_module_mka_module.so+0x11c219) #18 facebook::fboss::mka::MKASession::MKASession(facebook::fboss::mka::MKASessionConfig const&, folly::EventBase*, std::unique_ptr<facebook::fboss::AsyncPacketTransport::ReadCallback, std::default_delete<facebook::fboss::AsyncPacketTransport::ReadCallback> >, std::shared_ptr<facebook::fboss::AsyncPacketTransport>, std::shared_ptr<facebook::fboss::mka::MacsecHandler>, std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>)::$_37::operator()() const fboss/facebook/mka_service/mka_module/MKASession.cpp:133 (libfboss_facebook_mka_service_mka_module_mka_module.so+0x131375) #19 folly::detail::async_timeout_wrapper<facebook::fboss::mka::MKASession::MKASession(facebook::fboss::mka::MKASessionConfig const&, folly::EventBase*, std::unique_ptr<facebook::fboss::AsyncPacketTransport::ReadCallback, std::default_delete<facebook::fboss::AsyncPacketTransport::ReadCallback> >, std::shared_ptr<facebook::fboss::AsyncPacketTransport>, std::shared_ptr<facebook::fboss::mka::MacsecHandler>, std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>)::$_37>::timeoutExpired() folly/io/async/AsyncTimeout.h:255 (libfboss_facebook_mka_service_mka_module_mka_module.so+0x13132c) #20 folly::AsyncTimeout::libeventCallback(int, short, void*) folly/io/async/AsyncTimeout.cpp:174 (libfolly_io_async_async_base.so+0x41e6b) #21 event_process_active /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/event.c:390:5 (libevent-1.4.so.2+0x8608) #22 event_base_loop /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/event.c:532:4 (libevent-1.4.so.2+0x8608) #23 folly::EventBase::loopMain(int, bool) folly/io/async/EventBase.cpp:400 (libfolly_io_async_async_base.so+0x49c47) #24 folly::EventBase::loopBody(int, bool) folly/io/async/EventBase.cpp:326 (libfolly_io_async_async_base.so+0x4940f) #25 folly::EventBase::loop() folly/io/async/EventBase.cpp:305 (libfolly_io_async_async_base.so+0x49298) #26 folly::EventBase::loopForever() folly/io/async/EventBase.cpp:545 (libfolly_io_async_async_base.so+0x5c7bf) #27 folly::IOThreadPoolExecutor::threadRun(std::shared_ptr<folly::ThreadPoolExecutor::Thread>) folly/executors/IOThreadPoolExecutor.cpp:250 (libfolly_executors_io_thread_pool_executor.so+0x15b72) #28 void std::__invoke_impl<void, void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&>(std::__invoke_memfun_deref, void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/invoke.h:74 (libfolly_executors_thread_pool_executor.so+0x56191) #29 std::__invoke_result<void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&>::type std::__invoke<void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&>(void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/invoke.h:96 (libfolly_executors_thread_pool_executor.so+0x55f07) #30 void std::_Bind<void (folly::ThreadPoolExecutor::* (folly::ThreadPoolExecutor*, std::shared_ptr<folly::ThreadPoolExecutor::Thread>))(std::shared_ptr<folly::ThreadPoolExecutor::Thread>)>::__call<void, 0ul, 1ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul>) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/functional:420 (libfolly_executors_thread_pool_executor.so+0x55e6a) #31 void std::_Bind<void (folly::ThreadPoolExecutor::* (folly::ThreadPoolExecutor*, std::shared_ptr<folly::ThreadPoolExecutor::Thread>))(std::shared_ptr<folly::ThreadPoolExecutor::Thread>)>::operator()<void>() third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/functional:503 (libfolly_executors_thread_pool_executor.so+0x55dc6) #32 void folly::detail::function::FunctionTraits<void ()>::callSmall<std::_Bind<void (folly::ThreadPoolExecutor::* (folly::ThreadPoolExecutor*, std::shared_ptr<folly::ThreadPoolExecutor::Thread>))(std::shared_ptr<folly::ThreadPoolExecutor::Thread>)> >(folly::detail::function::Data&) folly/Function.h:363 (libfolly_executors_thread_pool_executor.so+0x55a5d) #33 folly::detail::function::FunctionTraits<void ()>::operator()() folly/Function.h:392 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xad423) #34 folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()::operator()() folly/executors/thread_factory/NamedThreadFactory.h:40 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xad3bc) #35 void std::__invoke_impl<void, folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()>(std::__invoke_other, folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()&&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/invoke.h:61 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xad34d) #36 std::__invoke_result<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()>::type std::__invoke<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()>(folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()&&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/invoke.h:96 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xad29d) #37 void std::thread::_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/std_thread.h:253 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xad245) #38 std::thread::_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()> >::operator()() third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/std_thread.h:260 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xad1e5) #39 std::thread::_State_impl<std::thread::_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()> > >::_M_run() third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/std_thread.h:211 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xacf79) #40 execute_native_thread_routine <null> (libstdc++.so.6+0xdf2e4) Previous write of size 8 at 0x7b84000a3548 by thread T44: #0 memcpy <null> (MKAModuleTest+0x26d4b6) #1 std::char_traits<char>::copy(char*, char const*, unsigned long) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/char_traits.h:409 (libfboss_agent_thrift_packet_stream_bidirectional_packet_stream.so+0x70881) #2 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_S_copy(char*, char const*, unsigned long) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/basic_string.h:359 (libfboss_agent_thrift_packet_stream_bidirectional_packet_stream.so+0x703d2) #3 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/basic_string.tcc:272 (libfboss_agent_thrift_packet_stream_bidirectional_packet_stream.so+0x6ff46) #4 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/basic_string.h:1386 (libfboss_agent_thrift_packet_stream_bidirectional_packet_stream.so+0x6fe19) #5 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::operator=(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/basic_string.h:680 (libfboss_agent_thrift_packet_stream_bidirectional_packet_stream.so+0x6e7b5) #6 facebook::fboss::BidirectionalPacketStream::clientConnected(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) fboss/agent/thrift_packet_stream/BidirectionalPacketStream.cpp:280 (libfboss_agent_thrift_packet_stream_bidirectional_packet_stream.so+0x77e3a) #7 facebook::fboss::PacketStreamService::connect(std::unique_ptr<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::default_delete<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >) fboss/agent/thrift_packet_stream/PacketStreamService.cpp:59 (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x5aab9) #8 virtual thunk to facebook::fboss::PacketStreamService::connect(std::unique_ptr<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::default_delete<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >) fboss/agent/thrift_packet_stream/PacketStreamService.cpp (libfboss_agent_thrift_packet_stream_packet_stream_server.so+0x5bf5d) #9 apache::thrift::ServiceHandler<facebook::fboss::PacketStream>::semifuture_connect(std::unique_ptr<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::default_delete<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >) buck-out/dev/gen/aab7ed39/fboss/agent/if/packet_stream-cpp2-packet_stream.thrift/gen-cpp2/PacketStream.cpp:35 (libfboss_agent_if_packet_stream-cpp2-services.so+0x91d45) #10 apache::thrift::ServiceHandler<facebook::fboss::PacketStream>::future_connect(std::unique_ptr<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::default_delete<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >) buck-out/dev/gen/aab7ed39/fboss/agent/if/packet_stream-cpp2-packet_stream.thrift/gen-cpp2/PacketStream.cpp:41 (libfboss_agent_if_packet_stream-cpp2-services.so+0x91b8e) #11 apache::thrift::ServiceHandler<facebook::fboss::PacketStream>::async_tm_connect(std::unique_ptr<apache::thrift::HandlerCallback<apache::thrift::ServerStream<facebook::fboss::TPacket> >, std::default_delete<apache::thrift::HandlerCallback<apache::thrift::ServerStream<facebook::fboss::TPacket> > > >, std::unique_ptr<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::default_delete<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >) buck-out/dev/gen/aab7ed39/fboss/agent/if/packet_stream-cpp2-packet_stream.thrift/gen-cpp2/PacketStream.cpp:60 (libfboss_agent_if_packet_stream-cpp2-services.so+0x8cc76) #12 void facebook::fboss::PacketStreamAsyncProcessor::executeRequest_connect<apache::thrift::CompactProtocolReader, apache::thrift::CompactProtocolWriter>(apache::thrift::ServerRequest&&) buck-out/dev/gen/aab7ed39/fboss/agent/if/packet_stream-cpp2-packet_stream.thrift/gen-cpp2/PacketStream.tcc:71 (libfboss_agent_if_packet_stream-cpp2-services.so+0xbdf18) #13 void apache::thrift::detail::ap::execute<facebook::fboss::PacketStreamAsyncProcessor>(facebook::fboss::PacketStreamAsyncProcessor*, apache::thrift::ServerRequest&&, apache::thrift::protocol::PROTOCOL_TYPES, apache::thrift::AsyncProcessorFactory::MethodMetadata const&) thrift/lib/cpp2/GeneratedCodeHelper.h:1235 (libfboss_agent_if_packet_stream-cpp2-services.so+0xba361) #14 facebook::fboss::PacketStreamAsyncProcessor::executeRequest(apache::thrift::ServerRequest&&, apache::thrift::AsyncProcessorFactory::MethodMetadata const&) buck-out/dev/gen/aab7ed39/fboss/agent/if/packet_stream-cpp2-packet_stream.thrift/gen-cpp2/PacketStream.cpp:295 (libfboss_agent_if_packet_stream-cpp2-services.so+0x9081c) #15 apache::thrift::(anonymous namespace)::MultiplexAsyncProcessor::executeRequest(apache::thrift::ServerRequest&&, apache::thrift::AsyncProcessorFactory::MethodMetadata const&) thrift/lib/cpp2/async/MultiplexAsyncProcessor.cpp:332 (libthrift_lib_cpp2_async_multiplex_async_processor.so+0x7f3df) #16 apache::thrift::detail::ap::processViaExecuteRequest(apache::thrift::AsyncProcessor*, std::unique_ptr<apache::thrift::ResponseChannelRequest, apache::thrift::RequestsRegistry::Deleter>, apache::thrift::SerializedCompressedRequest&&, apache::thrift::AsyncProcessorFactory::MethodMetadata const&, apache::thrift::protocol::PROTOCOL_TYPES, apache::thrift::Cpp2RequestContext*, apache::thrift::concurrency::ThreadManager*)::'lambda'(bool)::operator()(bool) thrift/lib/cpp2/GeneratedCodeHelper.h:1023 (libthrift_lib_cpp2_server.so+0x385b76) #17 apache::thrift::detail::ap::processViaExecuteRequest(apache::thrift::AsyncProcessor*, std::unique_ptr<apache::thrift::ResponseChannelRequest, apache::thrift::RequestsRegistry::Deleter>, apache::thrift::SerializedCompressedRequest&&, apache::thrift::AsyncProcessorFactory::MethodMetadata const&, apache::thrift::protocol::PROTOCOL_TYPES, apache::thrift::Cpp2RequestContext*, apache::thrift::concurrency::ThreadManager*)::'lambda'()::operator()() thrift/lib/cpp2/GeneratedCodeHelper.h:1032 (libthrift_lib_cpp2_server.so+0x385ed7) #18 void folly::detail::function::FunctionTraits<void ()>::callBig<apache::thrift::detail::ap::processViaExecuteRequest(apache::thrift::AsyncProcessor*, std::unique_ptr<apache::thrift::ResponseChannelRequest, apache::thrift::RequestsRegistry::Deleter>, apache::thrift::SerializedCompressedRequest&&, apache::thrift::AsyncProcessorFactory::MethodMetadata const&, apache::thrift::protocol::PROTOCOL_TYPES, apache::thrift::Cpp2RequestContext*, apache::thrift::concurrency::ThreadManager*)::'lambda'()>(folly::detail::function::Data&) folly/Function.h:377 (libthrift_lib_cpp2_server.so+0x385d7d) #19 folly::detail::function::FunctionTraits<void ()>::operator()() folly/Function.h:392 (libthrift_lib_cpp_concurrency_thread_manager.so+0x985b3) #20 apache::thrift::concurrency::FunctionRunner::run() thrift/lib/cpp/concurrency/FunctionRunner.h:138 (libthrift_lib_cpp_concurrency_thread_manager.so+0x97f1a) #21 virtual thunk to apache::thrift::concurrency::FunctionRunner::run() thrift/lib/cpp/concurrency/FunctionRunner.h (libthrift_lib_cpp_concurrency_thread_manager.so+0x98131) #22 apache::thrift::concurrency::ThreadManager::Task::run()::'lambda'()::operator()() const thrift/lib/cpp/concurrency/ThreadManager.cpp:178 (libthrift_lib_cpp_concurrency_thread_manager.so+0xb73d9) #23 apache::thrift::concurrency::ThreadManager::Task::run() folly/lang/Exception.h:285 (libthrift_lib_cpp_concurrency_thread_manager.so+0xb6a85) #24 apache::thrift::concurrency::ThreadManager::Impl::Worker::run() thrift/lib/cpp/concurrency/ThreadManager.cpp:653 (libthrift_lib_cpp_concurrency_thread_manager.so+0xb63b8) #25 apache::thrift::concurrency::PthreadThread::threadMain(void*) thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:210 (libthrift_lib_cpp_concurrency_posix_thread_factory.so+0xacd7) Location is heap block of size 4736 at 0x7b84000a2800 allocated by main thread: #0 operator new(unsigned long, std::align_val_t) <null> (MKAModuleTest+0x202b2b) #1 __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/ext/new_allocator.h:125 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xb1046) #2 std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/allocator.h:197 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xb0f09) #3 std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, (__gnu_cxx::_Lock_policy)2> >&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/allocated_ptr.h:97 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xb0b1e) #4 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, double&>(facebook::fboss::BidirectionalPacketStream*&, std::_Sp_alloc_shared_tag<std::allocator<facebook::fboss::BidirectionalPacketStream> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*&&, folly::EventBase*&&, double&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/shared_ptr_base.h:704 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xb082c) #5 std::__shared_ptr<facebook::fboss::BidirectionalPacketStream, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<facebook::fboss::BidirectionalPacketStream>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, double&>(std::_Sp_alloc_shared_tag<std::allocator<facebook::fboss::BidirectionalPacketStream> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*&&, folly::EventBase*&&, double&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/shared_ptr_base.h:1398 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xb073f) #6 std::shared_ptr<facebook::fboss::BidirectionalPacketStream>::shared_ptr<std::allocator<facebook::fboss::BidirectionalPacketStream>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, double&>(std::_Sp_alloc_shared_tag<std::allocator<facebook::fboss::BidirectionalPacketStream> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*&&, folly::EventBase*&&, double&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/shared_ptr.h:409 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xb0617) #7 std::shared_ptr<facebook::fboss::BidirectionalPacketStream> std::allocate_shared<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, double&>(std::allocator<facebook::fboss::BidirectionalPacketStream> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*&&, folly::EventBase*&&, double&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/shared_ptr.h:862 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xb0496) #8 std::shared_ptr<facebook::fboss::BidirectionalPacketStream> std::make_shared<facebook::fboss::BidirectionalPacketStream, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, double&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*&&, folly::EventBase*&&, double&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/shared_ptr.h:878 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xa7095) #9 facebook::fboss::mka::MKAModule::MKAModule(bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, double, unsigned short, unsigned short, unsigned short, bool) fboss/facebook/mka_service/mka_module/MKAModule.cpp:149 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xa5f5a) #10 std::_MakeUniq<facebook::fboss::mka::MKAModule>::__single_object std::make_unique<facebook::fboss::mka::MKAModule, bool, char const (&) [12], double const&, unsigned short&, unsigned short, unsigned short, bool&>(bool&&, char const (&) [12], double const&, unsigned short&, unsigned short&&, unsigned short&&, bool&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/unique_ptr.h:962 (MKAModuleTest+0x1dd436) #11 MKAModuleTest::startMKAModule(bool) fboss/facebook/mka_service/mka_module/tests/MKAModuleTest.cpp:149 (MKAModuleTest+0x1bc348) #12 MKAModuleTest_WarmBootConfigLoad_Test::TestBody() fboss/facebook/mka_service/mka_module/tests/MKAModuleTest.cpp:466 (MKAModuleTest+0x15e1c7) #13 testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2682:50 (libgtest.so.1.11.0+0x50f35) #14 testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2672:6 (libgtest.so.1.11.0+0x50f35) #15 main common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x2161) Mutex M341564816177216360 is already destroyed. Mutex M730281896454009712 is already destroyed. Thread T50 'MkaIOWorker0' (tid=1291160, running) created by main thread at: #0 pthread_create <null> (MKAModuleTest+0x26952d) #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xdf5ee) #2 folly::NamedThreadFactory::newThread(folly::Function<void ()>&&) folly/executors/thread_factory/NamedThreadFactory.h:37 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xaab23) #3 folly::ThreadPoolExecutor::addThreads(unsigned long) folly/executors/ThreadPoolExecutor.cpp:218 (libfolly_executors_thread_pool_executor.so+0x1ed65) #4 folly::ThreadPoolExecutor::ensureActiveThreads() folly/executors/ThreadPoolExecutor.cpp:502 (libfolly_executors_thread_pool_executor.so+0x217ab) #5 folly::IOThreadPoolExecutor::getEventBase() folly/executors/IOThreadPoolExecutor.cpp:173 (libfolly_executors_io_thread_pool_executor.so+0x163dc) #6 facebook::fboss::mka::MKAModule::MKAModule(bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, double, unsigned short, unsigned short, unsigned short, bool) fboss/facebook/mka_service/mka_module/MKAModule.cpp:152 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xa5f05) #7 std::_MakeUniq<facebook::fboss::mka::MKAModule>::__single_object std::make_unique<facebook::fboss::mka::MKAModule, bool, char const (&) [12], double const&, unsigned short&, unsigned short, unsigned short, bool&>(bool&&, char const (&) [12], double const&, unsigned short&, unsigned short&&, unsigned short&&, bool&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/unique_ptr.h:962 (MKAModuleTest+0x1dd436) #8 MKAModuleTest::startMKAModule(bool) fboss/facebook/mka_service/mka_module/tests/MKAModuleTest.cpp:149 (MKAModuleTest+0x1bc348) #9 MKAModuleTest_WarmBootConfigLoad_Test::TestBody() fboss/facebook/mka_service/mka_module/tests/MKAModuleTest.cpp:466 (MKAModuleTest+0x15e1c7) #10 testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2682:50 (libgtest.so.1.11.0+0x50f35) #11 testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2672:6 (libgtest.so.1.11.0+0x50f35) #12 main common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x2161) Thread T44 (tid=1291175, running) created by thread T45 at: #0 pthread_create <null> (MKAModuleTest+0x26952d) #1 apache::thrift::concurrency::PthreadThread::start() thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:110 (libthrift_lib_cpp_concurrency_posix_thread_factory.so+0xbf92) #2 apache::thrift::concurrency::ThreadManager::Impl::addWorker(unsigned long) thrift/lib/cpp/concurrency/ThreadManager.cpp:679 (libthrift_lib_cpp_concurrency_thread_manager.so+0x62028) #3 apache::thrift::concurrency::(anonymous namespace)::SimpleThreadManagerImpl::start() thrift/lib/cpp/concurrency/ThreadManager.cpp:446 (libthrift_lib_cpp_concurrency_thread_manager.so+0x776c4) #4 apache::thrift::ThriftServer::setupThreadManager() thrift/lib/cpp2/server/ThriftServer.cpp:725 (libthrift_lib_cpp2_server.so+0x42a776) #5 apache::thrift::ThriftServer::setup() thrift/lib/cpp2/server/ThriftServer.cpp:471 (libthrift_lib_cpp2_server.so+0x4270ba) #6 apache::thrift::ThriftServer::serve() thrift/lib/cpp2/server/ThriftServer.cpp:1079 (libthrift_lib_cpp2_server.so+0x43be58) #7 apache::thrift::util::ScopedServerThread::Helper::run() thrift/lib/cpp2/util/ScopedServerThread.cpp:186 (libthrift_lib_cpp2_util_scoped_server_thread.so+0x118f9) #8 apache::thrift::concurrency::PthreadThread::threadMain(void*) thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:210 (libthrift_lib_cpp_concurrency_posix_thread_factory.so+0xacd7) ``` In BidirectionalPacketStream::send(), we check clientConnected_. Then use connectedClientId_ to send. In BidirectionalPacketStream::clientConnected(), we immediately set clientConnected_ to true. Then store connectedClientId_. So it's possible for connectedClientId_ to be half baked when send() is using it. Setting clientConnected_ after connectedClientId_ will make sure we exit send() instead of using half baked members. Problem 2: ``` ================== WARNING: ThreadSanitizer: data race (pid=1423249) Write of size 8 at 0x7b840000bfc0 by thread T44: #0 std::__uniq_ptr_impl<apache::thrift::Client<facebook::fboss::PacketStream>, std::default_delete<apache::thrift::Client<facebook::fboss::PacketStream> > >::reset(apache::thrift::Client<facebook::fboss::PacketStream>*) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/unique_ptr.h:180 (libfboss_agent_thrift_packet_stream_packet_stream_client.so+0x70cc9) #1 std::unique_ptr<apache::thrift::Client<facebook::fboss::PacketStream>, std::default_delete<apache::thrift::Client<facebook::fboss::PacketStream> > >::reset(apache::thrift::Client<facebook::fboss::PacketStream>*) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/unique_ptr.h:456 (libfboss_agent_thrift_packet_stream_packet_stream_client.so+0x8677e) #2 facebook::fboss::PacketStreamClient::cancel()::$_5::operator()() const fboss/agent/thrift_packet_stream/PacketStreamClient.cpp:156 (libfboss_agent_thrift_packet_stream_packet_stream_client.so+0x8670d) #3 void folly::detail::function::FunctionTraits<void ()>::callSmall<facebook::fboss::PacketStreamClient::cancel()::$_5>(folly::detail::function::Data&) folly/Function.h:363 (libfboss_agent_thrift_packet_stream_packet_stream_client.so+0x866bd) #4 folly::detail::function::FunctionTraits<void ()>::operator()() folly/Function.h:392 (libfolly_io_async_async_base.so+0x4c4c3) #5 folly::EventBase::runInEventBaseThreadAndWait(folly::Function<void ()>)::$_4::operator()() folly/io/async/EventBase.cpp:681 (libfolly_io_async_async_base.so+0x72f02) #6 void folly::detail::function::FunctionTraits<void ()>::callBig<folly::EventBase::runInEventBaseThreadAndWait(folly::Function<void ()>)::$_4>(folly::detail::function::Data&) folly/Function.h:377 (libfolly_io_async_async_base.so+0x72d5d) #7 folly::detail::function::FunctionTraits<void ()>::operator()() folly/Function.h:392 (libfolly_io_async_async_base.so+0x4c4c3) #8 folly::EventBase::FuncRunner::operator()(folly::Function<void ()>) folly/io/async/EventBase.cpp:124 (libfolly_io_async_async_base.so+0x6ecd9) #9 folly::AtomicNotificationQueueTaskStatus folly::detail::invokeConsumerWithTask<folly::Function<void ()>, folly::EventBase::FuncRunner&, void, void, void>(folly::EventBase::FuncRunner&, folly::Function<void ()>&&, std::shared_ptr<folly::RequestContext>&&) folly/io/async/AtomicNotificationQueue-inl.h:281 (libfolly_io_async_async_base.so+0x6e45a) #10 bool folly::AtomicNotificationQueue<folly::Function<void ()> >::drive<folly::EventBase::FuncRunner&>(folly::EventBase::FuncRunner&) folly/io/async/AtomicNotificationQueue-inl.h:339 (libfolly_io_async_async_base.so+0x6e066) #11 bool folly::EventBaseAtomicNotificationQueue<folly::Function<void ()>, folly::EventBase::FuncRunner>::drive<folly::EventBase::FuncRunner&>(folly::EventBase::FuncRunner&) folly/io/async/EventBaseAtomicNotificationQueue-inl.h:250 (libfolly_io_async_async_base.so+0x6d9cf) #12 folly::EventBaseAtomicNotificationQueue<folly::Function<void ()>, folly::EventBase::FuncRunner>::execute() folly/io/async/EventBaseAtomicNotificationQueue-inl.h:269 (libfolly_io_async_async_base.so+0x4c747) #13 folly::EventBaseAtomicNotificationQueue<folly::Function<void ()>, folly::EventBase::FuncRunner>::handlerReady(unsigned short) folly/io/async/EventBaseAtomicNotificationQueue-inl.h:261 (libfolly_io_async_async_base.so+0x78194) #14 non-virtual thunk to folly::EventBaseAtomicNotificationQueue<folly::Function<void ()>, folly::EventBase::FuncRunner>::handlerReady(unsigned short) folly/io/async/EventBaseAtomicNotificationQueue.h (libfolly_io_async_async_base.so+0x7826c) #15 folly::EventHandler::libeventCallback(int, short, void*) folly/io/async/EventHandler.cpp:159 (libfolly_io_async_async_base.so+0x84b78) #16 event_process_active /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/event.c:390:5 (libevent-1.4.so.2+0x8608) #17 event_base_loop /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/event.c:532:4 (libevent-1.4.so.2+0x8608) #18 folly::EventBase::loopMain(int, bool) folly/io/async/EventBase.cpp:400 (libfolly_io_async_async_base.so+0x49c47) #19 folly::EventBase::loopBody(int, bool) folly/io/async/EventBase.cpp:326 (libfolly_io_async_async_base.so+0x4940f) #20 folly::EventBase::loop() folly/io/async/EventBase.cpp:305 (libfolly_io_async_async_base.so+0x49298) #21 folly::EventBase::loopForever() folly/io/async/EventBase.cpp:545 (libfolly_io_async_async_base.so+0x5c7bf) #22 folly::run(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&) folly/io/async/ScopedEventBaseThread.cpp:40 (libfolly_io_async_scoped_event_base_thread.so+0x8b78) #23 void std::__invoke_impl<void, void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> >(std::__invoke_other, void (*&&)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*&&, folly::EventBase*&&, folly::Baton<true, std::atomic>*&&, folly::Range<char const*>&&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/invoke.h:61 (libfolly_io_async_scoped_event_base_thread.so+0x101e0) #24 std::__invoke_result<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> >::type std::__invoke<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> >(void (*&&)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*&&, folly::EventBase*&&, folly::Baton<true, std::atomic>*&&, folly::Range<char const*>&&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/invoke.h:96 (libfolly_io_async_scoped_event_base_thread.so+0xff14) #25 void std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > >::_M_invoke<0ul, 1ul, 2ul, 3ul, 4ul>(std::_Index_tuple<0ul, 1ul, 2ul, 3ul, 4ul>) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/std_thread.h:253 (libfolly_io_async_scoped_event_base_thread.so+0xfe4d) #26 std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > >::operator()() third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/std_thread.h:260 (libfolly_io_async_scoped_event_base_thread.so+0xfd85) #27 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > > >::_M_run() third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/std_thread.h:211 (libfolly_io_async_scoped_event_base_thread.so+0xf759) #28 execute_native_thread_routine <null> (libstdc++.so.6+0xdf2e4) Previous read of size 8 at 0x7b840000bfc0 by thread T43: #0 std::__uniq_ptr_impl<apache::thrift::Client<facebook::fboss::PacketStream>, std::default_delete<apache::thrift::Client<facebook::fboss::PacketStream> > >::_M_ptr() const third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/unique_ptr.h:173 (libfboss_agent_thrift_packet_stream_packet_stream_client.so+0x6e211) #1 std::unique_ptr<apache::thrift::Client<facebook::fboss::PacketStream>, std::default_delete<apache::thrift::Client<facebook::fboss::PacketStream> > >::get() const third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/unique_ptr.h:422 (libfboss_agent_thrift_packet_stream_packet_stream_client.so+0x6e1b5) #2 std::unique_ptr<apache::thrift::Client<facebook::fboss::PacketStream>, std::default_delete<apache::thrift::Client<facebook::fboss::PacketStream> > >::operator bool() const third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/unique_ptr.h:436 (libfboss_agent_thrift_packet_stream_packet_stream_client.so+0x33fb5) #3 facebook::fboss::PacketStreamClient::isConnectedToServer() fboss/agent/thrift_packet_stream/PacketStreamClient.cpp:29 (libfboss_agent_thrift_packet_stream_packet_stream_client.so+0x33e41) #4 facebook::fboss::BidirectionalPacketStream::close(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) fboss/agent/thrift_packet_stream/BidirectionalPacketStream.cpp:208 (libfboss_agent_thrift_packet_stream_bidirectional_packet_stream.so+0xb5378) #5 facebook::fboss::AsyncThriftPacketTransport::close() fboss/agent/thrift_packet_stream/AsyncThriftPacketTransport.cpp:27 (libfboss_agent_thrift_packet_stream_bidirectional_packet_stream.so+0x6e543) #6 facebook::fboss::mka::MKASession::stop()::$_0::operator()() const fboss/facebook/mka_service/mka_module/MKASession.cpp:149 (libfboss_facebook_mka_service_mka_module_mka_module.so+0x157127) #7 void folly::detail::function::FunctionTraits<void ()>::callSmall<facebook::fboss::mka::MKASession::stop()::$_0>(folly::detail::function::Data&) folly/Function.h:363 (libfboss_facebook_mka_service_mka_module_mka_module.so+0x15704d) #8 folly::detail::function::FunctionTraits<void ()>::operator()() folly/Function.h:392 (libfolly_io_async_async_base.so+0x4c4c3) #9 folly::EventBase::runInEventBaseThreadAndWait(folly::Function<void ()>)::$_4::operator()() folly/io/async/EventBase.cpp:681 (libfolly_io_async_async_base.so+0x72f02) #10 void folly::detail::function::FunctionTraits<void ()>::callBig<folly::EventBase::runInEventBaseThreadAndWait(folly::Function<void ()>)::$_4>(folly::detail::function::Data&) folly/Function.h:377 (libfolly_io_async_async_base.so+0x72d5d) #11 folly::detail::function::FunctionTraits<void ()>::operator()() folly/Function.h:392 (libfolly_io_async_async_base.so+0x4c4c3) #12 folly::EventBase::FuncRunner::operator()(folly::Function<void ()>) folly/io/async/EventBase.cpp:124 (libfolly_io_async_async_base.so+0x6ecd9) #13 folly::AtomicNotificationQueueTaskStatus folly::detail::invokeConsumerWithTask<folly::Function<void ()>, folly::EventBase::FuncRunner&, void, void, void>(folly::EventBase::FuncRunner&, folly::Function<void ()>&&, std::shared_ptr<folly::RequestContext>&&) folly/io/async/AtomicNotificationQueue-inl.h:281 (libfolly_io_async_async_base.so+0x6e45a) #14 bool folly::AtomicNotificationQueue<folly::Function<void ()> >::drive<folly::EventBase::FuncRunner&>(folly::EventBase::FuncRunner&) folly/io/async/AtomicNotificationQueue-inl.h:339 (libfolly_io_async_async_base.so+0x6e066) #15 bool folly::EventBaseAtomicNotificationQueue<folly::Function<void ()>, folly::EventBase::FuncRunner>::drive<folly::EventBase::FuncRunner&>(folly::EventBase::FuncRunner&) folly/io/async/EventBaseAtomicNotificationQueue-inl.h:250 (libfolly_io_async_async_base.so+0x6d9cf) #16 folly::EventBaseAtomicNotificationQueue<folly::Function<void ()>, folly::EventBase::FuncRunner>::execute() folly/io/async/EventBaseAtomicNotificationQueue-inl.h:269 (libfolly_io_async_async_base.so+0x4c747) #17 folly::EventBaseAtomicNotificationQueue<folly::Function<void ()>, folly::EventBase::FuncRunner>::handlerReady(unsigned short) folly/io/async/EventBaseAtomicNotificationQueue-inl.h:261 (libfolly_io_async_async_base.so+0x78194) #18 non-virtual thunk to folly::EventBaseAtomicNotificationQueue<folly::Function<void ()>, folly::EventBase::FuncRunner>::handlerReady(unsigned short) folly/io/async/EventBaseAtomicNotificationQueue.h (libfolly_io_async_async_base.so+0x7826c) #19 folly::EventHandler::libeventCallback(int, short, void*) folly/io/async/EventHandler.cpp:159 (libfolly_io_async_async_base.so+0x84b78) #20 event_process_active /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/event.c:390:5 (libevent-1.4.so.2+0x8608) #21 event_base_loop /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/event.c:532:4 (libevent-1.4.so.2+0x8608) #22 folly::EventBase::loopMain(int, bool) folly/io/async/EventBase.cpp:400 (libfolly_io_async_async_base.so+0x49c47) #23 folly::EventBase::loopBody(int, bool) folly/io/async/EventBase.cpp:326 (libfolly_io_async_async_base.so+0x4940f) #24 folly::EventBase::loop() folly/io/async/EventBase.cpp:305 (libfolly_io_async_async_base.so+0x49298) #25 folly::EventBase::loopForever() folly/io/async/EventBase.cpp:545 (libfolly_io_async_async_base.so+0x5c7bf) #26 folly::IOThreadPoolExecutor::threadRun(std::shared_ptr<folly::ThreadPoolExecutor::Thread>) folly/executors/IOThreadPoolExecutor.cpp:250 (libfolly_executors_io_thread_pool_executor.so+0x15b72) #27 void std::__invoke_impl<void, void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&>(std::__invoke_memfun_deref, void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/invoke.h:74 (libfolly_executors_thread_pool_executor.so+0x56191) #28 std::__invoke_result<void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&>::type std::__invoke<void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&>(void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/invoke.h:96 (libfolly_executors_thread_pool_executor.so+0x55f07) #29 void std::_Bind<void (folly::ThreadPoolExecutor::* (folly::ThreadPoolExecutor*, std::shared_ptr<folly::ThreadPoolExecutor::Thread>))(std::shared_ptr<folly::ThreadPoolExecutor::Thread>)>::__call<void, 0ul, 1ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul>) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/functional:420 (libfolly_executors_thread_pool_executor.so+0x55e6a) #30 void std::_Bind<void (folly::ThreadPoolExecutor::* (folly::ThreadPoolExecutor*, std::shared_ptr<folly::ThreadPoolExecutor::Thread>))(std::shared_ptr<folly::ThreadPoolExecutor::Thread>)>::operator()<void>() third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/functional:503 (libfolly_executors_thread_pool_executor.so+0x55dc6) #31 void folly::detail::function::FunctionTraits<void ()>::callSmall<std::_Bind<void (folly::ThreadPoolExecutor::* (folly::ThreadPoolExecutor*, std::shared_ptr<folly::ThreadPoolExecutor::Thread>))(std::shared_ptr<folly::ThreadPoolExecutor::Thread>)> >(folly::detail::function::Data&) folly/Function.h:363 (libfolly_executors_thread_pool_executor.so+0x55a5d) #32 folly::detail::function::FunctionTraits<void ()>::operator()() folly/Function.h:392 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xa9603) #33 folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()::operator()() folly/executors/thread_factory/NamedThreadFactory.h:40 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xa959c) #34 void std::__invoke_impl<void, folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()>(std::__invoke_other, folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()&&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/invoke.h:61 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xa952d) #35 std::__invoke_result<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()>::type std::__invoke<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()>(folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()&&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/invoke.h:96 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xa947d) #36 void std::thread::_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/std_thread.h:253 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xa9425) #37 std::thread::_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()> >::operator()() third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/std_thread.h:260 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xa93c5) #38 std::thread::_State_impl<std::thread::_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::'lambda'()> > >::_M_run() third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/std_thread.h:211 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xa9159) #39 execute_native_thread_routine <null> (libstdc++.so.6+0xdf2e4) Location is heap block of size 4736 at 0x7b840000b400 allocated by main thread: #0 operator new(unsigned long, std::align_val_t) <null> (MKAModuleTest+0x1c960b) #1 __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/ext/new_allocator.h:125 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xad076) #2 std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/allocator.h:197 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xacf39) #3 std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, (__gnu_cxx::_Lock_policy)2> >&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/allocated_ptr.h:97 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xacb4e) #4 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, double&>(facebook::fboss::BidirectionalPacketStream*&, std::_Sp_alloc_shared_tag<std::allocator<facebook::fboss::BidirectionalPacketStream> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*&&, folly::EventBase*&&, double&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/shared_ptr_base.h:704 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xac85c) #5 std::__shared_ptr<facebook::fboss::BidirectionalPacketStream, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<facebook::fboss::BidirectionalPacketStream>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, double&>(std::_Sp_alloc_shared_tag<std::allocator<facebook::fboss::BidirectionalPacketStream> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*&&, folly::EventBase*&&, double&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/shared_ptr_base.h:1398 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xac76f) #6 std::shared_ptr<facebook::fboss::BidirectionalPacketStream>::shared_ptr<std::allocator<facebook::fboss::BidirectionalPacketStream>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, double&>(std::_Sp_alloc_shared_tag<std::allocator<facebook::fboss::BidirectionalPacketStream> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*&&, folly::EventBase*&&, double&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/shared_ptr.h:409 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xac647) #7 std::shared_ptr<facebook::fboss::BidirectionalPacketStream> std::allocate_shared<facebook::fboss::BidirectionalPacketStream, std::allocator<facebook::fboss::BidirectionalPacketStream>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, double&>(std::allocator<facebook::fboss::BidirectionalPacketStream> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*&&, folly::EventBase*&&, double&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/shared_ptr.h:862 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xac4c6) #8 std::shared_ptr<facebook::fboss::BidirectionalPacketStream> std::make_shared<facebook::fboss::BidirectionalPacketStream, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, double&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*&&, folly::EventBase*&&, double&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/shared_ptr.h:878 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xa34b5) #9 facebook::fboss::mka::MKAModule::MKAModule(bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, double, unsigned short, unsigned short, unsigned short, bool) fboss/facebook/mka_service/mka_module/MKAModule.cpp:149 (libfboss_facebook_mka_service_mka_module_mka_module.so+0xa24c0) #10 std::_MakeUniq<facebook::fboss::mka::MKAModule>::__single_object std::make_unique<facebook::fboss::mka::MKAModule, bool, char const (&) [12], double const&, unsigned short&, unsigned short, unsigned short, bool&>(bool&&, char const (&) [12], double const&, unsigned short&, unsigned short&&, unsigned short&&, bool&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/unique_ptr.h:962 (MKAModuleTest+0x1b7f66) #11 MKAModuleTest::startMKAModule(bool) fboss/facebook/mka_service/mka_module/tests/MKAModuleTest.cpp:129 (MKAModuleTest+0x194ff8) #12 MKAModuleTest::SetUp() fboss/facebook/mka_service/mka_module/tests/MKAModuleTest.cpp:178 (MKAModuleTest+0x14158c) #13 testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2677:48 (libgtest.so.1.11.0+0x50eb8) #14 testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2672:6 (libgtest.so.1.11.0+0x50eb8) #15 main common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x2161) Thread T44 'mka_service' (tid=1424413, running) created by main thread at: #0 pthread_create <null> (MKAModuleTest+0x23000d) #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xdf5ee) #2 folly::ScopedEventBaseThread::ScopedEventBaseThread(folly::EventBase::Options, folly::EventBaseManager*, folly::Range<char const*>) folly/io/async/ScopedEventBaseThread.cpp:68 (libfolly_io_async_scoped_event_base_thread.so+0x86f4) #3 folly::ScopedEventBaseThread::ScopedEventBaseThread(folly::EventBaseManager*, folly::Range<char const*>) folly/io/async/ScopedEventBaseThread.cpp:60 (…

Summary: We have a race between FsdbStreamClient::setServerToConnect() setting serverAddress_ and FsdbStreamClient::timeoutExpired() from reconnect thread checking if it has an address for us to reconnect to. With tests from D37376542: ``` ❯ buck test mode/dev-tsan //fboss/facebook/mka_service/fsdb/tests:MKAFsdbSyncerTest -- --stress-runs 10 ... Summary Pass: 10 Fatal: 10 ``` ``` ================== WARNING: ThreadSanitizer: data race (pid=708956) Read of size 2 at 0x7b4c00001c88 by thread T79: #0 folly::SocketAddress::getPort() const folly/SocketAddress.cpp:413 (libfolly_network_address.so+0x8bfda) #1 facebook::fboss::fsdb::FsdbStreamClient::timeoutExpired() fboss/fsdb/client/FsdbStreamClient.cpp:98 (libfboss_fsdb_client_fsdb_stream_client.so+0x2e858) #2 facebook::fboss::fsdb::FsdbStreamClient::FsdbStreamClient(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>)::$_16::operator()() const fboss/fsdb/client/FsdbStreamClient.cpp:34 (libfboss_fsdb_client_fsdb_stream_client.so+0x328e5) #3 folly::detail::async_timeout_wrapper<facebook::fboss::fsdb::FsdbStreamClient::FsdbStreamClient(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>)::$_16>::timeoutExpired() folly/io/async/AsyncTimeout.h:255 (libfboss_fsdb_client_fsdb_stream_client.so+0x3289c) #4 folly::AsyncTimeout::libeventCallback(int, short, void*) folly/io/async/AsyncTimeout.cpp:174 (libfolly_io_async_async_base.so+0x41e6b) #5 event_process_active /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/event.c:390:5 (libevent-1.4.so.2+0x8608) #6 event_base_loop /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/event.c:532:4 (libevent-1.4.so.2+0x8608) #7 folly::EventBase::loopMain(int, bool) folly/io/async/EventBase.cpp:400 (libfolly_io_async_async_base.so+0x49c47) #8 folly::EventBase::loopBody(int, bool) folly/io/async/EventBase.cpp:326 (libfolly_io_async_async_base.so+0x4940f) #9 folly::EventBase::loop() folly/io/async/EventBase.cpp:305 (libfolly_io_async_async_base.so+0x49298) #10 folly::EventBase::loopForever() folly/io/async/EventBase.cpp:545 (libfolly_io_async_async_base.so+0x5c7bf) #11 folly::run(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&) folly/io/async/ScopedEventBaseThread.cpp:40 (libfolly_io_async_scoped_event_base_thread.so+0x8b78) #12 void std::__invoke_impl<void, void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> >(std::__invoke_other, void (*&&)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*&&, folly::EventBase*&&, folly::Baton<true, std::atomic>*&&, folly::Range<char const*>&&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/invoke.h:61 (libfolly_io_async_scoped_event_base_thread.so+0x101e0) #13 std::__invoke_result<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> >::type std::__invoke<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> >(void (*&&)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*&&, folly::EventBase*&&, folly::Baton<true, std::atomic>*&&, folly::Range<char const*>&&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/invoke.h:96 (libfolly_io_async_scoped_event_base_thread.so+0xff14) #14 void std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > >::_M_invoke<0ul, 1ul, 2ul, 3ul, 4ul>(std::_Index_tuple<0ul, 1ul, 2ul, 3ul, 4ul>) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/std_thread.h:253 (libfolly_io_async_scoped_event_base_thread.so+0xfe4d) #15 std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > >::operator()() third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/std_thread.h:260 (libfolly_io_async_scoped_event_base_thread.so+0xfd85) #16 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > > >::_M_run() third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/std_thread.h:211 (libfolly_io_async_scoped_event_base_thread.so+0xf759) #17 execute_native_thread_routine <null> (libstdc++.so.6+0xdf2e4) Previous write of size 2 at 0x7b4c00001c88 by main thread (mutexes: write M477235840850862816): #0 folly::SocketAddress::SocketAddress(folly::SocketAddress&&) folly/SocketAddress.h:598 (libfboss_fsdb_client_fsdb_stream_client.so+0x4ca07) #1 void std::_Optional_base_impl<folly::SocketAddress, std::_Optional_base<folly::SocketAddress, false, false> >::_M_construct<folly::SocketAddress>(folly::SocketAddress&&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/optional:420 (libfboss_fsdb_client_fsdb_stream_client.so+0x4c70e) #2 std::enable_if<__and_v<std::__not_<std::is_same<std::optional<folly::SocketAddress>, std::remove_cv<std::remove_reference<folly::SocketAddress>::type>::type> >, std::__not_<std::__and_<std::is_scalar<folly::SocketAddress>, std::is_same<folly::SocketAddress, std::decay<folly::SocketAddress>::type> > >, std::is_constructible<folly::SocketAddress, folly::SocketAddress>, std::is_assignable<folly::SocketAddress&, folly::SocketAddress> >, std::optional<folly::SocketAddress>&>::type std::optional<folly::SocketAddress>::operator=<folly::SocketAddress>(folly::SocketAddress&&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/optional:810 (libfboss_fsdb_client_fsdb_stream_client.so+0x4c4a6) #3 facebook::fboss::fsdb::FsdbStreamClient::setServerToConnect(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned short, bool) fboss/fsdb/client/FsdbStreamClient.cpp:92 (libfboss_fsdb_client_fsdb_stream_client.so+0x4c2a7) #4 auto void facebook::fboss::fsdb::FsdbPubSubManager::addSubscriptionImpl<facebook::fboss::fsdb::FsdbStateSubscriber>(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, facebook::fboss::fsdb::FsdbStateSubscriber::FsdbSubUnitUpdateCb, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)::'lambda'(facebook::fboss::fsdb::FsdbStateSubscriber&)::operator()<std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::unique_ptr<facebook::fboss::fsdb::FsdbStreamClient, std::default_delete<facebook::fboss::fsdb::FsdbStreamClient> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::unique_ptr<facebook::fboss::fsdb::FsdbStreamClient, std::default_delete<facebook::fboss::fsdb::FsdbStreamClient> > > > > >(facebook::fboss::fsdb::FsdbStateSubscriber&) const fboss/fsdb/client/FsdbPubSubManager.cpp:263 (libfboss_fsdb_client_fsdb_pub_sub.so+0x155e9b) #5 auto folly::SynchronizedBase<folly::Synchronized<std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::unique_ptr<facebook::fboss::fsdb::FsdbStreamClient, std::default_delete<facebook::fboss::fsdb::FsdbStreamClient> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::unique_ptr<facebook::fboss::fsdb::FsdbStreamClient, std::default_delete<facebook::fboss::fsdb::FsdbStreamClient> > > > >, folly::SharedMutexImpl<false, void, std::atomic, folly::SharedMutexPolicyDefault> >, (folly::detail::SynchronizedMutexLevel)2>::withWLock<void facebook::fboss::fsdb::FsdbPubSubManager::addSubscriptionImpl<facebook::fboss::fsdb::FsdbStateSubscriber>(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, facebook::fboss::fsdb::FsdbStateSubscriber::FsdbSubUnitUpdateCb, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)::'lambda'(facebook::fboss::fsdb::FsdbStateSubscriber&)>(facebook::fboss::fsdb::FsdbStateSubscriber&&) folly/Synchronized.h:305 (libfboss_fsdb_client_fsdb_pub_sub.so+0x155814) #6 void facebook::fboss::fsdb::FsdbPubSubManager::addSubscriptionImpl<facebook::fboss::fsdb::FsdbStateSubscriber>(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, facebook::fboss::fsdb::FsdbStateSubscriber::FsdbSubUnitUpdateCb, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) fboss/fsdb/client/FsdbPubSubManager.cpp:247 (libfboss_fsdb_client_fsdb_pub_sub.so+0x1555ae) #7 facebook::fboss::fsdb::FsdbPubSubManager::addStatePathSubscription(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, std::function<void (facebook::fboss::fsdb::OperState&&)>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) fboss/fsdb/client/FsdbPubSubManager.cpp:197 (libfboss_fsdb_client_fsdb_pub_sub.so+0x15539c) #8 facebook::fboss::mka::TestSubscription<thriftpath::ChildThriftPath<facebook::fboss::mka::MkaState, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::MkaData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > > >::TestSubscription(facebook::fboss::fsdb::FsdbPubSubManager*, thriftpath::ChildThriftPath<facebook::fboss::mka::MkaState, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::MkaData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > >) fboss/facebook/mka_service/fsdb/tests/MKAFsdbTestUtils.h:57 (MKAFsdbSyncerTest+0x132899) #9 facebook::fboss::mka::TestSubscription<thriftpath::ChildThriftPath<facebook::fboss::mka::MkaState, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::MkaData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > > > facebook::fboss::mka::MKAFsdbSyncerTest::subscribe<thriftpath::ChildThriftPath<facebook::fboss::mka::MkaState, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::MkaData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > > >(thriftpath::ChildThriftPath<facebook::fboss::mka::MkaState, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::MkaData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > >) fboss/facebook/mka_service/fsdb/tests/MKAFsdbSyncerTest.cpp:79 (MKAFsdbSyncerTest+0x131b37) #10 facebook::fboss::mka::MKAFsdbSyncerTest_testSessionUpdate_Test::TestBody() fboss/facebook/mka_service/fsdb/tests/MKAFsdbSyncerTest.cpp:102 (MKAFsdbSyncerTest+0x10a03c) #11 testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2682:50 (libgtest.so.1.11.0+0x50f35) #12 testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2672:6 (libgtest.so.1.11.0+0x50f35) #13 main common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x2161) Location is heap block of size 408 at 0x7b4c00001c00 allocated by main thread: #0 operator new(unsigned long) <null> (MKAFsdbSyncerTest+0x14143c) #1 std::_MakeUniq<facebook::fboss::fsdb::FsdbStateSubscriber>::__single_object std::make_unique<facebook::fboss::fsdb::FsdbStateSubscriber, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, folly::EventBase*, folly::EventBase*, std::function<void (facebook::fboss::fsdb::OperState&&)>&, bool&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, folly::EventBase*&&, folly::EventBase*&&, std::function<void (facebook::fboss::fsdb::OperState&&)>&, bool&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/unique_ptr.h:962 (libfboss_fsdb_client_fsdb_pub_sub.so+0x1560f8) #2 auto void facebook::fboss::fsdb::FsdbPubSubManager::addSubscriptionImpl<facebook::fboss::fsdb::FsdbStateSubscriber>(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, facebook::fboss::fsdb::FsdbStateSubscriber::FsdbSubUnitUpdateCb, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)::'lambda'(facebook::fboss::fsdb::FsdbStateSubscriber&)::operator()<std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::unique_ptr<facebook::fboss::fsdb::FsdbStreamClient, std::default_delete<facebook::fboss::fsdb::FsdbStreamClient> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::unique_ptr<facebook::fboss::fsdb::FsdbStreamClient, std::default_delete<facebook::fboss::fsdb::FsdbStreamClient> > > > > >(facebook::fboss::fsdb::FsdbStateSubscriber&) const fboss/fsdb/client/FsdbPubSubManager.cpp:250 (libfboss_fsdb_client_fsdb_pub_sub.so+0x155a21) #3 auto folly::SynchronizedBase<folly::Synchronized<std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::unique_ptr<facebook::fboss::fsdb::FsdbStreamClient, std::default_delete<facebook::fboss::fsdb::FsdbStreamClient> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::unique_ptr<facebook::fboss::fsdb::FsdbStreamClient, std::default_delete<facebook::fboss::fsdb::FsdbStreamClient> > > > >, folly::SharedMutexImpl<false, void, std::atomic, folly::SharedMutexPolicyDefault> >, (folly::detail::SynchronizedMutexLevel)2>::withWLock<void facebook::fboss::fsdb::FsdbPubSubManager::addSubscriptionImpl<facebook::fboss::fsdb::FsdbStateSubscriber>(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, facebook::fboss::fsdb::FsdbStateSubscriber::FsdbSubUnitUpdateCb, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)::'lambda'(facebook::fboss::fsdb::FsdbStateSubscriber&)>(facebook::fboss::fsdb::FsdbStateSubscriber&&) folly/Synchronized.h:305 (libfboss_fsdb_client_fsdb_pub_sub.so+0x155814) #4 void facebook::fboss::fsdb::FsdbPubSubManager::addSubscriptionImpl<facebook::fboss::fsdb::FsdbStateSubscriber>(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, facebook::fboss::fsdb::FsdbStateSubscriber::FsdbSubUnitUpdateCb, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) fboss/fsdb/client/FsdbPubSubManager.cpp:247 (libfboss_fsdb_client_fsdb_pub_sub.so+0x1555ae) #5 facebook::fboss::fsdb::FsdbPubSubManager::addStatePathSubscription(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, std::function<void (facebook::fboss::fsdb::OperState&&)>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) fboss/fsdb/client/FsdbPubSubManager.cpp:197 (libfboss_fsdb_client_fsdb_pub_sub.so+0x15539c) #6 facebook::fboss::mka::TestSubscription<thriftpath::ChildThriftPath<facebook::fboss::mka::MkaState, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::MkaData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > > >::TestSubscription(facebook::fboss::fsdb::FsdbPubSubManager*, thriftpath::ChildThriftPath<facebook::fboss::mka::MkaState, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::MkaData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > >) fboss/facebook/mka_service/fsdb/tests/MKAFsdbTestUtils.h:57 (MKAFsdbSyncerTest+0x132899) #7 facebook::fboss::mka::TestSubscription<thriftpath::ChildThriftPath<facebook::fboss::mka::MkaState, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::MkaData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > > > facebook::fboss::mka::MKAFsdbSyncerTest::subscribe<thriftpath::ChildThriftPath<facebook::fboss::mka::MkaState, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::MkaData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > > >(thriftpath::ChildThriftPath<facebook::fboss::mka::MkaState, facebook::fboss::fsdb::FsdbOperStateRoot, thriftpath::Path<facebook::fboss::fsdb::MkaData, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, thriftpath::Path<facebook::fboss::fsdb::FsdbOperStateRoot, facebook::fboss::fsdb::FsdbOperStateRoot, apache::thrift::type_class::structure, folly::Unit> > >) fboss/facebook/mka_service/fsdb/tests/MKAFsdbSyncerTest.cpp:79 (MKAFsdbSyncerTest+0x131b37) #8 facebook::fboss::mka::MKAFsdbSyncerTest_testSessionUpdate_Test::TestBody() fboss/facebook/mka_service/fsdb/tests/MKAFsdbSyncerTest.cpp:102 (MKAFsdbSyncerTest+0x10a03c) #9 testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2682:50 (libgtest.so.1.11.0+0x50f35) #10 testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2672:6 (libgtest.so.1.11.0+0x50f35) #11 main common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x2161) Mutex M477235840850862816 is already destroyed. Thread T79 'FsdbReconnectTh' (tid=710754, running) created by main thread at: #0 pthread_create <null> (MKAFsdbSyncerTest+0x1a803d) #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xdf5ee) #2 folly::ScopedEventBaseThread::ScopedEventBaseThread(folly::EventBase::Options, folly::EventBaseManager*, folly::Range<char const*>) folly/io/async/ScopedEventBaseThread.cpp:68 (libfolly_io_async_scoped_event_base_thread.so+0x86f4) #3 folly::ScopedEventBaseThread::ScopedEventBaseThread(folly::EventBaseManager*, folly::Range<char const*>) folly/io/async/ScopedEventBaseThread.cpp:60 (libfolly_io_async_scoped_event_base_thread.so+0x83ed) #4 folly::ScopedEventBaseThread::ScopedEventBaseThread(folly::Range<char const*>) folly/io/async/ScopedEventBaseThread.cpp:53 (libfolly_io_async_scoped_event_base_thread.so+0xac91) #5 facebook::fboss::fsdb::FsdbPubSubManager::FsdbPubSubManager(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) fboss/fsdb/client/FsdbPubSubManager.h:139 (libfboss_fsdb_client_fsdb_pub_sub.so+0x124f18) #6 std::_MakeUniq<facebook::fboss::fsdb::FsdbPubSubManager>::__single_object std::make_unique<facebook::fboss::fsdb::FsdbPubSubManager, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/unique_ptr.h:962 (libfboss_fsdb_client_fsdb_syncer.so+0x9b52) #7 facebook::fboss::fsdb::FsdbSyncManager::FsdbSyncManager(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) fboss/fsdb/client/FsdbSyncManager.cpp:12 (libfboss_fsdb_client_fsdb_syncer.so+0x99d4) #8 facebook::fboss::mka::MKAFsdbSyncManager::MKAFsdbSyncManager() fboss/facebook/mka_service/fsdb/MKAFsdbSyncManager.cpp:30 (libfboss_facebook_mka_service_fsdb_fsdb_syncer.so+0x954a5) #9 std::_MakeUniq<facebook::fboss::mka::MKAFsdbSyncManager>::__single_object std::make_unique<facebook::fboss::mka::MKAFsdbSyncManager>() third-party-buck/platform010-compat/build/libgcc/include/c++/trunk/bits/unique_ptr.h:962 (MKAFsdbSyncerTest+0x13d1a9) #10 facebook::fboss::mka::MKAFsdbSyncerTest::SetUp() fboss/facebook/mka_service/fsdb/tests/MKAFsdbSyncerTest.cpp:51 (MKAFsdbSyncerTest+0x108d91) #11 testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2677:48 (libgtest.so.1.11.0+0x50eb8) #12 testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2672:6 (libgtest.so.1.11.0+0x50eb8) #13 main common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x2161) ThreadSanitizer: data race folly/SocketAddress.cpp:413 in folly::SocketAddress::getPort() const ``` Reviewed By: peygar Differential Revision: D38721759 fbshipit-source-id: 43570bd23ff2bf75cb812541ee7d022d1c012697

Summary: Kamet and Makalu don't support I2C_RDWR function that Sandia and Lassen do. For Kamet and Makalu, we need to use I2C_SMBUS ioctl. This code refactors the current logic in I2cDevIo so that we can support both these methods. I added an abstract class I2cDevIoImpl. We'll have two classes that will implement this - 1) I2cRdWrIo -> for I2C_RDWR based access (implemented in this diff. Code refactored from existing I2cDevIo) 2) I2cSmbusIo -> for I2C_SMBUS based access. (implemented in next diff) In this diff, I only implemented class #1. Next diff will implement #2. I2cDevIo will act as the master class and instantiate one of I2cRdWrIo or I2cSmbusIo depending on what the platform supports (added this logic in next diff) Differential Revision: D39213347 fbshipit-source-id: 01927f5503cdc7b44d1538df9fb142080c42dc7e

Summary: Our FsdbStreamClient service loop currently fails TSAN because we've been using separate thread for Thrift and service loop, which is not supported by Thrift team. As discussed in chat, this will require considerable restructuring to fix. So temporarily ignoring TSAN here for tests to pass. ``` WARNING: ThreadSanitizer: data race on vptr (ctor/dtor vs virtual call) (pid=3501845) Write of size 8 at 0x7b3000070280 by thread T209: #0 apache::thrift::transport::TTransportException::~TTransportException() <null> (setup_qsfp_test_bin-millenio-5.5+0x4cf57ab) #1 void std::__exception_ptr::__dest_thunk<apache::thrift::transport::TTransportException>(void*) <null> (setup_qsfp_test_bin-millenio-5.5+0x55f1ab1) #2 std::__exception_ptr::exception_ptr::_M_release() <null> (libstdc++.so.6+0xa193b) #3 apache::thrift::rocket::RequestContextQueue::failAllScheduledWrites(folly::exception_wrapper) <null> (setup_qsfp_test_bin-millenio-5.5+0xae221ad) #4 apache::thrift::rocket::RocketClient::close(folly::exception_wrapper) <null> (setup_qsfp_test_bin-millenio-5.5+0xadfbaa4) #5 apache::thrift::rocket::Parser<apache::thrift::rocket::RocketClient>::readErr(folly::AsyncSocketException const&) <null> (setup_qsfp_test_bin-millenio-5.5+0xae1b5d6) #6 folly::AsyncSocket::failConnect(char const*, folly::AsyncSocketException const&) <null> (setup_qsfp_test_bin-millenio-5.5+0xc2e0035) #7 folly::AsyncSocket::handleConnect() <null> (setup_qsfp_test_bin-millenio-5.5+0xc2ebdf0) #8 folly::AsyncSocket::handleWrite() <null> (setup_qsfp_test_bin-millenio-5.5+0xc2ea9c4) #9 folly::AsyncSocket::ioReady(unsigned short) <null> (setup_qsfp_test_bin-millenio-5.5+0xc2e692f) #10 folly::AsyncSocket::IoHandler::handlerReady(unsigned short) <null> (setup_qsfp_test_bin-millenio-5.5+0xc2f3320) #11 folly::EventHandler::libeventCallback(int, short, void*) <null> (setup_qsfp_test_bin-millenio-5.5+0xc402afc) #12 event_base_loop <null> (setup_qsfp_test_bin-millenio-5.5+0xcf5c1b8) #13 folly::EventBase::loopMain(int, bool) <null> (setup_qsfp_test_bin-millenio-5.5+0xc3f63a7) #14 folly::EventBase::loopForever() <null> (setup_qsfp_test_bin-millenio-5.5+0xc3fa224) #15 folly::run(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&) <null> (setup_qsfp_test_bin-millenio-5.5+0xc361bff) #16 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > > >::_M_run() <null> (setup_qsfp_test_bin-millenio-5.5+0xc36262b) #17 execute_native_thread_routine <null> (libstdc++.so.6+0xdf2e4) Previous read of size 8 at 0x7b3000070280 by thread T58: #0 folly::exceptionStr(std::exception const&) <null> (setup_qsfp_test_bin-millenio-5.5+0xc63fdb3) #1 facebook::fboss::fsdb::FsdbStreamClient::serviceLoopWrapper() (.resume) <null> (setup_qsfp_test_bin-millenio-5.5+0x4c69842) #2 folly::resumeCoroutineWithNewAsyncStackRoot(std::__n4861::coroutine_handle<void>, folly::AsyncStackFrame&) <null> (setup_qsfp_test_bin-millenio-5.5+0xc651151) #3 void folly::detail::function::FunctionTraits<void ()>::callSmall<folly::coro::detail::ViaCoroutinePromiseBase::scheduleContinuation()::'lambda'()>(folly::detail::function::Data&) <null> (setup_qsfp_test_bin-millenio-5.5+0x4c47168) #4 bool folly::AtomicNotificationQueue<folly::Function<void ()> >::drive<folly::EventBase::FuncRunner&>(folly::EventBase::FuncRunner&) <null> (setup_qsfp_test_bin-millenio-5.5+0xc3fcfa2) #5 folly::EventBaseAtomicNotificationQueue<folly::Function<void ()>, folly::EventBase::FuncRunner>::execute() <null> (setup_qsfp_test_bin-millenio-5.5+0xc3f74ac) #6 non-virtual thunk to folly::EventBaseAtomicNotificationQueue<folly::Function<void ()>, folly::EventBase::FuncRunner>::handlerReady(unsigned short) <null> (setup_qsfp_test_bin-millenio-5.5+0xc3ff07d) #7 folly::EventHandler::libeventCallback(int, short, void*) <null> (setup_qsfp_test_bin-millenio-5.5+0xc402afc) #8 event_base_loop <null> (setup_qsfp_test_bin-millenio-5.5+0xcf5c1b8) #9 folly::EventBase::loopMain(int, bool) <null> (setup_qsfp_test_bin-millenio-5.5+0xc3f63a7) #10 folly::EventBase::loopForever() <null> (setup_qsfp_test_bin-millenio-5.5+0xc3fa224) #11 folly::run(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&) <null> (setup_qsfp_test_bin-millenio-5.5+0xc361bff) #12 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > > >::_M_run() <null> (setup_qsfp_test_bin-millenio-5.5+0xc36262b) #13 execute_native_thread_routine <null> (libstdc++.so.6+0xdf2e4) Location is heap block of size 184 at 0x7b3000070200 allocated by thread T209: #0 malloc <null> (setup_qsfp_test_bin-millenio-5.5+0xd39d3c4) #1 __cxa_allocate_exception <null> (libstdc++.so.6+0xa045c) #2 folly::exception_wrapper::exception_wrapper<apache::thrift::transport::TTransportException, apache::thrift::transport::TTransportException, 325l>(apache::thrift::transport::TTransportException&&) <null> (setup_qsfp_test_bin-millenio-5.5+0xac66e06) #3 apache::thrift::rocket::RequestContextQueue::failAllScheduledWrites(folly::exception_wrapper) <null> (setup_qsfp_test_bin-millenio-5.5+0xae22180) #4 apache::thrift::rocket::RocketClient::close(folly::exception_wrapper) <null> (setup_qsfp_test_bin-millenio-5.5+0xadfbaa4) #5 apache::thrift::rocket::Parser<apache::thrift::rocket::RocketClient>::readErr(folly::AsyncSocketException const&) <null> (setup_qsfp_test_bin-millenio-5.5+0xae1b5d6) #6 folly::AsyncSocket::failConnect(char const*, folly::AsyncSocketException const&) <null> (setup_qsfp_test_bin-millenio-5.5+0xc2e0035) #7 folly::AsyncSocket::handleConnect() <null> (setup_qsfp_test_bin-millenio-5.5+0xc2ebdf0) #8 folly::AsyncSocket::handleWrite() <null> (setup_qsfp_test_bin-millenio-5.5+0xc2ea9c4) #9 folly::AsyncSocket::ioReady(unsigned short) <null> (setup_qsfp_test_bin-millenio-5.5+0xc2e692f) #10 folly::AsyncSocket::IoHandler::handlerReady(unsigned short) <null> (setup_qsfp_test_bin-millenio-5.5+0xc2f3320) #11 folly::EventHandler::libeventCallback(int, short, void*) <null> (setup_qsfp_test_bin-millenio-5.5+0xc402afc) #12 event_base_loop <null> (setup_qsfp_test_bin-millenio-5.5+0xcf5c1b8) #13 folly::EventBase::loopMain(int, bool) <null> (setup_qsfp_test_bin-millenio-5.5+0xc3f63a7) #14 folly::EventBase::loopForever() <null> (setup_qsfp_test_bin-millenio-5.5+0xc3fa224) #15 folly::run(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&) <null> (setup_qsfp_test_bin-millenio-5.5+0xc361bff) #16 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > > >::_M_run() <null> (setup_qsfp_test_bin-millenio-5.5+0xc36262b) #17 execute_native_thread_routine <null> (libstdc++.so.6+0xdf2e4) Thread T209 'qsfp_service' (tid=3503104, running) created by thread T56 at: #0 pthread_create <null> (setup_qsfp_test_bin-millenio-5.5+0xd3d5c0d) #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xdf5ee) #2 folly::ScopedEventBaseThread::ScopedEventBaseThread(folly::Range<char const*>) <null> (setup_qsfp_test_bin-millenio-5.5+0xc362370) #3 facebook::fboss::fsdb::FsdbStreamClient::FsdbStreamClient(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, folly::EventBase*, folly::EventBase*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>) <null> (setup_qsfp_test_bin-millenio-5.5+0x4c628b6) #4 facebook::fboss::fsdb::FsdbPublisher<facebook::fboss::fsdb::OperState>::FsdbPublisher(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, folly::EventBase*, folly::EventBase*, bool, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>) <null> (setup_qsfp_test_bin-millenio-5.5+0x4c3b669) #5 std::unique_ptr<facebook::fboss::fsdb::FsdbStatePublisher, std::default_delete<facebook::fboss::fsdb::FsdbStatePublisher> > facebook::fboss::fsdb::FsdbPubSubManager::createPublisherImpl<facebook::fboss::fsdb::FsdbStatePublisher>(std::lock_guard<std::mutex> const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, bool, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, int) const <null> (setup_qsfp_test_bin-millenio-5.5+0x4c3b05d) #6 facebook::fboss::fsdb::FsdbPubSubManager::createStatPathPublisher(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (facebook::fboss::fsdb::FsdbStreamClient::State, facebook::fboss::fsdb::FsdbStreamClient::State)>, int) <null> (setup_qsfp_test_bin-millenio-5.5+0x4c3e585) #7 facebook::fboss::fsdb::FsdbSyncManager::start() <null> (setup_qsfp_test_bin-millenio-5.5+0x4c35212) #8 void folly::detail::function::FunctionTraits<void (folly::futures::detail::CoreBase&, folly::Executor::KeepAlive<folly::Executor>&&, folly::exception_wrapper*)>::callSmall<void folly::futures::detail::Core<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >::setCallback<std::enable_if<!(folly::Future<folly::futures::detail::tryCallableResult<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > >, facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0, std::enable_if<is_invocable_v<facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0, folly::Try<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >&&>, void>::type>::value_type> folly::Future<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >::thenTry<facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0>(facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0&&) &&::'lambda'(folly::Executor::KeepAlive<folly::Executor>&&, folly::Try<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >&&)::ReturnsFuture::value), folly::Future<folly::futures::detail::tryCallableResult<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > >, facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0, std::enable_if<is_invocable_v<facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0, folly::Try<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >&&>, void>::type>::value_type> folly::Future<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >::thenTry<facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0>(facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0&&) &&::'lambda'(folly::Executor::KeepAlive<folly::Executor>&&, folly::Try<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >&&)::Return>::type folly::futures::detail::FutureBase<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >::thenImplementation<folly::Future<folly::futures::detail::tryCallableResult<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > >, facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0, std::enable_if<is_invocable_v<facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0, folly::Try<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >&&>, void>::type>::value_type> folly::Future<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >::thenTry<facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0>(facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0&&) &&::'lambda'(folly::Executor::KeepAlive<folly::Executor>&&, folly::Try<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >&&), folly::futures::detail::tryExecutorCallableResult<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > >, folly::Future<folly::futures::detail::tryCallableResult<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > >, facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0, std::enable_if<is_invocable_v<facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0, folly::Try<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >&&>, void>::type>::value_type> folly::Future<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >::thenTry<facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0>(facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0&&) &&::'lambda'(folly::Executor::KeepAlive<folly::Executor>&&, folly::Try<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >&&), void> >(facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0&&, folly::Future<folly::futures::detail::tryCallableResult<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > >, facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0, std::enable_if<is_invocable_v<facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0, folly::Try<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >&&>, void>::type>::value_type> folly::Future<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >::thenTry<facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0>(facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0&&) &&::'lambda'(folly::Executor::KeepAlive<folly::Executor>&&, folly::Try<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >&&), folly::futures::detail::InlineContinuation)::'lambda'(folly::Executor::KeepAlive<folly::Executor>&&, folly::Try<std::vector<folly::Try<folly::Unit>, std::allocator<folly::Try<folly::Unit> > > >&&)>(facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode)::$_0&&, std::shared_ptr<folly::RequestContext>&&, folly::futures::detail::InlineContinuation)::'lambda'(folly::futures::detail::CoreBase&, folly::Executor::KeepAlive<folly::Executor>&&, folly::exception_wrapper*)>(folly::futures::detail::CoreBase&, folly::Executor::KeepAlive<folly::Executor>&&, folly::exception_wrapper*, folly::detail::function::Data&) <null> (setup_qsfp_test_bin-millenio-5.5+0x3fa2bda) #9 void folly::detail::function::FunctionTraits<void ()>::callSmall<void folly::Executor::KeepAlive<folly::Executor>::add<folly::futures::detail::CoreBase::doCallback(folly::Executor::KeepAlive<folly::Executor>&&, folly::futures::detail::State)::$_0>(folly::futures::detail::CoreBase::doCallback(folly::Executor::KeepAlive<folly::Executor>&&, folly::futures::detail::State)::$_0&&) &&::'lambda'()>(folly::detail::function::Data&) <null> (setup_qsfp_test_bin-millenio-5.5+0xc410445) #10 folly::EventBase::FunctionLoopCallback::runLoopCallback() <null> (setup_qsfp_test_bin-millenio-5.5+0xc3fb315) #11 folly::EventBase::runLoopCallbacks() <null> (setup_qsfp_test_bin-millenio-5.5+0xc3f5590) #12 folly::EventBase::loopMain(int, bool) <null> (setup_qsfp_test_bin-millenio-5.5+0xc3f63b9) #13 folly::EventBase::loopForever() <null> (setup_qsfp_test_bin-millenio-5.5+0xc3fa224) #14 folly::run(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&) <null> (setup_qsfp_test_bin-millenio-5.5+0xc361bff) #15 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> const&), folly::EventBaseManager*, folly::EventBase*, folly::Baton<true, std::atomic>*, folly::Range<char const*> > > >::_M_run() <null> (setup_qsfp_test_bin-millenio-5.5+0xc36262b) #16 execute_native_thread_routine <null> (libstdc++.so.6+0xdf2e4) Thread T58 'FsdbStatsPublis' (tid=3501909, running) created by main thread at: #0 pthread_create <null> (setup_qsfp_test_bin-millenio-5.5+0xd3d5c0d) #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xdf5ee) #2 folly::ScopedEventBaseThread::ScopedEventBaseThread(folly::Range<char const*>) <null> (setup_qsfp_test_bin-millenio-5.5+0xc362370) #3 facebook::fboss::fsdb::FsdbPubSubManager::FsdbPubSubManager(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) <null> (setup_qsfp_test_bin-millenio-5.5+0x4c368b2) #4 facebook::fboss::fsdb::FsdbSyncManager::FsdbSyncManager(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) <null> (setup_qsfp_test_bin-millenio-5.5+0x4c34d3a) #5 facebook::fboss::QsfpFsdbSyncManager::QsfpFsdbSyncManager() <null> (setup_qsfp_test_bin-millenio-5.5+0x4c303d8) #6 facebook::fboss::WedgeManager::WedgeManager(std::unique_ptr<facebook::fboss::TransceiverPlatformApi, std::default_delete<facebook::fboss::TransceiverPlatformApi> >, std::unique_ptr<facebook::fboss::PlatformMapping, std::default_delete<facebook::fboss::PlatformMapping> >, facebook::fboss::PlatformMode) <null> (setup_qsfp_test_bin-millenio-5.5+0x3f9c26f) #7 facebook::fboss::Minipack16QManager::Minipack16QManager() <null> (setup_qsfp_test_bin-millenio-5.5+0x3fff408) #8 facebook::fboss::createFBWedgeManager(std::unique_ptr<facebook::fboss::PlatformProductInfo, std::default_delete<facebook::fboss::PlatformProductInfo> >) <null> (setup_qsfp_test_bin-millenio-5.5+0x3fbfa1c) #9 facebook::fboss::createWedgeManager() <null> (setup_qsfp_test_bin-millenio-5.5+0x4008efe) #10 facebook::fboss::HwQsfpEnsemble::init() <null> (setup_qsfp_test_bin-millenio-5.5+0x3f87775) #11 facebook::fboss::HwTest::SetUp() <null> (setup_qsfp_test_bin-millenio-5.5+0x3f7624e) #12 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) <null> (setup_qsfp_test_bin-millenio-5.5+0xc71b5cf) #13 __libc_start_call_main <null> (libc.so.6+0x2c656) ``` Reviewed By: peygar Differential Revision: D39152941 fbshipit-source-id: ed2351d691340eb5c1ca486fa7066b1a33f1411b

Summary: This is the first diff of the stack that adds a speed change test. The overall idea for the test is - 1. Create a copy of the config, and modify the speeds in the original config in the cold boot iteration. Need to modify the original config and not the copy so that the trunk step of prod->trunk->prod warm boots with the new speeds 2. Verify link sanity 3. Revert back the config at the end of warm boot iteration so that the next test works with the original config. This diff adds helper functions to do #1. Next diff adds #2 and #3. Reviewed By: rajank7 Differential Revision: D40725414 Privacy Context Container: L1125642 fbshipit-source-id: 83e4206db5bd9140ea5194909257ea8f2dec162e

Summary: Fixes the following TSAN data race - ``` Write of size 8 at 0x7b8400009888 by thread T40 (mutexes: write M507916304074681368, read M61215687833552496): #0 facebook::fboss::QsfpModule::markLastDownTime() fboss/qsfp_service/module/QsfpModule.cpp:951 (qsfp_service+0x206ea68) #1 facebook::fboss::TransceiverManager::markLastDownTime(facebook::fboss::TransceiverID) fboss/qsfp_service/TransceiverManager.cpp:1279 (qsfp_service+0x2143c4a) #2 void facebook::fboss::markLastDownTime_impl::operator()<facebook::fboss::ALL_PORTS_DOWN_helper Previous read of size 8 at 0x7b8400009888 by thread T75 (mutexes: read M61215687833552496, write M401237648678623312): #0 facebook::fboss::QsfpModule::shouldRemediateLocked() fboss/qsfp_service/module/QsfpModule.cpp:649 (qsfp_service+0x206dbeb) #1 facebook::fboss::QsfpModule::shouldRemediate()::$_36::operator()() const fboss/qsfp_service/module/QsfpModule.cpp:590 (qsfp_service+0x20722b0) #2 facebook::fboss::QsfpModule::shouldRemediate() fboss/qsfp_service/module/QsfpModule.cpp:596 (qsfp_service+0x20722b0) #3 facebook::fboss::TransceiverManager::triggerRemediateEvents(std::vector<facebook::fboss::TransceiverID, std::allocator<facebook::fboss::TransceiverID> > const&) fboss/qsfp_service/TransceiverManager.cpp:1257 (qsfp_service+0x212d79a) #4 facebook::fboss::TransceiverManager::refreshStateMachines() fboss/qsfp_service/TransceiverManager.cpp:978 (qsfp_service+0x2142a52) ``` Reviewed By: rajank7 Differential Revision: D43183960 Privacy Context Container: L1125642 fbshipit-source-id: 9169d1ea2f03db86cbfa20b9f3e615b0871ea2ec

Facebook: merge upstream FBOSS 1/30

Summary: Fixes the following TSAN error. isExiting_ is accessed by different threads ``` May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: WARNING: ThreadSanitizer: data race (pid=2492898) May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: Read of size 1 at 0x7b6400047670 by thread T105: May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: #0 facebook::fboss::TransceiverManager::updateState(std::unique_ptr<facebook::fboss::TransceiverStateMachineUpdate, std::default_delete<facebook::fboss::TransceiverStateMachineUpdate>>) <null> (qsfp_service+0x3ebb67e) (BuildId: b2acbcaa66db571598102646487722e03e26433d) May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: #1 facebook::fboss::TransceiverManager::updateStateBlockingWithoutWait(facebook::fboss::TransceiverID, facebook::fboss::TransceiverStateMachineEvent) <null> (qsfp_service+0x3ebb0d9) (BuildId: b2acbcaa66db571598102646487722e03e26433d) May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: #2 facebook::fboss::TransceiverManager::triggerProgrammingEvents() <null> (qsfp_service+0x3ebdc71) (BuildId: b2acbcaa66db571598102646487722e03e26433d) May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: #3 facebook::fboss::TransceiverManager::refreshStateMachines() <null> (qsfp_service+0x3ed0f97) (BuildId: b2acbcaa66db571598102646487722e03e26433d) May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: #4 void folly::detail::function::call_<main::$_1, true, false, void>(folly::detail::function::Data&) <null> (qsfp_service+0x2ae3ed1) (BuildId: b2acbcaa66db571598102646487722e03e26433d) May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: #5 folly::FunctionScheduler::runOneFunction(std::unique_lock<std::mutex>&, std::chrono::time_point<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l>>>) <null> (qsfp_service+0x2b99dce) (BuildId: b2acbcaa66db571598102646487722e03e26433d) May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: #6 folly::FunctionScheduler::run() <null> (qsfp_service+0x2b999ce) (BuildId: b2acbcaa66db571598102646487722e03e26433d) May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: #7 std::thread::_State_impl<std::thread::_Invoker<std::tuple<folly::FunctionScheduler::start()::$_2>>>::_M_run() <null> (qsfp_service+0x2b996b3) (BuildId: b2acbcaa66db571598102646487722e03e26433d) May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: #8 execute_native_thread_routine <null> (libstdc++.so.6+0xdf2e4) (BuildId: a70ae4d21aeb1a5dc28aa714c5135b5d4bf393c9) May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: Previous write of size 1 at 0x7b6400047670 by main thread: May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: #0 facebook::fboss::QsfpServiceSignalHandler::signalReceived(int) <null> (qsfp_service+0x2aecacf) (BuildId: b2acbcaa66db571598102646487722e03e26433d) May 24 20:13:06 fboss323865747.snc1.facebook.com qsfp_service_for_testing[2492898]: #1 folly::AsyncSignalHandler::libeventCallback(int, short, void*) <null> (qsfp_service+0x5d857d9) (BuildId: b2acbcaa66db571598102646487722e03e26433d) ``` Reviewed By: rajank7 Differential Revision: D57809004 Privacy Context Container: L1125642 fbshipit-source-id: 92efcd54b676a14badabe32ef86faf039a246fdb

Summary: Given a slotPath, we resolve a PmUnitConfig to explore. If productVersion == 1, use the default PmUnitConfig in `PlatformConfig::pmUnitConfigs` Otherwise, find a matching productVersion PmUnitConfig in `PlatformConfig::versionedPmUnitConfigs` ========= Next Steps: 1. Respin support on different slots. 2. Deprecate slotPathToPmUnitName_ in DataStore. For #1, I think we should forbid re-spin on PmUnits that don't have an IDPROM. Or at least enforce to have an IDPROM in the next versions. Reviewed By: somasun Differential Revision: D60025044 Privacy Context Container: L1125642 fbshipit-source-id: ef01fdaae3162d19ab6a1b0eba5f502f0752c28c

Summary: Test cases from different platform failed with this stack trace: ``` ==12772==ERROR: AddressSanitizer: SEGV on unknown address 0x0000000015b0 (pc 0x00004dfe8a72 bp 0x7ffd4ede0f70 sp 0x7ffd4ede0f50 T0) ==12772==The signal is caused by a READ memory access. SCARINESS: 20 (wild-addr-read) #0 0x4dfe8a72 in facebook::fboss::SwAgentInitializer::stopAgent(bool, bool) (/tmp/sai_setup_bin-11.7.0.0_dnx_odp+0x4dfe8a72) (BuildId: 0479bd9c683c0ce782f9ddd67f7bdff48e35ef9e) #1 0x59304e1a in facebook::fboss::MonoAgentEnsemble::~MonoAgentEnsemble() (/tmp/sai_setup_bin-11.7.0.0_dnx_odp+0x59304e1a) (BuildId: 0479bd9c683c0ce782f9ddd67f7bdff48e35ef9e) #2 0x59303b5d in facebook::fboss::MonoAgentEnsemble::~MonoAgentEnsemble() (/tmp/sai_setup_bin-11.7.0.0_dnx_odp+0x59303b5d) (BuildId: 0479bd9c683c0ce782f9ddd67f7bdff48e35ef9e) #3 0x5930574e in facebook::fboss::createAgentEnsemble(std::function<facebook::fboss::cfg::SwitchConfig (facebook::fboss::AgentEnsemble const&)>, bool, std::function<void (facebook::fboss::cfg::SwitchConfig const&, facebook::fboss::cfg::PlatformConfig&)>, unsigned int, bool) (/tmp/sai_setup_bin-11.7.0.0_dnx_odp+0x5930574e) (BuildId: 0479bd9c683c0ce782f9ddd67f7bdff48e35ef9e) #4 0x59293da3 in facebook::fboss::AgentHwTest::SetUp() (/tmp/sai_setup_bin-11.7.0.0_dnx_odp+0x59293da3) (BuildId: 0479bd9c683c0ce782f9ddd67f7bdff48e35ef9e) #5 0x171a104d in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/tmp/sai_setup_bin-11.7.0.0_dnx_odp+0x171a104d) (BuildId: 0479bd9c683c0ce782f9ddd67f7bdff48e35ef9e) ``` J3: https://www.internalfb.com/sandcastle/workflow/1301540292316074160/artifact/actionlog.1301540292424123608.stderr.1?selectedLines=922-931-1-3 TH4: https://www.internalfb.com/phabricator/paste/view/P1730442738 This can happens once exception in [createSwitch](https://www.internalfb.com/code/fbsource/[812e06694e40e840e99ed1c57e2816a2c90cbba9]/fbcode/fboss/agent/test/AgentEnsemble.cpp?lines=70), then ASAN/stack unwinding put memory error in the log and skipped the true issue (null pointer). Add pointer check here so we can log the error in createSwitch Reviewed By: vasantspatil Differential Revision: D69540697 Privacy Context Container: L1125642 fbshipit-source-id: 240e813b78f37a235e54a47bf4515cc54fdfeb68

Description: 1) Remove the following configuration from platform_manager.json: `/run/devmap/sensors/SMB_CPLD`: `/SMB_SLOT@0/[SMB_CPLD]` Motivation: 1) The 0xB8 register in SMB_CPLD has no effect because SP4 lacks a TEMP_CLK signal. So the max temperature defined in register 0xB8 cannot be detected. 2) Based on facebook#1, `temp1_input` in the SMB_CPLD driver has been removed, so this config needs to be updated accordingly. 3) This sensor was already removed from `sensor_service.json` earlier, so no changes are needed for the sensor_service config. Test plan: 1) Ran `platform_manager` with the updated config. 2) Confirmed that SMB_CPLD no longer exists in `/run/devmap/sensors/` by checking the directory

jaymzh pushed a commit to jaymzh/fboss that referenced this issue Jan 13, 2016

Merge pull request facebook#1 from sonoble/getdeps_update

8fd379d

Updated getdeps.sh to checkout specific versions of OpenNSL and iproute2

oridb closed this as completed Jan 15, 2016

nbellari mentioned this issue Oct 5, 2016

fboss build fails with "folly/folly-config.h" no such file or directory.. #45

Closed

arajeev-ARISTA pushed a commit to arajeev-ARISTA/fboss that referenced this issue Sep 26, 2023

Merge pull request facebook#1 from adamcalabrigo/fb_merge_1_30_2023

7b35f6a

Facebook: merge upstream FBOSS 1/30

kimdo8736 mentioned this issue Feb 19, 2025

janga: config: added patch files for rack fan service #337

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Exception in add route through fboss_route.py #1

Exception in add route through fboss_route.py #1

andrianyablonskyy commented Apr 7, 2015

oridb commented Apr 7, 2015

ghost commented Aug 4, 2015

Exception in add route through fboss_route.py #1

Exception in add route through fboss_route.py #1

Comments

andrianyablonskyy commented Apr 7, 2015

=================================================

=================================================

oridb commented Apr 7, 2015

ghost commented Aug 4, 2015