feat: Add support for dynamicaly reconfig rsync-timeout-ms and throttle-bytes-per-second #2610
Conversation
|
Testing: Platform： Testing Action: Result: No exceptions or errors were found. Attachments： 1: Network IO Trafiic of Master Machine(Captured in the middle of the test):
2: The Log of Slave Node(Captured in the middle of the test):
|
include/pika_conf.h
Outdated
| @@ -414,6 +414,10 @@ class PikaConf : public pstd::BaseConf { | |||
| std::shared_lock l(rwlock_); | |||
| return max_rsync_parallel_num_; | |||
| } | |||
| int64_t rsync_timeout_ms(){ | |||
There was a problem hiding this comment.
成员函数用驼峰方式命名吧，{跟入参的）加个空格吧，跟其他的保持一致。
include/throttle.h
Outdated
| int64_t GetWaitTimeoutMs(){ | ||
| return wait_timout_ms_.load(); | ||
| } | ||
| void ResetThrottleThroughputBytes(size_t new_throughput_bytes_per_s){ |
There was a problem hiding this comment.
格式问题，{之前的空格，还有函数体部分的缩进。
include/throttle.h
Outdated
| size_t check_cycle_ = 10; | ||
| pstd::Mutex keys_mutex_; | ||
| size_t caculate_check_time_us_(int64_t current_time_us, int64_t check_cycle) { | ||
| size_t base_aligning_time_us = 1000 * 1000 / check_cycle; | ||
| return current_time_us / base_aligning_time_us * base_aligning_time_us; | ||
| } | ||
| // Rsync timeout value used by WaitObject,defined by user, It's NOT protected by keys_mutex_ | ||
| std::atomic<int64_t> wait_timout_ms_{1000}; |
There was a problem hiding this comment.
这个等待超时的变量我理解定义在throttle里不太好。不行你直接在调用的地方实时获取pika_conf里的值。
src/rsync_client.cc
Outdated
| @@ -360,6 +360,7 @@ Status RsyncClient::PullRemoteMeta(std::string* snapshot_uuid, std::set<std::str | |||
|
|
|||
| if (resp.get() == nullptr || resp->code() != RsyncService::kOk) { | |||
| s = Status::IOError("kRsyncMeta request failed! db is not exist or doing bgsave"); | |||
| LOG(INFO) << s.ToString() << ", retries:" << retries; | |||
src/throttle.cc
Outdated
| @@ -19,12 +19,12 @@ Throttle::Throttle(size_t throttle_throughput_bytes, size_t check_cycle) | |||
| Throttle::~Throttle() {} | |||
|
|
|||
| size_t Throttle::ThrottledByThroughput(size_t bytes) { | |||
| std::unique_lock lock(keys_mutex_); | |||
| size_t available_size = bytes; | |||
| size_t now = pstd::NowMicros(); | |||
| size_t limit_throughput_bytes_s = std::max(static_cast<uint64_t>(throttle_throughput_bytes_), | |||
There was a problem hiding this comment.
这里的FLAGS_raft_minimal_throttle_threshold_mb去掉吧，应该是之前抄braft的代码忘了改。
include/throttle.h
Outdated
| if(last_rsync_config_updated_time_ms_.load() + 1000 < pstd::NowMilliSeconds()){ | ||
| //maximum update frequency of rsync config:1 time per sec | ||
| { | ||
| std::lock_guard guard(keys_mutex_); |
There was a problem hiding this comment.
这个地方用了原子变量，就不需要再用互斥锁了吧？
1 Add config item rsync-timeout-ms
2 Add support for dynamicaly reconfig rsync-timeout-ms and throttle-bytes-per-second
3 Add CI Test for it
fix #2263