Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Handle special case in trim_blocklog_front #10370

Merged
merged 9 commits into from
Sep 2, 2021

Conversation

nickjjzhao
Copy link
Contributor

@nickjjzhao nickjjzhao commented May 17, 2021

Change Description

EPE-895 - GitHub Issue]( #8948)

This line uint64_t& pos_content = *(uint64_t*)(buf + buffer_index); in method trim_blocklog_front() can cause bad memory access exception when there are less than 8 bytes right after address buf + buffer_index in the buffer, which is defined as auto buffer = make_unique<char[]>(detail::reverse_iterator::_buf_len);

Change Type

Select ONE:

  • Documentation
  • Stability bug fix
  • Other
  • Other - special case

Testing Changes

Select ANY that apply:

  • New Tests
  • Existing Tests
  • Test Framework
  • CI System
  • Other

Consensus Changes

  • Consensus Changes

API Changes

  • API Changes

Documentation Additions

  • Documentation Additions

uint64_t complete_pos_content;
bool re_write = false;
uint64_t re_write_offset;

// walk this memory section to adjust block position to match the adjusted location
// of the block start and store in the new index file
while(original_pos >= start_of_blk_buffer_pos) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The loop can be simplified to work within block boundaries only, for those doesn't fit into block boundaries, it should be re-read from the outer for-loop.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplified the while loop. Will make another update if it's not the way you expected. Thanks!

index.write(pos_content);
original_pos = start_of_this_block - pos_size;
if (buffer_index + pos_size <= detail::reverse_iterator::_buf_len){
uint64_t& pos_content = *(uint64_t*)(buf + buffer_index);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use memcpy() instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since reference var pos_content is defined for both reading and writing, I replaced this line with two methods read_buffer and write_buffer.

@@ -81,6 +81,9 @@ namespace eosio { namespace chain {

static bool trim_blocklog_front(const fc::path& block_dir, const fc::path& temp_dir, uint32_t truncate_at_block);

// used only for unit test to adjust the buffer length
static void set_buff_len(uint64_t len);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be in the public interface. Just implement it in block_log.cpp as a regular function and forward declare it inunittests/restart_chain_tests.cpp.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

}

BOOST_CHECK( block_log::trim_blocklog_front(temp1.path, temp2.path, truncate_at_block) == true);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you follow the example in

block_log new_log({ .log_dir = temp1.path});
// double check if the version has been set to the desired version
BOOST_CHECK(old_log.version() == version);
BOOST_CHECK(new_log.first_block_num() == 10);
BOOST_CHECK(new_log.head()->block_num() == old_log.head()->block_num());
int num_blocks_trimmed = 10 - 1;
BOOST_CHECK(fc::file_size(temp1.path / "blocks.index") == old_index_size - sizeof(uint64_t) * num_blocks_trimmed);
to verify the trimed block log is correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified.

@@ -1185,3 +1211,8 @@ namespace eosio { namespace chain {
}

} } /// eosio::chain

// used only for unit test to adjust the buffer length
void set_buff_len(uint64_t len){
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of the function is too generic without any namespace. Consider to use block_log_set_buff_len.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@nickjjzhao nickjjzhao merged commit f915db5 into release/2.0.x Sep 2, 2021
@nickjjzhao nickjjzhao deleted the jjz-epe895-trim-blocklog branch September 2, 2021 18:13
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants