-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CFSTORE Bugfix for realloc() moving KV area and cfstore_file_t data structures not being updated correctly #2624
Conversation
- issue 17: Heap corruption. - issue 23: Handles invalidated when realloc called. - issue 24: cfstore_find returns error when "previous" parameter is NULL. - issue 25: Memory leak when out of memory. With respect to issues 17 and 23: - A code defect existed for correctly updating cfstore_file_t data structures under the following conditions: -- the KV memory area contained some KV's. -- cfstore calls realloc() to increase the size of the KV area in memory because: * A new KV was being added to the KV area, or * the size of a pre-existing KV was being increased. -- The returned address from realloc() has changed from before the call (i.e. the location in memory of the KV area has changed) e.g. the presence of heap memory objects directly above the KV memory area in the memory address space causes realloc() to move the KV area so the newly increased area can be accommodated at contiguous addresses. -- In this scenario, the cfstore_file_t (structures for open files) head pointers do not get correctly updated. -- The defect was fixed by correctly updating the cfstore_file_t:: head pointer. -- A new add_del test case was added to the scenario where a new KV is being added to the KV area. -- A new create test case was added to the scenario where the size of a pre-existing KV is being increased in size. - A code defect for suppling a NULL handle as the previous argument to the Find() method (issue 24). -- Supply a null handle is valid, but it was being used to check for a valid hkey, which was incorrect. -- A new test case was added to check the case of supplying a NULL previous argument works correctly. - A code defect for a memory leak under the following conditions (issue 25): -- When realloc() fails to perform a requested change to the size of the KV area, the error handling sometimes incorrectly sets cfstore_context_t::area_0_head to NULL. Cfstore returns a suitable error to the client. If memory had previously been held at area_0_head, realloc(area_0_head, size) returning NULL means the memory at area_0_head is still retained. -- On receiving the error code, the client cleans up including a call to Uninitialize(). This should free the retained but as area_0_head == NULL this is not possible. Hence a memory leak occurred. -- This was fixed by not setting area_0_head = NULL on the realloc() failure. -- A create test case was modified to detect the leaking of memory in this way.
…ly removed to work around CFSTORE issue 17/23 (realloc()).
@@ -277,11 +279,104 @@ control_t cfstore_add_del_test_04(const size_t call_count) | |||
return CaseNext; | |||
} | |||
|
|||
/** @brief Delete and attribute after an internal realloc of the cfstore memory area |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
possible typo - did you mean to write an
rather than and
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Its a typo. Thanks. I'll fix in a subsequent commit if that's OK.
I skimmed over the patch and it looks good to me. Thanks for the fixes and tests to catch future regressions. |
/morph test |
Result: SUCCESSYour command has finished executing! Here's what you wrote!
Outputmbed Build Number: 816 All builds and test passed! |
Release mbed-os-5.1.4 Changes: New Targets: 2504: [Disco_F769NI] adding new target [#2504] 2654: DELTA_DFBM_NQ620 platform porting [#2654] 2615: [MTM_MTCONNECT04S] Added support for MTM_MTCONNECT04S [#2615] 2548: Nucleof303ze [#2548] Fixes: 2678: Fixing NCS36510 compile on Linux #2678 2657: [MAX326xx] Removed echoing of characters and carriage return. #2657 2651: Use lp_timer to count time in the deepsleep tests #2651 2645: NUCLEO_F446ZE - Enable mbed5 release version #2645 2643: Fix thread self termination #2643 2634: Updated USBHost for library changes #2634 2633: Updated USBDevice to use Callback #2633 2630: Test names not dependent on disk location of root #2630 2624: CFSTORE Bugfix for realloc() moving KV area and cfstore_file_t data structures not being updated correctly #2624 2623: DISCO_L476VG - Add Serial Flow Control pins + add SERIAL_FC macro #2623 2617: STM32F2xx - Enable Serial Flow Control #2617 2613: Correctly providing directories to build_apis #2613 2607: Fix uvisor memory tracing #2607 2604: Tools - Fix fill section size variation #2604 2601: Adding ON Semiconductor copyright notice to source and header files. #2601 2597: [HAL] Fixed "intrinsic is deprecated" warnings #2597 2596: [HAL] Improve memory tracer #2596 2594: Fix TCPServer constructor #2594 2593: Add app config command line switch for test and make #2593 2589: [NUC472] Fix heap configuration error with armcc #2589 2588: Timing tests drift refactor #2588 2587: add PTEx pins as option for SPI on Hexiwear - for SD Card Interface #2587 2584: Set size of callback irq array to IrqCnt #2584 2583: github issue and PR templates #2583 2582: [GCC_CR] fix runtime hang for baremetal build #2582 2580: lwip - Add check for previously-bound socket #2580 2579: lwip - Fix handling of max sockets in socket_accept #2579 2578: Fix double free in NanostackInterface #2578 2576: Add smoke test that builds example programs with mbed-cli #2576 2575: tools-config! - Allow an empty or mal-formed config to be passed to the config system #2575 2562: Fix GCC lazy init race condition and add test #2562 2559: [utest]: Allow the linker to remove any part of utest if not used #2559 2545: Added define guards for SEQUENTIAL_FLASH_JOURNAL_MAX_LOGGED_BLOBS so #2545 2538: STM32F4xx - Add support of ADC internal channels (Temp, VRef, VBat) #2538 2521: [NUCLEO_F207ZG] Add MBED5 capability #2521 2514: Updated FlexCan and SAI SDK drivers #2514 2487: Runtime dynamic memory tracing #2487 2442: Malloc heap info #2442 2419: [STM32F1] Add asynchronous serial #2419 2393: [tools] Prevent trace-backs from incomplete args #2393 2245: Refactor export subsystem #2245 2130: stm32 : reduce number of device.h files #2130
This commit contains CFSTORE fixes for the following related issues:
With respect to issues 17 and 23:
-- the KV memory area contained some KV's.
-- cfstore calls realloc() to increase the size of the KV area in memory because:
* A new KV was being added to the KV area, or
* The size of a pre-existing KV was being increased.
-- The returned address from realloc() has changed from before the call (i.e. the location in memory of the KV area has changed) e.g. the presence of heap memory objects directly above the KV memory area in the memory address space causes realloc() to move the KV area so the newly increased area can be accommodated at contiguous addresses.
-- In this scenario, the cfstore_file_t (structures for open files) head pointers do not get correctly updated.
-- The defect was fixed by correctly updating the cfstore_file_t:: head pointer.
-- A new add_del test case was added to the scenario where a new KV is being added to the KV area.
-- A new create test case was added to the scenario where the size of a pre-existing KV is being increased in size.
-- Supply a null handle is valid, but it was being used to check for a valid hkey, which was incorrect.
-- A new test case was added to check the case of supplying a NULL previous argument works correctly.
-- When realloc() fails to perform a requested change to the size of the KV area, the error handling sometimes incorrectly sets cfstore_context_t::area_0_head to NULL.
-- Cfstore returns a suitable error to the client. If memory had previously been held at area_0_head, realloc(area_0_head, size) returning NULL means the memory at area_0_head is still retained.
-- On receiving the error code, the client cleans up including a call to Uninitialize(). This should free the retained but as area_0_head == NULL this is not possible. Hence a memory leak occurred.
-- This was fixed by not setting area_0_head = NULL on the realloc() failure.
-- A create test case was modified to detect the leaking of memory in this way.
Notifications to interested parties:
@c1728p9 @jenia81 @mottigondabi