Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CFSTORE Bugfix for realloc() moving KV area and cfstore_file_t data structures not being updated correctly #2624

Merged
merged 2 commits into from
Sep 10, 2016

Conversation

simonqhughes
Copy link
Contributor

This commit contains CFSTORE fixes for the following related issues:

With respect to issues 17 and 23:

  • A code defect existed for correctly updating cfstore_file_t data structures under the following conditions:
    -- the KV memory area contained some KV's.
    -- cfstore calls realloc() to increase the size of the KV area in memory because:
    * A new KV was being added to the KV area, or
    * The size of a pre-existing KV was being increased.
    -- The returned address from realloc() has changed from before the call (i.e. the location in memory of the KV area has changed) e.g. the presence of heap memory objects directly above the KV memory area in the memory address space causes realloc() to move the KV area so the newly increased area can be accommodated at contiguous addresses.
    -- In this scenario, the cfstore_file_t (structures for open files) head pointers do not get correctly updated.
    -- The defect was fixed by correctly updating the cfstore_file_t:: head pointer.
    -- A new add_del test case was added to the scenario where a new KV is being added to the KV area.
    -- A new create test case was added to the scenario where the size of a pre-existing KV is being increased in size.
  • A code defect for supplying a NULL handle as the previous argument to the Find() method (issue 24).
    -- Supply a null handle is valid, but it was being used to check for a valid hkey, which was incorrect.
    -- A new test case was added to check the case of supplying a NULL previous argument works correctly.
  • A code defect for a memory leak under the following conditions (issue 25):
    -- When realloc() fails to perform a requested change to the size of the KV area, the error handling sometimes incorrectly sets cfstore_context_t::area_0_head to NULL.
    -- Cfstore returns a suitable error to the client. If memory had previously been held at area_0_head, realloc(area_0_head, size) returning NULL means the memory at area_0_head is still retained.
    -- On receiving the error code, the client cleans up including a call to Uninitialize(). This should free the retained but as area_0_head == NULL this is not possible. Hence a memory leak occurred.
    -- This was fixed by not setting area_0_head = NULL on the realloc() failure.
    -- A create test case was modified to detect the leaking of memory in this way.

Notifications to interested parties:
@c1728p9 @jenia81 @mottigondabi

- issue 17: Heap corruption.
- issue 23: Handles invalidated when realloc called.
- issue 24: cfstore_find returns error when "previous" parameter is NULL.
- issue 25: Memory leak when out of memory.

With respect to issues 17 and 23:
- A code defect existed for correctly updating cfstore_file_t data structures
  under the following conditions:
  -- the KV memory area contained some KV's.
  -- cfstore calls realloc() to increase the size of the KV area in
     memory because:
	  * A new KV was being added to the KV area, or
	  * the size of a pre-existing KV was being increased.
  -- The returned address from realloc() has changed from before the
     call (i.e. the location in memory of the KV area has changed)
	 e.g. the presence of heap memory objects directly above the KV memory
	 area in the memory address space causes realloc() to move the KV area
	 so the newly increased area can be accommodated at contiguous addresses.
  -- In this scenario, the cfstore_file_t (structures for open files) head pointers
     do not get correctly updated.
  -- The defect was fixed by correctly updating the cfstore_file_t:: head pointer.
  -- A new add_del test case was added to the scenario where a new KV is being added
     to the KV area.
  -- A new create test case was added to the scenario where the size of a
     pre-existing KV is being increased in size.

- A code defect for suppling a NULL handle as the previous argument to the Find() method
  (issue 24).
	-- Supply a null handle is valid, but it was being used to check for a valid hkey,
	   which was incorrect.
	-- A new test case was added to check the case of supplying a NULL previous argument
	   works correctly.

- A code defect for a memory leak under the following conditions (issue 25):
  -- When realloc() fails to perform a requested change to the size of the KV area, the
     error handling sometimes incorrectly sets cfstore_context_t::area_0_head to NULL.
	 Cfstore returns a suitable error to the client. If memory had previously been held
	 at area_0_head, realloc(area_0_head, size) returning NULL means the memory
	 at area_0_head is still retained.
  -- On receiving the error code, the client cleans up including a call to Uninitialize().
     This should free the retained but as area_0_head == NULL this is not possible. Hence
	 a memory leak occurred.
  -- This was fixed by not setting area_0_head = NULL on the realloc() failure.
  -- A create test case was modified to detect the leaking of memory in this way.
…ly removed to work around CFSTORE issue 17/23 (realloc()).
@@ -277,11 +279,104 @@ control_t cfstore_add_del_test_04(const size_t call_count)
return CaseNext;
}

/** @brief Delete and attribute after an internal realloc of the cfstore memory area
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

possible typo - did you mean to write an rather than and?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its a typo. Thanks. I'll fix in a subsequent commit if that's OK.

@c1728p9
Copy link
Contributor

c1728p9 commented Sep 5, 2016

I skimmed over the patch and it looks good to me. Thanks for the fixes and tests to catch future regressions.

@sg- sg- added the needs: CI label Sep 9, 2016
@sg-
Copy link
Contributor

sg- commented Sep 9, 2016

/morph test

@mbed-bot
Copy link

Result: SUCCESS

Your command has finished executing! Here's what you wrote!

/morph test

Output

mbed Build Number: 816

All builds and test passed!

@sg- sg- merged commit 5b90bf7 into ARMmbed:master Sep 10, 2016
theotherjimmy added a commit that referenced this pull request Sep 19, 2016
Release mbed-os-5.1.4

Changes:

New Targets:
2504: [Disco_F769NI] adding new target [#2504]
2654: DELTA_DFBM_NQ620 platform porting [#2654]
2615: [MTM_MTCONNECT04S] Added support for MTM_MTCONNECT04S [#2615]
2548: Nucleof303ze [#2548]

Fixes:

2678: Fixing NCS36510 compile on Linux #2678
2657: [MAX326xx] Removed echoing of characters and carriage return. #2657
2651: Use lp_timer to count time in the deepsleep tests #2651
2645: NUCLEO_F446ZE - Enable mbed5 release version #2645
2643: Fix thread self termination #2643
2634: Updated USBHost for library changes #2634
2633: Updated USBDevice to use Callback #2633
2630: Test names not dependent on disk location of root #2630
2624: CFSTORE Bugfix for realloc() moving KV area and cfstore_file_t data structures not being updated correctly #2624
2623: DISCO_L476VG - Add Serial Flow Control pins + add SERIAL_FC macro #2623
2617: STM32F2xx - Enable Serial Flow Control #2617
2613: Correctly providing directories to build_apis #2613
2607: Fix uvisor memory tracing #2607
2604: Tools - Fix fill section size variation #2604
2601: Adding ON Semiconductor copyright notice to source and header files. #2601
2597: [HAL] Fixed "intrinsic is deprecated" warnings #2597
2596: [HAL] Improve memory tracer #2596
2594: Fix TCPServer constructor #2594
2593: Add app config command line switch for test and make #2593
2589: [NUC472] Fix heap configuration error with armcc #2589
2588: Timing tests drift refactor #2588
2587: add PTEx pins as option for SPI on Hexiwear - for SD Card Interface #2587
2584: Set size of callback irq array to IrqCnt #2584
2583: github issue and PR templates #2583
2582: [GCC_CR] fix runtime hang for baremetal build #2582
2580: lwip - Add check for previously-bound socket #2580
2579: lwip - Fix handling of max sockets in socket_accept #2579
2578: Fix double free in NanostackInterface #2578
2576: Add smoke test that builds example programs with mbed-cli #2576
2575: tools-config! -  Allow an empty or mal-formed config to be passed to the config system #2575
2562: Fix GCC lazy init race condition and add test #2562
2559: [utest]: Allow the linker to remove any part of utest if not used #2559
2545: Added define guards for SEQUENTIAL_FLASH_JOURNAL_MAX_LOGGED_BLOBS so  #2545
2538: STM32F4xx - Add support of ADC internal channels (Temp, VRef, VBat) #2538
2521: [NUCLEO_F207ZG] Add MBED5 capability #2521
2514: Updated FlexCan and SAI SDK drivers #2514
2487: Runtime dynamic memory tracing #2487
2442: Malloc heap info #2442
2419: [STM32F1] Add asynchronous serial #2419
2393: [tools] Prevent trace-backs from incomplete args #2393
2245: Refactor export subsystem #2245
2130: stm32 : reduce number of device.h files #2130
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants