fix(caching): coalesce concurrent requests to wrap function

[douglascayers]

Please check if the PR fulfills these requirements

• Followed the Contributing guidelines.
• Tests for the changes have been added (for bug fixes/features) with 100% code coverage.
• Docs have been added / updated (for bug fixes / features)

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

Fixes #417

In the v4.x version, the caching.wrap() function would coalesce multiple concurrent requests to minimize how many times the generator function was invoked. This was a helpful optimization to reduce load on downstream systems like databases and APIs.

This pull request adds that cache miss relief logic back by using the promise-coalesce module, which is said logic in a generic, reusable function.

For tests, the #417 issue test demonstrates the undesired behavior. The updated wrap() tests demonstrate the desired behavior by coalescing requests.

Thanks

[jaredwray]

Hi @douglascayers - thanks for putting this together. A couple of questions around this:

• With this change is there anything breaking or should everything work as it with this added functionality?
• Since you added this module, will you be able to support and maintain it? If not, I would prefer adding the code directly into this project.
• Can you update the readme with an example of how this works?

[douglascayers]

@jaredwray Thanks for the follow up. I've rebased this PR and have answers to your questions below. Thanks

• With this change is there anything breaking or should everything work as it with this added functionality?

This is a drop-in patch update. No breaking changes. Users don't have to make any changes to benefit from this change.

The functional difference is that cache manager's wrap function is more efficient when multiple concurrent requests occur for the same cache key.

When there's a cache miss, instead of calling the user's function to fetch a value to cache once per request (which can exhaust downstream API rate limits, be slower, etc), one promise goes off to do the work and each of the other concurrent requests receive a promise that promises to resolve to the same value as the first promise that's actually executing the user's function.

• Since you added this module, will you be able to support and maintain it? If not, I would prefer adding the code directly into this project.

Yes, I intend to support it. I see value in the coalescing logic as a standalone helper library, which is agnostic to caching though that is the main driver for the solution.

With that said, I'm not opposed to adding the code directly to this project if you all want to maintain it going forward.

• Can you update the readme with an example of how this works?

The API for the wrap function to users is not changed, if that's what you're asking. And if so, the README for cache manager maybe doesn't need to change since users don't have to do anything differently than they do now. If this PR is merged, then users will notice an efficiency boost when concurrent requests call wrap function for the same cache-key.

This PR includes new tests to demonstrate the problem when wrap is called concurrently with and without coalescing logic.

I also have an example on the promise-coalesce project's README, too.

Let me know if you have any other questions or concerns, thanks

[codecov-commenter]

Codecov Report

Merging #599 (d1753dc) into master (66246e9) will not change coverage.
The diff coverage is 100.00%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

@@            Coverage Diff            @@
##            master      #599   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            5         5           
  Lines          292       295    +3     
  Branches        75        76    +1     
=========================================
+ Hits           292       295    +3     

Files	Coverage Δ
src/caching.ts	100.00% <100.00%> (ø)

[jaredwray]

@douglascayers - thanks so much for all the work on this and we will be working to get this deployed in the next couple weeks.

[douglascayers]

Fantastic, thanks!