Separated Room Statistics

[reivilibre]

This PR blocks:

• More efficient room directory queries

Not included in this PR:

• Purging of expired historical stats (i.e. the retention period is currently ignored)
• FIXME: we could/should get this from room_stats when matthew/stats lands (handlers/sync.py)

To consider:

• we may want to track current statistics and treat as zero-retention, if stats are disabled but the room directory is enabled – the room directory will depend on stats. This can be dealt with later, though, I suspect.
• it occurs to me now that the current way that old collection is performed could lead to CPU + DB I/O bursts at the end of each day (with bucket_size=1 day), because all dirty rows will become old at the same time.
  • As such, old collection should only be ran on a 'few' (~250?) rows at once.
  • How about: 250 rows, definitely including the one we want to update will be old collected when we go to add delta for a stats row which is old. This still ensures we pick up extra rows, but without having to do absolutely all of them at once.
    • In the future, a background job could be added to run to perform old collection slowly to hoover up the missing ones.

---

I still need to add a changelog and some documentation.

• Pull request is based on the develop branch
• Pull request includes a changelog file
• Pull request includes a sign off

[codecov]

Codecov Report

Merging #5847 into develop will increase coverage by 0.16%.
The diff coverage is 86.08%.

@@             Coverage Diff             @@
##           develop    #5847      +/-   ##
===========================================
+ Coverage    63.41%   63.58%   +0.16%     
===========================================
  Files          333      332       -1     
  Lines        36651    36494     -157     
  Branches      6057     6025      -32     
===========================================
- Hits         23244    23203      -41     
+ Misses       11755    11645     -110     
+ Partials      1652     1646       -6

[richvdh]

one line for this please, btw

[reivilibre]

Obsoleted by 8 other PRs.