-
-
Notifications
You must be signed in to change notification settings - Fork 32.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use orjson to parse json faster #32153
Conversation
fb02fba
to
5559c4c
Compare
Codecov Report
@@ Coverage Diff @@
## dev #32153 +/- ##
==========================================
+ Coverage 94.73% 94.75% +0.02%
==========================================
Files 767 768 +1
Lines 55505 55646 +141
==========================================
+ Hits 52580 52727 +147
+ Misses 2925 2919 -6
Continue to review full report at Codecov.
|
5559c4c
to
6e2cf1f
Compare
6e2cf1f
to
c2f1496
Compare
c2f1496
to
f4db411
Compare
@balloob I added it to http manifest because of failed CI job. |
It's a problem with CI that we should solve. We shouldn't hack around it. |
Can you fix it then? :) |
Trying but GitHub is not cooperating. |
Realized this is using Rust, which means a big pain point for the community to install on RPi's in a venv. I will have to revert this PR. |
I wonder if storing this in a sql native json data type is faster. Might just shift the work, but if it was smartly done it aught to be quicker. |
Proposed change
With #32128 and #32137 loading of logbook for the last 7 days improved from 22s to 11.3s.
I think it can be improved further. I noticed that
json.loads
takes quite a lot of time.https://github.com/ijl/orjson is about 2 times faster than standard
json.loads
and saves another 1.25s or 11% of overall time.Before:
After:
For encoding
json.dumps
is still used becauseorjson
doesn't support custom serialiser incls
arg.Type of change
Example entry for
configuration.yaml
:# Example configuration.yaml
Additional information
Checklist
black --fast homeassistant tests
)If user exposed functionality or configuration variables are added/changed:
If the code communicates with devices, web services, or third-party tools:
Updated and included derived files by running:
python3 -m script.hassfest
.requirements_all.txt
.Updated by running
python3 -m script.gen_requirements_all
..coveragerc
.The integration reached or maintains the following Integration Quality Scale: