Use ciso8601 library to parse datetime faster

[KapJI]

Proposed change

This is calltree of logbook query for all events for the last 7 days (numbers are nanoseconds):

hass is running on pretty fast machine with i7-6700K CPU @ 4.00GHz. And MariaDB is used for storing data.

But that request took insane 22 seconds!

• State.from_dict() took 9s.
• parse_datetime took 3.08s
• valid_entity_id took 4.6s

This change introduces https://github.com/closeio/ciso8601 which is library to parse datetime in ISO format written in C with very high performance. Exactly in format which is used by hass.

But just in case I left old code as a fallback option. I checked that it was never called on my setup. So it can be removed safely.

With this parse_datetime can do the same work in just 0.184s, 16.7x improvement! State.from_dict now takes 5.45s, 40% improvement there.

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Example entry for configuration.yaml:

# Example configuration.yaml

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• The code has been formatted using Black (black --fast homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• No score or internal
• 🥈 Silver
• 🥇 Gold
• 🏆 Platinum

[balloob]

Oh that's a great speedup!

What do you use to generate the call tree?

[balloob]

I am 👍 on this PR.

@pvizeli can you comment on if this pacakge can build on our Alpine images.

[pvizeli]

That is just a simple C extension that handles automatically by our wheel infrastructure after this PR is merged.

[KapJI]

@balloob To generate calltree I use cProfile and wrap humanify call with:

import cProfile
pr = cProfile.Profile()
pr.enable()
result = list(humanify(hass, yield_events(query)))
pr.disable()
pr.create_stats()
pr.dump_stats("logbook.cprof")
return result

Then I need to convert calltree to different format using pyprof2calltree:

pyprof2calltree -i logbook.cprof -o ~/logbook.calltree

And it can be viewed with qcachegrind:

qcachegrind logbook.calltree

[KapJI]

Fix lint warning and test. ciso8601 accepts just date as a valid string, I hope it's not a big deal.

[codecov]

Codecov Report

Merging #32128 into dev will increase coverage by 0.07%.
The diff coverage is 94.72%.

@@            Coverage Diff             @@
##              dev   #32128      +/-   ##
==========================================
+ Coverage   94.65%   94.73%   +0.07%     
==========================================
  Files         756      767      +11     
  Lines       54808    55509     +701     
==========================================
+ Hits        51877    52584     +707     
+ Misses       2931     2925       -6

Impacted Files	Coverage Δ
homeassistant/components/ecobee/config_flow.py	100% <ø> (ø)	⬆️
homeassistant/components/darksky/sensor.py	96.81% <ø> (ø)	⬆️
homeassistant/components/auth/indieauth.py	77.55% <ø> (ø)	⬆️
homeassistant/components/automation/config.py	100% <ø> (ø)	⬆️
homeassistant/components/google_wifi/sensor.py	100% <ø> (ø)	⬆️
homeassistant/components/deconz/binary_sensor.py	98.03% <ø> (ø)	⬆️
homeassistant/components/flux/switch.py	95.37% <ø> (ø)	⬆️
homeassistant/components/fido/sensor.py	100% <ø> (ø)	⬆️
homeassistant/components/deconz/sensor.py	96.82% <ø> (+0.79%)	⬆️
homeassistant/components/abode/const.py	100% <ø> (ø)	⬆️
... and 149 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 41f3fb2...8365e43. Read the comment docs.

[OnFreund]

@KapJI is it possible that this also needs to be added to requirements_test_all.txt?

[KapJI]

@OnFreund do you experience any issues with this?

@balloob what needs to be added in requirements_test_all.txt?

[balloob]

Nope, that file is maintained. Re-run pip3 install -e . to pick it up.

[OnFreund]

I already installed with requirements_all so everything is working now, but initially after this PR, following the docs for installing testing requirements did not install ciso8601. If no one else encountered this it could be just a local problem.

[balloob]

that's because as part of the tests we run pip3 install -e ., which installs Home Assistant and all of its requirements. They are not part of the test requirements.