🐛 Fixed issue with accumulating tracked services

[GitHK]

What do these changes do?

Addresses an issue where the dynamic-scheduler will keep track of services which have been stopped but are still tracked as requested_state=RUNNING. This can happen if services are manually removed or for other edge cases.

♻️ changed the bytes serialised for message-pack which more easily allows for model changes in the future
🐛 services which are reported IDLE by the platform and the dynamic-scheduler sees them as requested_state=RUNNING will be removed after 5 minutes of inactivity
⬆️ upgrade msgpack repo wide

Related issue/s

• fixes Faststream Broker creates millions of messages #6588

How to test

Dev-ops checklist

• No ENV changes or I properly updated ENV (read the instruction)

[sanderegg]

so question, if I get it correctly now you will have these entries for 5 minutes, that trigger an incredible amount of logs. is this really how it is suppose to work? I wonder if that rate of messages is not an overkill.

Thanks

[matusdrobuliak66]

Thanks, just have a few questions

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.20%. Comparing base (e994f5d) to head (d7a8ab9).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6631      +/-   ##
==========================================
- Coverage   87.61%   87.20%   -0.41%     
==========================================
  Files        1525      765     -760     
  Lines       60678    36871   -23807     
  Branches     2085      265    -1820     
==========================================
- Hits        53165    32155   -21010     
+ Misses       7195     4655    -2540     
+ Partials      318       61     -257     

Flag	Coverage Δ
integrationtests	79.75% <ø> (+3.74%)	⬆️
unittests	87.54% <100.00%> (+1.60%)	⬆️

Components	Coverage Δ
api	∅ <ø> (∅)
pkg_aws_library	∅ <ø> (∅)
pkg_dask_task_models_library	∅ <ø> (∅)
pkg_models_library	∅ <ø> (∅)
pkg_notifications_library	∅ <ø> (∅)
pkg_postgres_database	∅ <ø> (∅)
pkg_service_integration	∅ <ø> (∅)
pkg_service_library	∅ <ø> (∅)
pkg_settings_library	∅ <ø> (∅)
pkg_simcore_sdk	77.44% <ø> (-7.84%)	⬇️
agent	∅ <ø> (∅)
api_server	∅ <ø> (∅)
autoscaling	95.26% <ø> (∅)
catalog	89.51% <ø> (ø)
clusters_keeper	98.85% <ø> (ø)
dask_sidecar	91.30% <ø> (ø)
datcore_adapter	∅ <ø> (∅)
director	∅ <ø> (∅)
director_v2	90.83% <ø> (ø)
dynamic_scheduler	96.71% <100.00%> (+0.08%)	⬆️
dynamic_sidecar	59.70% <ø> (-30.05%)	⬇️
efs_guardian	∅ <ø> (∅)
invitations	∅ <ø> (∅)
osparc_gateway_server	85.41% <ø> (+0.26%)	⬆️
payments	∅ <ø> (∅)
resource_usage_tracker	∅ <ø> (∅)
storage	∅ <ø> (∅)
webclient	∅ <ø> (∅)
webserver	89.30% <ø> (-0.02%)	⬇️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e994f5d...d7a8ab9. Read the comment docs.

[GitHK]

@matusdrobuliak66 @pcrespov @sanderegg

If you ever need to use msgpack for serialising any object that is not json serialisable you are going to have a miserable time.
Pleas use https://github.com/vsergeev/u-msgpack-python (which is an official library linked form they official page it's just listed in a strange way).

Answers to questions that might come out:

• Yes it is the one with les starts (which means nothing).
• Yes it has less releases (which could also mean less buggy other than less used).
• Yes it is much more simpler and straight forward to use.

I state my case here and revere back for it for the dynamic-scheduler.

[sanderegg]

I agree that the number of stars is not everything. Nevertheless that shows a bit the usage of a library. so one with 2k stars is probably more used than one with 200. Also when I check that repo, I see Test: No Status, that does not really seem maintained.
Also note the last release is from May 2023.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud