🎨 introduce logger filtering with ENV var ⚠️

[matusdrobuliak66]

What do these changes do?

• We will change the log level to INFO in production. I thought this might be a good approach to interactively filter out some repetitive logs that are not useful and might be polluting the logs.
• 🎨 introduce logger filtering with ENV var LOG_FILTER_MAPPING
  Example:

LOG_FILTER_MAPPING = {
    "uvicorn.access": [
        '"GET / HTTP/1.1" 200',
        '"GET /metrics HTTP/1.1" 200',
    ],
    "gunicorn.access": [
        '"GET /v0/ HTTP/1.1" 200',
        '"GET /v0/health HTTP/1.1" 200',
    ],
}

Related issue/s

How to test

Dev-ops checklist

• https://git.speag.com/oSparc/osparc-ops-deployment-configuration/-/merge_requests/1029
• No ENV changes or I properly updated ENV (read the instruction)

[codecov]

Codecov Report

Attention: Patch coverage is 85.71429% with 8 lines in your changes missing coverage. Please review.

Project coverage is 87.89%. Comparing base (87a9b7e) to head (1a7e3ed).
Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff            @@
##           master    #6596    +/-   ##
========================================
  Coverage   87.88%   87.89%            
========================================
  Files        1562     1556     -6     
  Lines       62829    62680   -149     
  Branches     2085     2088     +3     
========================================
- Hits        55218    55091   -127     
+ Misses       7293     7271    -22     
  Partials      318      318            

Flag	Coverage Δ
integrationtests	64.83% <85.71%> (+0.02%)	⬆️
unittests	85.83% <85.71%> (-1.19%)	⬇️

Components	Coverage Δ
api	∅ <ø> (∅)
pkg_aws_library	93.38% <ø> (ø)
pkg_dask_task_models_library	96.87% <ø> (ø)
pkg_models_library	92.14% <ø> (ø)
pkg_notifications_library	83.79% <ø> (ø)
pkg_postgres_database	87.77% <ø> (ø)
pkg_service_integration	71.44% <ø> (ø)
pkg_service_library	76.63% <65.21%> (-0.05%)	⬇️
pkg_settings_library	91.42% <ø> (ø)
pkg_simcore_sdk	85.27% <ø> (ø)
agent	97.17% <100.00%> (+0.01%)	⬆️
api_server	89.82% <100.00%> (+<0.01%)	⬆️
autoscaling	95.26% <100.00%> (+<0.01%)	⬆️
catalog	89.52% <100.00%> (+0.01%)	⬆️
clusters_keeper	98.85% <100.00%> (+<0.01%)	⬆️
dask_sidecar	91.32% <100.00%> (+0.02%)	⬆️
datcore_adapter	94.05% <100.00%> (+0.02%)	⬆️
director	58.38% <ø> (ø)
director_v2	90.75% <100.00%> (-0.13%)	⬇️
dynamic_scheduler	96.72% <100.00%> (+<0.01%)	⬆️
dynamic_sidecar	89.72% <100.00%> (-0.03%)	⬇️
efs_guardian	90.00% <100.00%> (+0.03%)	⬆️
invitations	93.51% <100.00%> (+0.04%)	⬆️
osparc_gateway_server	85.15% <ø> (ø)
payments	93.09% <100.00%> (+<0.01%)	⬆️
resource_usage_tracker	87.38% <100.00%> (+0.01%)	⬆️
storage	89.75% <100.00%> (+0.01%)	⬆️
webclient	∅ <ø> (∅)
webserver	89.30% <100.00%> (-0.01%)	⬇️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 87a9b7e...1a7e3ed. Read the comment docs.

[sanderegg]

added a comment

[mrnicegyu11]

I have a subtle change request that I personally find reasonable, but I have formulated as a comment. If you dont want to do it, please bring me a coffee or tell me a compliment, then I might overlook my precautions and approve 😽

[sanderegg]

thanks, just a few comments

[YuryHrytsuk]

@matusdrobuliak66 this is amazing. I like it!

I wonder if it further slows down general performance because of logging. It would be nice to see a benchmark but we don't have anything set up yet to compare.

[YuryHrytsuk]

I think before it is merged, we need a PR in osparc-config with all the new required ENV.

[mrnicegyu11]

For the record re regex speed: you are right @matusdrobuliak66 https://stackoverflow.com/questions/19911508/python-speed-for-in-vs-regular-expression

[pcrespov]

I like the idea. I have some comments regarding the implementation. Lots of thoughts :-)
I am happy to discuss it offline and even help implementing some parts (now or in further iterations)

[pcrespov]

Please consider the comments before merging. Either by taking action in this PR or by creating an issue.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[GitHK]

Thanks. You missed one.