Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Sync task that auto-registers items from VIP as licensed items in the system 🗃️ 🚨 #7140

Merged
merged 69 commits into from
Feb 5, 2025

Conversation

pcrespov
Copy link
Member

@pcrespov pcrespov commented Jan 29, 2025

What do these changes do?

ReDoc

Fetches models from vip-api, validates them, and registers them in licensed_items through a periodic background task.

  • ♻️ Moved domain models from licenses to models_library.licenses
  • ♻️ Enhanced schema model for LicensedItemRestGet (refer to OAS above)
    • Replaced object in licensed_resource_data with a more specific definition [PINED!]
  • 🗃️ postgres-database
    • Added unique constraint ("licensed_resource_name", "licensed_resource_type") for resource identification
  • ♻️ Change: WEBSERVER_LICENSES is no longer a flag. To determine if it is disabled, the front-end should refer to pluginDisabled in the statics instead.
  • 🐛 Fixed retry exception in packages/service-library/src/servicelib/background_task.py
  • 🐛 Fixed absolute import in services/api-server/src/simcore_service_api_server/main.py (required for CLI)
  • 🐛 Fixed incorrect entrypoint name in services/resource-usage-tracker/docker/boot.sh

Related issue/s

How to test

Driving tests

cd services/web/server
make install-dev
pytest -vv -ff tests/unit/**/test_*license*.py

Dev-ops

  • 🚨 After the first sync, make sure to enable new items by assigning corresponding product (product_name) and princing plans (pricing_plan_id)
  • New envs. example
WEBSERVER_LICENSES={}
LICENSES_ITIS_VIP_SYNCER_ENABLED=false
ITIS_VIP_API_URL=https://some-api-url/{category}
ITIS_VIP_CATEGORIES='{"HumanWholeBody": "Humans", "HumanBodyRegion": "Humans (Region)", "AnimalWholeBody": "Animal"}'

@pcrespov pcrespov added this to the Singularity milestone Jan 29, 2025
@pcrespov pcrespov self-assigned this Jan 29, 2025
@pcrespov pcrespov added a:webserver issue related to the webserver service a:database associated to postgres service and postgres-database package labels Jan 29, 2025
@pcrespov pcrespov changed the title WIP: ✨ Is7097/fetch vip api 🗃️ WIP: ✨ Is7097/fetch vip api 🗃️ 🚨 Jan 29, 2025
Copy link

codecov bot commented Jan 29, 2025

Codecov Report

Attention: Patch coverage is 90.76433% with 29 lines in your changes missing coverage. Please review.

Project coverage is 87.69%. Comparing base (4fa6ee2) to head (35041a5).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7140      +/-   ##
==========================================
+ Coverage   87.66%   87.69%   +0.03%     
==========================================
  Files        1636     1634       -2     
  Lines       63958    63981      +23     
  Branches     1179     1179              
==========================================
+ Hits        56069    56111      +42     
+ Misses       7577     7559      -18     
+ Partials      312      311       -1     
Flag Coverage Δ
integrationtests 64.66% <55.23%> (+0.01%) ⬆️
unittests 85.99% <90.76%> (+0.03%) ⬆️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library 94.02% <ø> (ø)
pkg_dask_task_models_library 97.09% <ø> (ø)
pkg_models_library 91.43% <98.86%> (-0.09%) ⬇️
pkg_notifications_library 84.57% <ø> (ø)
pkg_postgres_database 88.39% <ø> (ø)
pkg_service_integration 70.03% <ø> (ø)
pkg_service_library 74.13% <60.00%> (+<0.01%) ⬆️
pkg_settings_library 90.71% <ø> (ø)
pkg_simcore_sdk 85.50% <ø> (ø)
agent 96.46% <ø> (ø)
api_server 90.55% <100.00%> (ø)
autoscaling 96.08% <ø> (ø)
catalog 90.33% <ø> (ø)
clusters_keeper 99.24% <ø> (ø)
dask_sidecar 91.26% <ø> (ø)
datcore_adapter 93.19% <ø> (ø)
director 76.92% <ø> (-0.09%) ⬇️
director_v2 91.30% <ø> (ø)
dynamic_scheduler 97.17% <ø> (ø)
dynamic_sidecar 89.75% <ø> (ø)
efs_guardian 90.25% <ø> (ø)
invitations 93.28% <ø> (ø)
osparc_gateway_server ∅ <ø> (∅)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.11% <100.00%> (+0.32%) ⬆️
storage 89.57% <ø> (ø)
webclient ∅ <ø> (∅)
webserver 86.21% <87.61%> (+<0.01%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4fa6ee2...35041a5. Read the comment docs.

@pcrespov pcrespov force-pushed the is7097/fetch-vip-api branch from 8fac934 to f266505 Compare January 30, 2025 00:02
@pcrespov pcrespov force-pushed the is7097/fetch-vip-api branch from 17762ce to 77be4fc Compare January 30, 2025 15:50
@pcrespov pcrespov force-pushed the is7097/fetch-vip-api branch from 5285367 to 5d45c0b Compare February 4, 2025 16:14
Copy link
Member

@odeimaiz odeimaiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the js part looks good to me

@pcrespov pcrespov enabled auto-merge (squash) February 4, 2025 17:09
Copy link
Contributor

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, thanks a lot! 🥇

Copy link
Member

@mrnicegyu11 mrnicegyu11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving without reading to unblock. I am currently fixing a server incident sorry. should be fine :--)

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, just a few minor things

Copy link
Contributor

@YuryHrytsuk YuryHrytsuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If applicable: don't forget about ENV for STAG / PROD environments

Copy link

sonarqubecloud bot commented Feb 5, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
1 Accepted issue

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@pcrespov pcrespov disabled auto-merge February 5, 2025 16:55
@pcrespov pcrespov merged commit 1b17b3d into ITISFoundation:master Feb 5, 2025
92 of 94 checks passed
@pcrespov pcrespov deleted the is7097/fetch-vip-api branch February 5, 2025 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:database associated to postgres service and postgres-database package a:webserver issue related to the webserver service
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants