Implement API endpoints for collections #625

richaagarwal · 2022-08-05T08:11:46Z

Implements #506 and #507

To run locally, follow instructions at https://github.com/chanzuckerberg/napari-hub/wiki/Local-development-guide#running-frontend--backend-e2e and then navigate to http://localhost:8080/collections or a particular collection, e.g. http://localhost:8080/collections/alfa-cohort.

You can also browse the dev branch at https://dev-collections-backend-frontend.dev.imaging.cziscience.com/collections

richaagarwal · 2022-08-05T08:12:31Z

backend/api/app.py

@@ -4,8 +4,8 @@
 from werkzeug.middleware.dispatcher import DispatcherMiddleware
 from flask import Flask, Response, jsonify, render_template
 from flask_githubapp.core import GitHubApp
-import yaml


I ran flake8 on this file and found this was an unused import

ooh nice, do we already have flake8 in the repo? if not it would be great to have it as a GitHub check like we do with eslint for the frontend 🤩

Agreed! I'll open up an issue to improve our linting

codemonkey800 · 2022-08-05T17:30:45Z

What is needed to connect this to your implementation in #521?

As long as the API is located at /collections and /collections/<name>, then no additional work is required on the frontend 👌

If you want to test this E2E, you can set up a dev deployment by pushing to a dev-* branch. I already went ahead and created one, but looks like there's issues:
https://dev-collections-api-frontend.dev.imaging.cziscience.com/collections

@potating-potato does dev deployment setup both frontend and backend together?

richaagarwal · 2022-08-05T17:46:57Z

@codemonkey800 The branch you created looks like it doesn't have any of this branch's commits, so maybe that's why?

richaagarwal · 2022-08-05T18:42:04Z

Pushed to a dev branch, viewable here: https://dev-collections-backend-frontend.dev.imaging.cziscience.com/collections

liu-ziyang · 2022-08-05T18:48:48Z

@potating-potato does dev deployment setup both frontend and backend together?

Yes, there is also dev API endpoint separately that you can use to verify the backend implementations

richaagarwal · 2022-08-05T18:57:02Z

@potating-potato Where can I find the naming convention for the dev API endpoint? Jeremy mentioned the frontend_url is in the build logs but it's a little hard to search through that if you don't know what you're looking for.

liu-ziyang · 2022-08-05T20:46:07Z

the frontend URL is: https://dev-collections-backend-frontend.dev.imaging.cziscience.com/
and backend URL is: https://6vwmwypt38.execute-api.us-west-2.amazonaws.com/dev-collections-backend (I think we should create a DNS alias for the backend URL)

codemonkey800

overall LGTM, but did have a few questions on error handling. feel free to add to this PR or in a follow up 🔥 🤩

codemonkey800 · 2022-10-18T17:27:20Z

backend/api/app.py

@@ -4,8 +4,8 @@
 from werkzeug.middleware.dispatcher import DispatcherMiddleware
 from flask import Flask, Response, jsonify, render_template
 from flask_githubapp.core import GitHubApp
-import yaml


ooh nice, do we already have flake8 in the repo? if not it would be great to have it as a GitHub check like we do with eslint for the frontend 🤩

codemonkey800 · 2022-10-18T17:41:25Z

backend/api/collections.py

+
+
+def get_collections():
+    resp = requests.get(COLLECTIONS_URL)


should we wrap this in try / catch to handle network errors gracefully, or are we fine with errors here? 🤔

Ah good point! I'm re-working this to use the utilities in utils/github.py so that we handle these as we do elsewhere.

codemonkey800 · 2022-10-18T17:43:26Z

backend/api/collections.py

+def get_collection(slug, index_page=False):
+    collection_url = "https://raw.githubusercontent.com/chanzuckerberg/napari-hub-collections/main/collections/{slug}.yml".format(slug=slug)
+    resp = requests.get(collection_url)
+    if resp.status_code == 404:


how should we handle network errors or non 404s? 🤔 maybe we can log the error and return None

richaagarwal · 2022-11-02T21:59:35Z

frontend/src/utils/featureFlags.ts

@@ -28,7 +28,7 @@ export const FEATURE_FLAGS = createFeatureFlags({
  },

  collections: {
-    environments: ['dev'],
+    environments: ['dev', 'staging', 'prod'],


Adding prod here as well as this should go out with our next release cycle, if approved!

klai95 · 2022-11-03T00:26:40Z

backend/api/collections.py

+    return collections
+
+
+def get_yaml_data(slug, visibility_requirements):


What does the slug variable represent in this and the below methods?

i think it's the sluggified name of the file, but I feel name or maybe name_slug would be clearer here

This is a good overview of what a slug is - https://en.wikipedia.org/wiki/Clean_URL#Slug. For this work, taking the example of the Alfa Cohort, alfa-cohort is parsed from https://github.com/chanzuckerberg/napari-hub-collections/blob/main/collections/alfa-cohort.yml, and then used as the slug on our site, e.g. https://dev-collections-backend-frontend.dev.imaging.cziscience.com/collections/alfa-cohort.

I'll update the naming to collection_name since this is similar to our naming pattern for plugins if that helps avoid confusion!

codemonkey800

💯

codemonkey800 · 2022-11-03T16:04:22Z

backend/api/collections.py

+    return collections
+
+
+def get_yaml_data(slug, visibility_requirements):


i think it's the sluggified name of the file, but I feel name or maybe name_slug would be clearer here

richaagarwal commented Aug 5, 2022

View reviewed changes

richaagarwal force-pushed the collections-backend branch 2 times, most recently from b065675 to 041a31f Compare October 15, 2022 02:57

richaagarwal changed the title ~~Implement API endpoint for collections index~~ Implement API endpoints for collections Oct 17, 2022

codemonkey800 approved these changes Oct 18, 2022

View reviewed changes

richaagarwal force-pushed the collections-backend branch from 920d443 to c2ed845 Compare November 2, 2022 21:51

richaagarwal marked this pull request as ready for review November 2, 2022 21:51

richaagarwal commented Nov 2, 2022

View reviewed changes

richaagarwal requested review from klai95, manasaV3 and neuromusic November 2, 2022 22:00

klai95 reviewed Nov 3, 2022

View reviewed changes

codemonkey800 added the new-feature Release Label: Used for categorizing features in automated release notes label Nov 3, 2022

codemonkey800 approved these changes Nov 3, 2022

View reviewed changes

Richa Agarwal and others added 6 commits November 3, 2022 16:46

Implement API endpoint for collections index

eebd641

Implement API endpoint for collection page

7fc5bc2

Implement visibility conditions

dfdcfea

Extend github utility and code cleanup

a6c2228

Update feature flags

48f4e04

Update naming from slug to collection_name

07ec9e7

richaagarwal force-pushed the collections-backend branch from 9585e22 to 07ec9e7 Compare November 3, 2022 23:46

richaagarwal merged commit d8ef930 into main Nov 4, 2022

richaagarwal deleted the collections-backend branch November 4, 2022 18:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement API endpoints for collections #625

Implement API endpoints for collections #625

richaagarwal commented Aug 5, 2022 •

edited

Loading

richaagarwal Aug 5, 2022

codemonkey800 Oct 18, 2022

richaagarwal Nov 2, 2022

codemonkey800 commented Aug 5, 2022

richaagarwal commented Aug 5, 2022

richaagarwal commented Aug 5, 2022

liu-ziyang commented Aug 5, 2022 •

edited

Loading

richaagarwal commented Aug 5, 2022

liu-ziyang commented Aug 5, 2022

codemonkey800 left a comment

codemonkey800 Oct 18, 2022

codemonkey800 Oct 18, 2022

richaagarwal Nov 2, 2022

codemonkey800 Oct 18, 2022

richaagarwal Nov 2, 2022

klai95 Nov 3, 2022

codemonkey800 Nov 3, 2022

richaagarwal Nov 3, 2022 •

edited

Loading

codemonkey800 left a comment

codemonkey800 Nov 3, 2022

		return collections


		def get_yaml_data(slug, visibility_requirements):

Implement API endpoints for collections #625

Implement API endpoints for collections #625

Conversation

richaagarwal commented Aug 5, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codemonkey800 commented Aug 5, 2022

richaagarwal commented Aug 5, 2022

richaagarwal commented Aug 5, 2022

liu-ziyang commented Aug 5, 2022 • edited Loading

richaagarwal commented Aug 5, 2022

liu-ziyang commented Aug 5, 2022

codemonkey800 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richaagarwal Nov 3, 2022 • edited Loading

Choose a reason for hiding this comment

codemonkey800 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richaagarwal commented Aug 5, 2022 •

edited

Loading

liu-ziyang commented Aug 5, 2022 •

edited

Loading

richaagarwal Nov 3, 2022 •

edited

Loading