Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add golang purl service #316

Merged
merged 15 commits into from
Mar 14, 2024
8 changes: 8 additions & 0 deletions packagedb/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -414,3 +414,11 @@ class PurlValidateResponseSerializer(Serializer):
class PurlValidateSerializer(Serializer):
purl = CharField(required=True)
check_existence = BooleanField(required=False, default=False)


class GoLangPurlSerializer(Serializer):
go_package = CharField(required=True)


class GoLangPurlResponseSerializer(Serializer):
golang_purl = CharField()
8 changes: 8 additions & 0 deletions packagedb/tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -1179,3 +1179,11 @@ def test_api_purl_validation_empty_request(self):
}

self.assertAlmostEquals(expected, response1.data)


class ToGolangPurlTestCase(TestCase):

def test_to_golang_purl(self):
response = self.client.get("/api/to_purl/go", data={"go_package": "github.com/gorilla/[email protected]"}, follow=True)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, github.com/gorilla/[email protected] is not a valid import statement.
Does Go allow specifying a particular package version using an @ symbol in an import statement?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, not a go import https://github.com/moby/moby/blob/6c10086976d07d4746e03dcfd188972a2f07e1c9/vendor.mod#L51 but *.mod files does have version representation like this? should I change the code in packageurl-python to accept this kind of inputs as well ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should I change the code in packageurl-python to accept this kind of inputs as well ?

Update the code to also accept this would make sense. With proper documentation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, not a go import https://github.com/moby/moby/blob/6c10086976d07d4746e03dcfd188972a2f07e1c9/vendor.mod#L51 but *.mod files does have version representation like this? should I change the code in packageurl-python to accept this kind of inputs as well ?

IMO it makes sense to support getting PURLs from both Go import and Go Manifest.
However, I'm not sure if we want to support obtaining PURLs from Go dependency management commands like...

go get github.com/gorilla/[email protected]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@keshav-space so should we throw an error when we encounter any package like this ? when there is "@" in package, we should error out ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added a PR here https://github.com/package-url/packageurl-python/pull/148/files, please have a look

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@keshav-space so should we throw an error when we encounter any package like this ? when there is "@" in package, we should error out ?

If we don't support them in packageurl-python, then we should raise an error here.

Copy link
Member

@pombredanne pombredanne Mar 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is all OK and handled in the packageurl library. And we do not support the @ notation from a "go get"

expected = "pkg:golang/github.com/gorilla/mux%40v1.7.3"
self.assertEqual(expected, response.data["golang_purl"])
58 changes: 58 additions & 0 deletions packagedb/to_purl.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#
# Copyright (c) nexB Inc. and others. All rights reserved.
# purldb is a trademark of nexB Inc.
# SPDX-License-Identifier: Apache-2.0
# See http://www.apache.org/licenses/LICENSE-2.0 for the license text.
# See https://github.com/nexB/purldb for support or download.
# See https://aboutcode.org for more information about nexB OSS projects.
#

from django.db import router
from drf_spectacular.utils import OpenApiParameter, extend_schema
from packageurl.utils import get_golang_purl
from rest_framework import status
from rest_framework import viewsets
from rest_framework.response import Response

from packagedb.serializers import GoLangPurlResponseSerializer
from packagedb.serializers import GoLangPurlSerializer
from rest_framework import routers


@extend_schema(
parameters=[
OpenApiParameter("go_package", str, "query", description="go import package"),
],
responses={200: GoLangPurlResponseSerializer()},
)
class GolangPurlViewSet(viewsets.ViewSet):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need some docsting. Being more explicit about the fact that the endpoint accepts a standard Go import string and returns the corresponding PackageURL will be helpful.

"""
Return a ``golang_purl`` from a standard go import string
``go_package``.
"""

serializer_class = GoLangPurlSerializer

def get_view_name(self):
return "GoLang purl"

def list(self, request):
serializer = self.serializer_class(data=request.query_params)
response = {}

if not serializer.is_valid():
return Response(
{"errors": serializer.errors}, status=status.HTTP_400_BAD_REQUEST
)

validated_data = serializer.validated_data
go_import = validated_data.get("go_package")
purl = get_golang_purl(go_import)
response["golang_purl"] = purl
serializer = GoLangPurlResponseSerializer(
response, context={"request": request}
)
return Response(serializer.data)

api_to_purl_router = routers.DefaultRouter()
api_to_purl_router.register("go", GolangPurlViewSet, "go")
2 changes: 2 additions & 0 deletions purldb_project/urls.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
from matchcode.api import ExactPackageArchiveIndexViewSet
from minecode.api import PriorityResourceURIViewSet
from packagedb.api import PurlValidateViewSet
from packagedb.to_purl import api_to_purl_router
from packagedb.api import CollectViewSet
from drf_spectacular.views import SpectacularAPIView
from drf_spectacular.views import SpectacularSwaggerView
Expand All @@ -51,6 +52,7 @@
TemplateView.as_view(template_name='robots.txt', content_type='text/plain'),
),
path('api/', include((api_router.urls, 'api'))),
path('api/to_purl/', include((api_to_purl_router.urls, 'api_to'))),
path("", RedirectView.as_view(url="api/")),
path('api/schema/', SpectacularAPIView.as_view(), name='schema'),
path('api/docs/', SpectacularSwaggerView.as_view(url_name='schema'), name='swagger-ui'),
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ natsort==8.2.0
normality==2.5.0
openpyxl==3.1.2
packagedcode-msitools==0.101.210706
packageurl-python==0.13.4
packageurl-python==0.14.0
packaging==23.2
packvers==21.5
parameter-expansion-patched==0.3.1
Expand Down
2 changes: 1 addition & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ install_requires =
jawa == 2.2.0
markdown == 3.5.1
natsort == 8.2.0
packageurl-python == 0.13.4
packageurl-python == 0.14.0
psycopg[binary]==3.1.17
PyGithub == 1.56
reppy2 == 0.3.6
Expand Down
Loading