Skip to content

vcio-api-v3: Create the API v3 structure, align with DejaCode #2057

Closed

Description

@TG1999

The current API has a lot's of different endpoints that can be combined into one, instead of supporting vulnerabilities can support advisories and group advisories into using heuristics like common identifiers and affected package range.

Activity

  1. self-assigned this
    on Dec 17, 2025
  2. moved this to Todo ready in 00-AboutCodePlanneron Dec 17, 2025
  3. TG1999 commented on Dec 23, 2025

    @TG1999
    ContributorAuthor

    Use cases:

    • I have a PURL, getting a list of advisories for that PURL
    • I have a list of PURLS, and I want list of advisories for all the PURLs

    Minimal set of useful data:

    For advisory

    • TBD

    For package

    • TBD

    Research other APIs like vulnerabilitylookup, osv.dev

    The current API mostly looks fine. We will get it reviewed by other consumers

  4. TG1999 commented on Jan 28, 2026

    @TG1999
    ContributorAuthor

    We will be grouping advisories on basis of content these are the fields summary, severity and packages.

  5. moved this from Todo ready to In progress in 00-AboutCodePlanneron Jan 28, 2026
  6. moved this from In progress to Done in 00-AboutCodePlanneron Feb 16, 2026
  7. closed this as completedby moving to Done in 00-AboutCodePlanneron Feb 16, 2026
  8. TG1999 commented on Feb 16, 2026

    @TG1999
    ContributorAuthor

    This is done now

    Reference:

    Advisory structure is changed from now

    {
    "packages": [
                {
                    "purl": "pkg:pypi/extractor@0.5",
                    "affected_by_vulnerabilities": [
                        {
                            "advisory_id": "pypa_importer_v2/PYSEC-2006-4",
                            "fixed_by_packages": [],
                            "duplicate_advisory_ids": [
                                "pysec_importer_v2/PYSEC-2006-4"
                            ]
                        }
                    ],
                    "fixing_vulnerabilities": [],
                    "next_non_vulnerable_version": null,
                    "latest_non_vulnerable_version": null,
                    "risk_score": null
                },
    ],
    "advisories_by_id": {
                "pypa_importer_v2/PYSEC-2006-4": {
                    "advisory_id": "pypa_importer_v2/PYSEC-2006-4",
                    "url": "https://github.com/pypa/advisory-database/blob/main/vulns/extractor/PYSEC-2006-4.yaml",
                    "aliases": [
                        "CVE-2006-2458",
                        "GHSA-f836-7jqw-3684"
                    ],
                    "summary": "Multiple heap-based buffer overflows in Libextractor 0.5.13 and earlier allow remote attackers to execute arbitrary code via (1) the asf_read_header function in the ASF plugin (plugins/asfextractor.c), and (2) the parse_trak_atom function in the QT plugin (plugins/qtextractor.c).",
                    "severities": [],
                    "weaknesses": [],
    }
    }
    
  9. moved this from Done to In progress in 00-AboutCodePlanneron Mar 6, 2026
  10. reopened this on Mar 9, 2026
  11. 13 remaining items

  12. TG1999 commented on May 7, 2026

    @TG1999
    ContributorAuthor
    >>> from django.db.models.functions import Length
    >>> 
    >>> from django.db.models import Max
    >>> max_length = (
    ...     AdvisoryV2.objects
    ...     .annotate(advisory_len=Length("advisory_id"))
    ...     .aggregate(Max("advisory_len"))
    ... )
    >>> print(max_length)
    {'advisory_len__max': 120}
    

    Currently this is the biggest advisory_id we store, we can reduce from 500 to 200 for now and increase if needed in future

  13. changed the title [-]Review the API structure for advisories[/-] [+]vcio-api-v3: Review the API structure for advisories[/+] on May 11, 2026
  14. changed the title [-]vcio-api-v3: Review the API structure for advisories[/-] [+]vcio-api-v3: Review the API structure for advisories, align with DejaCode[/+] on May 12, 2026
  15. changed the title [-]vcio-api-v3: Review the API structure for advisories, align with DejaCode[/-] [+]vcio-api-v3: Create the API v3 structure, align with DejaCode[/+] on May 12, 2026
  16. TG1999 commented on May 28, 2026

    @TG1999
    ContributorAuthor

    The "fetch 101 IDs to count, then run the queryset again" pattern loads the same data twice. Use .count() with a cap, or reuse the rows already loaded.

    When we fetch 101 IDs to count, we already capping our count 101, from what I found

    count = MyModel.objects.filter(... )[:1000].count()
    

    translates to

    SELECT COUNT(*) FROM (
        SELECT ... LIMIT 1000
    ) subquery;
    

    We can do qs[100:101] for even a more faster check if needed

  17. TG1999 commented on May 28, 2026

    @TG1999
    ContributorAuthor

    to_representation builds both the inline data and the URL fields for every object, then drops half. Pick one upfront.

    This is not possible to pick one upfront, it depends on data. If a package has more than 100 advisories then only we give a URL, otherwise we give advisories as is. For that we need to do computations at package level first.

  18. TG1999 commented on Jun 4, 2026

    @TG1999
    ContributorAuthor

    Dev work is done: #2287

    PR is in review now after testing it on staging

  19. TG1999 commented on Jun 10, 2026

    @TG1999
    ContributorAuthor

    All comments have been reviewed in #2287, hence we can review this on staging and close it once verfied and tested

  20. TG1999 commented on Jun 10, 2026

    @TG1999
    ContributorAuthor

    This is done now

    Reference:

    New response for grouped advisories, usage of API is here in https://github.com/aboutcode-org/vulnerablecode/blob/main/api_v3_usage.rst and https://public.vulnerablecode.io/api/docs/#/v3

    {
                "purl": "pkg:alpm/archlinux/389-ds-base@1.4.4.4-5",
                "affected_by_vulnerabilities": [
                    {
                        "advisory_id": "AVG-1482",
                        "advisory_uid": "archlinux/AVG-1482",
                        "aliases": [
                            "CVE-2020-35518"
                        ],
                        "summary": "information disclosure",
                        "weighted_severity": 6.2,
                        "exploitability": 0.5,
                        "risk_score": 3.1,
                        "fixed_by_packages": [
                            "pkg:alpm/archlinux/389-ds-base@2.0.2-1"
                        ],
                        "ssvc_trees": [],
                        "resource_url": "http://public2.vulnerablecode.io/advisories/archlinux/AVG-1482"
                    }
                ],
                "fixing_vulnerabilities": [],
                "next_non_vulnerable_version": "2.0.2-1",
                "latest_non_vulnerable_version": "2.0.7-1",
                "risk_score": 3.1
            },
            {
                "purl": "pkg:alpm/archlinux/389-ds-base@2.0.3-2",
                "affected_by_vulnerabilities": [
                    {
                        "advisory_id": "AVG-2206",
                        "advisory_uid": "archlinux/AVG-2206",
                        "aliases": [
                            "ASA-202107-72",
                            "CVE-2021-3514",
                            "CVE-2021-3652"
                        ],
                        "summary": "multiple issues",
                        "weighted_severity": 6.2,
                        "exploitability": 0.5,
                        "risk_score": 3.1,
                        "fixed_by_packages": [
                            "pkg:alpm/archlinux/389-ds-base@2.0.7-1"
                        ],
                        "ssvc_trees": [],
                        "resource_url": "http://public2.vulnerablecode.io/advisories/archlinux/AVG-2206"
                    }
                ],
                "fixing_vulnerabilities": [],
                "next_non_vulnerable_version": "2.0.7-1",
                "latest_non_vulnerable_version": "2.0.7-1",
                "risk_score": 3.1
            }
    

    This has been a major re-architecture on how we mark advisories as completely imported and usable. We have added a new pipeline mark_unfurl_version_range_v2 that can mark the completely unfurled advisories as usable and do grouping and risk score calculation only once an advisory is completely unfurled. This is an approach to ensure data is consistent, consumable and user friendly as well.

    We have also added SSVC trees, resource_url support in our API

  21. moved this from Done to Reviewed in 00-AboutCodePlanneron Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions