Repository navigation
Use the SCIO Pipeline Mechanism for v1 Importers/Improvers #1509
Description
We should adopt/reuse the pipeline mechanism in SCIO for Importers and Improvers.
With this, we would have a common programming model across tools, including ScanCode.io and PurlDB
See also:
- VCIO-next: Design new Improver #1395
- RFC: Adopt scancode.io "pipeline" approach for visit/map/collect purldb#521
Migrate the improvers to aboutcode.pipeline:
- vulnerabilities/improvers/valid_versions.py
- vulnerabilities/improvers/vulnerability_kev.py
- vulnerabilities/improvers/vulnerability_status.py
- vulnerabilities/improvers/default.py
merge this as a step in new importer pipeline- Completed in Add base pipeline for importers and migrate PyPa importer to aboutcode pipeline #1559
- This is now available as a
import_new_advisoriesstep inVulnerableCodeBaseImporterPipeline.
Migrate the importers to aboutcode.pipeline:
- nvd Migrate NVD importer to aboutcode pipeline #1587
- github Migrate GitHub importer to aboutcode pipeline #1584
- gitlab Migrate GitLab importer to aboutcode pipeline #1580
- npm Migrate Npm importer to aboutcode pipeline #1574
- pypa Add base pipeline for importers and migrate PyPa importer to aboutcode pipeline #1559
- nginx Migrate Nginx importer to aboutcode pipeline #1575
- pysec Migrate pysec importer to aboutcode pipeline #1627
- alpine_linux VCIO-next: Migrate Alpine Linux importer to aboutcode pipeline #1633
- openssl
- redhat
- debian
- postgresql
- archlinux
- ubuntu
- debian_oval
- retiredotnet
- apache_httpd
- mozilla
- gentoo
- istio
- project_kb_msr2019
- suse_scores
- elixir_security
- apache_tomcat
- xen
- ubuntu_usn
- fireeye
- apache_kafka
- oss_fuzz
- ruby
- github_osv
- epss
Activity
pombredanne commentedon Aug 5, 2024
@keshav-space do you have any commit for this yet?
keshav-space commentedon Aug 5, 2024
@pombredanne here is WIP PR #1533
tdruez commentedon Aug 12, 2024
aboutcode.pipeline 0.1 published to PyPI at https://pypi.org/project/aboutcode.pipeline/
[-]Use the SCIO Pipeline Mechanism for Importers/Improvers[/-][+]VCIO-next: Use the SCIO Pipeline Mechanism for Importers/Improvers[/+][-]VCIO-next: Use the SCIO Pipeline Mechanism for Importers/Improvers[/-][+]Use the SCIO Pipeline Mechanism for Importers/Improvers[/+][-]Use the SCIO Pipeline Mechanism for Importers/Improvers[/-][+]Use the SCIO Pipeline Mechanism for v1 Importers/Improvers[/+]keshav-space commentedon Jan 26, 2026
Superseded by #1881 remaining legacy importer/improver will be migrated to v2 pipeline.