Ancestors

Toot

Written by Jörn Franke on 2024-11-16 at 21:12

released a new version of scrapy-contrib-bigexporter (https://github.com/ZuInnoTe/scrapy-contrib-bigexporters/releases/tag/scb-0.5.0) for exporting web crawled data with scrapy to big data formats, such as parquet, orc and avro.

now also with provenance using pypi trusted publisher (https://blog.trailofbits.com/2024/11/14/attestations-a-new-generation-of-signatures-on-pypi/). You can be now ensure which CI job build the artifact and how it was built.

It uses short-living token to publish to pypi and not static credentials.

See the provenance e.g. for the *.whl on pypi: https://pypi.org/project/scrapy-contrib-bigexporters/#scrapy_contrib_bigexporters-0.5.0-py3-none-any.whl

[#]pypi

=> More informations about this toot | More toots from jornfranke@mastodon.online

Descendants