Commoncrawl mirror

As part of our data processing pipeline, we extensively use Common Crawl data. It would be highly beneficial to have an option to download this data directly from your cloud platform. This feature would help us avoid the costs associated with external traffic billing