SecretScraper is a highly configurable web scraping tool that crawls links from target websites and extracts sensitive data using regular expressions.
Instructions for use: Python version >= 3.9
Features:
Web scraping: Extract links through DOM hierarchy and regular expressions.
Support for domain whitelist and blacklist.
Support for multiple targets, input target URLs from a file.
Support for local file scanning and customizable extensions: headers, proxies, timeouts, cookies, crawl depth, follow redirects, etc.
Built-in regular expressions for searching sensitive information, flexibly configured in yaml format.
Installation:
pip install secretscraper
Update:
pip install --upgrade secretscraper
Single target:
secretscraper -u https://xxxxxxx.com/