Search the Community
Showing results for tags 'url-crawler'.
-
URL-CRAWLER V1.0.0 URL-CRAWLER is a Python script that extracts all third-party links from a given domain. It uses the BeautifulSoup library to parse the HTML and filter out links that belong to the same domain as the original domain. # Extract links from a single domain and display them in the console python urlcrawler.py -d example.com # Extract links from a single domain and save them to a file python urlcrawler.py -d example.com -o links.txt # Extract links from multiple domains and save them to a file python urlcrawler.py -t -o links.txt # Extract links from a single domain and display their status codes in the console python urlcrawler.py -d example.com -s #For more detailed usage instructions, run the script with the -h or --help option. [Hidden Content]