itsMe Posted November 13, 2023 Share Posted November 13, 2023 This is the hidden content, please Sign In or Sign Up Crawlector (the name Crawlector is a combination of Crawler & Detector) is a threat-hunting framework designed for scanning websites for malicious objects. Features Supports spidering websites for finding additional links for scanning (up to 2 levels only) Integrates Yara as a backend engine for rule scanning Supports online and offline scanning Supports crawling for domains/sites' digital certificate Supports querying URLhaus for finding malicious URLs on the page Deep Object Extraction (DOE) Slack Alert Notification Parametrized support for HTTP redirection Retreiving Whois information Supports hashing the page's content with TLSH (Trend Micro Locality Sensitive Hash), and other standard cryptographic hash functions such as md5, sha1, sha256, and ripemd128, among others TLSH won't return a value if the page size is less than 50 bytes or not "enough amount of randomness" is present in the data Supports querying the rating and category of every URL Supports expanding on a given site, by attempting to find all available TLDs and/or subdomains for the same domain This feature uses the Omnisint Labs API (this site is down as of March 10, 2023) and RapidAPI APIs TLD expansion implementation is native This feature along with the rating and categorization, provides the capability to find scam/phishing/malicious domains for the original domain Supports domain resolution (IPv4 and IPv6) Saves scanned website pages for later scanning (can be saved as a zip compressed) The entirety of the framework’s settings is controlled via a single customizable configuration file All scanning sessions are saved into a well-structured CSV file with a plethora of information about the website being scanned, in addition to information about the Yara rules that have triggered Many other features... All HTTP(S) communications are proxy-aware One executable Written in C++ This is the hidden content, please Sign In or Sign Up Link to comment Share on other sites More sharing options...
Recommended Posts