Blocklist for newly created scam, phishing, and malware domains automatically retrieved daily using Google Search API, automated detection, and public databases.
Since the project began, the blocklist has expanded to include not only scam websites but also malware domains.
This blocklist aims to be an alternative to blocking all newly registered domains (NRDs) seeing how many, but not all, NRDs are malicious. This is done by detecting new malicious domains within a short period of their registration date.
For blocking all NRDs, use xRuffKez's NRD Lists.
Sources include:
- Public databases
- Google Search indexing to find common scam site templates
- Detection of common cybersquatting techniques like typosquatting, doppelganger domains, and IDN homograph attacks using dnstwist and URLCrazy
- Domain generation algorithm (DGA) domain detection using DGA Detector
- Regex expression matching for phishing NRDs. See the list of expressions here
A list of all sources can be found in SOURCES.md.
The automated retrieval is done daily at 16:00 UTC.
| Format | Syntax |
|---|---|
| Adblock Plus | ||scam.com^ |
| Wildcard Domains | scam.com |
This blocklist is integrated into Hagezi's Threat Intelligence Feed (full version). For extended protection, please use that list instead.
Total domains: 208581
Light version: 18873
New domains after filtering:
Today | Monthly | %Monthly | %Filtered | Source
7 | 11634 | 7 % | 20 % | 165 Anti-fraud
40 | 1960 | 1 % | 1 % | Cybersquatting
679 | 29818 | 20 % | 0 % | DGA Detector
30 | 2197 | 1 % | 23 % | Emerging Threats
6 | 2540 | 1 % | 19 % | FakeWebshopListHUN
71 | 879 | 0 % | 3 % | Google Search
342 | 21637 | 14 % | 9 % | Jeroengui phishing feed
0 | 159 | 0 % | 8 % | Jeroengui scam feed
1136 | 43920 | 30 % | 25 % | PhishStats
285 | 10595 | 7 % | 0 % | PhishStats (NRDs)
0 | 213 | 0 % | 0 % | PuppyScams.org
610 | 28713 | 19 % | 1 % | Regex Matching
3 | 279 | 0 % | 12 % | aa419.org
0 | 371 | 0 % | 7 % | scam.directory
1 | 48 | 0 % | 32 % | scamadviser.com
0 | 8 | 0 % | 4 % | stopgunscams.com
2925 | 146183 | 100 % | 20 % | All sources
- %Monthly: percentage out of total domains from all sources.
- %Filtered: percentage of dead, whitelisted, and parked domains.
Dead domains removed today: 2744
Resurrected domains added today: 954
Parked domains removed today: 289
Unparked domains added today: 123
Domains over time (days)
Courtesy of iam-py-test/blocklist_stats.
- Domains are filtered against an actively maintained whitelist
- Domains are checked against the Tranco Top Sites Ranking for potential false positives which are then vetted manually
- Common subdomains like 'www' are stripped
- Non-domain entries are removed
- Redundant rules are removed via wildcard matching. For example, 'abc.example.com' is a wildcard match of 'example.com' and, therefore, is redundant and removed. Wildcards are occasionally added to the blocklist manually to further optimize the number of entries
Entries that require manual verification/intervention are notified to the maintainer for fast remediations.
The full filtering process can be viewed in the repository's code.
Dead domains are removed daily using AdGuard's Dead Domains Linter.
Dead domains that are resolving again are included back into the blocklist.
Parked domains are removed daily. A list of common parked domain messages is used to automatically detect these domains. This list can be viewed here: parked_terms.txt.
Parked sites no longer containing any of the parked messages are assumed to be unparked and are included back into the blocklist.
Tip
For list maintainers interested in integrating the parked domains as a source, the list of parked domains can be found here: parked_domains.txt (capped to newest 50000 entries).
For collated blocklists cautious about size, a light version of the blocklist is available in the lists directory. Sources excluded from the light version are marked in SOURCES.md.
Note that dead and parked domains that become alive/unparked are not added back into the light version due to limitations in how these domains are recorded.
A blocklist for NSFW domains is available in Adblock Plus format here: nsfw.txt.
Details
- Domains are automatically retrieved from the Tranco Top Sites Ranking daily
- Dead domains are removed daily
- Note that resurrected domains are not added back
- Note that parked domains are not checked for
This blocklist does not just include adult videos, but also NSFW content of the artistic variety (rule34, illustrations, etc).
- AdGuard's Hostlist Compiler: simple tool that compiles hosts blocklists and removes redundant rules
- Elliotwutingfeng's repositories: various original blocklists
- Google's Shell Style Guide: Shell script style guide
- Grammarly: spelling and grammar checker
- Hagezi's DNS Blocklists: various curated blocklists including threat intelligence feeds
- Jarelllama's Blocklist Checker: generate a simple static report for blocklists or see previous reports of requested blocklists
- ShellCheck: static analysis tool for Shell scripts
- VirusTotal: analyze suspicious files, domains, IPs, and URLs to detect malware (also includes WHOIS lookup)
- iam-py-test/blocklist_stats: statistics on various blocklists