Robots Disallowed Top 10000
Paths harvested from Disallow directives across the most popular sites' robots.txt files, ranked by frequency. These paths often point to sensitive or interesting content site owners intended to hide.
- Size
- 195,167 lines (6.1 MB)
- Category
- Web Content
- Source
- danielmiessler/SecLists
- License
- MIT
- Recommended tools
- ffufgobusterferoxbusterdirsearch
Preview
First 15 entries. Download or copy the full list (195,167 lines (6.1 MB)) using the buttons above.
-132817115$ -126190193$ -123616669$ -117784433$ -116942273$ -97578199$ -95659321$ -90176551$ -81069985$ -78772737$ -77789465$ -70966931$ -70430875$ -66093527$ -65817689$
Paths harvested from Disallow directives across the most popular sites' robots.txt files, ranked by frequency. These paths often point to sensitive or interesting content site owners intended to hide.
This list is geared toward web content and directory discovery. It contains roughly 195,167 lines (6.1 MB) and pairs well with tools such as ffuf, gobuster, feroxbuster, dirsearch. Pick the smallest list that fits your engagement: shorter lists are faster and quieter for online attacks, while larger lists give broader coverage for offline work where speed is less of a constraint.
You can copy the sample preview straight from this page, copy the entire list to
your clipboard, or download the raw .txt file. The full list is served directly
from its upstream source on GitHub.
Sourced from danielmiessler/SecLists and distributed under MIT. Only use wordlists against systems you are explicitly authorized to test.