Urlhunter – A Recon Tool That Allows Searching On URLs That Are Exposed Via Shortener Services
urlhunter is a recon device that will allow seeking on URLs that are uncovered through shortener products and services these kinds of as bit.ly and goo.gl. The challenge is penned in Go.
How?
A team named URLTeam (kudos to them) are brute forcing the URL shortener companies and publishing matched success on a daily basis. urlhunter downloads their collections and lets you assess them.
Installation
From Binary
You can obtain the pre-developed binaries from the releases page and operate. For example:
tar xzvf urlhunter_.1._Linux_amd64.tar.gz
./urlhunter --assistance
From Supply
-
Put in Go on your program
-
Operate:
go get -u github.com/utkusen/urlhunter
Note For The Home windows People: urlhunter makes use of XZ Utils which is pre-mounted on Linux and macOS programs. For Windows devices, you require to download it from https://tukaani.org/xz/
Utilization
urlhunter demands 3 parameters to operate: -keyword phrases, -date and -o.
For illustration: urlhunter -keyword phrases search phrases.txt -date 2020-11-20 -o out.txt
-keywords and phrases
You require to specify the txt file that contains key phrases to research on URLs. Search phrases need to be written line by line. You have three various approaches to specify key terms:
One Search term: urlhunter will look for the given key word as a substring. For illustration:
acme.com keyword will equally match https://acme.com/blabla and https://another.com/?referrer=acme.com
Several Keywords: urlhunter will lookup the provided key terms with an AND logic. Which implies, a URL should incorporate all the supplied key phrases. Key terms ought to be divided with , character. For example:
acme.com,admin will match https://acme.com/secret/adminpanel but will not likely match https://acme.com/somethingelse
Regex Values: urlhunter will research for the specified regex benefit. In the keyword file, the line that incorporates a regular expression formulation ought to commence with regex string. The format is: regex REGEXFORMULA. For example:
regex 1d10 will match https://case in point.com/index.php?id=12938454312 but won’t match https://instance.com/index.php?id=abc223
-day
urlhunter downloads the archive data files of the provided day(s). You have a few various methods to specify the day:
Most up-to-date: urlhunter will down load the most up-to-date archive. -date most up-to-date
One Day: urlhunter will down load the archive of the supplied date. Date format is YYYY-MM-DD.
For instance: -day 2020-11-20
Day Selection: urlhunter will download all the archives among provided get started and conclusion dates.
For instance: -day 2020-11-10:2020-11-20
-o
You can specify the output file with -o parameter. For instance -o out.txt
Demonstration Movie
The Speed Trouble
Archive.org throttles the pace when downloading information. Consequently, downloading an archive takes extra time than typical. As a workaround, you can down load the archives via Torrent and set them under the archive/ folder which is located in the similar directory with the urlhunter’s binary. The directory tree will glimpse like:
|-urlhunter
|---urlhunter(binary)
|---archive
|-----urlteam_2020-11-20-11-17-04
|-----urlteam_2020-11-17-11-17-04
Case in point Use Conditions
urlhunter could possibly be helpful for cyber intelligence and bug bounty uses. For instance:
docs.google.com/a/acme.com drive.google.com/a/acme.com key phrases permit you to come across community Google Docs&Drive share inbound links of Acme enterprise.
acme.com,password_reset_token search phrase may make it possible for you to find the performing password reset tokens of acme.com
trello.com lets you to come across general public Trello addresses.
Thanks
Special many thanks to Samet(@cyanpiny) who gave me the notion of this tool.
