Awesome

<br /> <div align="center"> <a href="https://raw.githubusercontent.com/iudicium/pryingdeep/main/web/static/logo.png"> <img src="web/static/logo.png" alt="Logo" width="350" height="200"> </a> <h3 align="center">Prying Deep</h3> <p align="center"> <b>OSINT intellegence gathering tool for the dark/clear web <br /> <br /> <a href="https://github.com/iudicium/pryingdeep/issues">Report Bug</a> <a href="https://github.com/iudicium/pryingdeep/issues">Request Feature</a> </p> </div>  <details> <summary>Table of Contents</summary> <ol> <li> <a href="#about-the-project">About The Project</a> </li> <li> <a href="#getting-started">Getting Started</a> <ul> <li><a href="#prerequisites">Prerequisites</a></li> <li><a href="#manual-installation">Manual Installation</a></li> <li><a href="#binary-installation">Binary Installation</a></li> </ul> </li> <li> <a href="#docker">Docker</a> <ul> <li><a href="#tor">Tor</a></li> </ul> </li> <li><a href="#usage">Usage</a></li> <li><a href="#roadmap">Roadmap</a></li> <li><a href="#contributing">Contributing</a></li> <li><a href="#license">License</a></li> <li><a href="#preview">Preview</a></li> </ol> </details>

Translation (Перевод)

Click

IMPORTANT

This repo is no longer active, sorry guys, I no longer want to keep working on this :)

About The Project

I've seen multiple projects out there in GitHub, that are crawlers for the deep web, but most of them did not meet my standards of OSINT on the deep web. So I decided to create my own deep web OSINT tool.

This tool serves as a reminder that the best practices of OPSEC should always be followed in the deep web.

<u>The author of this project is not responsible for any possible harm caused by the usage of this tool.</u>

What does this tool do?

Prying Deep crawls dark/clear net platforms and extracts as much intelligence as possible effectively.

Getting Started

Prerequisites

Before you can use our OSINT tool, please ensure you have the following dependencies installed:

Docker: (optional)
- You can download and install Docker by following the official installation instructions for your specific operating system:
- Docker Installation Guide.
Go: (required)
- Golang Installation Guide.
PostgresSQL: (required if you don't use docker)
- Make sure your pryingdeep.yaml environment variables match the environment in docker-compose.yaml
- PostgreSQL Installation

Binary Installation

Install the binary via:

go install -v github.com/iudicium/pryingdeep/cmd/pryingdeep@latest

Run the following command:

pryingdeep install

Adjust the values inside the config folder to your needs.

Manual Installation

Clone the repo:

 git clone https://github.com/iudicium/pryingdeep.git

Adjust the values in the .yaml configuration either through flags or manually. Database, logger, tor all require manual configuration. <br>

<u>You will need to read</u> Colly Docs. Also, refer to Config Explanation

Build the binary via:
- go build -> inside the cmd/pryingdeep directory
- go build cmd/pryingdeep/pryingdeep.go -> root directory, binary will also be there.

Docker

To start run pryingdeep inside a docker container use this command:

docker-compose up

Config

Tor

Read more about building and running our tor container here: Tor

Usage

Pryingdeep specializes in collecting information about dark-web/clearnet websites.
This tool was specifically built to extract as much information as possible from a .onion website

Usage:
pryingdeep [command]

Available Commands:
completion  Generate the autocompletion script for the specified shell
crawl       Start the crawling process
export      Export the collected data into a file.
help        Help about any command
install     Installation of config files

Flags:
-c, --config string   Path to the .yaml configuration. (default "pryingdeep.yaml")
-h, --help            help for pryingdeep
-z, --save-config     Save chosen options to your .yaml configuration
-s, --silent          -s to disable logging and run silently