Home

Awesome

git-all-secrets

About

git-all-secrets is a tool that can:

Scanning is done by multiple open source tools such as:

NOTE - More such tools can be added in future, if desired! NOTE - Scanning can be done by all the tools or any one of them by specifying the toolName flag.

If all the tools are used to scan, the final output from the tool combines the output from all files from all the tools into one consolidated output file.

Getting started

The easiest way to run git-all-secrets is via Docker and I highly recommend installing Docker if you don't already have it. Once you have Docker installed,

Flags/Options

Note

Scanning Private Repositories

The most secure way to scan private repositories is to clone using the SSH URLs. To accomplish this, one needs to place an appropriate SSH key which has been added to a Github User. Github has helpful documentation for configuring your account. Make sure this key does not have any passphrase set on it. Once you have the SSH key, simply mount it to the Docker container via a volume. It is as simple as typing the below commands:

docker run -it -v ~/.ssh/id_rsa_personal:/root/.ssh/id_rsa abhartiya/tools_gitallsecrets -token=<> -user=<> -scanPrivateReposOnly

OR

docker run -it -v ~/.ssh/id_rsa_personal:/root/.ssh/id_rsa abhartiya/tools_gitallsecrets -token=<> -repoURL=<> -scanPrivateReposOnly

Here, I am mapping my personal SSH key id_rsa_personal stored locally to /root/.ssh/id_rsa inside the container so that git-all-secrets will try to clone the repo via ssh and will use the SSH key stored at /root/.ssh/id_rsa inside the container. This way, you are not really storing anything sensitive inside the container. You are just using a file from your local machine. Once the container is destroyed, it no longer has access to this key.

Scanning an Organization Team

The Github API limits the circumstances where a private repository is reported. If one is trying to scan an Organization with a user which is not an admin, you may need to provide the team which provides repository access to the user. In order to do this, use the teamName flag along with the org flag. Example is below:

docker run --it -v ~/.ssh/id_rsa_personal:/root/.ssh/id_rsa abhartiya/tools_gitallsecrets -token=<> -org=<> -teamName <>

Scanning Github Enterprise

git-all-secrets now supports scanning Github Enterprise as well. If you have your own Github Enterprise hosted behind a VPN or something, make sure you are connected on the VPN or on the correct network that has access to the Github Enterprise repos. The enterpriseURL is what you'd need to scan your Github Enterprise repos. Below are some examples:

Example 1:

docker run -it -v ~/.ssh/id_rsa_gitenterprise:/root/.ssh/id_rsa -token <token> -enterpriseURL https://github.<org>.com/api/v3 -repoURL https://github.<org>.com/<user>/<repo>.git

Here, I am now mounting my github enterprise SSH key onto the container, followed by my personal access token, the enterprise URL to which the requests will be sent and the repo I want to scan.

Example 2:

docker run -it -v ~/.ssh/id_rsa_gitenterprise:/root/.ssh/id_rsa -token <token> -enterpriseURL https://github.<org>.com/api/v3 -repoURL https://github.<org>.com/<user>/<repo>.git -toolName thog -thogEntropy

Above, I am now just running truffleHog against the repository with the Entropy settings.

Example 3:

docker run -it -v ~/.ssh/id_rsa_gitenterprise:/root/.ssh/id_rsa -token <token> -enterpriseURL https://github.<org>.com/api/v3 -user <username> -scanPrivateReposOnly

Above, I am scanning only the private repositories of the user whose token is provided with all the tools (repo-supevisor and thog), but without the entropy setting of truffleHog.

TODO

Details

Features

Motivation

I looked at a large number of open source tools that could be potentially used to look for secrets in github repositories. Some of the top tools that I thought were good are: gitrob, truffleHog and git-secrets.

Gitrob is meant to be a standalone tool that is pretty difficult to integrate with other tools because it has its own database and UI to see all the secrets discovered. It also produces a ton of false positives, more than truffleHog. And, it doesn't really highlight the secrets discovered. It just looks at the files and their extensions, not the actual content. So, although Gitrob is a great tool to get started with, I would recommend running it every once in a while to understand what the attack surface looks like and see if it has changed.

Then, there is truffleHog that looks for secrets in the actual contents of the file by looking at Shannon's entropy and prints the output on the screen. It takes in a repository URL or a repository directory as an argument. This is a pretty good tool although it does have its share of false positives. Some of the other drawbacks are:

Finally, there is git-secrets which can flag things like AWS secrets. The best part is that you can add your own regular expressions as well for secrets that you know it should be looking for. A major drawback is that it doesn't do a good job on finding high entropy strings like truffleHog does. You can also only scan a particular directory that is a repository so no recursion scanning from a directory of repositories either.

So, as you can see, there are decent tools out there, but they had to be combined somehow. There was also a need to recursively scan multiple repositories and not just one. And, what about gists? There are organizations and users. Then, there are repositories for organizations and users. There are also gists by users. All of these should be scanned. And, scanned such that it could be automated and easily consumed by other tools/frameworks.

Changelog

Donate

If you want to show some love, my BTC wallet address is 1PtMhXWCcMZCitcDfaEBe7jnV9sjKoNvq7.