Awesome
Awesome GitHub
Explore your favorite Awesome GitHub repositories via the command-line tool awg!
Awesome Lists
What is Awesome Lists?
Such as famous awesome-go, which is a member of Awesome Lists, where we can quickly find frameworks, libraries, software, and other resources related to Go.
Furthermore, we can find more awesome things from Awesome Lists.
Command-line Tool awg
At the moment, Awesome List usually exhibit lots of GitHub repositories. For example, awesome-go contains thousands of GitHub repositories. However, an Awesome List doesn't include information like the count of stars or the last commit date for those repositories. In many cases, we demand that information and have to manually open links to see the them.
The command-line tool, awg, helps us dig a little deeper into the Awesome List to find out more about all the GitHub repositories on it!
awg will fetch information of GitHub repositories listed on an Awesome List, and output the data to a file of your choice. You can use awg to generate a browser page to view the data later, or use your favorite tools like jq or python to analyze.
The final output:
{
"data": {
"Command Line": [
{
"id": {
"owner": "urfave",
"name": "cli"
},
"owner": "urfave",
"awesome_name": "urfave/cli",
"link": "https://github.com/urfave/cli",
"watch": 295,
"star": 14171,
"fork": 1134,
"last_commit": "2020-07-12T13:32:01Z",
"description": "A simple, fast, and fun package for building command line apps in Go",
"awesome_description": "urfave/cli - Simple, fast, and fun package for building command line apps in Go (formerly codegangsta/cli)."
},
// ...
]
// ...
}
}
Data Analysis
We can use any tool to analyze the obtained data file.
For example, after getting data file awg.json
, which is concerned with awesome-go
View in Browser
The page effect is shown below
By runing this command
awg view awg.json
It will be running a simple web server locally,
listening at 127.0.0.1:3000
by default.
Open this page with your browser to view.
Can use --listen
to specify other address.
Note: This does not mean that it can be viewed offline. An Internet connection is necessary.
To replace the implementation for viewing,
you can replace the embedded JS script address via --script
,
which supports local path.
View in Terminal
By using the popular command line tool jq, we can:
View Command Line section of awesome-go, sorted by the count of stars
cat awg.json | jq '.data | ."Command Line" | sort_by(.star)'
Installation
Get command-line tool awg
go get github.com/rydesun/awesome-github/cmd/awg
Fetch Data
First prepare the following before running awg:
- GitHub personal access token
- awg configuration file
Access Token
awg fetch information of GitHub repository by calling the GitHub GraphQL API. This official API requires your personal access token to be verified before you can use it. So, you need to provide awg with a GitHub personal access token.
If you do not have the token, please view the article Creating a personal access token。
awg does not need the token to have any scopes or permissions. Therefore, do not grant the token any scopes or permissions.
Configuration
A configuration file is necessary.
See configuration file template
in the directory configs
as a reference.
awg will read the personal access token from the
environment variable GITHUB_ACCESS_TOKEN
first.
So it is not necessary to store this value in the configuration file.
Increasing the number of concurrent query requests will increase the speed of the query, but the number should not be too large (the current recommended value is 3), otherwise this behavior will be viewed as abusing, and then blocked.
All relative paths in a configuration file are relative to awg's current working directory, not to the directory where the configuration file is located. Absolute paths are encouraged.
Run
Fetch JSON data
awg fetch --config path/to/config.yaml
(Recommended) Specify the GitHub Personal Access Token from an environment variable
GITHUB_ACCESS_TOKEN=<Your Token> awg fetch --config path/to/config.yaml
Note the rate limit, which is not identical to the number of concurrent requests. Currently awg can query up to 5000 GitHub repositories per hour. If there are too many queries, GitHub will impose limits on requests and responses. See GitHub Resource limitations for more.
Notes
This project is currently only testing awesome-go's lists, other testing for Awesome List results are pending.
awg does not support Windows platforms.