Awesome

<br/>A no-stone-unturned Ansible playbook you can use to set up the ultimate home lab or on-premise addition to your cloud!<br/><br/>

<a href="#table-of-contents" style="width:100%"><img style="width:100%" src="https://gitlab.com/megabyte-labs/assets/-/raw/master/png/aqua-divider.png" /></a>

Table of Contents

• Introduction
• Quick Start
  • Vagrant (Recommended)
  • macOS/Linux
  • Windows
• Supported Operating Systems
• Requirements
  • Host Requirements
    • Easier Method of Installing the Host Requirements
  • Operating System
  • Connection
  • MAS on Mac OS X
• Software
  • Binaries
  • NPM Packages
  • Python Packages
  • Ruby Gems
  • Visual Studio Code Extensions
  • Chrome Extensions
  • Homebrew Formulae (macOS and Linux only)
  • Homebrew Casks (macOS only)
  • Go, Rust, and System-Specific Packages
• Web Applications
  • Helm Charts
  • Host Applications
    • HTPC
• Philosophy
• Architecture
• Managing Environments
  • Switching Between Environments
• Contributing
• License

<a href="#introduction" style="width:100%"><img style="width:100%" src="https://gitlab.com/megabyte-labs/assets/-/raw/master/png/aqua-divider.png" /></a>

Introduction

Welcome to a new way of doing things. Born out of complete paranoia and a relentless pursuit of the best of GitHub Awesome lists, Gas Station aims to add the capability of being able to completely wipe whole networks and restore them on a regular basis. It takes a unique approach to network provisioning because it supports desktop provisioning as a first-class citizen. By default, without much configuration, it is meant to provision and maintain the state of a network that includes development workstations and servers. One type of user that might benefit from this project is a web developer who wants to start saving the state of their desktop as code. Another type of user is one who wants to start hosting RAM-intensive web applications in their home-lab environment to save huge amounts on cloud costs. This project is also meant to be maintainable by a single person. Granted, if you look through our eco-system you will see we are well-equipped for supporting entire teams as well.

Gas Station a collection of Ansible playbooks, configurations, scripts, and roles meant to provision computers and networks with the "best of GitHub". By leveraging Ansible, you can provision your whole network relatively fast in the event of a disaster or scheduled network reset. This project is also intended to increase the security of your network by allowing you to frequently wipe, reinstall, and re-provision your network, bringing it back to its original state. This is done by backing up container storage volumes (like database files and Docker volumes) to encrypted S3 buckets, storing configurations in encrypted git repositories, and leveraging GitHub-sourced power tools to make the job easy-peasy.

This project started when a certain somebody changed their desktop wallpaper to an cute picture of a cat 🐱 when, all of a sudden, their computer meowed. Well, it actually started before that but no one believes someone who claims that time travelers hacked them on a regular basis. Tip: If you are stuck in spiritual darkness involving time travelers, save yourself some headaches by adopting an other-people first mentality that may include volunteering, tithing, and surrendering to Jesus Christ. Anyway, enough preaching!

Gas Station is:

• Highly configurable - most roles come with optional variables that you can configure to change the behavior of the role
• Highly configured - in-depth research is done to ensure each software component is configured with bash completions, plugins that are well-received by the community, and integrated with other software used in the playbook
• Compatible with all major operating systems (i.e. Windows, Mac OS X, Ubuntu, Fedora, CentOS, Debian, and even Archlinux)
• The product of a team of experts
• An amazing way to learn about developer tools that many would consider to be "the best of GitHub"
• Open to new ideas - feel free to open an issue or contribute with a pull request!

<a href="#quick-start" style="width:100%"><img style="width:100%" src="https://gitlab.com/megabyte-labs/assets/-/raw/master/png/aqua-divider.png" /></a>

Quick Start

The easiest way to run the entire playbook, outlined in the main.yml file, is to run the appropriate command listed below. These commands will run the playbook on the machine you run the command on. This is probably the best way to get your feet wet before you decide to give us a ⭐ and customize the playbook for your own needs. Ideally, this command should be run on the machine that you plan on running Ansible with to provision the other computers on your network. It is only guaranteed to work on fresh installs so testing it out with Vagrant is highly encouraged.

Vagrant (Recommended)

To test it out with Vagrant, you can run the following commands which will open up an interactive dialog where you can pick which operating system and virtualization provider you wish to test the installation with:

bash .config/scripts/start.sh # Only required if you do not have the dependencies (i.e. Task) already installed
task ansible:test:vagrant

macOS/Linux

curl -sS https://gitlab.com/megabyte-labs/gas-station/-/raw/master/files/quickstart.sh | bash

Windows

In an administrative PowerShell session, run:

Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://gitlab.com/megabyte-labs/gas-station/-/raw/master/files/quickstart.ps1'))

<a href="#supported-operating-systems" style="width:100%"><img style="width:100%" src="https://gitlab.com/megabyte-labs/assets/-/raw/master/png/aqua-divider.png" /></a>

Supported Operating Systems

The following chart shows the operating systems that have been tested for compatibility using the environments/dev/ environment. This chart is automatically generated using the Ansible Molecule tests you can view in the molecule/default/ folder. We currently have logic in place to automatically handle the testing of Windows, Mac OS X, Ubuntu, Fedora, CentOS, Debian, and Archlinux. If your operating system is not listed but is a variant of one of the systems we test (i.e. a Debian-flavored system or a RedHat-flavored system) then it might still work.

compatibility_matrix

<a href="#requirements" style="width:100%"><img style="width:100%" src="https://gitlab.com/megabyte-labs/assets/-/raw/master/png/aqua-divider.png" /></a>

Requirements

• Python >=3.7
• Ansible >=2.9
• Ansible controller should be a macOS/Linux environment (WSL/Docker can be used on Windows)

Host Requirements

There are Python and Ansible package requirements need to be installed by running the following command (or equivalent) in the root of this repository:

if type poetry &> /dev/null; then poetry install --no-root; else pip3 install -r .config/requirements.txt; fi
ansible-galaxy install requirements.yml

Easier Method of Installing the Host Requirements

You can also run bash .config/scripts/start.sh if you do not mind development dependencies being installed as well. This method will even handle installing Python 3 and Ansible.

Operating System

This playbook is built and tested to run on fresh installs of Windows, Mac OS X, Ubuntu, Fedora, Debian, CentOS, and Archlinux. It may still be possible to run the playbook on your current machine. However, installing the playbook on a fresh install is the only thing we actively support. That said, if you come across an issue with an environment that already has configurations and software present, please do not hesitate to open an issue.

Connection

SSH (or WinRM in the case of Windows) and Python should be available on the target systems you would like to provision. If you are attempting to provision a Windows machine, you can ensure that WinRM is enabled and configured so that you can remotely provision the Windows target by running the following command with PowerShell:

Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://gitlab.com/megabyte-labs/gas-station/-/raw/master/files/client.ps1'))

MAS on Mac OS X

We use mas to install apps from the App Store in some of our roles. Sadly, automatically signing into the App Store is not possible on OS X 10.13+ via mas. This is because mas no longer supports login functionality on OS X 10.13+.

There is another caveat with mas. In order to install an application using mas, the application has to have already been added via the App Store GUI. This means that the first time around you will have to install the apps via the App Store GUI so they are associated with your App Store account.

<a href="#software" style="width:100%"><img style="width:100%" src="https://gitlab.com/megabyte-labs/assets/-/raw/master/png/aqua-divider.png" /></a>

Software

This project breaks down software into a role (found in the subdirectories of the roles/ folder) if the software requires anything other than being added to the PATH variable. Below is a quick description of what each role does. Browsing through this list, along with the conditions laid out in main.yml, you will be able to get a better picture of what software will be installed by the default main.yml playbook.

role_descriptions

We encourage you to browse through the repositories that are linked to in the table above to learn about the configuration options they support.

Binaries

A lot of nifty software does not require any configuration other than being added to the PATH or being installed with an installer like brew. For this kind of software that requires no configuration, we list the software we would like installed by the playbook as a variable in group_vars/ or host_vars/ as an array of keys assigned to the software variable (example here). With those keys, we install the software using the [professormanhattan.genericinstaller](https://galaxy.ansible.com/professormanhattan/genericinstaller) role which determines how to install the binaries by looking up the keys against the software_package object (example here). For your convienience, the software we recommend and install by default is listed below:

binary_var_chart

NPM Packages

NPM provides a huge catalog of useful CLIs and libraries so we also include a useful and interesting default set of NPM-hosted CLIs for hosts in the desktop group (defined here, for example):

npm_var_chart

Python Packages

In a similar fashion to the NPM packages, we include a great set of default Python packages that are included by default for the desktop group (defined here):

pypi_var_chart

Ruby Gems

A handful of Ruby gems are also installed on targets in the desktop group (defined here):

gem_var_chart

Visual Studio Code Extensions

A considerable amount of effort has gone into researching and finding the "best" VS Code extensions. They are defined here and Gas Station also installs a good baseline configuration which includes settings for these extensions:

vscode_var_chart

Chrome Extensions

To reduce the amount of time it takes to configure Chromium-based browsers like Brave, Chromium, and Chrome, we also include the capability of automatically installing Chromium-based browser extensions (via a variable defined here):

chrome_var_chart

Homebrew Formulae (macOS and Linux only)

Although most of the brew installs are handled by the Binaries installer, some brew packages are also installed using this configuration. The default Homebrew formulae include:

brew_var_chart

Homebrew Casks (macOS only)

On macOS, some software is installed using Homebrew casks. These include:

cask_var_chart

Go, Rust, and System-Specific Packages

Go packages, Rust crates, and system-specific packages like .deb and .rpm bundles are all handled by the [professormanhattan.genericinstaller](https://galaxy.ansible.com/professormanhattan/genericinstaller) role described above in the Binaries section. There are also ways of installing Go and Rust packages directly by using configuration options provided by their corresponding roles outlined in the Roles section.

<a href="#web-applications" style="width:100%"><img style="width:100%" src="https://gitlab.com/megabyte-labs/assets/-/raw/master/png/aqua-divider.png" /></a>

Web Applications

This playbook does a bit more than just install software. It also optionally sets up web applications too. If you choose to deploy the default Gas Station web applications on your network, you should probably do it on a computer/server that has a lot of RAM (e.g. 64GB+).

Although a production environment will always be more stable and performant if it is hosted with a major cloud provider, sometimes it makes more sense to locally host web applications. Some applications have abnormally large RAM requirements that could potentially cost thousands per month to host with a legit cloud provider.

We use Kubernetes as the provider for the majority of the applications. It is a production-grade system and although there is a steeper learning curve it is well worth it. Each application we install is packaged as a Helm chart. All of the data is backed up regularly to an encrypted cloud S3 bucket of your choice.

Helm Charts

The available Helm charts that this playbook completely handles the set up for are listed below.

helm_var_chart

Host Applications

By default, on each computer provisioned using the default settings of Gas Station, several apps are installed on each host. Docker Compose is used to manage the deployment. The default apps include:

You can, of course, disable deploying these apps. However, we include them because they have a small footprint and include useful features. You can also customize the list of apps you wish to include on each host.

HTPC

We do not maintain any of the host applications except the ones listed above. However, we do provide the capability of marking a computer being provisioned as an HTPC. Doing this will include a suite of web applications with powerful auto-downloading, organizing, tagging, and media-serving capabilities. Since most people will probably be stepping outside the confines of the law for this, it is not recommended. If you still want to experiment then you can find descriptions of the applications below. The applications are intended to be hosted on a single computer via Docker Compose. The backend for Kodi is included but you should still use the regular installation method for Plex and the front-end of Kodi to view your media collection.

<a href="#philosophy" style="width:100%"><img style="width:100%" src="https://gitlab.com/megabyte-labs/assets/-/raw/master/png/aqua-divider.png" /></a>

Philosophy

The philosophy of this project basically boils down to "automate everything" and include the best development tools that might be useful without over-bloating the machine with services. Automating everything should include tasks like automatically accepting software terms in advance or pre-populating Portainer with certificates of all the Docker hosts you would like to control. One problem we face is that there are so many great tools offered on GitHub. A lot of research has to go into what to include and what to pass on. The decision of whether or not to include a piece of software in the default playbook basically boils down to:

• Project popularity - If one project has 10k stars and a similar alternative has 500 stars then 9 times of out 10 the more popular project is selected.
• Last commit date - We prefer software that is being actively maintained, for obvious reasons.
• Cross platform - Our playbook supports the majority of popular operating systems so we opt for cross-platform software. However, in some cases, we will include software that has limited cross-platform support like Xcode (which is only available on Mac OS X). If a piece of software is too good to pass up, it is added and only installed on the system(s) that support it.
• Usefulness - If a tool could potentially improve developer effectiveness then we are more likely to include it.
• System Impact - Software that can be run with a small RAM footprint and software that does not need a service to load on boot is much more likely to be included.

One of the goals of this project is to be able to re-provision a network with the click of a button. This might not be feasible since consumer-grade hardware usually does not include features like IPMI (which is a feature included in high-end motherboards that lets you control the power state remotely). However, we aim to reduce the amount of interaction required when re-provisioning an entire network down to the bare minimum. In the worst case scenario, you will have to reformat, reinstall the operating system, and ensure that OpenSSH is running (or WinRM in the case of Windows) on each of the computers in your network. However, the long term goal is to allow the user to reformat and reinstall the operating system used as your Ansible host using an automated USB installer and then automatically re-provision everything else on the network by utilizing IPMI.

You might ask, "But how can I retain application-level configurations?" We currently handle this by:

• Pre-defining dotfiles in a customizable Git repository
• Backing up to encrypted S3 buckets
• Syncing files to private git repositories
• Utilizing tools that synchronize settings like mackup or macprefs in the case of macOS

However, we intentionally keep this synchronization to a minimum (i.e. only back up what is necessary). After all, one of the goals of this project is to be able to regularly flush the bad stuff off a system. By keeping what we back up to a minimum, we reduce the attack surface.

<a href="#architecture" style="width:100%"><img style="width:100%" src="https://gitlab.com/megabyte-labs/assets/-/raw/master/png/aqua-divider.png" /></a>

Architecture

You can find a high-level overview of what each folder and file does in the ARCHITECTURE.md file.

<a href="#managing-environments" style="width:100%"><img style="width:100%" src="https://gitlab.com/megabyte-labs/assets/-/raw/master/png/aqua-divider.png" /></a>

Managing Environments

We accomplish managing different environments by symlinking all the folders that should be unique to each network environment (e.g. host_vars/, group_vars/, inventories/, files/vpn/, and files/ssh/). In the environments/ folder, you will see multiple folders. In our case, environments/dev/ contains sensible configurations for testing the playbook and its' roles. The production environment is a seperate git submodule that links to a private git repository that contains our Ansible-vaulted API keys and passwords. When you are ready to set up your production configurations, you can use this method of storing your environment-specific folders in the environments/ folder as well. But if you are just starting off, you do not have to worry about this since, by default, this playbook is configured to run with the settings included in the /environments/dev/ folder.

Switching Between Environments

If you already have the project bootstrapped (i.e. already ran bash .config/scripts/start.sh), you can switch environments with an interactive prompt by running:

task ansible:playbook:environment

Alternatively, you can run the following if you would like to bypass the prompt:

task ansible:playbook:environment -- environmentName

<a href="#contributing" style="width:100%"><img style="width:100%" src="https://gitlab.com/megabyte-labs/assets/-/raw/master/png/aqua-divider.png" /></a>

Contributing

Contributions, issues, and feature requests are welcome! Feel free to check the issues page. If you would like to contribute, please take a look at the contributing guide.

Brian Zalewski<br/><br/>

<a href="#license" style="width:100%"><img style="width:100%" src="https://gitlab.com/megabyte-labs/assets/-/raw/master/png/aqua-divider.png" /></a>

License

Copyright © 2020-2021 Megabyte LLC. This project is MIT licensed.