Home

Awesome

<img alt="NGS logo - cloud and UNIX shell icon" align="right" src="img/ngs-logo-300.png" />

Discord Commits in dev since last release Build Status Docker Build Status Homebrew snapcraft license

Next Generation Shell is a modern programming language for DevOps.

The Problem

bash or Python? That's a square pegs for a round hole situation. Both are inadequate for Ops tasks. NGS aims to reduce your frustration and let you be more productive. There is no reason that you should be in this situation. (Python here represents also other general purpose programming languages).

bash does not meet any modern expectations for syntax, error handling nor has ability to work with structured data (beyond arrays and associative arrays which can not be nested). Let it go. You are not usually coding in assembly, FORTRAN, C, or C++, do you? They just don’t match the typical Ops tasks. Don’t make your life harder than it should be. Let it go. (Let’s not make it a blanket statement. Use your own judgement when to make an exception).

Python along with many other languages are general purpose programming languages which were not intended to solve specifically Ops problems. The consequence is longer and less readable scripts when dealing with files or running external programs, which are both pretty common for Ops.

See bash or Python? The Square Pegs and a Round Hole Situation blog post.

Shell user interface, the atrocity. Current shells as well as proposed alternatives treat UI as if nothing happened since the 70-s: mostly typing commands and getting some text back. Watch Unix shell - We can do better now 15 minutes video about what's wrong with the user interface and how NGS is fixing (WIP) that.

Suggested Solution - NGS

I like helping people. Reducing frustration and making people more productive just feels good. A scalable way to help is thorugh tools. That is why I created NGS. It is a programming language with typical Ops tasks in mind. See the use cases below. The next big planned part is the interactive shell.

NGS Use Cases

You as the Ops person, probably wondering whether NGS would be a good fit for your tasks. NGS aims to be best fit for the use cases below.

See Use Cases wiki page for more information about the use cases and examples.

Project Status

The language is very useful. See the the bin folder for examples. NGS is used in Beame.io for miscellaneous scripting such as testing CLI tools, performance tests orchestration, cloud manipulation, etc.

The work on the shell has just started. It is not usable yet. The shell is being implemented in NGS. See the design document.

From George Nachman, creator of iTerm2:

Neat! This is totally the project I would do if I had unlimited free time :) I've wished for something like this for a long time. I think there's a lot of room for improvement over a basic TTY, and you've nailed some of the important things

Installing

Using Script

On Linux: make sure curl and sudo are installed

On MacOS: make sure you have brew installed.

curl https://ngs-lang.org/install.sh | bash
ngs -pi 'sum(0..10)'

Using Homebrew

brew install ngs

Using Snap

sudo snap install ngs

Using Docker

Docker repository is at https://hub.docker.com/r/ngslang/ngs/

docker run -it --rm ngslang/ngs

Using GitHub Action

Add to your Github Action the following (make sure to release the version as required)

steps:
	- uses: ngs-lang/ngs@v0.2.13

After that, ngs can be used by simply calling ngs in any run step

steps:
	- uses: ngs-lang/ngs@v0.2.13
	- run: ngs -pi 'sum(10..100)'

Running NGS as AWS Lambda Custom Runtime

To run NGS as AWS Lambda Function see https://github.com/ngs-lang/ngs-aws-lambda

Using iPython or Jupyter Notebook

Please refer to extension located in https://github.com/ngs-lang/ngs-ipython-extension

Using AWS Lambda

Please refer to extension located in https://github.com/ngs-lang/ngs-aws-lambda

Manually Compiling and Running

Clone from Git

git clone https://github.com/ngs-lang/ngs.git

# or if using git over ssh
git clone git@github.com:ngs-lang/ngs.git

cd ngs

Install with Dependencies - Debian-based Linux and MacOS

./install.sh

Install without Dependencies

sudo make install

Run Tests

make tests

Compile

make build

Build is currently tested with the GitHub action in .github/workflows/build.yml and we currently test the following 64-bit architectures:

systemgccclang
ubuntu 20.049, 10, 119, 10, 11
debian 11.19, 109, 11
macOS 10.159, 10, 1110
centos 79, 10, 11--
amazonlinux 2default (7.3)default (11)
fedora 34default (11)default (12)
fedora 35default (11)default (13)
archlinuxdefault (11.1)default (13)

Notes:

Run

# If NGS is not installed (from the root of ngs project):
NGS_PATH=lib ./build/ngs SCRIPT_NAME.ngs

# If NGS is installed:
ngs SCRIPT_NAME.ngs

Tested as follows (some time ago):

If you have troubles compiling, please try to compile the commit tagged tested.

Uninstall

cd build
for i in $(<install_manifest.txt);do rm "$i";done

Debug - Mac

# Debug when stuck (note to self mostly)
killall -SIGSEGV ngs
lldb --core /cores/core.XXXXX

Debug - Homebrew builds

Build and run docker

# Build the docker
docker build -t ngs .
# Run the container
docker run -it --rm ngs
# Use NGS inside the container
ngs -pi 'sum(0..10)'

Generate Documentation

On Linux

# --- Linux ---
cd doc
rm -r out
mkdir out
./make.ngs out

On Mac, follow the instructions to create case sensitive volume.

# --- Mac --- One time setup ---
cd doc/
ln -s /Volumes/YOUR-VOLUME-NAME out

# --- Mac --- Each time ---
cd doc/
rm -r out/*;
./make.ngs out

Code Examples

Arrays

a = [1, 2, 3]
arr = a.map(X*2) # arr is now [2, 4, 6]
for i in arr {
	echo(i)
}

Hashes

h = {"a": 1, "b1": 2, "b2": 3}
echo(h.filterk(/^b/).mapv(X+10))  # {b1=12, b2=13}

Functions (multimethods) and multi-dispatch

F my_func(x:Int) x*10 # Single expression does not require { ... } syntax

doc This method is documented!
F my_func(s:Str) {
	t = s * 2
	"[" + t + "]" # Last value returned as the result
}

echo(my_func(1))      # 10
echo(my_func("xyz"))  # [xyzxyz]
echo(my_func)         # <MultiMethod with 2 method(s)>

More information about the language and syntax in particular is in ngslang.1

Basic Cloud

NGS has AWS library based on concept of Declarative Primitives

TLDR:

This is how an instance can be created using NGS (real working code).

{
	NGS_BUILD_CIDR = '192.168.120.0/24'
	NGS_BUILD_TAGS = {'Name': 'ngs-build'}

	vpc    = AWS::Vpc(NGS_BUILD_TAGS).converge(CidrBlock=NGS_BUILD_CIDR, Tags=NGS_BUILD_TAGS)
	gw     = AWS::Igw(Attachments=[{'VpcId': vpc}]).converge(Tags=NGS_BUILD_TAGS)
	rtb    = AWS::RouteTable(VpcId=vpc).converge(Routes=Present({"DestinationCidrBlock": "0.0.0.0/0", "GatewayId": gw}))
	subnet = AWS::Subnet(VpcId=vpc, CidrBlock=NGS_BUILD_CIDR).converge()

	sg = AWS::SecGroup("ngs-build-sg", vpc).converge(
		Description = "ngs-build-sg"
		Tags = NGS_BUILD_TAGS
		IpPermissions = [ AWS::util::world_open_port(22) ]
	)

	ami = AWS::Image(OwnerId=AWS::AMI_OWNER_DEBIAN, Name=Pfx('debian-jessie-amd64-hvm'), RootDeviceType='ebs', VolumeType='gp2').latest()

	instance = AWS::Instance(
		ImageId = ami
		State = null
		KeyName = ENV.get('AWS_NGS_BUILD_KEY', 'ngs-build')
		SecurityGroups = sg
		SubnetId = subnet
		PublicIpAddress = true
		Tags = NGS_BUILD_TAGS
	).converge(
		State = 'running'
	)

	# Get SSH fingerprint from machine's console
	AWS::add_to_known_hosts(instance, 'PublicIpAddress')
}

Sample Scripts

Contributing

If the change is big or involves modifying the syntax, it's better to coordinate with Ilya before you start.

Planned Features

( This section is moving to Wiki and Issues )

See "feature" issues and wiki: https://github.com/ngs-lang/ngs/issues?q=is%3Aissue+is%3Aopen+label%3Afeature

You are welcome to open new issues with feature-request label if there is something you would like to see in NGS.

Cross-system

Development

Later / Unformed / Unfinished Thoughts

How to Run the POC

Following instructions should work (tested on Debian)

cd small-poc
mkdir ssl
cd ssl
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout mysitename.key -out mysitename.crt
cd ..
npm install
nodejs server.js

Have you heard of project X? How it compares to NGS?

Discussion / requests / comments