Awesome
Learn Circle CI (Continuous Integration)
A step-by-step tutorial for using Circle CI for Continuous Integration & Deployment!
Note: This guide/tutorial is a vendor-specific set of instructions for how to enable CI/CD for a Node.js project hosted on GitHub. For a more general introduction to Continuous Integration, please read/follow our "Learn Travis" tutorial first: https://github.com/dwyl/learn-travis
Why?
In a nutshell: the reason you would use Circle-CI
is because it's "free"<sup>1</sup>. see: https://circleci.com/pricing
If you have a product owner / client
who wants/needs their code to remain private
and does not want to pay $49/month for one of the other CI/CD services,
then Circle-CI is your go-to choice.
<sup>1</sup>Most of our client
private
projects that use Circle-CI don't pay a penny. However one of our clients is paying for Circle-CI because tests in their 3 (actively developed) apps take up more than 1500 "free minutes" to build/test per month. So we feel they are "breaking even" on us...
What?
Run your (application/unit/integration/UI) tests using a container on Circle-CI and deploy your app to your choice of environment if they pass.
Who?
This tutorial is aimed at developers who need a way of automatically testing
and deploying their private
apps/code.
Prerequisites?
- Basic knowledge of CI concepts. If you are new to (or "rusty") on CI, please read/follow our "Learn Travis" tutorial first: https://github.com/dwyl/learn-travis
- Basic JavaScript/Node.js knowledge.
- 10 Minutes of Time.
No other knowledge/skill/experience is assumed or implied.
Stuck? If you have any questions related to getting started with Circle-CI or you are "stuck" deploying using the script(s) we demo here, please open an issue/question and we will try our best to help! https://github.com/dwyl/learn-circleci/issues
How
If you do not already have an App that you want to test and deploy, see:
1. Register
Register using your GitHub account: https://circleci.com (if you haven't already)
2. Add Project
Click the "add project" +
button from the left-hand side menu.
Then select the project you want Circle-CI to test for you,
and click the "Setup Project" button:
You should see something like this now: <br />
3. Setup .circleci/config.yml
File
Next, follow the on-screen instructions
to create a config.yml
folder
and setup your .circleci/config.yml
file.
mkdir .circleci
vi .circleci/config.yml
Click on the "Copy To Clipboard" button to get the code.
paste the sample code into the .circleci/config.yml
file you have open in your text editor.
e.g:
Save the file, commit it and push to GitHub
.
e.g: https://github.com/nelsonic/circleci-hello-world-nodejs/commit/ad368321f3fa05686c97a8147f1e5570870bb527
Note: Circle-CI installs Yarn by default. This is NOT required anymore in 2018. It adds a needless step to the build process and sends your
package.json
to Facebook HQ. I've left it as thedefault
for this example, but we don't use Yarn for any of our Node.js projects, there is zero benefit.
4. Start Building
Once you have created the .circleci/config.yml
,
scroll down the page and click on "Start building":
You should be redirected to a build progress/status page:
e.g: https://circleci.com/gh/nelsonic/circleci-hello-world-nodejs/1
The build and running the (single unit) test, took 3 seconds and ended in "SUCCESS". (it's a "hello world" example, it should be fast and pass!)
5. Status Badge!
At this point we can add a "Badge" to our project README.md
e.g: https://circleci.com/gh/nelsonic/circleci-hello-world-nodejs/edit#badges
For why this is a good idea, see: https://github.com/dwyl/repo-badges
Part 2 - Continuous Deployment
Continuous Deployment (CD) is the perfect complement to Continuous Integration (CI) and people usually use the same "pipeline" (system) to automate it. It just means that your application is automatically deployed by the CI system (in this case CircleCI) when the PR is merged.
If you are
new
to Continuous Deployment (CD) in general we wrote an introductory post in: https://github.com/dwyl/learn-devops
<sup>1</sup>As always, we recommend using Heroku for "MVP" App(s) because it's much easier/faster to get started and traction. see: https://github.com/dwyl/learn-heroku and: https://circleci.com/docs/2.0/deployment-integrations/#heroku once you need "more power" than Heroku (or need to reduce costs), take a look at: https://github.com/dwyl/learn-devops
6. Pre-Deployment Checks
- Ensure that you have a
"start"
script in yourpackage.json
. e.g:
{
"scripts": {
"start": "node server.js",
"test": "node hello.test.js"
}
}
- Ensure that your app runs on your
localhost
before attempting to deploy it to your chosen "cloud" provider.
Run the start
command:
npm start
Visit: http://localhost:5000
Ok, so it's working on localhost
; we have a good "baseline". Onwards!
Note: there are several ways of starting node.js servers, (e.g: PM2, Forever or SystemD) we are using
"node server.js"
for simplicity. But if you are running your own infrastructure, you will have a well defined system for running/monitoring node.js.
7. Deploy!
For this section we are doing an "advanced" deployment to a DigitalOcean Instance using Dokku (which is an Open Source Heroku "clone"). It's only "advanced" in that there are a few more "steps" than deploying to Heroku, but don't worry, we have documented everything "step-by-step" so it should only take you about 10 minutes to setup.
Note: as always, if you get "stuck", just open an issue and we will try our best to help! https://github.com/dwyl/learn-circleci/issues
7.1 Add RSA Key
In order to deploy the app via SSH, we need to add the SSH (RSA) key to CircleCI. Thankfully this is a lot easier thank on Travis-CI!
In your app's Settings page,
- Scroll down to the "Permissions" section.
- Click on SSH Permissions
- Click on the
Add SSH Key
button.
A "modal" window will appear which allows you to paste your Private Key.
<!-- note: update this link to the specific section in the Dokku deployment guide once https://github.com/dwyl/learn-devops/pull/29 is merged! -->Note: we do NOT recommend using your personal
private
key for deployments! rather you should create a key that is specific to the server you are deploying to and is only "known" (used) by your CI/CD system. see: https://github.com/dwyl/learn-devops
We have successfully added our SSH Deployment RSA Key!
7.2 Add Deployment Scripts to your Project
Create a /bin
directory in your project.
mkdir bin
Copy the deployement scripts from: https://github.com/nelsonic/circleci-hello-world-nodejs/tree/master/bin
You can either do this manually by downloading each file to your project, or ... using a script!
7.3 Add Required Environment Variables
Add any Environment Variables you need to your .circleci/config.yml
file.
The IP Address of the server is required as is the DOKKU_APP
name.
e.g:
environment:
SERVER_IP_ADDRESS: 138.68.163.126
DOKKU_APP: circlecidemo
TRAVIS: "false"
working example: https://github.com/nelsonic/circleci-hello-world-nodejs/blob/ecfab4a49141da87f36519e50ecda593f01aaf48/.circleci/config.yml#L8
Note: if you need your environment variables to not be in Version Control, add them via the CircleCI Web Interface. more details: https://circleci.com/docs/2.0/env-vars
If you are
new
to Environment Variables, checkout: https://github.com/dwyl/learn-environment-variables
7.4 Add Command to Run Deployment Script
Add the following lines to your .circleci/config.yml
file:
# Use BASH instead of DASH! see: https://ubuntuforums.org/showthread.php?t=1932504
- run: ls -al /bin/sh && sudo rm /bin/sh && sudo ln -s /bin/bash /bin/sh && ls -al /bin/sh
# test shell script:
- run: echo ${SERVER_IP_ADDRESS}
- run: echo -e "Host $SERVER_IP_ADDRESS\n\tStrictHostKeyChecking no\n" >> ~/.ssh/config
- run: sh bin/deploy.sh
Working example: https://github.com/nelsonic/circleci-hello-world-nodejs/blob/ecfab4a49141da87f36519e50ecda593f01aaf48/.circleci/config.yml#L43-L48
Push your code to GitHub and let CircleCI do the rest! It usually takes a minute to deploy (depending on your dependencies)
In the build log you should see something like:
e.g: https://circleci.com/gh/nelsonic/circleci-hello-world-nodejs/18
8. Check it Worked!
In our case our DOKKU_APP
was defined as circlecidemo
(see: section 7.3 above)
So the deployed app is available on:
https://circlecidemo.ademo.app
If we visit this URL in our browser we see:
Notice the "Git Hash" value: dc933373388911d9405fb452c9753f0250766bb3
This corresponds to the latest commit on the master
branch:
https://github.com/nelsonic/circleci-hello-world-nodejs/commits/master
(GitHub only shows the first 7 characters of the has in the UI
in this case: dc93337
)
So it worked! (the latest master
has been deployed by CircleCI!)
done
Relevant / Background Reading
- How to run Bash Scripts on CircleCI: https://circleci.com/docs/2.0/using-shell-scripts
- Permission denied running bash script ... https://stackoverflow.com/questions/33942926/circleci-permission-denied-running-bash-script
Official "Getting Started" Video?
The official
get started on CircleCI 2.0 video: https://youtu.be/KhjwnTD4oec
uses a Python example:
https://github.com/bellkev/circleci-demo-python-django
it's not really "hello world" (beginner friendly) ...
So just focus on the UI/navigation, then come back here for a simple example!