Home

Awesome

footer: © Node.University, 2018 theme: Simple slidenumbers: true build-lists: true [.hide-footer] [.slidenumbers: false]

You Don't Know Node, v2

Quick Intro to Core Node

Turn OFF laptops, phones, distractors and LISTEN

Who am I? ¯\_(ツ)_/¯

Not Kyle Simpson

(Hint: He wrote "You Don't Know JS" series)

inline

^You are looking for Kyle Simpson, you're in a wrong room

My name is Azat Mardan and I am the creator of Node University

Azat Mardan and numbers

inline

^Macy's, Intuit, Northwestern Mutual, Apple, DocuSign, UC Davis, Salesforce, The University of Arizona, The Orchard, M3, Twilio, Fox Studios, Michael Kors

inline

Fun Facts

I know Node

I dislike compiled languages

Actually, just one. Java!

inline

My Mission is to Make a World a Better Place

My Plan

Software helps people

+

Node is a good tool to build good software

=

I need to teach as many people Node as possible

My Goal (for this talk)

Spike your interest in core Node features

Starting with basics: Why Use Node?

Input/output is one of the most expensive type tasks (>CPU) 💰

Node has non-blocking I/O

inline

^Docs are not very ilustrative

inline

^Good but too much tech details

inline

^This allows processing other tasks while IO calls are unfinished like this ^Nginx vs. Apache ^Blocking I/O is expensive!

Java Sleep

System.out.println("Step: 1");
System.out.println("Step: 2");
Thread.sleep(1000);
System.out.println("Step: 3");

Node "Sleep"

console.log('Step: 1')
setTimeout(function () {
  console.log('Step: 3')
}, 1000)
console.log('Step: 2')

Process Multiple Tasks

console.log('Step: 1')
setTimeout(function () {
  console.log('Step: 3')
  // console.log('Step 5')
}, 1000);
console.log('Step: 2')
// console.log('Step 4')

Blocking Web Server

inline

inline

Non-Blocking Web Server

inline

^This is in contrast to today's more common concurrency model where OS threads are employed. Thread-based networking is relatively inefficient and very difficult to use. Furthermore, users of Node are free from worries of dead-locking the process --- there are no locks

inline

[Multi-threading] is the software equivalent of a nuclear device because if it is used incorrectly, it can blow up in your face.

http://blog.codinghorror.com/threading-concurrency-and-the-most-powerful-psychokinetic-explosive-in-the-univ

Blocking systems have to be multi-threaded

Node is single threaded... and that's good! 😄

It's still possible to write blocking code in Node.js. :flushed:

Blocking Node.js Code

// blocking.js
console.log('Step: 1')
for (var i = 1; i<1000000000; i++) {
  // This will take 100-1000ms
}
console.log('Step: 2')

Blocking Node.js Code

var fs = require('fs')

var contents = fs.readFileSync('accounts.txt','utf8')
console.log(contents)
console.log('Hello Ruby\n')

var contents = fs.readFileSync('ips.txt','utf8')
console.log(contents)
console.log('Hello Node!')
//accounts.txt->Hello Ruby->ips.txt->Hello Node!

Non-Blocking Node.js Code

var fs = require('fs')

fs.readFile('accounts.txt','utf8', function(error, contents){
   console.log(contents)
})
console.log('Hello Ruby\n')

fs.readFile('ips.txt','utf8', function(error, contents){
   console.log(contents)
})
console.log('Hello Node!')
//Hello Ruby->Hello Node->... accounts.txt->ips.txt or ips.txt->accounts.txt

Node typically is much faster than other platforms

How many of you reach the performance limitations of apps built with blocking I/O systems?

Probably not many

My Fav Node Benefit

JavaScript everywhere. One language to rule 'em all!

When one language is used everywhere

Most of Node is JavaScript

Node !== Browser JavaScript

Node Core Modules

inline

How to create global variables (no window in Node), work with modules, get path to my script?

global

Now available everywhere in your code

It has properties!

global.__filename

global.__dirname

global.module

global.require()

How do I...?

global.process or process

process.pid

process.versions

process.arch

process.argv

process.env

process.uptime()

process.memoryUsage()

process.cwd()

process.exit()

process.kill()

Who likes and understands callbacks? 🙋

http://callbackhell.com

fs.readdir(source, function (err, files) {
  if (err) {
    console.log('Error finding files: ' + err)
  } else {
    files.forEach(function (filename, fileIndex) {
      console.log(filename)
      gm(source + filename).size(function (err, values) {
        if (err) {
          console.log('Error identifying file size: ' + err)
        } else {
          console.log(filename + ' : ' + values)
          aspect = (values.width / values.height)
          widths.forEach(function (width, widthIndex) {
            height = Math.round(width / aspect)
            console.log('resizing ' + filename + 'to ' + height + 'x' + height)
            this.resize(width, height).write(dest + 'w' + width + '_' + filename, function(err) {
              if (err) console.log('Error writing file: ' + err)
            })
          }.bind(this))
        }
      })
    })
  }
})

Callbacks are not very developmental scalable 😞

Me When Working With Deeply Nested Callbacks

inline

Events

Events are part of core and supported by most of the core modules while more advanced patterns such as promises, generators, async/await are not.

Events == Node Observer Pattern

Events

var events = require('events')
var emitter = new events.EventEmitter()

Events

In node.js an event can be described simply as a string with a corresponding callback.

emitter.on('done', function(results) {
  console.log('Done: ', results)
})

Using Event Emitters

var events = require('events')
var emitter = new events.EventEmitter()

emitter.on('knock', function() {
  console.log('Who\'s there?')
})

emitter.on('knock', function() {
  console.log('Go away!')
})

emitter.emit('knock')

Inheriting from EventEmitter

// job.js
var util = require('util')
var Job = function Job() {
  // ...
  this.process = function() {
    // ...
    job.emit('done', { completedOn: new Date() })
  }
}

util.inherits(Job, require('events').EventEmitter)
module.exports = Job

Inheriting from EventEmitter

// weekly.js
var Job = require('./job.js')
var job = new Job()

job.on('done', function(details){
  console.log('Job was completed at', details.completedOn)
  job.removeAllListeners()
})

job.process()

Listeners

emitter.listeners(eventName)
emitter.on(eventName, listener)
emitter.once(eventName, listener)
emitter.removeListener(eventName, listener)

Resources to Learn Node Patterns

Problems with Large Data

Streams

Abstractions for continuous chunking of data

No need to wait for the entire resource to load

Types of Streams

Streams Inherit from Event Emitter

Streams are Everywhere!

Readable Stream Example

process.stdin

Standard input streams contain data going into applications.

This is achieved via a read operation.

Input typically comes from the keyboard used to start the process.

To listen in on data from stdin, use the data and end events:

// stdin.js
process.stdin.resume()
process.stdin.setEncoding('utf8')

process.stdin.on('data', function (chunk) {
  console.log('chunk: ', chunk)
})

process.stdin.on('end', function () {
  console.log('--- END ---')
})

Demo

$ node stdin.js

New Interface read()

var readable = getReadableStreamSomehow()
readable.on('readable', () => {
  var chunk
  while (null !== (chunk = readable.read())) {
    console.log('got %d bytes of data', chunk.length)
  }
})

^readable.read is sync but the chunks are small

Writable Stream Example

process.stdout

Standard output streams contain data going out of the applications.

This is done via a write operation.

Data written to standard output is visible on the command line.

Writable Stream

To write to stdout, use the write function:

process.stdout.write('A simple message\n')

What about HTTP?

const http = require('http')
var server = http.createServer( (req, res) => {
  req.setEncoding('utf8')
  req.on('data', (chunk) => {
    transform(chunk) // This functions is defined somewhere else
  })
  req.on('end', () => {  
    var data = JSON.parse(body)
    res.end()
  })
})

server.listen(1337)

Pipe

var r = fs.createReadStream('file.txt')
var z = zlib.createGzip()
var w = fs.createWriteStream('file.txt.gz')
r.pipe(z).pipe(w)

^Readable.pipe takes writable and returns destination

What data type to use for binary data?

Buffers

Binary data type, to create:

Docs: http://bit.ly/1IeAcZ1

Working with Buffer

// buf.js
var buf = Buffer.alloc(26)
for (var i = 0 ; i < 26 ; i++) {
  buf[i] = i + 97 // 97 is ASCII a
}
console.log(buf) // <Buffer 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71 72 73 74 75 76 77 78 79 7a>
console.log(buf.toString('utf8')) // abcdefghijklmnopqrstuvwxyz

Buffer Convertion

buf.toString('ascii') // outputs: abcdefghijklmnopqrstuvwxyz
buf.toString('ascii', 0, 5) // outputs: abcde
buf.toString('utf8', 0, 5) // outputs: abcde
buf.toString(undefined, 0, 5) // encoding defaults to 'utf8', outputs abcde

Remember fs?

fs.readFile('/etc/passwd', function (err, data) {
  if (err) return console.error(err)
  console.log(data)
});

data is buffer!

Demo

$ node server-stream

Streams and Buffer Demo

// server-stream.js
app.get('/stream', function(req, res) {
  var stream = fs.createReadStream(largeImagePath)
  stream.pipe(res)
})

$ node server-stream

http://localhost:3000/stream http://localhost:3000/non-stream

Results in DevTools

/stream responds faster!

X-Response-Time
~300ms vs. 3-5s

Stream Resources

Stream automated workshop: https://github.com/substack/stream-adventure

$ sudo npm install -g stream-adventure
$ stream-adventure

https://github.com/substack/stream-handbook

inline

How to scale a single threaded system?

Cluster Usage

Number of processes = number of CPUs

Clusters

var cluster = require('cluster')
var numCPUs = require('os').cpus().length
if (cluster.isMaster) {
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork()
  }
} else if (cluster.isWorker) {
  // your server code
})

Cluster Demo

  1. Run code/cluster.js with node ($ node cluster.js).
  2. Install loadtest with npm: $ npm install -g loadtest
  3. Run load testing with: $ loadtest http://localhost:3000 -t 20 —c 10

Press control+c on the server terminal

Cluster Libraries

pm2

https://github.com/Unitech/pm2

http://pm2.keymetrics.io

Advantages:

pm2 Demo: Typical Express Server

var express = require('express')
var port = 3000
global.stats = {}
console.log('worker (%s) is now listening to http://localhost:%s',
 process.pid, port)
var app = express()
app.get('*', function(req, res) {
  if (!global.stats[process.pid]) global.stats[process.pid] = 1
  else global.stats[process.pid] += 1;
  var l ='cluser '
    + process.pid
    + ' responded \n';
  console.log(l, global.stats)
  res.status(200).send(l)
})
app.listen(port)

pm2 Demo

Using server.js:

$ pm2 start server.js -i 0

In a new window:

$ loadtest  http://localhost:3000 -t 20 -c 10
$ pm2 list

Spawn vs Fork vs Exec

Spawn Example

fs = require('fs')
process = require('child_process')
var p = process.spawn('node', 'program.js')
p.stdout.on('data', function(data)) {
  console.log('stdout: ' + data)
})

Fork Example

fs = require('fs')
process = require('child_process')
var p = process.fork('program.js')
p.stdout.on('data', function(data)) {
  console.log('stdout: ' + data)
})

Exec Example

fs = require('fs')
process = require('child_process')
var p = process.exec('node program.js', function (error, stdout, stderr) {
  if (error) console.log(error.code)
})

There are few more methods

^child_process.execFile(): similar to child_process.exec() except that it spawns the command directly without first spawning a shell. child_process.execSync(): a synchronous version of child_process.exec() that will block the Node.js event loop. child_process.execFileSync(): a synchronous version of child_process.execFile() that will block the Node.js event loop.

How to handle async errors?

Handling Async Errors

Event Loop: Async errors are harder to handle/debug, because system loses context of the error. Then, application crashes.

Try/catch is not good enough.

Synchronous Error in Node

try {
  throw new Error('Fail!')
} catch (e) {
  console.log('Custom Error: ' + e.message)
}

For sync errors try/catch works fine.

Async Error Example

try {
  setTimeout(function () {
    throw new Error('Fail!')
  }, Math.round(Math.random()*100))
} catch (e) {
  console.log('Custom Error: ' + e.message)
}

The app crashes!

Me When Async Error's Thrown

inline

Async Errors

How to deal with it?

:sweat_smile:

Best Practices for Async Errors?

on('error')

Anything that inherits from or creates an instance of the above: Express, LoopBack, Sails, Hapi, etc.

server.on('error', function (err) {
  console.error(err)
})

on('error') Chained Method Example

var http = require(‘http’)
var server = http.createServer(app)
  .on('error', function(e) {
    console.log(‘Failed to create server’)
    console.error(e)
    process.exit(1)
  })

on(‘error’) Named Variable Example

var req = http.request(options, function(res) {
  // … processing the response
})

req.on('error', function(e) {
  console.log('problem with request: ' + e.message)
})

uncaughtException

uncaughtException is a very crude mechanism for exception handling. An unhandled exception means your application - and by extension Node.js itself - is in an undefined state. Blindly resuming means anything could happen.

uncaughtException

Always listen to uncaughtException!

process.on(‘uncaughtException’, handle)

or

process.addListener('uncaughtException', handle)

uncaughtException Expanded Examples

process.on('uncaughtException', function (err) {
  console.error('uncaughtException: ', err.message)
  console.error(err.stack)
  process.exit(1)
})

or

process.addListener('uncaughtException', function (err) {
  console.error('uncaughtException: ', err.message)
  console.error(err.stack)
  process.exit(1)

Domain

(Just out of curiosity)

Domain Example

let domain = require('domain').create()
domain.on('error', function(error){
  console.log(error)
})
domain.run(function(){
  throw new Error('Failed!')
})

Domain with Async Error Demo

domain-async.js:

let d = require('domain').create()
d.on('error', function(e) {
   console.log('Custom Error: ' + e)
})
d.run(function() {
  setTimeout(function () {
    throw new Error('Failed!')
  }, Math.round(Math.random()*100))
})

Source code for Domain

https://github.com/nodejs/node/blob/master/lib/domain.js

var _domain = [null];
Object.defineProperty(process, 'domain', {
  enumerable: true,
  get: function() {
    return _domain[0];
  },
  set: function(arg) {
    return _domain[0] = arg;
  }
});

Domain overwrites uncaught exception

if (process.hasUncaughtExceptionCaptureCallback()) {
  throw new errors.Error('ERR_DOMAIN_CALLBACK_NOT_AVAILABLE');
}

process.on('removeListener', (name, listener) => {
  if (name === 'uncaughtException' &&
      listener !== domainUncaughtExceptionClear) {
    // If the domain listener would be the only remaining one, remove it.
    const listeners = process.listeners('uncaughtException');
    if (listeners.length === 1 && listeners[0] === domainUncaughtExceptionClear)
      process.removeListener(name, domainUncaughtExceptionClear);
  }
});

^When domains are in use, they claim full ownership of the uncaught exception capture callback.

C++ Addons

How to Write C/C++ binding for your IoT, hardware, drone, smartdevice, etc.?

Node and C++

Create the hello.cc file:

#include <node.h>

namespace demo {

using v8::FunctionCallbackInfo;
using v8::HandleScope;
using v8::Isolate;
using v8::Local;
using v8::Object;
using v8::String;
using v8::Value;

Node and C++

Create the hello.cc file:

void Method(const FunctionCallbackInfo<Value>& args) {
  Isolate* isolate = args.GetIsolate();
  args.GetReturnValue().Set(String::NewFromUtf8(isolate, "capital one"));
}

void init(Local<Object> exports) {
  NODE_SET_METHOD(exports, "hello", Method);
}

NODE_MODULE(addon, init)

}  // namespace demo

Creating binding.gyp

Create binding.gyp:

{
  "targets": [
    {
      "target_name": "addon",
      "sources": [ "hello.cc" ]
    }
  ]
}

node-gyp

$ npm install -g node-gyp

https://github.com/nodejs/node-gyp

^Needs Python

Configuring and Building

$ node-gyp configure
$ node-gyp build

Check for compiled .node files in build/Release/

C++ Addons Examples

https://github.com/nodejs/node-addon-examples

Including Addon

Create hello.js and include your C++ addon:

var addon = require('./build/Release/addon')
console.log(addon.hello()) // 'capital one'

Run

$ node hello.js

Moaaarr core Node

30-Second Summary

  1. Event Emitters
  2. Streams
  3. Buffers
  4. Clusters
  5. C++ Addons
  6. Domain

^We have covered a lot in the last 60 minutes

There are big, big, big problems

^2 days - won't remember my name and most of what I said ^5 days - remember only a joke ^8 days - won't remember being here

New information

Solution: You Don't Know Node Online Course

inline

https://node.university/p/you-dont-know-node

Sign up for Node University NOW!

inline

Node.University is the ultimate resource for the best online Node education!

https://node.university 🚀

This is how to do it

Take out your laptops or phones :computer: 📱

Seriously. Do it now!

Go to node.university

(https://node.university in Safari, Chrome, Edge)

[.hide-footer] [.slidenumbers: false]

fit

^Click/press on Free courses

[.hide-footer] [.slidenumbers: false]

fit

^ Click/press on You Don't Know Node

[.hide-footer] [.slidenumbers: false]

fit

^Click/Press on enroll

[.hide-footer] [.slidenumbers: false]

fit

^Put your email and create a password

One Last Thing 👉

CodingHorror.com

inline

The End