Home

Awesome

GitHub license GitHub issues GitHub stars

electron-re


Test on electron@8.2.0 / 9.3.5

Contents

├── Contents (you are here!)
│
├── Architecture
│
├── * What can be used for?
│   ├── In Electron Project
│   └── In Nodejs/Electron Project
│
├── * Install
│
├── * Instruction1: ProcessManager
│   ├── Require it in main.js(electron)
│   └── Open process-manager window for your application
│
├── * Instruction2: Service
│   ├── The arguments to create a service
│   ├── Enable service auto reload after code changed
│   └── The methods of a Service instance
│
├── * Instruction3: MessageChannel
│   ├── The methods of MessageChannel
│   └── A full usage
│
├── * Instruction4: ChildProcessPool
│   ├── Create a childprocess pool
│   ├── Send request to a process instance
│   ├── Send request to all process instances
│   ├── Destroy the child processes of the process pool
│   └── Set the max instance limit of pool
│
├── * Instruction5: ProcessHost
│   ├── Require it in a sub process
│   ├── Registry a task with unique name
│   ├── Working with ChildProcessPool
│   └── Unregistry a task with unique name
│
├── * Instruction6: WorkerThreadPool
│   ├── Create a static WorkerThreadPool pool
│   ├── Create a static WorkerThreadPool excutor
│   ├── Create a dynamic WorkerThreadPool pool
│   └── Create a dynamic WorkerThreadPool excutor
│
├── Examples

Architecture


architecture

I. What can be used for?


II. Install


$: npm install electron-re --save
# or
$: yarn add electron-re --save

III. Instruction 1: ProcessManager


Used in Electron project, build for ChildProcessPool/BrowserService.

Compatible with native IpcRenderer/Main.

All functions:

  1. Show all alive processes in your Electron application: main process, renderer process, the service process (imported by electron-re), and the child process created by ChildProcessPool (imported by electron-re).

  2. The process list displays info: process ID, process type(mark), parent process ID, memory, CPU. All processes type include main (main process), service (service process), renderer (renderer process) , node (child process in process pool). click on table header to sort an item in increasing/decreasing order.

  3. You can kill a process, view process console data, check CPU/memory occupancy within 1 min.

  4. If a process marked as renderer, pressing the DevTools button then the built-in debugging tool will open as an undocked window. Besides the child-processes are created by ChildProcessPool with --inspect parameter, DevTools is not supported, just visit chrome://inspect in chrome for remote debugging.

  5. Try to use MessageChannel for sending/receiving ipc messages, there is a ui pannel area that show activities of it (logger).

Require it in main.js(electron)

const {
  MessageChannel, // remember to require it in main.js even if you don't use it
  ProcessManager
} = require('electron-re');

Open process-manager window

ProcessManager.openWindow();
  1. Main

The main ui

main

  1. Console

Show console info of all processes

console

  1. DevTools

Open devtools for electron renderer window

devtools

  1. Trends

Show cpu/memory occupancy trends

trends

trends2

  1. Kill

Kill process from one-click

kill

  1. Signals Pannel

Activities logger for MessageChannel tool

signals

IV. Instruction 2: Service


Used in Electron project, working with MessageChannel, remember to check "Instruction 3".

1. The arguments to create a service

The service process is a customized render process that works in the background, receiving path, options as arguments:

/* --- main.js --- */
const { BrowserService } = require('electron-re');
const myService = new BrowserService('app', 'path/to/app.service.js', options);app.service.js'));

2. Enable service auto reload after code changed

The auto-reload feature is based on nodejs - fs.watch api. When webSecurity closed and in dev mode, service will reload when service code changed.

1.Set dev mode in new BrowserService() options
2.Get webSecurity closed

/* --- main.js --- */
const myService = new BrowserService('app', 'path/to/app.service.js', {
  ...options,
  // set dev mode with webSecurity closed
  dev: true,
  webPreferences: { webSecurity: false }
});

3. The methods of a Service instance

The service instance is a customized BrowserWindow instance too, initialized by a file worked with commonJs module, so you can use require('name') and can't use import some from 'name' syntax. It has two extension methods:

suggest to put some business-related code into a service.

/* --- main.js --- */
  
const { 
  BrowserService,
  MessageChannel // must required in main.js even if you don't use it
} = require('electron-re');
...

app.whenReady().then(async() => {
  // after app is ready in main process
  const myService = new BrowserService('app', 'path/to/app.service.js');
  // async
  await myService.connected();
  mhyService.openDevTools();
  /* work with webContents method, also you can use MessageChannel instead */
  mhyService.webContents.send('channel1', { value: 'test1' });
});
...

/* --- app.service.js --- */
const { ipcRenderer } = require('electron');
/* working with ipc method, also you can use MessageChannel instead */
ipcRenderer.on('channel1', (event, result) => {
  // works
  ...
});

V. Instruction 3: MessageChannel


Used in Electron project, working with Service.

When sending data from main/other process to a service you need to use MesssageChannel, such as: MessageChannel.send('service-name', 'channel', 'params'), And also it can be used to replace other build-in ipc methods, more flexible.

The methods of MessageChannel

1.Public methods,used in Main-Pocess / Renderer-Process / Service

/* send data to a service - like the build-in ipcMain.send */
MessageChannel.send('service-name', channel, params);
/* send data to a service and return a Promise - extension method */
MessageChannel.invoke('service-name', channel, params);
/*
  send data to a renderer/servcie which id is same as the given windowId/webContentsId,
  same as ipcRenderer.sendTo,
  recommend to use it when you want to send data from main/service to a renderer window
*/
MessageChannel.sendTo('windowId/webContentsId', channel, params);
/* listen a channel and return unlisten function, probably same as ipcMain.on/ipcRenderer.on */
const removeOnListener = MessageChannel.on(channel, func);
/* listen a channel once and return unlisten function, probably same as ipcMain.once/ipcRenderer.once */
const removeOnceListener = MessageChannel.once(channel, func);

2.Only used in Renderer-process / Service

/* send data to main process - like the build-in ipcRender.send */
MessageChannel.send('main', channel, params);
/* send data to main process and return a Promise - extension method */
MessageChannel.invoke('main', channel, params);

3.Only used in Main-process / Service

/*
  handle a channel signal, extension method,
  and you can return data directly or return a Promise instance
*/
MessageChannel.handle(channel, processorFunc);

Full Usage

const {
  BrowserService,
  MessageChannel // must required in main.js even if you don't use it
} = require('electron-re');
const isInDev = process.env.NODE_ENV === 'dev';
...

/* use MessageChannel instead of build-in method */
app.whenReady().then(() => {
  const myService = new BrowserService('app', 'path/to/app.service.js');
  myService.connected().then(() => {
    // open devtools in dev mode for debugging
    if (isInDev) myService.openDevTools();
    MessageChannel.send('app', 'channel1', { value: 'test1' });
    MessageChannel.invoke('app', 'channel2', { value: 'test2' }).then((response) => {
      console.log(response);
    });

    MessageChannel.on('channel3', (event, response) => {
      console.log(response);
    });

    MessageChannel.handle('channel4', (event, response) => {
      console.log(response);
      return { res: 'channel4-res' };
    });

  })
});
const { ipcRenderer } = require('electron');
const { MessageChannel } = require('electron-re');

MessageChannel.on('channel1', (event, result) => {
  console.log(result);
});

MessageChannel.handle('channel2', (event, result) => {
  console.log(result);
  return { response: 'channel2-response' }
});

MessageChannel.invoke('app2', 'channel3', { value: 'channel3' }).then((event, result) => {
  console.log(result);
});

MessageChannel.send('app2', 'channel4', { value: 'channel4' });
MessageChannel.handle('channel3', (event, result) => {
  console.log(result);
  return { response: 'channel3-response' }
});

MessageChannel.once('channel4', (event, result) => {
  console.log(result);
});

MessageChannel.send('main', 'channel3', { value: 'channel3' });
MessageChannel.send('main', 'channel3', { value: 'channel3' });
MessageChannel.invoke('main', 'channel4', { value: 'channel4' });

const { ipcRenderer } = require('electron');
const { MessageChannel } = require('electron-re');

MessageChannel.send('app', 'channel1', { value: 'test1'});
MessageChannel.invoke('app2', 'channel3', { value: 'test2' });
MessageChannel.send('main', 'channel3', { value: 'test3' });
MessageChannel.invoke('main', 'channel4', { value: 'test4' });

VI. Instruction 4: ChildProcessPool


Used in Nodejs/Electron project, working with ProcessHost, remember to check "Instruction 5".

Multi-process helps to make full use of multi-core CPU, let's see some differences between multi-process and multi-thread:

  1. It is difficult to share data between different processes, but threads can share memory.
  2. Processes consume more computer resources than threads.
  3. The processes will not affect each other, a thread hanging up will cause the whole process to hang up.

Attention of Electron Bugs !!

DO NOT USE require('electron') in child_process js exec file, this will cause fatal error in the production environment!

Besides that, In order to use ChildProcessPool, you need to place your child_process exec js file in an external directory such as ~/.config/. Otherwise, when you packaged your app, Node.js can not find that exec file.

The another way to solve this problem is to set asar to false in the electron-builder.json, this is not recommended but works.

{
   ...
   "asar": false,
   ...
}

1. Create a childprocess pool

const { ChildProcessPool, LoadBalancer } = require('electron-re');

global.ipcUploadProcess = new ChildProcessPool({
  path: path.join(app.getAppPath(), 'app/services/child/upload.js'),
  max: 3,
  env: { lang: global.lang, NODE_ENV: nodeEnv },
  strategy: LoadBalancer.ALGORITHM.WEIGHTS_POLLING, // loadbalance strategy
  weights: [1, 2, 3],
});

2. Send request to a process instance

global.ipcUploadProcess.send(
  'init-works',
  {
    name: 'fileName',
    type: 'fileType',
    size: 'fileSize',
  },
  'id-number' // optional and it's given by you
)
.then((rsp) => {
  console.log(rsp);
});

3. Send request to all process instances

All sub processes will receive a request, and you can get a response data array from all sub processes.

global.ipcUploadProcess.sendToAll(
  'task-get-all',
  { key: 'test' }
)
.then((rsp) => {
  console.log(rsp);
});

4. Destroy the child processes of the process pool

// destroy a process with id value
global.ipcUploadProcess.disconnect(id);
// destroy all processes
global.ipcUploadProcess.disconnect();

5. Set the max instance limitation of pool

In addition to using the max parameter to specify the maximum number of child process instances created by the process pool, you can also call this method to dynamically set the number of child process instances that need to be created.

global.ipcUploadProcess.setMaxInstanceLimit(number);

VII. Instruction 5: ProcessHost


Used in Nodejs/Electron project, working with ChildProcessPool.

In Instruction 4, We already know how to create a sub-process pool and send request using it. Now let's figure out how to registry a task and handle process messages in a sub process(created by ChildProcessPool constructor with param - path).

Using ProcessHost we will no longer pay attention to the message sending/receiving between main process and sub processes. Just declaring a task with a unique service-name and put your processing code into a function. And remember that if the code is async, return a Promise instance instead.

1. Require it in a sub process

const { ProcessHost } = require('electron-re');

2. Registry a task with unique name

Support chain call

ProcessHost
  .registry('init-works', (params) => {
    return initWorks(params);
  })
  .registry('async-works', (params) => {
    return asyncWorks(params);
  });

function initWorks(params) {
  console.log(params);
  return params;
}

function asyncWorks(params) {
  console.log(params);
  return fetch(url);
}

3. Working with ChildProcessPool


/* 1. send a request in main process */
global.ipcUploadProcess.send(
  'init-works',
  {
    name: 'fileName',
    type: 'fileType',
    size: 'fileSize',
  }
);

...

/* 2. handle this request in sub process */
...

4. Unregistry a task with unique name(if necessary)

Support chain call

ProcessHost
  .unregistry('init-works')
  .unregistry('async-works')
  ...

VIII. WorkerThreadPool


Multi Processes help to make full use of cpu, Multi Threads improve task parallelism ability of Node.js.

In Node.js, there is only one main process which has single main thread, the main thread run event loops and executes macro/micro tasks. In theory, macro/micro task should be short and quick, if we use main thread for some cpu-sensitive heavy tasks, this will block event loop on main thread.

So, try to put your heavy tasks into worker threads will be better in Node.js. The worker thread pool is effective for creating and managing threads, besides, it provides a task queue. When pool has no idle thread, more coming tasks are placed in queue and be taken out from queue after while to be excuted by new idle thread.

Create a static WorkerThreadPool pool

  1. Options of StaticThreadPool:
const uint8Array = new Uint8Array([ 1, 2, 3, 4 ]);
const staticPool = new StaticThreadPool(
  {
    // execPath: 'path/to/executable.js',
    execString: 'module.exports = (payload) => `res:${payload}`',
    // execFunction: (payload) => payload,
    lazyLoad: true,
    maxThreads: 24,
    maxTasks: 48,
    taskRetry: 1,
    taskLoopTime: 1e3,
    taskTimeout: 5e3,
  },
  {
    transferList: [uint8Array.buffer]
  }
);
  1. Attributes of a StaticThreadPool instance
  1. Methods of a StaticThreadPool instance
staticPool
  .setTaskRetry(1)
  .exec('payload-data', {
    taskTimeout: 5e3,
    taskRetry: 1,
  })
  .then((rsp) => {
    console.log(rsp);
  });

Create a static WorkerThreadPool excutor

  1. Options of StaticThreadPool Executor
const uint8Array = new Uint8Array([ 1, 2, 3, 4 ]);
const staticExecutor = staticPool.createExecutor({
  taskRetry: 2,
  taskTimeout: 2e3,
  transferList: [unit8Array.buffer]
});
  1. Methods of a StaticThreadPool Executor
staticExecutor
  .setTaskRetry(2)
  .exec('test')
  .then((rsp) => {
    console.log(rsp);
  });

Create a dynamic WorkerThreadPool pool

  1. Options of DynamicThreadPool:
const dynamicPool = new DynamicThreadPool({
  maxThreads: 24,
  maxTasks: 48,
  taskRetry: 1,
  taskLoopTime: 1e3,
  taskTimeout: 5e3,
});
  1. Attributes of a DynamicThreadPool instance
  1. Methods of a DynamicThreadPool instance
dynamicPool
  .setExecString(`module.exports = (payload) => console.log(payload);`)
  .setTaskRetry(1)
  .exec('payload-data', {
    taskTimeout: 5e3,
    taskRetry: 1,
  })
  .then((rsp) => {
    console.log(rsp);
  });

Create a dynamic WorkerThreadPool excutor

  1. Options of DynamicThreadPool Executor
const uint8Array = new Uint8Array([ 1, 2, 3, 4 ]);
const dynamicExecutor = dynamicPool.createExecutor({
  execString: `module.exports = (payload) => payload`,
  // execFunction: (payload) => payload,
  // execPath: 'path/to/executable.js',
  taskRetry: 2,
  taskTimeout: 2e3,
  transferList: [unit8Array.buffer]
});
  1. Methods of a DynamicThreadPool Executor

IX. Examples


  1. electronux - A project of mine that uses BroserService and MessageChannel of electron-re.

  2. file-slice-upload - A demo about parallel upload of multiple files, it uses ChildProcessPool and ProcessHost of electron-re, based on Electron@9.3.5.

  3. Also you can check the index.dev.js and test dir in root, there are some useful cases.

X. Test Coverage

------------------------------------|---------|----------|---------|---------|---------------------------------------------------------------------------------------------
File                                | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                                                                           
------------------------------------|---------|----------|---------|---------|---------------------------------------------------------------------------------------------
All files                           |   75.27 |    55.73 |   68.15 |   79.29 |                                                                                             
 lib                                |   96.42 |    66.66 |     100 |   96.42 |                                                                                             
  index.js                          |   96.42 |    66.66 |     100 |   96.42 | 23                                                                                          
 lib/libs                           |   66.83 |    44.61 |   59.12 |   72.59 |                                                                                             
  BrowserService.class.js           |   76.99 |    55.26 |   55.17 |   77.22 | 61-64,69-92,106-108,172,184-185,191,209-210,231,235                                         
  EventCenter.class.js              |      80 |    58.33 |   83.33 |     100 | 15-35                                                                                       
  FileWatcher.class.js              |      60 |    41.66 |   42.85 |   66.66 | 29-49                                                                                       
  MessageChannel.class.js           |   52.11 |    32.35 |    46.8 |   58.49 | 60-61,75-256,292-297,312-313,394-409,540-547                                                
  ProcessHost.class.js              |   70.73 |    35.71 |    62.5 |   76.31 | 27,51-59,80,92-108                                                                          
  ProcessLifeCycle.class.js         |   88.46 |    67.64 |   94.11 |   98.36 | 95                                                                                          
  consts.js                         |     100 |      100 |     100 |     100 |                                                                                             
  utils.js                          |   65.16 |    45.83 |      60 |   68.67 | 75-93,107-114,120-131,180,188                                                               
 lib/libs/ChildProcessPool          |   84.84 |       62 |      80 |   89.91 |                                                                                             
  ForkedProcess.js                  |   78.33 |       50 |   72.72 |   82.45 | 22,64-75,97,103-105                                                                         
  index.js                          |   86.76 |    65.78 |   82.35 |   92.39 | 119,141,219-223,231-233,287,306-315                                                         
 lib/libs/LoadBalancer              |   80.29 |       50 |   84.37 |   80.91 |                                                                                             
  consts.js                         |     100 |      100 |     100 |     100 |                                                                                             
  index.js                          |   77.58 |       50 |   82.14 |   78.18 | 63-69,83,97,103,113,126,153,158-162,178-193,203                                             
  scheduler.js                      |      95 |       50 |     100 |      95 | 28                                                                                          
 lib/libs/LoadBalancer/algorithm    |   94.79 |     67.3 |     100 |     100 |                                                                                             
  MINIMUM_CONNECTION.js             |    92.3 |    64.28 |     100 |     100 | 5-6,19                                                                                      
  POLLING.js                        |   85.71 |       50 |     100 |     100 | 5-9                                                                                         
  RANDOM.js                         |     100 |       50 |     100 |     100 | 7                                                                                           
  SPECIFY.js                        |     100 |       75 |     100 |     100 | 14                                                                                          
  WEIGHTS.js                        |   92.85 |    66.66 |     100 |     100 | 5-11                                                                                        
  WEIGHTS_MINIMUM_CONNECTION.js     |   94.11 |       80 |     100 |     100 | 5,15                                                                                        
  WEIGHTS_POLLING.js                |    92.3 |    66.66 |     100 |     100 | 5-10                                                                                        
  WEIGHTS_RANDOM.js                 |     100 |    66.66 |     100 |     100 | 9,17                                                                                        
  index.js                          |     100 |      100 |     100 |     100 |                                                                                             
 lib/libs/ProcessManager            |   51.08 |       25 |   38.46 |   51.77 |                                                                                             
  index.js                          |   57.21 |    31.39 |      48 |    58.6 | 66,71,76,81,110,123,132,143,150-169,175-224,228-230,236-245,285-287,295-298,303-306,311-336 
  ui.js                             |   32.35 |        0 |    6.66 |   32.83 | 34-122,130-137                                                                              
 lib/libs/WorkerThreadPool          |   81.52 |     57.5 |      78 |   84.72 |                                                                                             
  Task.js                           |   85.36 |    83.33 |   61.53 |   85.36 | 44-45,77-92                                                                                 
  TaskQueue.js                      |   66.19 |    39.28 |   86.66 |   69.23 | 78-79,98-139                                                                                
  Thread.js                         |      85 |     64.7 |      80 |   91.78 | 61-63,150-156,165                                                                           
  consts.js                         |     100 |      100 |     100 |     100 |                                                                                             
  index.js                          |     100 |      100 |     100 |     100 |                                                                                             
  utils.js                          |   91.66 |       50 |     100 |   91.66 | 23                                                                                          
 lib/libs/WorkerThreadPool/Executor |   81.53 |     67.5 |   80.76 |   89.79 |                                                                                             
  DynamicExecutor.js                |   76.66 |    71.42 |      75 |   84.09 | 50-53,66-67,97                                                                              
  Executor.js                       |   93.75 |    66.66 |     100 |   93.75 | 70,74                                                                                       
  StaticExecutor.js                 |   78.94 |       60 |      75 |   95.45 | 39                                                                                          
 lib/libs/WorkerThreadPool/Pool     |   83.37 |    72.41 |   81.01 |   87.95 |                                                                                             
  DynamicThreadPool.js              |    82.5 |    66.12 |   76.47 |   90.56 | 87-89,129,141                                                                               
  StaticThreadPool.js               |   87.69 |    70.83 |   92.85 |   97.95 | 144                                                                                         
  ThreadPool.js                     |   82.57 |    75.49 |   79.16 |   85.21 | 116-118,144-145,165,213-214,224,256-258,356,494-533,564,568,572,576,587                     
 lib/libs/WorkerThreadPool/Worker   |   77.35 |       60 |    62.5 |   81.01 |                                                                                             
  index.js                          |   84.28 |    67.85 |   76.47 |   95.34 | 49,73                                                                                       
  worker-runner.js                  |   63.88 |    41.66 |   28.57 |   63.88 | 19-22,32,36-42,54-59,65,76                                                                  
 lib/tasks                          |   83.33 |       50 |     100 |   88.23 |                                                                                             
  app.init.js                       |   83.33 |       50 |     100 |   88.23 | 33-39                                                                                       
------------------------------------|---------|----------|---------|---------|---------------------------------------------------------------------------------------------