NODEJS(13)Platform - Async.js

NODEJS(13)Platform - Async.js

1. Async.js Introduction
It is designed for use with Node.js, but it can also be used directly in the browser.

It support series, parallel, waterfall.

>sudo npm install -g async

There are 3 main parts, Flows, Collections, Utils.

2. Flows
series(tasks, [callback])
We have multiple functions which need to execute, they do not need to talk to each other, just need them execute one by one.
step1(function(err,v1){
     step2(function(err,v2){
          step3(function(err,v3){
               //do something, we can get the err or values v1/v2/v3
          }
     }
}
———>
var async = require(‘async’)
async.series([step1, step2, step3],
     function(err, values){
          //err and values including v1/v2/v3
});

Support JSON format

Functions will be executed by turns, if one of the functions came across err, the series will stop and call the callback method with error information immediately, the left functions will not be executed.

Some examples
https://github.com/freewind/async_demo

parallel(tasks, [callback])
all the functions will be executed in parallel, the callback result will include the results in order of the tasks defined, NOT in the order of who executed complete first who be the first.
async.parallel([
     function(cb){ t.fire(‘a400’, cb, 400) },
     function(cb){ t.fire(‘a200’, cb, 200) },
     function(cb){ t.fire(‘a300’, cb, 300) } 
], function (err, results) {
     log(‘1.1 err: ‘, err); // undefined
     log(‘1.1 results: ‘, results); // [‘a400’, ‘a200’, ‘a300’ ]
})

Support JSON format

https://github.com/freewind/async_demo/blob/master/parallel.js

waterfall(tasks, [callback])
execute in turns, the next function need the result from previous function as parameter.

NOT support JSON
async.waterfall([
     function(cb) { log(‘1.1.1: }, ‘start’); cb(null, 3),
     function(n, cb) { log(‘1.1.2: ', n); t.inc(n,cb); },
     function(n, cb) { log(‘1.1.3: ‘, n); t.fire(n*n, cb); }
], function(err, result){
     log(‘1.1 err: ‘, err); // null
     log(‘1.1 result: ‘ , result); // 16  ——> 3 + 1, 4*4 = 16
});

auto(tasks, [callback])
https://github.com/freewind/async_demo/blob/master/auto.js

Some series, some parallel.

whilst(test, fn, callback) 
util(test, fn, callback)

queue
powerful parallel, there is an idea of workers, system will not execute the tasks at one time, it depends on the workers.

var q = async.queue(function(task, callback) {
     log(‘worker is processing task: ‘, task.name);
     task.run(callbck);
}, 2); 

When we use out all the workers, system will call saturated
q.saturated = function(){
     log(‘ all workers to be used’);
}

When last tasks is processing, system will call empty
q.empty = function(){
     log(‘ no more tasks waiting’);
}

When all tasks are done, system will call drain
q.dran = function(){
     log(‘all tasks have been done’);
}

How to Put the Tasks
q.push({name: ‘t1’, run: function(cb) {
     log(‘t1 is running, waiting tasks: ‘, q.length());
     t.fire(‘t1’, cb, 400);//400ms delay
}}, function (err){
     log(‘t1 executed’);
});

https://github.com/freewind/async_demo/blob/master/queue.js

q.push([{name:'t3', run: function(cb){

    log('t3 is running, waiting tasks: ', q.length());
    t.fire('t3', cb, 300); // 300ms后执行
}},{name:'t4', run: function(cb){
    log('t4 is running, waiting tasks: ', q.length());
    t.fire('t4', cb, 500); // 500ms后执行
}}], function(err) {
    log('t3/4 executed');
});

iterator(tasks)

3. Utils
memoize(fn, [hasher])
something like cache, and we can define the hasher to decide how to generate the cache key.

Image we have a function which is very slow. slow_fn(x, y, callback)

var fn = async.memoize(slow_fn);
That will make fn to be a cache fn.
fn(‘x’,’b’, function(err,result){
     console.log(result);
});
fn(‘x’,’b’, function(err,result){
     console.log(result);
}); // the cache will hit next time when we call 

Change the cache key rules
var fn_hasher = async.memoize(slow_fn, function(x, y){
     return x+y;
});

unmemoize(fn) <——> memoize(fn, hasher)
var fn_slow_2 = async.unmemoize(fn);

3. Collections
forEach(arr, iterator(item, callback), callback(err))
all the function in iterator will run in parallel

var arr = [{name:’Jack’, delay:200},
               {name:’Mike’, delay:100},
               {name:’Freeman’, delay:300}];
async.forEach(arr, function(item,callback){
     log(‘1.1 enter:’ + item.name);
     setTimeout(function(){
          log(‘1.1 handle:’ + item.name);
          callback();
     },item.delay);
},function(err){
     log(‘1.1 err:’+err);
});

If we want them run series mode, we need to use forEachSeries

If we want to make them more complex, some run in parallel, some run series. We can use forEachLimit
async.forEachLimit(arr, 2, function(item, callback){
     …snip...
}, function(err){
     …snip...
});

every 2 will be a group, they will run in parallel, between groups, they will be series.

https://github.com/freewind/async_demo/blob/master/forEach.js

map(arr, iterator(item, callback), callback(err, results))
The most different part is the results, if you need the results then use map, No care about result, use forEach.

async.map(arr, function(item, callback){
     setTimeout(function(){ callback(null, item.name + ‘!!!”);},item.delay);
}, function(err, results){
     log(‘1.1 err: ‘, err); log(‘1.1 results:’, results);
});

The same things mapSeries, more example https://github.com/freewind/async_demo/blob/master/map.js

filter(arr, iterator(item, callback(test)),callback(results))
filter all the items >=3
async.filter([1,2,3,4,5], function(item,callback){
     setTimeout(function(){ callback(item>=3); },200);
},function(results){
     log(‘1.1 results: ‘, results);  //[3,4,5]
});
The same for filterSeries  https://github.com/freewind/async_demo/blob/master/filter_reject.js

reject(arr, iterator(item, callback(test)),callback(results)) <—> filter

detect(array, iterator(item, callback(test)), callback(result))
Find the first one match the test case. <——>detectSeries

sortBy(array, iterator(item, callback(err, result)), callback(err, results))
sort all the items based on the result, from small to large number.

some/any(arr, iterator(item, callback(test)),callback(result))
Find at least one item in array match the test case

every/all(arr, iterator(item, callback(test)),callback(result))
Find all/every item in array match the test case

concat(arr, iterator(item, callback(err, result)), callback(err, result))
concat all the results together to make a big array

References:
Async
https://github.com/caolan/async
http://blog.fens.me/nodejs-async/

http://freewind.me/blog/20120515/917.html
http://freewind.me/blog/20120517/931.html
http://freewind.me/blog/20120518/932.html