Dispatch tasks among muliple workors

Sometimes we many have many tasks to be executed by several workors. we wish tasks can be executed by workors in equal division

Basic solution:

1. put all tasks into a PIPO pipeline.

2. Bring up several workors.

3. Each workors do loop to fetch a task from pipeline, until no task in the pipeline, quit workd

#!/bin/ksh

export PIPEFD=9
export PIPEFILE=pipefile
export PIPEEND=TASKEND
export MAXWORKER=3

echo "=== BEGIN ==="

# Define an FIFO pipeline
rm -f $PIPEFILE
mkfifo $PIPEFILE

# Redirect stdin/stdout to pipeline file
eval "exec ${PIPEFD}<>${PIPEFILE}"

# Simulate the task list into pipeline file
echo "aaa bbb ccc ddd eee fff ggg hhh" | tr ' ' '\n' >&${PIPEFD}

# Add task end flag into pipeline file
for WORKOR in {1..$MAXWORKER}; do
  echo "${PIPEEND}" >&${PIPEFD}
done

# Launch workers in parallel
for WORKOR in {1..$MAXWORKER}; do
  while read TASK; do
    [[ "${TASK}" == "${PIPEEND}" ]] && exit 0
    sleep 1
    echo "WORKOR["${WORKOR}"]="${TASK}
    # TODO: execute TASK using WORKOR
  done <&${PIPEFD} &
  PIDS="$PIDS $!"
done

# Wait all workor done
wait $PIDS

# Close FD for pipeline
eval "exec ${PIPEFD}>&-"
eval "exec ${PIPEFD}<&-"

echo "===  END  ==="

Sample execute output:

=== BEGIN ===
WORKOR[3]=ccc
WORKOR[2]=bbb
WORKOR[1]=aaa
WORKOR[3]=ddd
WORKOR[1]=fff
WORKOR[2]=eee
WORKOR[1]=hhh
WORKOR[3]=ggg
===  END  ===