Skip to content

Concurrency in shell scripts with timeout

并发执行回顾

#!/usr/bin/env bash

PIDARR=()
date
for c in {1..10};do
{
    echo $c
    sleep $c
} &
    PIDARR+=("$!")
done
wait ${PIDARR[@]}
date

目标

以可控制的多进程方式并发执行某任务,达到最大执行时长后停止脚本。

思路

  1. 使用fifo管道文件,并预填充n个值(与并发数相等)
  2. 记录脚本本身PID并启动计时器进程(计时终止后杀脚本本身PID
  3. 并发执行
  4. 正常执行完毕后,杀掉计时器子进程,并退出主进程
  5. 回收fd

控制参数

  • PROC 进程数
  • MAX_EXEC_TIME 最大执行时长

不带时间控制版本

#!/usr/bin/env bash

trap "exec 10>&-;exec 10<&-;exit 0" 2
mkfifo testfifo
exec 10<>testfifo
rm -f testfifo
MAX_CONCURRENCY=10
for x in $(seq 1 ${MAX_CONCURRENCY}); do
    echo >&10
done
for x in {001..100}; do
    read -u10
    {
        ### do something
        echo "success:$x"
        sleep 2
        echo >&10
    } &
done
wait
exec 10>&-
exec 10<&-
exit 0

带时间控制版本

#!/usr/bin/env bash

## create fd
[[ -e ./fd1 ]] || mkfifo ./fd1
exec 10<>./fd1
rm -rf ./fd1

MAX_CONCURRENCY=3
for i in $(seq 1 ${MAX_CONCURRENCY}); do
    echo >&10
done

## Define max execuate time here
MAX_EXEC_TIME=30

## echo $$ -> PPID
## pass PID of this process to backgroud process
T_PPID=$(echo $$)

## time tic-toc
{
    sleep ${MAX_EXEC_TIME} && exec 10<&- && exec 10>&- && echo "[ERROR] ABORT and EXIT" && kill ${T_PPID}
} &

## echo $! -> PID
## remember latest backgroud process PID
T_PID=$(echo $!)

## task list
for TASK in $(seq 1 10); do
    read -u10
    {
        ## do something here
        echo "start_task_${TASK}: $!"
        sleep 3
        echo >&10
    } &
done
kill ${T_PID}
wait

## close fd read & write
exec 10<&-
exec 10>&-

## normal exit
echo "[INFO] NORMAL EXIT" && exit 0