Concurrency programming in bash script with timeout control
Review of Concurrent Execution in Bash
#!/usr/bin/env bash
PIDARR=()
date
for c in {1..10};do
{
echo $c
sleep $c
} &
PIDARR+=("$!")
done
wait ${PIDARR[@]}
date
Objective
To execute a task in a controllable, multi-process, concurrent manner, and stop the script after the maximum execution time.
Idea
- Use
fifo
pipeline files and pre-filln
values (equal to the number of concurrent processes). - Record the script’s own
PID
and start the timer process (terminate the timer and kill the script’s ownPID
after timing out). - Concurrent execution.
- After normal completion, kill the timer subprocess and exit the main process.
- Recycle
fd
.
Control Parameters
MAX_CONCURRENCY
Number of processesMAX_EXEC_TIME
Maximum execution time
Concurrency without time control
#!/usr/bin/env bash
trap "exec 10>&-;exec 10<&-;exit 0" 2
mkfifo testfifo
exec 10<>testfifo
rm -f testfifo
MAX_CONCURRENCY=10
for x in $(seq 1 ${MAX_CONCURRENCY}); do
echo >&10
done
for x in {001..100}; do
read -u10
{
### do something
echo "success:$x"
sleep 2
echo >&10
} &
done
wait
exec 10>&-
exec 10<&-
exit 0
Concurrency with time control
#!/usr/bin/env bash
## create fd
[[ -e ./fd1 ]] || mkfifo ./fd1
exec 10<>./fd1
rm -rf ./fd1
MAX_CONCURRENCY=3
for i in $(seq 1 ${MAX_CONCURRENCY}); do
echo >&10
done
## Define max execuate time here
MAX_EXEC_TIME=30
## echo $$ -> PPID
## pass PID of this process to backgroud process
T_PPID=$(echo $$)
## time tic-toc
{
sleep ${MAX_EXEC_TIME} && exec 10<&- && exec 10>&- && echo "[ERROR] ABORT and EXIT" && kill ${T_PPID}
} &
## echo $! -> PID
## remember latest backgroud process PID
T_PID=$(echo $!)
## task list
for TASK in $(seq 1 10); do
read -u10
{
## do something here
echo "start_task_${TASK}: $!"
sleep 3
echo >&10
} &
done
kill ${T_PID}
wait
## close fd read & write
exec 10<&-
exec 10>&-
## normal exit
echo "[INFO] NORMAL EXIT" && exit 0
Leave a message
Disclaimer
- Welcome to visit the knowledge base of SRE and DevOps!
- License under CC BY-NC 4.0
- Made with Material for MkDocs and improve writing by generative AI tools
- Copyright issue feedback me#imzye.com, replace # with @