shell 语言高级技巧

【外文翻译】shell 语言高级技巧

Advanced Shell Scripting Techniques: Automating Complex Tasks with Bash

Bash scripting, a cornerstone of Unix and Linux system administration, offers powerful tools to automate repetitive tasks, streamline workflows, and handle complex operations. For those already comfortable with basic scripting, diving into advanced techniques can unlock new levels of efficiency and capability. This post will explore advanced shell scripting techniques in Bash, focusing on script optimization, robust error handling, and automating complex system administration tasks.

Script Optimization

Optimization is crucial for ensuring that your scripts run efficiently, especially when dealing with large datasets or intensive tasks. Here are some key techniques to optimize your Bash scripts.

Use Built-in Commands

Whenever possible, leverage built-in shell commands rather than external binaries. Built-in commands execute faster because they don’t require loading an external process. For example, use [[ ]] for conditionals instead of [ ] or test.

最好使用内置的命令，少用外部的

# Inefficient
if [ "$var" -eq 1 ]; then
    echo "Equal to 1"
fi

# Efficient
if [[ "$var" -eq 1 ]]; then
    echo "Equal to 1"
fi

Minimize Subshells

Subshells can be expensive in terms of performance. Avoid them when possible by using built-in commands or parameter expansion.

通过使用内置命令或者是参数扩展来达到尽量少用子shell的目的

# Inefficient
output=$(cat file.txt)

# Efficient
output=$(<file.txt)

上面这两个都是变量替换，第二个我也是才见到！

aa=`cat /etc/hosts`
cat $aa
echo $aa
bb=`< /etc/hosts`
cat $bb
echo $bb
cc=1234
echo $cc
dd="123\nasdasd\n"
echo $dd
# cat $dd

Use Arrays for Bulk Data

When handling a large amount of data, arrays can be more efficient and easier to manage than multiple variables.

# Inefficient
item1="apple"
item2="banana"
item3="cherry"

# Efficient
items=("apple" "banana" "cherry")
for item in "${items[@]}"; do
    echo "$item"
done

数据需要记录很多的话，最好使用array来保存

Enable Noclobber

To prevent accidental overwriting of files, use the noclobber option. This is particularly useful in scripts that generate temporary files.

set -o noclobber

这个我也不清楚，知道的读者可以说一下

Efficient File Operations

When performing file operations, use efficient techniques to minimize resource usage.

# Inefficient
while read -r line; do
    echo "$line"
done < file.txt

# Efficient
while IFS= read -r line; do
    echo "$line"
done < file.txt

最小化使用范围

Parallel Processing

For tasks that can be executed concurrently, consider using parallel processing to speed up your scripts. Tools like xargs and GNU parallel can be incredibly useful.

# Using xargs for parallel processing
cat urls.txt | xargs -n 1 -P 4 curl -O

能同时执行的就最好并发处理

Error Handling

Robust error handling is critical for creating reliable and maintainable scripts. Here are some techniques to enhance error handling in your Bash scripts.

Exit on Error

Using set -e ensures that your script exits immediately if any command fails, preventing cascading errors.

set -e

脚本中最好加入set -e 这个选项，因为可以在命令行发生错误的时候立即结束脚本，防止之后的错误产生。

Custom Error Messages

Implement custom error messages to provide more context when something goes wrong.

command1 || { echo "command1 failed"; exit 1; }

发生错误的时候最好有详细的提示

Trap Signals

Use the trap command to catch and handle signals and errors gracefully.

trap 'echo "Error occurred"; cleanup; exit 1' ERR

function cleanup() {
    # Cleanup code
}

多使用trap处理信号和errors

Validate Inputs

Always validate user inputs and script arguments to prevent unexpected behavior.

if [[ -z "$1" ]]; then
    echo "Usage: $0 <argument>"
    exit 1
fi

用户输入一定要核实

Logging

Implement logging to keep track of script execution and diagnose issues.

logfile="script.log"
exec > >(tee -i $logfile)
exec 2>&1

echo "Script started"

日志记录脚本的执行状态.

exec 重定义文件描述符，标准输出到后面的进程的标准输入，标准错误和标准输出一样重定向到tee这个进程中。

Automating Complex System Administration Tasks

Advanced shell scripting can greatly simplify complex system administration tasks. Here are a few examples.

Automated Backups

Creating automated backup scripts ensures that critical data is regularly saved and can be restored in case of failure.

#!/bin/bash

set -e
trap 'echo "Backup failed"; exit 1' ERR

backup_dir="/backup"
timestamp=$(date +%Y%m%d%H%M%S)
backup_file="${backup_dir}/backup_${timestamp}.tar.gz"

# Create a backup
tar -czf "$backup_file" /important_data

echo "Backup completed: $backup_file"

自动备份脚本

System Monitoring

Automate system monitoring to detect and respond to issues proactively.

#!/bin/bash

threshold=80
partition="/dev/sda1"

usage=$(df -h | grep "$partition" | awk '{print $5}' | sed 's/%//')

if [[ "$usage" -gt "$threshold" ]]; then
    echo "Disk usage on $partition is above $threshold%"
    # Add code to handle high disk usage
fi

自动监控脚本

User Management

Streamline user management tasks such as adding or removing users.

#!/bin/bash

function add_user() {
    local username=$1
    useradd "$username" && echo "User $username added successfully"
}

function remove_user() {
    local username=$1
    userdel "$username" && echo "User $username removed successfully"
}

case $1 in
    add)
        add_user "$2"
        ;;
    remove)
        remove_user "$2"
        ;;
    *)
        echo "Usage: $0 {add|remove} <username>"
        exit 1
        ;;
esac

目前没有理解这个脚本到底有啥用。

Automated Updates

Ensure your systems are always up-to-date with automated update scripts.

#!/bin/bash

set -e
trap 'echo "Update failed"; exit 1' ERR

apt-get update && apt-get upgrade -y

echo "System updated successfully"

Network Configuration

Automate network configuration tasks to quickly set up new systems.

#!/bin/bash

function configure_network() {
    local interface=$1
    local ip_address=$2
    local gateway=$3

    cat <<EOF > /etc/network/interfaces
auto $interface
iface $interface inet static
    address $ip_address
    gateway $gateway
EOF

    systemctl restart networking
    echo "Network configured on $interface"
}

configure_network "eth0" "192.168.1.100" "192.168.1.1"

文本连接

Advanced Shell Scripting Techniques: Automating Complex Tasks with Bash | Omid Farhang

Further Reading

• Advanced Bash-Scripting Guide
• Bash Reference Manual
• Linux Command Line and Shell Scripting Bible