shell 常用例子(持续更新)

本文介绍了多个Shell脚本的实用技巧,包括检查文件中是否存在特定字符串并添加,检测端口占用情况,判断类是否在JAR文件中,处理命令执行顺序,从yaml文件中提取值,实现命令执行重试机制以及如何设置变量的默认值。这些技巧对于系统管理和自动化任务非常有用。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

1. 判断文件中是否存在某个字符串,如果不存在则添加

#!/usr/bin/env bash


strs=( "username soft nproc 65535" "username hard nproc 65535" )
file="/etc/security/limits.conf"
# 当数组中有空格时,数组引用需要用双引号包住
for str in "${strs[@]}"
do
# 判断 所需的字符串 是否在文件中存在如果存在则不进行操作
  is_exist=$(cat "${file}"|grep "${str}")
  if [ "${is_exist}"x == "x" ]; then
   echo "Ready to write ${str} to ${file}"
   echo "${str}" >> ${file}
   if [ $? == 0 ]; then
     echo "Write successfully"
   else
     echo "Write failure!"
   fi
  else
   echo "The ${str} already exists in the  ${file}"
  fi
done

 

2. 判断某台机器下是否有这些端口被占用

#!/usr/bin/env bash

ports=( 8030 9020 9030 9010 9060 8040 9050 8060 8000 )
for num in ${ports[@]}
do
  netstat -tunlp |grep $num
done

 

t:显示TCP传输协议的连线状况
u: 显示UDP传输协议的连线状况。
n:展示ip和端口为程序地址,而不是域名。
l: 显示正在listen的Socket
p:显示pid和程序名称

查找此时在被listen的(TCP、UDP传输协议)程序,并展示(以ip+port为格式的)地址、pid和程序名称

 

3. 判断一个类是否在lib目录(包含jar)下存在

#!/usr/bin/env bash


ls /${hive_home}/lib | while read one_line
do
 class_name=$(jar -vtf $one_line |grep CustomInDBAuthenticationProviderImpl)
 if [[  ${class_name}x != "x"  ]]; then
   echo "jar:$one_line  contains the ${class_name}"
 fi
done

v:详细输出,即类的全限定名
t: 展示jar中的内容,比如包含哪些类、有哪些文件等
f: 需要指定jar包的文件名

 

4.当前面的命令执行成功影响后面命令的执行时

判断上一条命令是否执行成功,如果没有执行成功,则下面的逻辑肯定不会执行成功

。。。
exit_code=$?
if [ $exit_code -ne 0 ];then
  exit $exit_code
fi
if  [  "${20}"  ]; then
    echo "starting to load data to mysql using mysql load"
    bash  $F_HOME/bin/load_function.sh  "${20}"
fi

 

5. 获取yaml文件的值

#!/usr/bin/env bash

declare -A config_map

while read line
do
  #行为空时跳过这行
  if  [  ! "${line}"  ]; then
    continue
  fi
  conf_name=$(echo ${line} | cut -d : -f 1 | sed 's/ //g' )
  conf_value=$(echo ${line} | cut -d : -f 2 |  sed 's/ //g' )
  # 如果值为空则设置默认值:bigdata
  config_map[${conf_name}]=${conf_value:-bigdata}
  echo "get config_map [ ${conf_name} : ${config_map[${conf_name}]} ]"
done < bigdata_conf.yaml

bigdata_conf.yaml 文件

zookeeper:
  zookeeper1: node1
  zookeeper2: node2
  zookeeper3: node3

hdfs:
  namenode1: node1
  namenode2: node2
  journalnode1: node1
  journalnode2: node2
  journalnode3: node3
# gao
yarn:
  resourcemanager1: node1
  resourcemanager2: node2
  jobhistorynode: node3

输出

。。。
get config_map [ zookeeper : bigdata ]
get config_map [ zookeeper1 : node1 ]
get config_map [ zookeeper2 : node2 ]
get config_map [ zookeeper3 : node3 ]
。。。

 

6. shell重试脚本

#!/bin/sh
#记录重试次数
count=0     
# 重试标识,flag=0 表示任务正常,flag=1 表示需要进行重试
flag=0      
while [ 0 -eq 0 ]
do
    echo ".................. job begin  ..................."
    # ...... 添加要执行的内容,flag 的值在这个逻辑中更改为1,或者不变......
    
    sleep 10
    port_cnt=`netstat -anp | grep 9083 | grep LISTEN | wc -l`
    thread_cnt=`ps -ef | grep -v "grep " | grep "org.apache.hadoop.hive.metastore.HiveMetaStore" -c`
    if [[ $port_cnt -lt 1 || $thread_cnt -lt 1 ]];then
        flag=1
    else
        flag=0
    fi
    
    # 检查和重试过程   
    if [ flag -eq 0 ]; then     #执行成功,不重试
        echo "--------------- job complete ---------------"
        break;
    else                        #执行失败,重试
        count=$[${count}+1]
        if [ ${count} -eq 3 ]; then     #指定重试次数,重试超过4次即失败
            echo 'timeout,exit.'
            break
        fi
        echo "...............retry in 1 seconds .........."
        sleep 1
    fi
done

 

7. 变量设定默认值

语法说明:

#当变量a为null时则var=b  
var=${a-b}  

#当变量a为null或为空字符串时则var=b  
var=${a:-b}  
### 如果第16个入参为null或者为空字符串,则赋值为30
idle_state_retention_time=", \"table.exec.state.ttl\":\"$(( ${16:-30}* 60000 ))\""

 

打包

mac文件夹打包

#!/usr/bin/env bash

# 用于mac电脑下的打包脚本

# todo: 如未下载,先下载软件:brew install gnu-tar

#脚本位置锚定
cur_dir=$(dirname $(readlink -f $0))
cd $cur_dir
cd ..



# 删除系统文件
find ./docker_middle_lakehouse -name ".DS_Store" -delete

# 打包到此项目的上一级目录
ARCHIVE_FILE=docker_middle_lakehouse.tar.gz
gtar -zcvf $ARCHIVE_FILE \
  --exclude='docker_middle_lakehouse/.idea' \
  --exclude='docker_middle_lakehouse/.git' \
  --exclude='docker_middle_lakehouse/lakehouse/minio/data' \
  --exclude='docker_middle_lakehouse/lakehouse/neo4j/config' \
  --exclude='docker_middle_lakehouse/lakehouse/neo4j/data' \
  --exclude='docker_middle_lakehouse/lakehouse/neo4j/logs' \
  --exclude='docker_middle_lakehouse/lakehouse/neo4j/plugins' \
  --exclude='docker_middle_lakehouse/lakehouse/starrocks/data' \
  --exclude='docker_middle_lakehouse/lakehouse/weaviate/archive' \
  --exclude='docker_middle_lakehouse/lakehouse/weaviate/backup' \
  --exclude='docker_middle_lakehouse/lakehouse/weaviate/data' \
  --exclude='docker_middle_lakehouse/.gitignore' \
  --exclude='docker_middle_lakehouse/docker_middle_lakehouse.iml' \
  --exclude='docker_middle_lakehouse/package.sh' \
  docker_middle_lakehouse


# 检查打包命令是否成功执行
if [ $? -eq 0 ]; then
    # 获取当前工作目录
    CURRENT_DIR=$(pwd)
    # 打印打包文件的存放位置
    echo "打包成功!打包文件存放在:$CURRENT_DIR/$ARCHIVE_FILE"
else
    echo "打包失败,请检查相关错误信息。"
fi


Shell脚本高级编程教程,希望对你有所帮助。 Example 10-23. Using continue N in an actual task: 1 # Albert Reiner gives an example of how to use "continue N": 2 # --------------------------------------------------------- 3 4 # Suppose I have a large number of jobs that need to be run, with 5 #+ any data that is to be treated in files of a given name pattern in a 6 #+ directory. There are several machines that access this directory, and 7 #+ I want to distribute the work over these different boxen. Then I 8 #+ usually nohup something like the following on every box: 9 10 while true 11 do 12 for n in .iso.* 13 do 14 [ "$n" = ".iso.opts" ] && continue 15 beta=${n#.iso.} 16 [ -r .Iso.$beta ] && continue 17 [ -r .lock.$beta ] && sleep 10 && continue 18 lockfile -r0 .lock.$beta || continue 19 echo -n "$beta: " `date` 20 run-isotherm $beta 21 date 22 ls -alF .Iso.$beta 23 [ -r .Iso.$beta ] && rm -f .lock.$beta 24 continue 2 25 done 26 break 27 done 28 29 # The details, in particular the sleep N, are particular to my 30 #+ application, but the general pattern is: 31 32 while true 33 do 34 for job in {pattern} 35 do 36 {job already done or running} && continue 37 {mark job as running, do job, mark job as done} 38 continue 2 39 done 40 break # Or something like `sleep 600' to avoid termination. 41 done 42 43 # This way the script will stop only when there are no more jobs to do 44 #+ (including jobs that were added during runtime). Through the use 45 #+ of appropriate lockfiles it can be run on several machines 46 #+ concurrently without duplication of calculations [which run a couple 47 #+ of hours in my case, so I really want to avoid this]. Also, as search 48 #+ always starts again from the beginning, one can encode priorities in 49 #+ the file names. Of course, one could also do this without `continue 2', 50 #+ but then one would have to actually check whether or not some job 51 #+ was done (so that we should immediately look for the next job) or not 52 #+ (in which case we terminate or sleep for a long time before checking 53 #+ for a new job).
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

roman_日积跬步-终至千里

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值