Linux环境Shell脚本上传下载阿里云OSS文件
背景
工作中由于我们项目生成的日志文件比较重要,而本地磁盘空间有限存储不了多久,因此考虑备份方案,我们原本打算保存在nas上,然而由于各种原因与运维沟通下来建议保存到oss上面。
由于linux原生支持shell,而网上大多数方案基于python-sdk,因此我们为了减少依赖,考虑直接使用shell脚本上传OSS,网上找了些资料,参见:
坑
然而脚本试用下来有坑,特地记录一下:
- 字符比较提示异常
上面截图字符比较会提示:
./oss.sh: 13: ./oss.sh: [get: not found
./oss.sh: 16: ./oss.sh: [put: not found
./oss.sh: 32: ./oss.sh: [put: not found
应该改成上面的格式
2.拼接url的时候把bucket也带进去了。 3.拼接签名不对,研究了很久发现不应该用“#!/bin/sh”,而需要使用“#!/bin/bash”,这是个大坑。。。
修改版本
下面给出修改版本,需要自取:
#!/bin/bash
host="oss-cn-shanghai.aliyuncs.com"
bucket="bucket名"
Id="AccessKey ID"
Key="Access Key Secret"
# 参数1,PUT:上传,GET:下载
method=$1
# 参数2,上传时为本地源文件路径,下载时为oss源文件路径
source=$2
# 参数3,上传时为OSS目标文件路径,下载时为本地目标文件路径
dest=$3
osshost=$bucket.$host
# 校验method
if test -z "$method"
then
method=GET
fi
if [ "${method}"x = "get"x ] || [ "${method}"x = "GET"x ]
then
method=GET
elif [ "${method}"x = "put"x ] || [ "${method}"x = "PUT"x ]
then
method=PUT
else
method=GET
fi
#校验上传目标路径
if test -z "$dest"
then
dest=$source
fi
echo "method:"$method
echo "source:"$source
echo "dest:"$dest
#校验参数是否为空
if test -z "$method" || test -z "$source" || test -z "$dest"
then
echo $0 put localfile objectname
echo $0 get objectname localfile
exit -1
fi
if [ "${method}"x = "PUT"x ]
then
resource="/${bucket}/${dest}"
contentType=`file -ib ${source} |awk -F ";" '{print $1}'`
dateValue="`TZ=GMT date +'%a, %d %b %Y %H:%M:%S GMT'`"
stringToSign="${method}\n\n${contentType}\n${dateValue}\n${resource}"
signature=`echo -en $stringToSign | openssl sha1 -hmac ${Key} -binary | base64`
echo $stringToSign
echo $signature
url=http://${osshost}/${dest}
echo "upload ${source} to ${url}"
curl -i -q -X PUT -T "${source}" \
-H "Host: ${osshost}" \
-H "Date: ${dateValue}" \
-H "Content-Type: ${contentType}" \
-H "Authorization: OSS ${Id}:${signature}" \
${url}
else
resource="/${bucket}/${source}"
contentType=""
dateValue="`TZ=GMT date +'%a, %d %b %Y %H:%M:%S GMT'`"
stringToSign="${method}\n\n${contentType}\n${dateValue}\n${resource}"
signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${Key} -binary | base64`
url=http://${osshost}/${source}
echo "download ${url} to ${dest}"
curl --create-dirs \
-H "Host: ${osshost}" \
-H "Date: ${dateValue}" \
-H "Content-Type: ${contentType}" \
-H "Authorization: OSS ${Id}:${signature}" \
${url} -o ${dest}
fi
执行命令:
#上传
$ ./oss.sh put a.gz c.gz
#下载
$ ./oss.sh get c.gz d.gz
2018-11-21更新:
今天看到阿里云提供ossutil64,详见:https://help.aliyun.com/document_detail/50452.html 有了这个方便很多。