swift 安装历程

安装环境：ubuntu 11.10 desktop版本
参考文献： http://docs.openstack.org/developer/swift/development_saio.html#loopback-section
                 http://limu713.blog.163.com/blog/static/150869042011645655986/
                 http://wenku.baidu.com/view/aa04a27401f69e314332940d.html

                 http://www.cnblogs.com/yuxc/archive/2011/09/24/2188295.html

NOTE：step1--5 均需以root身份进行相关操作。当前用户修改root用户的命令可采取sudo passwd root命令

step1.  安装依赖软件：
            1.apt-get install python-software-properties(安装源仓库添加工具包)
            2.add-apt-repository ppa:swift-core/release(添加swift仓库 add-apt-repository ppa:swift-core/ppa不知道这两者之间有什么区别，我选择了前者)
            3.apt-get update(更新源列表，最好先把源改为最佳源，我用的是163的源。参考文章linux常见问题集）
            4.apt-get install curl gcc git-core memcached python-coverage python-devpython-nose python-setuptools python-simplejson python-xattr sqlite3xfsprogs python-eventlet python-greenlet python-pastedeploypython-netifaces python-pip
            5.pip install mock(note: pip is a tool for installing and managing Python packages

step2：创建swift用户

sudo useradd -mk /home/swift -s /bin/bash swift

sudo password swift
       groupadd swift
       usermod -a -G swift swift（由于我创建的swift组里面仅仅包含swift用户，而swift用户也仅仅属于swift组，所以我没有添加-a选项）
        note:groupadd swift ; useradd swift -g swift 也是可行的。

 

编辑/etc/sudoers，添加以下一行

# User alias specification

swift   ALL=(ALL) NOPASSWD:ALL
note:在这一步的时候我出现了不能添加的问题，这是由于sudoers只是可读文件，不具有写权限。这个也很简单，直接用chmod命令修改即可添加，添加完成后还得修改为回原来的只读属性。我是后来在swift用户下运行sudo XX XX命令的时候提示出错的，按照提示错处再次通过chmod命令修改文件属性即可解决该问题。



step3. 建立存储区域
note:值得注意的是这里有两种方式建立存储区域，可以使用单独的分区或者环路设备来作为存储区域。之前采用了单独分区结果一直出错，特别是在分出一个区来格式化为xfs格式的时候出现了一系列问题。究其根本是对于linux文件系统的不熟悉与linux这种命令行操作方式的不熟悉导致的。虽然经过痛苦的出错纠错后成功了解决了这个问题。却在后面tempauth认证中出错了。最后忍痛直接重新开始做，采用了环路设备来存储，结果得到顺利进行下去了。与此同时为了后面需要用到新建用户来使用swift，自己也安装网上创建了swift：swift，当然你也可以创建其他用户名和用户组，

        dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000

       mkfs.xfs -i size=1024 /srv/swift-disk
       Edit /etc/fstab and add   /srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0
        mkdir /mnt/sdb1
       mount  /mnt/sdb1

mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4

chown -R swift:swift /mnt/sdb1/*

mkdir /srv

for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done

mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server

/srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift

chown -R swift:swift /etc/swift /srv/[1-4]/ /var/run/swift 

//注意不要忘记/srv/[1-4]/后面的这个斜杠

//请务必验证/srv下各个目录的属主正确性，以及链接的正确性

        最后，把下面两行加入到/etc/rc.local文件的exit 0之前。

mkdir /var/run/swift

chown –R swift:swift /var/run/swift

step4. 配置rsync(配置文件见附件）

编辑/etc/default/rsync:

RSYNC_ENABLE设置为true

重启服务：

service rsync restart



step5.（可选）创建单独的日志文件
        Create /etc/rsyslog.d/10-swift.conf:（见附件）
        修改/etc/rsyslog.conf：

$PrivDropToGroup adm

 

mkdir -p /var/log/swift/hourly

chown -R syslog.adm /var/log/swif
       chmod -R g+w /var/log/swift
        service rsyslog restar


step6.下载swift源码和设置测试环境（step1---5是root权限下进行的，下面的将以swift用户进行操作）
   

su - swift

mkdir ~/bin
       //笨笨熊的办法是采用bzr进行版本下载的。

    bzr init-repo swift(建立共享仓库swift)

cd ~/swift; bzr branch lp:swift trunk(建立swift的拷贝，并下载code。（branch alias get and clone）)

cd ~/swift/trunk; sudo python setup.py develop
      但是在进行code下载的时候出现了错误，由于有其他办法所以也就没去深究解决之。下面是我采用直接从openstack里clone出源码//

      git clone  https://github.com/openstack/swift.git
        cd  ~/swift;
        sudo python setup.py develop

git clone https://github.com/openstack/python-swiftclient.git
cd ~/python-swiftclient; sudo python setup.py develop

编辑~/.bashrc，在最后加入：

export SWIFT_TEST_CONFIG_FILE=/etc/swift/func_test.conf

export PATH=${PATH}:~/bin

 

. ~/.bashrc
note:在进行编辑.bashrc文件的时候发现swift用户没有这个文件。使用find -name .bashrc 找到了三个文件，其中root用户和开机创建的用户有一个，对比两个文件发现文件是一样的。于是直接复制里面的内容在swift的/home/swift目录下创建.bashrc文件并将上面的两句加入进文件最后面。由于创建时是root权限，因此创建完成后还需要使用chown命令将之修改为swift：swift。然后保存。



step7.配置各个节点
note:所有文件均采用touch XX先行创建，然后再用vi编辑。当然也可以直接用vi编辑，为了确保编辑正确我还用cat命令进行了显示核查。配置文件见附件。

Create /etc/swift/proxy-server.conf:
Create /etc/swift/swift.conf:
/etc/swift/account-server/1.conf:
/etc/swift/account-server/2.conf:
/etc/swift/account-server/3.conf:
/etc/swift/account-server/4.conf:
/etc/swift/container-server/1.conf:
/etc/swift/container-server/2.conf:
/etc/swift/container-server/3.conf:
/etc/swift/container-server/4.conf:
/etc/swift/object-server/1.conf:
/etc/swift/object-server/2.conf:
/etc/swift/object-server/3.conf:
/etc/swift/object-server/4.conf:


step8.创建运行swift的脚本(见附件）

Create ~/bin/resetswift.
Create ~/bin/remakerings:
Create ~/bin/startmain:
Create ~/bin/startrest:

step9.脚本测试
chmod +x ~/bin/*

重新产生rings
 remakerings

单元测试
 cp ~/swift/test/sample.conf /etc/swift/test.conf
cd ~/swift; ./.unittests

启动startmain
 startmain


获取一个 X-Storage-Url 和 X-Auth-Token:
curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0

检查swift工作:
  swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat

上传/下载文件测试
以下内容仅仅作为参考，实际操作的时候需要用实际得到的来替换例子里面相对应的内容。

获得test用户的Storage_Auth_Token和URL
       # curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0

得到以下回复信息：

* About to connect() to 127.0.0.1 port 8080 (#0)

*   Trying 127.0.0.1... connected

* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)

> GET /auth/v1.0 HTTP/1.1

> User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15

> Host: 127.0.0.1:8080

> Accept: */*

> X-Storage-User: test:tester

> X-Storage-Pass: testing

< HTTP/1.1 200 OK

< X-Storage-Url: http://127.0.0.1:8080/v1/AUTH_test

< X-Storage-Token: AUTH_tk6474e5ee0cb04832b9d2a168e1a164d8

< X-Auth-Token: AUTH_tk6474e5ee0cb04832b9d2a168e1a164d8

< Content-Length: 0

< Date: Mon, 04 Jul 2011 01:36:57 GMT

* Connection #0 to host 127.0.0.1 left intact

Closing connection #0

创建container

创建一个名称为ljl1的container（目录）

 # curl -X PUT -D -  -H "X-Auth_Token:AUTH_tk6474e5ee0cb04832b9d2a168e1a164d8" http://127.0.0.1:8080/v1/AUTH_test/ljl1

//正确情况下，应该得到以下信息：

HTTP/1.1 201 Created

Content-Length: 18

Content-Type: text/html; charset=UTF-8

Date: Mon, 04 Jul 2011 01:39:38 GMT

 

查看test用户的container列表，发现只有一个目录：ljl1：

# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list

ljl1

上传Object（文件）

# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing upload ljl1 ./cvs/CVSROOT/config            //上传config文件到ljl1目录中

cvs/CVSROOT/config

# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list ljl1   //查看ljl1目录内的信息

bin/startmain    

cvs/CVSROOT/config

下载Object（文件）

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing download  ljl1

 
note:

 /var/log/syslog 中记载了client实时的操作，并记录了运行错误，可以作为查错的参考。

由于不能直接传附件，只好直接将内容显示如下：

/etc/rsyncd.conf


uid = swift

gid = swift

log file = /var/log/rsyncd.log

pid file = /var/run/rsyncd.pid

address = 127.0.0.1

[account6012]

max connections = 25

path = /srv/1/node/

read only = false

lock file = /var/lock/account6012.lock

[account6022]

max connections = 25

path = /srv/2/node/

read only = false

lock file = /var/lock/account6022.lock

[account6032]

max connections = 25

path = /srv/3/node/

read only = false

lock file = /var/lock/account6032.lock

[account6042]

max connections = 25

path = /srv/4/node/

read only = false

lock file = /var/lock/account6042.lock

[container6011]

max connections = 25

path = /srv/1/node/

read only = false

lock file = /var/lock/container6011.lock

[container6021]

max connections = 25

path = /srv/2/node/

read only = false

lock file = /var/lock/container6021.lock

[container6031]

max connections = 25

path = /srv/3/node/

read only = false

lock file = /var/lock/container6031.lock

[container6041]

max connections = 25

path = /srv/4/node/

read only = false

lock file = /var/lock/container6041.lock

[object6010]

max connections = 25

path = /srv/1/node/

read only = false

lock file = /var/lock/object6010.lock

[object6020]

max connections = 25

path = /srv/2/node/

read only = false

lock file = /var/lock/object6020.lock

[object6030]

max connections = 25

path = /srv/3/node/

read only = false

lock file = /var/lock/object6030.lock

[object6040]

max connections = 25

path = /srv/4/node/

read only = false

lock file = /var/lock/object6040.lock





/etc/rsyslog.d/10-swift.conf
# Uncomment the following to have a log containing all logs together

#local1,local2,local3,local4,local5.* /var/log/swift/all.log

# Uncomment the following to have hourly proxy logs for stats processing

#$template HourlyProxyLog,"/var/log/swift/hourly/%$YEAR%%$MONTH%%$DAY%%

$HOUR%"

#local1.*;local1.!notice ?HourlyProxyLog

local1.*;local1.!notice /var/log/swift/proxy.log

local1.notice /var/log/swift/proxy.error

local1.* ~

local2.*;local2.!notice /var/log/swift/storage1.log

local2.notice /var/log/swift/storage1.error

local2.* ~

local3.*;local3.!notice /var/log/swift/storage2.log

local3.notice /var/log/swift/storage2.error

local3.* ~

local4.*;local4.!notice /var/log/swift/storage3.log

local4.notice /var/log/swift/storage3.error

local4.* ~

local5.*;local5.!notice /var/log/swift/storage4.log

local5.notice /var/log/swift/storage4.error

local5.* ~







 



/etc/swift/proxy-server.conf

[DEFAULT]
bind_port = 8080
user = swift
log_facility = LOG_LOCAL1
eventlet_debug = true

[pipeline:main]
pipeline = healthcheck cache tempauth proxy-logging proxy-server

[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true
account_autocreate = true

[filter:tempauth]
use = egg:swift#tempauth
user_admin_admin = admin .admin .reseller_admin
user_test_tester = testing .admin
user_test2_tester2 = testing2 .admin
user_test_tester3 = testing3

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:cache]
use = egg:swift#memcache

[filter:proxy-logging]
use = egg:swift#proxy_logging








/etc/swift/swift.conf

[swift-hash]
# random unique string that can never change (DO NOT LOSE)
swift_hash_path_suffix = changeme





account-server(1--4)
NO1:


[DEFAULT]
devices = /srv/1/node
mount_check = false
disable_fallocate = true
bind_port = 6012
user = swift
log_facility = LOG_LOCAL2
recon_cache_path = /var/cache/swift
eventlet_debug = true

[pipeline:main]
pipeline = recon account-server

[app:account-server]
use = egg:swift#account

[filter:recon]
use = egg:swift#recon

[account-replicator]
vm_test_mode = yes

[account-auditor]

[account-reaper]


NO2:


[DEFAULT]
devices = /srv/2/node
mount_check = false
disable_fallocate = true
bind_port = 6022
user = swift
log_facility = LOG_LOCAL3
recon_cache_path = /var/cache/swift2
eventlet_debug = true

[pipeline:main]
pipeline = recon account-server

[app:account-server]
use = egg:swift#account

[filter:recon]
use = egg:swift#recon

[account-replicator]
vm_test_mode = yes

[account-auditor]

[account-reaper]


NO3:


[DEFAULT]
devices = /srv/3/node
mount_check = false
disable_fallocate = true
bind_port = 6032
user = swift
log_facility = LOG_LOCAL4
recon_cache_path = /var/cache/swift3
eventlet_debug = true

[pipeline:main]
pipeline = recon account-server

[app:account-server]
use = egg:swift#account

[filter:recon]
use = egg:swift#recon

[account-replicator]
vm_test_mode = yes

[account-auditor]

[account-reaper]

NO4:


[DEFAULT]
devices = /srv/4/node
mount_check = false
disable_fallocate = true
bind_port = 6042
user = swift
log_facility = LOG_LOCAL5
recon_cache_path = /var/cache/swift4
eventlet_debug = true

[pipeline:main]
pipeline = recon account-server

[app:account-server]
use = egg:swift#account

[filter:recon]
use = egg:swift#recon

[account-replicator]
vm_test_mode = yes

[account-auditor]

[account-reaper]







container-server(1--4)
NO1:


[DEFAULT]
devices = /srv/1/node
mount_check = false
disable_fallocate = true
bind_port = 6011
user = swift
log_facility = LOG_LOCAL2
recon_cache_path = /var/cache/swift
eventlet_debug = true

[pipeline:main]
pipeline = recon container-server

[app:container-server]
use = egg:swift#container

[filter:recon]
use = egg:swift#recon

[container-replicator]
vm_test_mode = yes

[container-updater]

[container-auditor]

[container-sync]


NO2:


[DEFAULT]
devices = /srv/2/node
mount_check = false
disable_fallocate = true
bind_port = 6021
user =swift
log_facility = LOG_LOCAL3
recon_cache_path = /var/cache/swift2
eventlet_debug = true

[pipeline:main]
pipeline = recon container-server

[app:container-server]
use = egg:swift#container

[filter:recon]
use = egg:swift#recon

[container-replicator]
vm_test_mode = yes

[container-updater]

[container-auditor]

[container-sync]


NO3:

[DEFAULT]
devices = /srv/3/node
mount_check = false
disable_fallocate = true
bind_port = 6031
user = swift
log_facility = LOG_LOCAL4
recon_cache_path = /var/cache/swift3
eventlet_debug = true

[pipeline:main]
pipeline = recon container-server

[app:container-server]
use = egg:swift#container

[filter:recon]
use = egg:swift#recon

[container-replicator]
vm_test_mode = yes

[container-updater]

[container-auditor]

[container-sync]


NO4:

[DEFAULT]
devices = /srv/4/node
mount_check = false
disable_fallocate = true
bind_port = 6041
user = swift
log_facility = LOG_LOCAL5
recon_cache_path = /var/cache/swift4
eventlet_debug = true

[pipeline:main]
pipeline = recon container-server

[app:container-server]
use = egg:swift#container

[filter:recon]
use = egg:swift#recon

[container-replicator]
vm_test_mode = yes

[container-updater]

[container-auditor]

[container-sync]








object-server(1--4)
NO1:

[DEFAULT]
devices = /srv/1/node
mount_check = false
disable_fallocate = true
bind_port = 6010
user = swift
log_facility = LOG_LOCAL2
recon_cache_path = /var/cache/swift
eventlet_debug = true

[pipeline:main]
pipeline = recon object-server

[app:object-server]
use = egg:swift#object

[filter:recon]
use = egg:swift#recon

[object-replicator]
vm_test_mode = yes

[object-updater]

[object-auditor]


NO2:


[DEFAULT]
devices = /srv/2/node
mount_check = false
disable_fallocate = true
bind_port = 6020
user = swift
log_facility = LOG_LOCAL3
recon_cache_path = /var/cache/swift2
eventlet_debug = true

[pipeline:main]
pipeline = recon object-server

[app:object-server]
use = egg:swift#object

[filter:recon]
use = egg:swift#recon

[object-replicator]
vm_test_mode = yes

[object-updater]

[object-auditor]


NO3:

[DEFAULT]
devices = /srv/3/node
mount_check = false
disable_fallocate = true
bind_port = 6030
user = swift
log_facility = LOG_LOCAL4
recon_cache_path = /var/cache/swift3
eventlet_debug = true

[pipeline:main]
pipeline = recon object-server

[app:object-server]
use = egg:swift#object

[filter:recon]
use = egg:swift#recon

[object-replicator]
vm_test_mode = yes

[object-updater]

[object-auditor]


NO4:

[DEFAULT]
devices = /srv/4/node
mount_check = false
disable_fallocate = true
bind_port = 6040
user = swift
log_facility = LOG_LOCAL5
recon_cache_path = /var/cache/swift4
eventlet_debug = true

[pipeline:main]
pipeline = recon object-server

[app:object-server]
use = egg:swift#object

[filter:recon]
use = egg:swift#recon

[object-replicator]
vm_test_mode = yes

[object-updater]

[object-auditor]








resetswift:


#!/bin/bash

swift-init all stop
find /var/log/swift -type f -exec rm -f {} \;
sudo umount /mnt/sdb1
sudo mkfs.xfs -f -i size=1024 /srv/swift-disk
sudo mount /mnt/sdb1
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
sudo chown swift:swift /mnt/sdb1/*
mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4
sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log /var/log/syslog
find /var/cache/swift* -type f -name *.recon -exec rm -f {} \;
sudo service rsyslog restart
sudo service memcached restart







remakerings:

#!/bin/bash

cd /etc/swift

rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz

swift-ring-builder object.builder create 18 3 1
swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1
swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1
swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1
swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1
swift-ring-builder object.builder rebalance
swift-ring-builder container.builder create 18 3 1
swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1
swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1
swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1
swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1
swift-ring-builder container.builder rebalance
swift-ring-builder account.builder create 18 3 1
swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1
swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1
swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1
swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1
swift-ring-builder account.builder rebalance


startmain:
#!/bin/bash

swift-init main start


startrest:
#!/bin/bash

swift-init rest start