简介
本次部署nova组件,分别部署nova和nova-compute,nova部署到controller节点,nova-compute部署到另一台设备,这样部署说明nova-compute是可以部署多个进行横向扩展的。下边开始部署nova组件,controller节点IP 192.168.46.130,nova-compute节点IP 192.168.46.133
部署controller nova
创建数据库
与其他组件的部署一样,首先需要创建nova需要的数据库,操作如下:
$ mysql -u root -p
创建nova_api、nova、nova_cell0数据库
MariaDB [(none)]> CREATE DATABASE nova_api;
MariaDB [(none)]> CREATE DATABASE nova;
MariaDB [(none)]> CREATE DATABASE nova_cell0;
授权数据库
MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'localhost' IDENTIFIED BY 'nova';
MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'%' IDENTIFIED BY 'nova';
MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' IDENTIFIED BY 'nova';
MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' IDENTIFIED BY 'nova';
MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'localhost' IDENTIFIED BY 'nova';
MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'%' IDENTIFIED BY 'nova';
组件部署
首先设置所需要的环境变量,这个在前边组件的部署中有过说明:
[root@localhost ~]# source admin-openstack.sh
1、创建nova user
$ openstack user create --domain default --password-prompt nova
User Password:
Repeat User Password:
+---------------------+----------------------------------+
| Field | Value |
+---------------------+----------------------------------+
| domain_id | default |
| enabled | True |
| id | 8a7dbf5279404537b1c7b86c033620fe |
| name | nova |
| options | {} |
| password_expires_at | None |
+---------------------+----------------------------------+
2、为nova用户设置admin角色和项目
$ openstack role add --project service --user nova admin
3、注册服务
$ openstack service create --name nova --description "OpenStack Compute" compute
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | OpenStack Compute |
| enabled | True |
| id | 060d59eac51b4594815603d75a00aba2 |
| name | nova |
| type | compute |
+-------------+----------------------------------+
4、注册endpoints
$ openstack endpoint create --region RegionOne compute public http://192.168.46.130:8774/v2.1
+--------------+-------------------------------------------+
| Field | Value |
+--------------+-------------------------------------------+
| enabled | True |
| id | 3c1caa473bfe4390a11e7177894bcc7b |
| interface | public |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 060d59eac51b4594815603d75a00aba2 |
| service_name | nova |
| service_type | compute |
| url | http://192.168.46.130:8774/v2.1 |
+--------------+-------------------------------------------+
$ openstack endpoint create --region RegionOne compute internal http://192.168.46.130:8774/v2.1
+--------------+-------------------------------------------+
| Field | Value |
+--------------+-------------------------------------------+
| enabled | True |
| id | e3c918de680746a586eac1f2d9bc10ab |
| interface | internal |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 060d59eac51b4594815603d75a00aba2 |
| service_name | nova |
| service_type | compute |
| url | http://192.168.46.130:8774/v2.1 |
+--------------+-------------------------------------------+
$ openstack endpoint create --region RegionOne compute admin http://192.168.46.130:8774/v2.1
+--------------+-------------------------------------------+
| Field | Value |
+--------------+-------------------------------------------+
| enabled | True |
| id | 38f7af91666a47cfb97b4dc790b94424 |
| interface | admin |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 060d59eac51b4594815603d75a00aba2 |
| service_name | nova |
| service_type | compute |
| url | http://192.168.46.130:8774/v2.1 |
+--------------+-------------------------------------------+
5、创建Placement service
$ openstack user create --domain default --password-prompt placement
User Password:
Repeat User Password:
+---------------------+----------------------------------+
| Field | Value |
+---------------------+----------------------------------+
| domain_id | default |
| enabled | True |
| id | fa742015a6494a949f67629884fc7ec8 |
| name | placement |
| options | {} |
| password_expires_at | None |
+---------------------+----------------------------------+
6、为Placement service添加项目和角色
$ openstack role add --project service --user placement admin
7、注册Placement服务
$ openstack service create --name placement --description "Placement API" placement
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | Placement API |
| enabled | True |
| id | 2d1a27022e6e4185b86adac4444c495f |
| name | placement |
| type | placement |
+-------------+----------------------------------+
8、为Placement服务注册endpoints
$ openstack endpoint create --region RegionOne placement public http://192.168.46.130:8778
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| enabled | True |
| id | 2b1b2637908b4137a9c2e0470487cbc0 |
| interface | public |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 2d1a27022e6e4185b86adac4444c495f |
| service_name | placement |
| service_type | placement |
| url | http://192.168.46.130:8778 |
+--------------+----------------------------------+
$ openstack endpoint create --region RegionOne placement internal http://192.168.46.130:8778
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| enabled | True |
| id | 02bcda9a150a4bd7993ff4879df971ab |
| interface | internal |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 2d1a27022e6e4185b86adac4444c495f |
| service_name | placement |
| service_type | placement |
| url | http://192.168.46.130:8778 |
+--------------+----------------------------------+
$ openstack endpoint create --region RegionOne placement admin http://192.168.46.130:8778
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| enabled | True |
| id | 3d71177b9e0f406f98cbff198d74b182 |
| interface | admin |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 2d1a27022e6e4185b86adac4444c495f |
| service_name | placement |
| service_type | placement |
| url | http://192.168.46.130:8778 |
+--------------+----------------------------------+
配置组件
- 安装组件包
# yum install openstack-nova-api openstack-nova-conductor \
openstack-nova-console openstack-nova-novncproxy \
openstack-nova-scheduler openstack-nova-placement-api
- 修改/etc/nova/nova.conf配置文件
1、设置对外的API
[DEFAULT]
# ...
enabled_apis = osapi_compute,metadata
2、配置数据库
[api_database]
# ...
connection = mysql+pymysql://nova:nova@192.168.46.130/nova_api
[database]
# ...
connection = mysql+pymysql://nova:nova@192.168.46.130/nova
3、配置RabbitMQ
[DEFAULT]
# ...
transport_url = rabbit://openstack:openstack@192.168.46.130
4、打开keystone认证和配置认证token
[api]
# ...
auth_strategy = keystone
[keystone_authtoken]
auth_uri = http://192.168.46.130:5000
auth_url = http://192.168.46.130:35357
memcached_servers = 192.168.46.130:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = nova
password = nova
5、关闭防火墙配置
[DEFAULT]
# ...
use_neutron = True
firewall_driver = nova.virt.firewall.NoopFirewallDriver
6、vnc配置
[vnc]
enabled = true
# ...
vncserver_listen = 192.168.46.130
vncserver_proxyclient_address = 192.168.46.130
7、配置本地镜像服务API
[glance]
# ...
api_servers= http://192.168.46.130:9292
8、锁文件路径
[oslo_concurrency]
# ...
lock_path = /var/lib/nova/tmp
9、配置Placement API
[placement]
os_region_name = RegionOne
project_domain_name = Default
project_name = service
auth_type = password
user_domain_name = Default
auth_url = http://192.168.46.130:35357/v3
username = placement
password = placement
10、修改当前版本bug,编辑/etc/httpd/conf.d/00-nova-placement-api.conf文件,增加以下代码
<Directory /usr/bin>
<IfVersion >= 2.4>
Require all granted
</IfVersion>
<IfVersion < 2.4>
Order allow,deny
Allow from all
</IfVersion>
</Directory>
11、启动httpd服务
# systemctl restart httpd
最后的配置
1、同步cell0数据库
# su -s /bin/sh -c "nova-manage cell_v2 map_cell0" nova
2、创建cell1
# su -s /bin/sh -c "nova-manage cell_v2 create_cell --name=cell1 --verbose" nova
109e1d4b-536a-40d0-83c6-5f121b82b650
3、同步nova数据库
# su -s /bin/sh -c "nova-manage db sync" nova
4、验证cell0和cell1
# nova-manage cell_v2 list_cells
+-------+--------------------------------------+
| Name | UUID |
+-------+--------------------------------------+
| cell1 | 109e1d4b-536a-40d0-83c6-5f121b82b650 |
| cell0 | 00000000-0000-0000-0000-000000000000 |
+-------+--------------------------------------+
配置服务自动启动
# systemctl enable openstack-nova-api.service \
openstack-nova-consoleauth.service openstack-nova-scheduler.service \
openstack-nova-conductor.service openstack-nova-novncproxy.service
# systemctl start openstack-nova-api.service \
openstack-nova-consoleauth.service openstack-nova-scheduler.service \
openstack-nova-conductor.service openstack-nova-novncproxy.service
部署compute node
1、安装包
# yum install openstack-nova-compute
2、编辑/etc/nova/nova.conf配置文件
- 打开API
[DEFAULT]
# ...
enabled_apis = osapi_compute,metadata
- 配置RabbitMQ
[DEFAULT]
# ...
transport_url= rabbit://openstack:openstack@192.168.46.130
- 配置keystone认证
[api]
# ...
auth_strategy = keystone
[keystone_authtoken]
auth_uri = http://192.168.46.130:5000
auth_url = http://192.168.46.130:35357
memcached_servers = 192.168.46.130:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = nova
password = nova
- 关闭防火墙驱动
[DEFAULT]
# ...
use_neutron = True
firewall_driver = nova.virt.firewall.NoopFirewallDriver
- vnc配置
[vnc]
# ...
enabled = True
vncserver_listen= 192.168.46.133
vncserver_proxyclient_address= 192.168.46.133
novncproxy_base_url=http://192.168.46.130:6080/vnc_auto.html
- 配置Glance服务API
[glance]
# ...
api_servers= http://192.168.46.130:9292
- 锁文件路径
[oslo_concurrency]
# ...
lock_path = /var/lib/nova/tmp
- 配置Placement API
[placement]
os_region_name = RegionOne
project_domain_name = Default
project_name = service
auth_type = password
user_domain_name = Default
auth_url = http://192.168.46.130:35357/v3
username = placement
password = placement
- 配置libvirt,在生产环境中一般使用kvm
[libvirt]
# ...
virt_type = qemu
- 启动服务
# systemctl enable libvirtd.service openstack-nova-compute.service //设置自动启动
# systemctl start libvirtd.service openstack-nova-compute.service //启动服务
问题记录
在以上执行启动服务时,命令行长时间卡住,一般说明连接RabbitMQ失败,可以在/var/log/nova/nova-compute.log文件中查看详细报错,报错如下:
2019-05-11 12:02:57.432 5308 ERROR oslo.messaging._drivers.impl_rabbit [req-503bd955-7075-44a8-b553-f70b9437ff3b - - - - -] [84517112-2f15-4dfa-b9f0-e195a80a67c2] AMQP server on 192.168.46.130:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds.: error: [Errno 113] EHOSTUNREACH
2019-05-11 12:03:29.606 5308 ERROR oslo.messaging._drivers.impl_rabbit [req-503bd955-7075-44a8-b553-f70b9437ff3b - - - - -] [84517112-2f15-4dfa-b9f0-e195a80a67c2] AMQP server on 192.168.46.130:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds.: error: [Errno 113] EHOSTUNREACH
2019-05-11 12:04:01.792 5308 ERROR oslo.messaging._drivers.impl_rabbit [req-503bd955-7075-44a8-b553-f70b9437ff3b - - - - -] [84517112-2f15-4dfa-b9f0-e195a80a67c2] AMQP server on 192.168.46.130:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds.: error: [Errno 113] EHOSTUNREACH
以上错误说明nova-compute节点连接controller节点的RabbitMQ失败了,一般原因是controller上配置了iptables,需要设置iptables把RabbitMQ的5672端口打开。设置如下:
# iptables -I INPUT -p tcp --dport 5672 -j ACCEPT
#添加规则
# service iptables save
#保存设置
# service iptables restart
# 重启iptables,生效规则
添加compute node到cell数据库
- 在controller node上查看,确定在数据库中有nova-compute
[root@localhost ~]# source admin-openstack.sh [root@localhost ~]# openstack compute service list +----+------------------+-----------------------+----------+---------+-------+----------------------------+ | ID | Binary | Host | Zone | Status | State | Updated At | +----+------------------+-----------------------+----------+---------+-------+----------------------------+ | 1 | nova-consoleauth | localhost.localdomain | internal | enabled | up | 2019-05-12T10:36:09.000000 | | 2 | nova-conductor | localhost.localdomain | internal | enabled | up | 2019-05-12T10:36:09.000000 | | 3 | nova-scheduler | localhost.localdomain | internal | enabled | up | 2019-05-12T10:36:10.000000 | | 8 | nova-compute | node2 | nova | enabled | up | 2019-05-12T10:36:07.000000 | +----+------------------+-----------------------+----------+---------+-------+----------------------------+
- 发现compute node,每次添加新的compute nodes时,都需要在controller node上执行nova-manage cell_v2 discover_hosts注册新的compute nodes
# su -s /bin/sh -c "nova-manage cell_v2 discover_hosts --verbose" nova
Found 2 cell mappings.
Skipping cell0 since it does not contain hosts.
Getting compute nodes from cell 'cell1': ad5a5985-a719-4567-98d8-8d148aaae4bc
Found 1 computes in cell: ad5a5985-a719-4567-98d8-8d148aaae4bc
Checking host mapping for compute host 'compute': fe58ddc1-1d65-4f87-9456-bc040dc106b3
Creating host mapping for compute host 'compute': fe58ddc1-1d65-4f87-9456-bc040dc106b3
通过以上的组件部署,nova-compute可以有多个,并且当配置完一个nova-compute时,其他的与此类似。