在配置新的OpenStack环境中，尝试创建实例时遇到'No valid host was found. There are not enough hosts available.'错误。通过检查horizon和nova的日志，发现与权限认证相关的问题。在keystone日志中找到多个'Unauthorized'警告，原因是计算节点的PLACEMENT口令为空。修复该口令后，问题得到解决。
一套新起的OpenStack环境，在创建实例是发现报错。No valid host was found. There are not enough hosts available.
查看日之后发现 httpd有报错日志，日志指向horizon和nova。
[root@controller httpd]# cat error_log
[Wed Dec 16 14:46:09.105540 2020] [:error] [pid 3409] INFO openstack_auth.views Logging out user "".
[Wed Dec 16 14:46:12.922388 2020] [:error] [pid 3409] INFO openstack_auth.plugin.base Attempted scope to domain default failed, will attempt to scope to another domain.
[Wed Dec 16 14:46:13.193729 2020] [:error] [pid 3409] INFO openstack_auth.forms Login successful for user "Admin" using domain "default", remote address 172.31.7.247.
[Wed Dec 16 15:13:21.280021 2020] [:error] [pid 3409] INFO horizon.tables.actions \xe5\xb7\xb2\xe8\xb0\x83\xe5\xba\xa6\xe5\x88\xa0\xe9\x99\xa4\xe5\xae\x9e\xe4\xbe\x8b: "test"
[Wed Dec 16 15:13:21.641190 2020] [:error] [pid 3409] INFO horizon.tables.actions \xe5\xb7\xb2\xe8\xb0\x83\xe5\xba\xa6\xe5\x88\xa0\xe9\x99\xa4\xe5\xae\x9e\xe4\xbe\x8b: "test"
[root@controller nova]# egrep -n 'error|ERROR' nova-conductor.log
1253:2020-12-16 13:45:18.870 3211 WARNING nova.scheduler.utils [req-ebff204d-0c0a-4014-b647-9197eab36cb3 17423e2a06fe404e82c27530c95196a7 dc83162c60ed4184b33b4d9130e606e0 - default default] [instance: 5580bd2e-0926-4b09-825d-a36b630bc690] Setting instance to ERROR state.: NoValidHost_Remote: No valid host was found. There are not enough hosts available.
1254:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager [req-4900bea5-f27c-4bed-ade3-57f5c578e134 17423e2a06fe404e82c27530c95196a7 dc83162c60ed4184b33b4d9130e606e0 - default default] Failed to schedule instances: NoValidHost_Remote: No valid host was found. There are not enough hosts available.
1273:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager Traceback (most recent call last):
1274:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 1271, in schedule_and_build_instances
1275:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager instance_uuids, return_alternates=True)
1276:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 784, in _schedule_instances
1277:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager return_alternates=return_alternates)
1278:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 50, in select_destinations
1279:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager instance_uuids, return_objects, return_alternates)
1280:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 35, in __run_method
1281:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager return getattr(self.instance, __name)(*args, **kwargs)
1282:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations
1283:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager instance_uuids, return_objects, return_alternates)
1284:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations
1285:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args)
1286:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 179, in call
1287:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager retry=self.retry)
1288:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 133, in _send
1289:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager retry=retry)
1290:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 645, in send
1291:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager call_monitor_timeout, retry=retry)
1292:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send
1293:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager raise result
1294:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager NoValidHost_Remote: No valid host was found. There are not enough hosts available.
1295:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager Traceback (most recent call last):
1296:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager
1297:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 229, in inner
1298:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager return func(*args, **kwargs)
1299:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager
1300:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 169, in select_destinations
1301:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager allocation_request_version, return_alternates)
1302:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager
1303:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 91, in select_destinations
1304:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager allocation_request_version, return_alternates)
1305:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager
1306:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 256, in _schedule
1307:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager claimed_instance_uuids)
1308:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager
1309:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 293, in _ensure_sufficient_hosts
1310:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason)
1311:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager
1312:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager NoValidHost: No valid host was found. There are not enough hosts available.
1313:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager
1314:2020-12-16 15:13:07.092 3211 ERROR nova.conductor.manager
1335:2020-12-16 15:13:07.152 3211 WARNING nova.scheduler.utils [req-4900bea5-f27c-4bed-ade3-57f5c578e134 17423e2a06fe404e82c27530c95196a7 dc83162c60ed4184b33b4d9130e606e0 - default default] [instance: 7c77e3ce-5e20-4a71-bc81-6de2024112ee] Setting instance to ERROR state.: NoValidHost_Remote: No valid host was found. There are not enough hosts available.
查看nova计算节点服务显示正常
[root@controller nova]# openstack compute service list
重刷数据库及重启openstack服务，重启计算节点nova服务，问题依旧。到此基本没什么招数了....
[root@controller nova]# su -s /bin/sh -c "nova-manage cell_v2 discover_hosts --verbose" nova
[root@controller nova]# systemctl restart openstack-nova-api.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service
[root@compute-1 ~]# systemctl restart libvirtd.service openstack-nova-compute.service
再次排查发现keystone下面有个 WARNING。二不是ERROR(吐血)。发现是一个计算节点PLACEMENT 口令为空所导致的问题。。。浪费我几天时间
[root@controller keystone]# cat keystone.log | grep WARNING
172.31.240.201: Unauthorized: The request you have made requires authentication.
2020-12-16 18:07:06.974 3480 WARNING keystone.common.wsgi [req-05cdb02c-ca09-43d0-8a8d-8d5d3e1526b9 - - - - -] Authorization failed. The request you have made requires authentication. from 172.31.240.201: Unauthorized: The request you have made requires authentication.
2020-12-16 18:08:07.133 3478 WARNING keystone.common.wsgi [req-e3550c13-d37b-46c3-b355-87ac76edecae - - - - -] Authorization failed. The request you have made requires authentication. from 172.31.240.201: Unauthorized: The request you have made requires authentication.
2020-12-16 18:08:07.550 3476 WARNING keystone.common.wsgi [req-246e0bb6-d4b9-4f22-9053-44e4579e6555 - - - - -] Authorization failed. The request you have made requires authentication. from 172.31.240.201: Unauthorized: The request you have made requires authentication.
2020-12-16 18:08:07.979 3477 WARNING keystone.common.wsgi [req-c64835c6-f4a1-47d9-8229-a0ac2476a0f6 - - - - -] Authorization failed. The request you have made requires authentication. from 172.31.240.201: Unauthorized: The request you have made requires authentication.
2020-12-16 18:09:06.967 3476 WARNING keystone.common.wsgi [req-f7a32efe-4b80-4196-ab95-35fce1f48684 - - - - -] Authorization failed. The request you have made requires authentication. from 172.31.240.201: Unauthorized: The request you have made requires authentication.
2020-12-16 18:09:07.347 3478 WARNING keystone.common.wsgi [req-d0f89687-d019-4983-9d51-a116406a141e - - - - -] Authorization failed. The request you have made requires authentication. from 172.31.240.201: Unauthorized: The request you have made requires authentication.
2020-12-16 18:09:07.760 3480 WARNING keystone.common.wsgi [req-239fe092-e773-40d0-a1b2-035882a96e6c - - - - -] Authorization failed. The request you have made requires authentication. from 172.31.240.201: Unauthorized: The request you have made requires authentication.
2020-12-16 18:10:07.958 3476 WARNING keystone.common.wsgi [req-351ef95c-cc2c-46d0-af9c-79a0d2845366 - - - - -] Authorization failed. The request you have made requires authentication. from 172.31.240.201: Unauthorized: The request you have made requires authentication.
2020-12-16 18:10:08.378 3480 WARNING keystone.common.wsgi [req-79617ae8-9740-4fa0-9ac3-d80211919ae4 - - - - -] Authorization failed. The request you have made requires authentication. from 172.31.240.201: Unauthorized: The request you have made requires authentication.
2020-12-16 18:10:08.794 3478 WARNING keystone.common.wsgi [req-1ec8adce-9564-4855-beff-451032546d41 - - - - -] Authorization failed. The request you have made requires authentication. from 172.31.240.201: Unauthorized: The request you have made requires authentication.
扫一扫
1847
到【灌水乐园】发言
