Развертывание Kolla-Ansible завершается сбоем в Ubuntu 22.04 во время «Ожидания регистрации сервисов nova-computer»
Постановка проблемы: установка Kolla Ansible завершается сбоем при «развертывании» игры с указанными ниже ошибками (раздел «Ошибки»).
О развертывании:
- Лабораторное развертывание Openstack с использованием Kolla Ansible
- 1 контроллер, 2 вычислительных узла, 1 узел блочного хранилища с использованием томов LVM/Cinder.
- Все узлы под управлением Ubuntu 22.04.
- «Bootstrap-серверы» и «предварительные проверки» работают успешно. Развертывание не удалось
Устранение неполадок выполнено:
- https://lists.openstack.org/pipermail/openstack-discuss/2022-September/030641.html
- Кажется, эта ссылка точно соответствует моей проблеме.
- Однако я не пытаюсь использовать SSL на прокси-сервере HA.
- Развертывание завершается с одной и той же ошибкой независимо от того, включен или выключен параметр «enable_haproxy».
- Развертывание завершается с той же ошибкой независимо от того, прокомментированы или нет/комментированы «внешний fqdn» и «внешний vip».
- Мой вывод globals.yml ниже.
Ошибки:
TASK [nova-cell : Waiting for nova-compute services to register themselves] ******************************************************************************************************************************************************************
skipping: [compute2]
skipping: [controller]
fatal: [compute1 -> controller]: FAILED! => {"msg": "The conditional check '(nova_compute_services.stdout | from_json | map(attribute='Host') | list) is superset(expected_compute_service_hosts)' failed. The error was: Expecting value: line 1 column 1 (char 0)"}
TASK [nova-cell : Fail if nova-compute service failed to register] ***************************************************************************************************************************************************************************
fatal: [compute2]: FAILED! => {"msg": "The conditional check 'any_failed_services or (nova_compute_registration_fatal | bool and\n failed_compute_service_hosts | length > 0)' failed. The error was: error while evaluating conditional (any_failed_services or (nova_compute_registration_fatal | bool and\n failed_compute_service_hosts | length > 0)): {{ ansible_facts.nodename in failed_compute_service_hosts or\n (ansible_facts.hostname ~ \"-ironic\") in failed_compute_service_hosts }}: {{ expected_compute_service_hosts | difference(nova_compute_service_hosts) | list }}: {{ hostvars[all_computes_in_batch[0]].nova_compute_services.stdout |\n from_json |\n map(attribute='Host') |\n list }}: Unable to look up a name or access an attribute in template string ({{ hostvars[all_computes_in_batch[0]].nova_compute_services.stdout |\n from_json |\n map(attribute='Host') |\n list }}).\nMake sure your variable name does not contain invalid characters like '-': the JSON object must be str, bytes or bytearray, not AnsibleUndefined\n\nThe error appears to be in '/usr/local/share/kolla-ansible/ansible/roles/nova-cell/tasks/wait_discover_computes.yml': line 45, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n # that failed to register.\n - name: Fail if nova-compute service failed to register\n ^ here\n"}
fatal: [controller]: FAILED! => {"msg": "The conditional check 'any_failed_services or (nova_compute_registration_fatal | bool and\n failed_compute_service_hosts | length > 0)' failed. The error was: error while evaluating conditional (any_failed_services or (nova_compute_registration_fatal | bool and\n failed_compute_service_hosts | length > 0)): {{ ansible_facts.nodename in failed_compute_service_hosts or\n (ansible_facts.hostname ~ \"-ironic\") in failed_compute_service_hosts }}: {{ expected_compute_service_hosts | difference(nova_compute_service_hosts) | list }}: {{ hostvars[all_computes_in_batch[0]].nova_compute_services.stdout |\n from_json |\n map(attribute='Host') |\n list }}: Unable to look up a name or access an attribute in template string ({{ hostvars[all_computes_in_batch[0]].nova_compute_services.stdout |\n from_json |\n map(attribute='Host') |\n list }}).\nMake sure your variable name does not contain invalid characters like '-': the JSON object must be str, bytes or bytearray, not AnsibleUndefined\n\nThe error appears to be in '/usr/local/share/kolla-ansible/ansible/roles/nova-cell/tasks/wait_discover_computes.yml': line 45, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n # that failed to register.\n - name: Fail if nova-compute service failed to register\n ^ here\n"}
PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
block1 : ok=31 changed=5 unreachable=0 failed=0 skipped=9 rescued=0 ignored=0
compute1 : ok=37 changed=5 unreachable=0 failed=1 skipped=16 rescued=0 ignored=0
compute2 : ok=37 changed=5 unreachable=0 failed=1 skipped=16 rescued=0 ignored=0
controller : ok=163 changed=15 unreachable=0 failed=1 skipped=93 rescued=0 ignored=0
localhost : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Вывод globals.yml:
zindagi@deployment:~$ grep -v '^\s*$\|^\s*\#' /etc/kolla/globals.yml
---
workaround_ansible_issue_8743: yes
config_strategy: "COPY_ALWAYS"
kolla_base_distro: "ubuntu"
openstack_release: "master"
kolla_internal_vip_address: "10.0.0.10"
kolla_internal_fqdn: "{{ kolla_internal_vip_address }}"
kolla_external_vip_address: "{{ kolla_internal_vip_address }}"
kolla_external_fqdn: "{{ kolla_external_vip_address }}"
network_interface: "eth1"
neutron_external_interface: "eth2"
neutron_plugin_agent: "openvswitch"
keepalived_virtual_router_id: "51"
enable_openstack_core: "yes"
enable_glance: "{{ enable_openstack_core | bool }}"
enable_haproxy: "no"
enable_keystone: "{{ enable_openstack_core | bool }}"
enable_neutron: "{{ enable_openstack_core | bool }}"
enable_cinder: "yes"
enable_cinder_backup: "no"
enable_cinder_backend_iscsi: "{{ enable_cinder_backend_lvm | bool }}"
enable_cinder_backend_lvm: "yes"
enable_heat: "{{ enable_openstack_core | bool }}"
enable_horizon: "{{ enable_openstack_core | bool }}"
enable_openvswitch: "{{ enable_neutron | bool and neutron_plugin_agent != 'linuxbridge' }}"
fernet_token_expiry: 86400
glance_backend_file: "yes"
cinder_volume_group: "cinder-volumes"
nova_compute_virt_type: "qemu"
nova_console: "novnc"
ОБНОВЛЕНИЕ с 1 по 8 мая в 16:00 по восточному стандартному времени:
- Я перестроил это развертывание с нуля.
- Предыдущее развертывание было на VirtualBox - переключено на ESXi 8.x (не должно иметь значения - все еще указываю на это)
- Повышенные характеристики используемых виртуальных машин.
- На этот раз я использовал «rocky» в качестве «базового дистрибутива» в «globals.yml».
- Еще несколько изменений в моем globals.yml (ниже)
- На этот раз развертывание НЕ ПРОИЗОШЛО из-за того, что службы nova-compute не смогли зарегистрироваться. Однако он сломался из-за другого сообщения об ошибке :
ОШИБКА:
RUNNING HANDLER [iscsi : Restart tgtd container] *********************************************************************************************************************************************************************************************
fatal: [kolla-block1]: FAILED! => {"changed": true, "msg": "'Traceback (most recent call last):\\n File \"/usr/local/lib/python3.10/dist-packages/docker/api/client.py\", line 268, in _raise_for_status\\n response.raise_for_status()\\n File \"/usr/lib/python3/dist-packages/requests/models.py\", line 943, in raise_for_status\\n raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.42/images/create?tag=master-rocky-9&fromImage=quay.io%2Fopenstack.kolla%2Ftgtd\\n\\nDuring handling of the above exception, another exception occurred:\\n\\nTraceback (most recent call last):\\n File \"/tmp/ansible_kolla_docker_payload_y2hdgwrz/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 406, in main\\n File \"/tmp/ansible_kolla_docker_payload_y2hdgwrz/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 370, in recreate_or_restart_container\\n self.start_container()\\n File \"/tmp/ansible_kolla_docker_payload_y2hdgwrz/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 388, in start_container\\n self.pull_image()\\n File \"/tmp/ansible_kolla_docker_payload_y2hdgwrz/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 211, in pull_image\\n json.loads(line.strip().decode(\\'utf-8\\')) for line in self.dc.pull(\\n File \"/usr/local/lib/python3.10/dist-packages/docker/api/image.py\", line 430, in pull\\n self._raise_for_status(response)\\n File \"/usr/local/lib/python3.10/dist-packages/docker/api/client.py\", line 270, in _raise_for_status\\n raise create_api_error_from_http_exception(e)\\n File \"/usr/local/lib/python3.10/dist-packages/docker/errors.py\", line 31, in create_api_error_from_http_exception\\n raise cls(e, response=response, explanation=explanation)\\ndocker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.42/images/create?tag=master-rocky-9&fromImage=quay.io%2Fopenstack.kolla%2Ftgtd: Not Found (\"manifest for quay.io/openstack.kolla/tgtd:master-rocky-9 not found: manifest unknown: manifest unknown\")\\n'"}
Пересмотренный globals.yml
zindagi@kolla-deployment:~$ grep -v '^\s*$\|^\s*\#' /etc/kolla/globals.yml
---
workaround_ansible_issue_8743: yes
kolla_base_distro: "rocky"
kolla_internal_vip_address: "192.168.1.250"
network_interface: "eth1"
neutron_external_interface: "eth2"
neutron_plugin_agent: "openvswitch"
enable_openstack_core: "yes"
enable_haproxy: "yes"
enable_cinder: "yes"
enable_cinder_backup: "no"
enable_cinder_backend_lvm: "yes"
enable_openvswitch: "{{ enable_neutron | bool and neutron_plugin_agent != 'linuxbridge' }}"
fernet_token_expiry: 86400
glance_backend_file: "yes"
cinder_volume_group: "cinder-volumes"
nova_compute_virt_type: "qemu"
nova_console: "novnc"
ОБНОВЛЕНИЕ 1–8 мая, 17:30:
- Я считаю, что теперь эта проблема решена.
- Изменен «базовый дистрибутив» обратно на Ubuntu.
- Enable_cinder_backend_iscsi изменен обратно на «да».
- Самым большим изменением, которое я сделал, было изменение версии с «master» на «zed».
Выход «развертывания» пьесы Коллы
PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
kolla-block1 : ok=34 changed=23 unreachable=0 failed=0 skipped=11 rescued=0 ignored=0
kolla-compute1 : ok=69 changed=43 unreachable=0 failed=0 skipped=51 rescued=0 ignored=0
kolla-compute2 : ok=64 changed=43 unreachable=0 failed=0 skipped=50 rescued=0 ignored=0
kolla-controller : ok=323 changed=215 unreachable=0 failed=0 skipped=242 rescued=0 ignored=1
localhost : ok=4 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
Окончательный файл globals.yml:
root@kolla-deployment:/home/zindagi# grep -v '^\s*$\|^\s*\#' /etc/kolla/globals.yml
---
workaround_ansible_issue_8743: yes
kolla_base_distro: "ubuntu"
kolla_internal_vip_address: "192.168.1.250"
network_interface: "eth1"
neutron_external_interface: "eth2"
neutron_plugin_agent: "openvswitch"
enable_openstack_core: "yes"
enable_haproxy: "yes"
enable_cinder: "yes"
enable_cinder_backup: "no"
enable_cinder_backend_iscsi: "yes"
enable_cinder_backend_lvm: "yes"
enable_openvswitch: "{{ enable_neutron | bool and neutron_plugin_agent != 'linuxbridge' }}"
fernet_token_expiry: 86400
glance_backend_file: "yes"
cinder_volume_group: "cinder-volumes"
nova_compute_virt_type: "qemu"
nova_console: "novnc"
Выходные данные docker ps -a от вычислительных, контроллерных и блочных/хранилищных узлов:
root@kolla-compute1:/home/zindagi# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9f5130d4135b quay.io/openstack.kolla/neutron-openvswitch-agent:master-ubuntu-jammy "dumb-init --single-…" 3 minutes ago Up 3 minutes (healthy) neutron_openvswitch_agent
bdff1895f3af quay.io/openstack.kolla/openvswitch-vswitchd:master-ubuntu-jammy "dumb-init --single-…" 7 minutes ago Up 7 minutes (healthy) openvswitch_vswitchd
11ab834e7db4 quay.io/openstack.kolla/openvswitch-db-server:master-ubuntu-jammy "dumb-init --single-…" 8 minutes ago Up 8 minutes (healthy) openvswitch_db
8f2734cb33f4 quay.io/openstack.kolla/nova-compute:master-ubuntu-jammy "dumb-init --single-…" 9 minutes ago Up 9 minutes (healthy) nova_compute
2bbf5b3bbc35 quay.io/openstack.kolla/nova-libvirt:master-ubuntu-jammy "dumb-init --single-…" 10 minutes ago Up 10 minutes (healthy) nova_libvirt
5b5281614f7d quay.io/openstack.kolla/nova-ssh:master-ubuntu-jammy "dumb-init --single-…" 12 minutes ago Up 11 minutes (healthy) nova_ssh
32cf358b66de quay.io/openstack.kolla/iscsid:master-ubuntu-jammy "dumb-init --single-…" 29 minutes ago Up 29 minutes iscsid
2a5adf6983a4 quay.io/openstack.kolla/cron:master-ubuntu-jammy "dumb-init --single-…" 34 minutes ago Up 34 minutes cron
a1a2ed0aa775 quay.io/openstack.kolla/kolla-toolbox:master-ubuntu-jammy "dumb-init --single-…" 36 minutes ago Up 35 minutes kolla_toolbox
4ed9854f1826 quay.io/openstack.kolla/fluentd:master-ubuntu-jammy "dumb-init --single-…" 37 minutes ago Up 37 minutes fluentd
root@kolla-controller:/home/zindagi# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
01750827b82b quay.io/openstack.kolla/heat-engine:master-ubuntu-jammy "dumb-init --single-…" About a minute ago Up 57 seconds (healthy) heat_engine
dbba9ef19e19 quay.io/openstack.kolla/heat-api-cfn:master-ubuntu-jammy "dumb-init --single-…" About a minute ago Up About a minute (healthy) heat_api_cfn
c42514c10c14 quay.io/openstack.kolla/heat-api:master-ubuntu-jammy "dumb-init --single-…" About a minute ago Up About a minute (healthy) heat_api
60fc15caabe4 quay.io/openstack.kolla/neutron-metadata-agent:master-ubuntu-jammy "dumb-init --single-…" 3 minutes ago Up 3 minutes (healthy) neutron_metadata_agent
affd17658759 quay.io/openstack.kolla/neutron-l3-agent:master-ubuntu-jammy "dumb-init --single-…" 3 minutes ago Up 3 minutes (healthy) neutron_l3_agent
cbb256538649 quay.io/openstack.kolla/neutron-dhcp-agent:master-ubuntu-jammy "dumb-init --single-…" 4 minutes ago Up 3 minutes (healthy) neutron_dhcp_agent
67d4c8f1b6b6 quay.io/openstack.kolla/neutron-openvswitch-agent:master-ubuntu-jammy "dumb-init --single-…" 4 minutes ago Up 4 minutes (healthy) neutron_openvswitch_agent
f34d7a7db2ee quay.io/openstack.kolla/neutron-server:master-ubuntu-jammy "dumb-init --single-…" 4 minutes ago Up 4 minutes (healthy) neutron_server
3beefd2cd97a quay.io/openstack.kolla/openvswitch-vswitchd:master-ubuntu-jammy "dumb-init --single-…" 8 minutes ago Up 8 minutes (healthy) openvswitch_vswitchd
83fc1eee9ae1 quay.io/openstack.kolla/openvswitch-db-server:master-ubuntu-jammy "dumb-init --single-…" 8 minutes ago Up 8 minutes (healthy) openvswitch_db
65992167e59a quay.io/openstack.kolla/nova-novncproxy:master-ubuntu-jammy "dumb-init --single-…" 14 minutes ago Up 13 minutes (healthy) nova_novncproxy
cb58408a8728 quay.io/openstack.kolla/nova-conductor:master-ubuntu-jammy "dumb-init --single-…" 14 minutes ago Up 14 minutes (healthy) nova_conductor
6caa3a38cb4e quay.io/openstack.kolla/nova-api:master-ubuntu-jammy "dumb-init --single-…" 15 minutes ago Up 15 minutes (healthy) nova_api
7341fbb3ec59 quay.io/openstack.kolla/nova-scheduler:master-ubuntu-jammy "dumb-init --single-…" 15 minutes ago Up 15 minutes (healthy) nova_scheduler
ad3580d98c0b quay.io/openstack.kolla/placement-api:master-ubuntu-jammy "dumb-init --single-…" 18 minutes ago Up 18 minutes (healthy) placement_api
0701d637d49d quay.io/openstack.kolla/cinder-scheduler:master-ubuntu-jammy "dumb-init --single-…" 21 minutes ago Up 21 minutes (healthy) cinder_scheduler
6320ffe512fc quay.io/openstack.kolla/cinder-api:master-ubuntu-jammy "dumb-init --single-…" 21 minutes ago Up 21 minutes (healthy) cinder_api
ad0f2adb96a7 quay.io/openstack.kolla/glance-api:master-ubuntu-jammy "dumb-init --single-…" 23 minutes ago Up 23 minutes (healthy) glance_api
f01034594a50 quay.io/openstack.kolla/keystone:master-ubuntu-jammy "dumb-init --single-…" 26 minutes ago Up 26 minutes (healthy) keystone
2c4ed3eb1553 quay.io/openstack.kolla/keystone-fernet:master-ubuntu-jammy "dumb-init --single-…" 26 minutes ago Up 26 minutes (healthy) keystone_fernet
8b63d3416385 quay.io/openstack.kolla/keystone-ssh:master-ubuntu-jammy "dumb-init --single-…" 26 minutes ago Up 26 minutes (healthy) keystone_ssh
dc2ae69f6196 quay.io/openstack.kolla/rabbitmq:master-ubuntu-jammy "dumb-init --single-…" 29 minutes ago Up 29 minutes (healthy) rabbitmq
8329d6a73929 quay.io/openstack.kolla/memcached:master-ubuntu-jammy "dumb-init --single-…" 30 minutes ago Up 30 minutes (healthy) memcached
35acc9734a53 quay.io/openstack.kolla/mariadb-clustercheck:master-ubuntu-jammy "dumb-init --single-…" 31 minutes ago Up 31 minutes mariadb_clustercheck
8d94088d4c74 quay.io/openstack.kolla/mariadb-server:master-ubuntu-jammy "dumb-init -- kolla_…" 31 minutes ago Up 31 minutes (healthy) mariadb
4aebfdba90a9 quay.io/openstack.kolla/keepalived:master-ubuntu-jammy "dumb-init --single-…" 33 minutes ago Up 33 minutes keepalived
123a4fc71aae quay.io/openstack.kolla/haproxy:master-ubuntu-jammy "dumb-init --single-…" 33 minutes ago Up 33 minutes (healthy) haproxy
7f6c2adc5e41 quay.io/openstack.kolla/cron:master-ubuntu-jammy "dumb-init --single-…" 35 minutes ago Up 35 minutes cron
f1abce91f91a quay.io/openstack.kolla/kolla-toolbox:master-ubuntu-jammy "dumb-init --single-…" 35 minutes ago Up 35 minutes kolla_toolbox
8056c8a6a470 quay.io/openstack.kolla/fluentd:master-ubuntu-jammy "dumb-init --single-…" 38 minutes ago Up 37 minutes fluentd
root@kolla-block1:/home/zindagi# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
34eb37634e70 quay.io/openstack.kolla/cinder-volume:master-ubuntu-jammy "dumb-init --single-…" 26 minutes ago Up 26 minutes (healthy) cinder_volume
140d5029b8c7 quay.io/openstack.kolla/tgtd:master-ubuntu-jammy "dumb-init --single-…" 36 minutes ago Up 36 minutes tgtd
3e7eb6195b1f quay.io/openstack.kolla/iscsid:master-ubuntu-jammy "dumb-init --single-…" 36 minutes ago Up 36 minutes iscsid
a7e1464b43df quay.io/openstack.kolla/cron:master-ubuntu-jammy "dumb-init --single-…" 41 minutes ago Up 41 minutes cron
837217ad632d quay.io/openstack.kolla/kolla-toolbox:master-ubuntu-jammy "dumb-init --single-…" 42 minutes ago Up 41 minutes kolla_toolbox
f1cb0457cf12 quay.io/openstack.kolla/fluentd:master-ubuntu-jammy "dumb-init --single-…" 44 minutes ago Up 44 minutes fluentd
root@kolla-block1:/home/zindagi#