Cloudlet does not create overlay

[MSoares]

Dear Tom,

I tried to follow your advice, and in this way, I was trying to do a little test: put 2 cloudlets through Virtualbox.

I installed Virtualbox, OS is Ubuntu 14.04 and I installed elijah-provisioning.

I was able to import a VM base, this base appears if I type the cloudlet list-base command. However, it gives an error:
"Failed to initialize KVM: No such file or directory
No accelerator found!
Error, make sure previous VM is closed and check QEMU_ARGUMENT
Failed to create overlay "

> mickael@mickael-VirtualBox:~$ cloudlet list-base
> hash value                    path
> ------------------------------------------------------------------------------------------
> abda52a61692094b3b7d45c9647d022f5e297d1b788679eb93735374007576b8    /home/mickael/.cloudlet/abda52a61692094b3b7d45c9647d022f5e297d1b788679eb93735374007576b8/precise.raw
> ------------------------------------------------------------------------------------------
> mickael@mickael-VirtualBox:~$ cloudlet overlay /home/mickael/.clou692094b3b7d45c9647d022f5e297d1b788679eb93735374007576b8/precise.raw -- -redir tcp:2222::22 -redir tcp:8080::80
> INFO     QEMU access file : /tmp/cloudlet-qemu-7A05Ug/qemu-trim-log
> INFO     start monitoring at /var/tmp/cloudletfs-Yq6vtn/disk/streams/chunks_modified
> INFO     start monitoring at /var/tmp/cloudletfs-Yq6vtn/disk/streams/chunks_accessed
> INFO     start monitoring at /var/tmp/cloudletfs-Yq6vtn/memory/streams/chunks_accessed
> INFO     start monitoring at /tmp/cloudlet-qemu-7A05Ug/qemu-trim-log
> INFO     * Overlay creation configuration
> INFO       - {'DISK_ONLY': False,
>  'FREE_SUPPORT': False,
>  'TRIM_SUPPORT': True,
>  'XRAY_SUPPORT': False,
>  'ZIP_CONTAINER': True}
> INFO     restoring VM...
> ERROR    internal error: Child process (LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games HOME=/home/mickael USER=mickael LOGNAME=mickael /usr/local/bin/cloudlet_qemu-system-x86_64 -no-user-config -device '?' -device 'pci-assign,?' -device 'virtio-blk-pci,?' -device 'virtio-net-pci,?' -device 'scsi-disk,?' -device 'PIIX4_PM,?' -device 'usb-redir,?' -device 'ide-drive,?' -device 'usb-host,?' -device 'scsi-generic,?' -device 'usb-storage,?') unexpected exit status 1: Could not access KVM kernel module: No such file or directory
> failed to initialize KVM: No such file or directory
> No accelerator found!

> ERROR    failed : reasons - <domain xmlns:ns0="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
>name>cloudlet-543793584db34094af90406bcad9c2c8</name>
>uuid>54379358-4db3-4094-af90-406bcad9c2c8</uuid>
>memory unit="KiB">1048576</memory>
>currentMemory unit="KiB">1048576</currentMemory>
>vcpu placement="static">4</vcpu>
>   <os>
>     <type arch="x86_64" machine="pc-1.0">hvm</type>
>     <boot dev="hd" />
>   </os>
>   <features>
>     <acpi />
>   </features>
>   <cpu match="exact" mode="custom">
>     <topology cores="4" sockets="1" threads="1" />
>   <model fallback="forbid">core2duo</model></cpu>
>   <clock offset="utc" />
>   <on_poweroff>destroy</on_poweroff>
>   <on_reboot>restart</on_reboot>
>   <on_crash>destroy</on_crash>
>   <devices>
>     <emulator>/usr/local/bin/cloudlet_qemu-system-x86_64</emulator>
>     <disk device="disk" type="file">
>       <driver name="qemu" type="raw" />
>       <source file="/var/tmp/cloudletfs-Yq6vtn/disk/image" />
>       <target bus="ide" dev="hda" />
>       <address bus="0" controller="0" target="0" type="drive" unit="0" />
>     </disk>
>     <disk device="cdrom" type="file">
>       <driver name="qemu" type="raw" />
>       <source file="/var/lib/cloudlet/conf/ovftransport.iso" />
>       <target bus="ide" dev="hdc" />
>       <readonly />
>       <address bus="1" controller="0" target="0" type="drive" unit="0" />
>     </disk>
>     <controller index="0" type="ide">
>       <address bus="0x00" domain="0x0000" function="0x1" slot="0x01" type="pci" />
>     </controller>
>     <interface type="user">
>       <mac address="52:54:00:9f:a8:da" />
>       <model type="virtio" />
>       <address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci" />
>     </interface>
>     <input bus="ps2" type="mouse" />
>     <graphics autoport="yes" keymap="en-us" listen="127.0.0.1" port="-1" type="vnc">
>       <listen address="127.0.0.1" type="address" />
>     </graphics>
>     <video>
>       <model heads="1" type="cirrus" vram="9216" />
>       <address bus="0x00" domain="0x0000" function="0x0" slot="0x02" type="pci" />
>     </video>
>     <memballoon model="virtio">
>       <address bus="0x00" domain="0x0000" function="0x0" slot="0x04" type="pci" />
>     </memballoon>
>   </devices>
>   <ns0:commandline><ns0:arg value="-cloudlet" /><ns0:arg value="raw=suspend,logfile=/tmp/cloudlet-qemu-7A05Ug/qemu-trim-log" /><ns0:arg value="-redir" /><ns0:arg value="tcp:2222::22" /><ns0:arg value="-redir" /><ns0:arg value="tcp:8080::80" /></ns0:commandline><seclabel relabel="yes" type="dynamic" /></domain>
> XML: internal error: Child process (LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games HOME=/home/mickael USER=mickael LOGNAME=mickael /usr/local/bin/cloudlet_qemu-system-x86_64 -no-user-config -device '?' -device 'pci-assign,?' -device 'virtio-blk-pci,?' -device 'virtio-net-pci,?' -device 'scsi-disk,?' -device 'PIIX4_PM,?' -device 'usb-redir,?' -device 'ide-drive,?' -device 'usb-host,?' -device 'scsi-generic,?' -device 'usb-storage,?') unexpected exit status 1: Could not access KVM kernel module: No such file or directory
> failed to initialize KVM: No such file or directory
> No accelerator found!

> Error, make sure previous VM is closed and check QEMU_ARGUMENT, args - {}
> ERROR    failed at Traceback (most recent call last):
>   File "/usr/local/lib/python2.7/dist-packages/elijah/provisioning/synthesis.py", line 93, in decorated_function
>     return function(self, *args, **kwargs)
>   File "/usr/local/lib/python2.7/dist-packages/elijah/provisioning/synthesis.py", line 229, in resume_basevm
>     self.base_mem_fuse, self.new_xml_str)
>   File "/usr/local/lib/python2.7/dist-packages/elijah/provisioning/synthesis.py", line 1245, in run_snapshot
>     restore_with_config(conn, mem_snapshot, new_xml_string)
>   File "/usr/local/lib/python2.7/dist-packages/elijah/provisioning/synthesis.py", line 1337, in restore_with_config
>     raise CloudletGenerationError(message)
> CloudletGenerationError: <domain xmlns:ns0="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
>   <name>cloudlet-543793584db34094af90406bcad9c2c8</name>
>   <uuid>54379358-4db3-4094-af90-406bcad9c2c8</uuid>
>   <memory unit="KiB">1048576</memory>
>   <currentMemory unit="KiB">1048576</currentMemory>
>   <vcpu placement="static">4</vcpu>
>   <os>
>     <type arch="x86_64" machine="pc-1.0">hvm</type>
>     <boot dev="hd" />
>   </os>
>   <features>
>     <acpi />
>   </features>
>   <cpu match="exact" mode="custom">
>     <topology cores="4" sockets="1" threads="1" />
>   <model fallback="forbid">core2duo</model></cpu>
>   <clock offset="utc" />
>   <on_poweroff>destroy</on_poweroff>
>   <on_reboot>restart</on_reboot>
>   <on_crash>destroy</on_crash>
>   <devices>
>     <emulator>/usr/local/bin/cloudlet_qemu-system-x86_64</emulator>
>     <disk device="disk" type="file">
>       <driver name="qemu" type="raw" />
>       <source file="/var/tmp/cloudletfs-Yq6vtn/disk/image" />
>       <target bus="ide" dev="hda" />
>       <address bus="0" controller="0" target="0" type="drive" unit="0" />
>     </disk>
>     <disk device="cdrom" type="file">
>       <driver name="qemu" type="raw" />
>       <source file="/var/lib/cloudlet/conf/ovftransport.iso" />
>       <target bus="ide" dev="hdc" />
>       <readonly />
>       <address bus="1" controller="0" target="0" type="drive" unit="0" />
>     </disk>
>     <controller index="0" type="ide">
>       <address bus="0x00" domain="0x0000" function="0x1" slot="0x01" type="pci" />
>     </controller>
>     <interface type="user">
>       <mac address="52:54:00:9f:a8:da" />
>       <model type="virtio" />
>       <address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci" />
>     </interface>
>     <input bus="ps2" type="mouse" />
>     <graphics autoport="yes" keymap="en-us" listen="127.0.0.1" port="-1" type="vnc">
>       <listen address="127.0.0.1" type="address" />
>     </graphics>
>     <video>
>       <model heads="1" type="cirrus" vram="9216" />
>       <address bus="0x00" domain="0x0000" function="0x0" slot="0x02" type="pci" />
>     </video>
>     <memballoon model="virtio">
>       <address bus="0x00" domain="0x0000" function="0x0" slot="0x04" type="pci" />
>     </memballoon>
>   </devices>
>   <ns0:commandline><ns0:arg value="-cloudlet" /><ns0:arg value="raw=suspend,logfile=/tmp/cloudlet-qemu-7A05Ug/qemu-trim-log" /><ns0:arg value="-redir" /><ns0:arg value="tcp:2222::22" /><ns0:arg value="-redir" /><ns0:arg value="tcp:8080::80" /></ns0:commandline><seclabel relabel="yes" type="dynamic" /></domain>
> XML: internal error: Child process (LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games HOME=/home/mickael USER=mickael LOGNAME=mickael /usr/local/bin/cloudlet_qemu-system-x86_64 -no-user-config -device '?' -device 'pci-assign,?' -device 'virtio-blk-pci,?' -device 'virtio-net-pci,?' -device 'scsi-disk,?' -device 'PIIX4_PM,?' -device 'usb-redir,?' -device 'ide-drive,?' -device 'usb-host,?' -device 'scsi-generic,?' -device 'usb-storage,?') unexpected exit status 1: Could not access KVM kernel module: No such file or directory
> failed to initialize KVM: No such file or directory
> No accelerator found!

> Error, make sure previous VM is closed and check QEMU_ARGUMENT

> INFO     close Stream monitoring thread
> INFO     Fuse close pipe
> INFO     NO chunks has been waited at FUSE
> INFO     close Fuse Exec thread
> INFO     close File monitoring thread
> ERROR    Traceback (most recent call last):
>   File "/usr/local/bin/cloudlet", line 193, in main
>     machine = vm_overlay.resume_basevm()
>   File "/usr/local/lib/python2.7/dist-packages/elijah/provisioning/synthesis.py", line 102, in decorated_function
>     raise e
> CloudletGenerationError: <domain xmlns:ns0="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
>   <name>cloudlet-543793584db34094af90406bcad9c2c8</name>
>   <uuid>54379358-4db3-4094-af90-406bcad9c2c8</uuid>
>   <memory unit="KiB">1048576</memory>
>   <currentMemory unit="KiB">1048576</currentMemory>
>   <vcpu placement="static">4</vcpu>
>   <os>
>     <type arch="x86_64" machine="pc-1.0">hvm</type>
>     <boot dev="hd" />
>   </os>
>   <features>
>     <acpi />
>   </features>
>   <cpu match="exact" mode="custom">
>     <topology cores="4" sockets="1" threads="1" />
>   <model fallback="forbid">core2duo</model></cpu>
>   <clock offset="utc" />
>   <on_poweroff>destroy</on_poweroff>
>   <on_reboot>restart</on_reboot>
>   <on_crash>destroy</on_crash>
>   <devices>
>     <emulator>/usr/local/bin/cloudlet_qemu-system-x86_64</emulator>
>     <disk device="disk" type="file">
>       <driver name="qemu" type="raw" />
>       <source file="/var/tmp/cloudletfs-Yq6vtn/disk/image" />
>       <target bus="ide" dev="hda" />
>       <address bus="0" controller="0" target="0" type="drive" unit="0" />
>     </disk>
>     <disk device="cdrom" type="file">
>       <driver name="qemu" type="raw" />
>       <source file="/var/lib/cloudlet/conf/ovftransport.iso" />
>       <target bus="ide" dev="hdc" />
>       <readonly />
>       <address bus="1" controller="0" target="0" type="drive" unit="0" />
>     </disk>
>     <controller index="0" type="ide">
>       <address bus="0x00" domain="0x0000" function="0x1" slot="0x01" type="pci" />
>     </controller>
>     <interface type="user">
>       <mac address="52:54:00:9f:a8:da" />
>       <model type="virtio" />
>       <address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci" />
>     </interface>
>     <input bus="ps2" type="mouse" />
>     <graphics autoport="yes" keymap="en-us" listen="127.0.0.1" port="-1" type="vnc">
>       <listen address="127.0.0.1" type="address" />
>     </graphics>
>     <video>
>       <model heads="1" type="cirrus" vram="9216" />
>       <address bus="0x00" domain="0x0000" function="0x0" slot="0x02" type="pci" />
>     </video>
>     <memballoon model="virtio">
>       <address bus="0x00" domain="0x0000" function="0x0" slot="0x04" type="pci" />
>     </memballoon>
>   </devices>
>   <ns0:commandline><ns0:arg value="-cloudlet" /><ns0:arg value="raw=suspend,logfile=/tmp/cloudlet-qemu-7A05Ug/qemu-trim-log" /><ns0:arg value="-redir" /><ns0:arg value="tcp:2222::22" /><ns0:arg value="-redir" /><ns0:arg value="tcp:8080::80" /></ns0:commandline><seclabel relabel="yes" type="dynamic" /></domain>
> XML: internal error: Child process (LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games HOME=/home/mickael USER=mickael LOGNAME=mickael /usr/local/bin/cloudlet_qemu-system-x86_64 -no-user-config -device '?' -device 'pci-assign,?' -device 'virtio-blk-pci,?' -device 'virtio-net-pci,?' -device 'scsi-disk,?' -device 'PIIX4_PM,?' -device 'usb-redir,?' -device 'ide-drive,?' -device 'usb-host,?' -device 'scsi-generic,?' -device 'usb-storage,?') unexpected exit status 1: Could not access KVM kernel module: No such file or directory
> failed to initialize KVM: No such file or directory
> No accelerator found!

> Error, make sure previous VM is closed and check QEMU_ARGUMENT

I try many times to close Virtualbox VM and restart again, create new Virtualbox VM and it appears allways.Virtualbox is working properly, in the PC BIOS the virtualization function is on, I'm not really sure what's happening.

Thank you again for your help in overcoming this problem.

[teiszler]

Micael,

It seems this may only be possible if the nested Guest OS is 32-bit and not 64. See the second response in this article:
https://www.quora.com/VirtualBox-inside-VirtualBox-Is-that-possible

You could try using a 32-bit base VM and see if you could get that working.

Regards,
Tom

[MSoares]

Dear Tom,

I have finally receive a computer from my university in order to test my thesys scheme for 2 situations: 2 cloudlets and 2 openstack++.

So in my new computer, I have installed Ubuntu 14.04, elijah provisioning and elijah devstack and openstack++ like I had done in my laptop before.
-There are no errors in the g-reg.log file.
-This is a plain devstack installation on a Fresh Ubuntu install on university desktop.
-Glance is just using the filesystem as a backend for image storage by default.
-I am running all this in a "stack" user account I created on the machine.
-I recreated this running on an Ubuntu 14.04 VirtualBox running on another laptop I have; exactly the same errors.

Most of the installation runs fine, but I run into the error in elijah devstack: Found no valid images to use!

2017-02-15 14:27:06.453 | ++ declare -a images
2017-02-15 14:27:06.453 | ++ is_service_enabled glance
2017-02-15 14:27:06.455 | ++ return 0
2017-02-15 14:27:06.455 | ++ read -r IMAGE_NAME IMAGE_UUID
2017-02-15 14:27:06.456 | +++ awk '-F|' '!/^(+--)|ID|aki|ari/ { print $3,$2 }'
2017-02-15 14:27:06.456 | +++ glance image-list --status=active
2017-02-15 14:27:07.523 | ++ case "${#images[*]}" in
2017-02-15 14:27:07.523 | ++ echo 'Found no valid images to use!'
2017-02-15 14:27:07.523 | Found no valid images to use!
2017-02-15 14:27:07.523 | ++ exit 1
2017-02-15 14:27:07.523 | + exit_trap
2017-02-15 14:27:07.523 | + local r=1
2017-02-15 14:27:07.524 | ++ jobs -p
2017-02-15 14:27:07.524 | + jobs=
2017-02-15 14:27:07.524 | + [[ -n '' ]]
2017-02-15 14:27:07.524 | + kill_spinner
2017-02-15 14:27:07.524 | + '[' '!' -z '' ']'
2017-02-15 14:27:07.524 | + [[ 1 -ne 0 ]]
2017-02-15 14:27:07.524 | + echo 'Error on exit'
2017-02-15 14:27:07.524 | Error on exit
2017-02-15 14:27:07.524 | + generate-subunit 1487166685 2142 fail
2017-02-15 14:27:08.156 | + [[ -z /opt/stack/logs ]]
2017-02-15 14:27:08.156 | + /home/stack/devstack/tools/worlddump.py -d /opt/stack/logs
2017-02-15 14:27:08.838 | + exit 1

I think what is going wrong is that Glance is being used to find active images, but there are no images. If I login to Horizon, and go to the Images page, I can see the three Cirros images, but they are stuck in "Queued" state indefinetly; as such they are not active.

Further back in the stack.sh output, I can see the glance calls to upload the Cirros images, and they return ok (with the image status "Queued" as well.

I took a look at the glance-api log in /opt/stack/logs/g-api.log, and indeed, I am seeing three blocks of errors; one for each of the three Cirros images:

2017-02-15 14:24:26.455 e[00;32mDEBUG glance.common.client [e[01;36mreq-cc6f031f-1ed3-42fc-91dd-1ea88b56218d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mConstructed URL: http://0.0.0.0:9191/images/cirros-0.3.4-x86_64-uec-kernele[00m e[00;33mfrom (pid=5906) constructurl /opt/stack/glance/glance/common/client.py:401e[00m
2017-02-15 14:24:26.723 e[01;31mERROR glance.registry.client.v1.client [e[01;36mreq-cc6f031f-1ed3-42fc-91dd-1ea88b56218d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[01;31m] e[01;35me[01;31mRegistry client request GET /images/cirros-0.3.4-x86_64-uec-kernel raised NotFounde[00m
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00mTraceback (most recent call last):
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/registry/client/v1/client.py", line 117, in do_request
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m **kwargs)
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/common/client.py", line 71, in wrapped
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m return func(self, *args, **kwargs)
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/common/client.py", line 376, in do_request
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m headers=copy.deepcopy(headers))
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/common/client.py", line 88, in wrapped
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m return func(self, method, url, body, headers)
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/common/client.py", line 523, in dorequest
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m raise exception.NotFound(res.read())
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00mNotFound: 404 Not Found
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00mThe resource could not be found.
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m
e[01;31m2017-02-15 14:24:26.723 TRACE glance.registry.client.v1.client e[01;35me[00m
2017-02-15 14:24:26.724 e[00;32mDEBUG glance.api.v1.controller [e[01;36mreq-cc6f031f-1ed3-42fc-91dd-1ea88b56218d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mImage with identifier cirros-0.3.4-x86_64-uec-kernel not founde[00m e[00;33mfrom (pid=5906) get_image_meta_or_404 /opt/stack/glance/glance/api/v1/controller.py:45e[00m
2017-02-15 14:24:26.724 e[00;36mINFO eventlet.wsgi.server [e[01;36mreq-cc6f031f-1ed3-42fc-91dd-1ea88b56218d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;36m] e[01;35me[00;36m10.0.32.60 - - [15/Feb/2017 14:24:26] "HEAD /v1/images/cirros-0.3.4-x86_64-uec-kernel HTTP/1.1" 404 189 0.600849e[00m
2017-02-15 14:24:26.726 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-cc6f031f-1ed3-42fc-91dd-1ea88b56218d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mDetermining version of request: GET /v1/images/detail Accept: /e[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:47e[00m
2017-02-15 14:24:26.726 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-cc6f031f-1ed3-42fc-91dd-1ea88b56218d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mUsing url versioninge[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:60e[00m
2017-02-15 14:24:26.726 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-cc6f031f-1ed3-42fc-91dd-1ea88b56218d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mMatched version: v1e[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:72e[00m
2017-02-15 14:24:26.727 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-cc6f031f-1ed3-42fc-91dd-1ea88b56218d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mnew path /v1/images/detaile[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:73e[00m
2017-02-15 14:24:26.728 e[00;32mDEBUG oslo_policy.policy [e[01;36mreq-620ca48e-a1a1-4799-916a-7a8591d3fd66 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mReloaded policy file: /etc/glance/policy.jsone[00m e[00;33mfrom (pid=5906) loadpolicy_file /usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py:425e[00m
2017-02-15 14:24:26.728 e[00;32mDEBUG glance.common.client [e[01;36mreq-620ca48e-a1a1-4799-916a-7a8591d3fd66 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mConstructed URL: http://0.0.0.0:9191/images/detail?limit=20e[00m e[00;33mfrom (pid=5906) constructurl /opt/stack/glance/glance/common/client.py:401e[00m
2017-02-15 14:24:26.745 e[00;32mDEBUG glance.registry.client.v1.client [e[01;36mreq-620ca48e-a1a1-4799-916a-7a8591d3fd66 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mRegistry request GET /images/detail HTTP 200 request id req-0047f60f-16b6-4c2a-8109-c780340f2160e[00m e[00;33mfrom (pid=5906) do_request /opt/stack/glance/glance/registry/client/v1/client.py:124e[00m
2017-02-15 14:24:26.746 e[00;36mINFO eventlet.wsgi.server [e[01;36mreq-620ca48e-a1a1-4799-916a-7a8591d3fd66 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;36m] e[01;35me[00;36m10.0.32.60 - - [15/Feb/2017 14:24:26] "GET /v1/images/detail?limit=20 HTTP/1.1" 200 188 0.020170e[00m
2017-02-15 14:24:26.752 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-620ca48e-a1a1-4799-916a-7a8591d3fd66 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mDetermining version of request: POST /v1/images Accept: /e[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:47e[00m
2017-02-15 14:24:26.752 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-620ca48e-a1a1-4799-916a-7a8591d3fd66 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mUsing url versioninge[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:60e[00m
2017-02-15 14:24:26.752 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-620ca48e-a1a1-4799-916a-7a8591d3fd66 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mMatched version: v1e[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:72e[00m
2017-02-15 14:24:26.752 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-620ca48e-a1a1-4799-916a-7a8591d3fd66 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mnew path /v1/imagese[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:73e[00m
2017-02-15 14:24:26.754 e[00;32mDEBUG oslo_policy.policy [e[01;36mreq-1c378dd5-c5dd-4f26-8a2f-2171b115495d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mReloaded policy file: /etc/glance/policy.jsone[00m e[00;33mfrom (pid=5906) loadpolicy_file /usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py:425e[00m
2017-02-15 14:24:26.755 e[00;32mDEBUG oslo_policy.policy [e[01;36mreq-1c378dd5-c5dd-4f26-8a2f-2171b115495d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mReloaded policy file: /etc/glance/policy.jsone[00m e[00;33mfrom (pid=5906) loadpolicy_file /usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py:425e[00m
2017-02-15 14:24:26.755 e[00;32mDEBUG glance.registry.client.v1.api [e[01;36mreq-1c378dd5-c5dd-4f26-8a2f-2171b115495d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mAdding image metadata...e[00m e[00;33mfrom (pid=5906) add_image_metadata /opt/stack/glance/glance/registry/client/v1/api.py:161e[00m
2017-02-15 14:24:26.755 e[00;32mDEBUG glance.common.client [e[01;36mreq-1c378dd5-c5dd-4f26-8a2f-2171b115495d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mConstructed URL: http://0.0.0.0:9191/imagese[00m e[00;33mfrom (pid=5906) constructurl /opt/stack/glance/glance/common/client.py:401e[00m
2017-02-15 14:24:27.302 e[00;32mDEBUG glance.registry.client.v1.client [e[01;36mreq-1c378dd5-c5dd-4f26-8a2f-2171b115495d e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mRegistry request POST /images HTTP 200 request id req-fecd8d1f-2c73-49d3-8dd6-2b87c01c4df0e[00m e[00;33mfrom (pid=5906) do_request /opt/stack/glance/glance/registry/client/v1/client.py:124e[00m
...
2017-02-15 14:24:29.104 e[01;31mERROR glance.registry.client.v1.client [e[01;36mreq-d798d5bc-b01b-4016-ad3f-d8580d26cf96 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[01;31m] e[01;35me[01;31mRegistry client request GET /images/cirros-0.3.4-x86_64-uec-ramdisk raised NotFounde[00m
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00mTraceback (most recent call last):
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/registry/client/v1/client.py", line 117, in do_request
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m **kwargs)
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/common/client.py", line 71, in wrapped
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m return func(self, *args, **kwargs)
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/common/client.py", line 376, in do_request
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m headers=copy.deepcopy(headers))
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/common/client.py", line 88, in wrapped
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m return func(self, method, url, body, headers)
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/common/client.py", line 523, in dorequest
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m raise exception.NotFound(res.read())
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00mNotFound: 404 Not Found
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00mThe resource could not be found.
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m
e[01;31m2017-02-15 14:24:29.104 TRACE glance.registry.client.v1.client e[01;35me[00m
2017-02-15 14:24:29.105 e[00;32mDEBUG glance.api.v1.controller [e[01;36mreq-d798d5bc-b01b-4016-ad3f-d8580d26cf96 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mImage with identifier cirros-0.3.4-x86_64-uec-ramdisk not founde[00m e[00;33mfrom (pid=5906) get_image_meta_or_404 /opt/stack/glance/glance/api/v1/controller.py:45e[00m
2017-02-15 14:24:29.105 e[00;36mINFO eventlet.wsgi.server [e[01;36mreq-d798d5bc-b01b-4016-ad3f-d8580d26cf96 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;36m] e[01;35me[00;36m10.0.32.60 - - [15/Feb/2017 14:24:29] "HEAD /v1/images/cirros-0.3.4-x86_64-uec-ramdisk HTTP/1.1" 404 189 0.012802e[00m
2017-02-15 14:24:29.107 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-d798d5bc-b01b-4016-ad3f-d8580d26cf96 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mDetermining version of request: GET /v1/images/detail Accept: /e[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:47e[00m
2017-02-15 14:24:29.107 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-d798d5bc-b01b-4016-ad3f-d8580d26cf96 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mUsing url versioninge[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:60e[00m
2017-02-15 14:24:29.107 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-d798d5bc-b01b-4016-ad3f-d8580d26cf96 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mMatched version: v1e[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:72e[00m
2017-02-15 14:24:29.107 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-d798d5bc-b01b-4016-ad3f-d8580d26cf96 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mnew path /v1/images/detaile[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:73e[00m
2017-02-15 14:24:29.109 e[00;32mDEBUG oslo_policy.policy [e[01;36mreq-c5e53f94-19f5-4554-bea1-faa7b9d5753e e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mReloaded policy file: /etc/glance/policy.jsone[00m e[00;33mfrom (pid=5906) loadpolicy_file /usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py:425e[00m
2017-02-15 14:24:29.109 e[00;32mDEBUG glance.common.client [e[01;36mreq-c5e53f94-19f5-4554-bea1-faa7b9d5753e e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mConstructed URL: http://0.0.0.0:9191/images/detail?limit=20e[00m e[00;33mfrom (pid=5906) constructurl /opt/stack/glance/glance/common/client.py:401e[00m
2017-02-15 14:24:29.130 e[00;32mDEBUG glance.registry.client.v1.client [e[01;36mreq-c5e53f94-19f5-4554-bea1-faa7b9d5753e e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mRegistry request GET /images/detail HTTP 200 request id req-7078d072-7f3f-416c-933b-64a9b14082a2e[00m e[00;33mfrom (pid=5906) do_request /opt/stack/glance/glance/registry/client/v1/client.py:124e[00m
2017-02-15 14:24:29.131 e[00;36mINFO eventlet.wsgi.server [e[01;36mreq-c5e53f94-19f5-4554-bea1-faa7b9d5753e e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;36m] e[01;35me[00;36m10.0.32.60 - - [15/Feb/2017 14:24:29] "GET /v1/images/detail?limit=20 HTTP/1.1" 200 660 0.024116e[00m
2017-02-15 14:24:29.136 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-c5e53f94-19f5-4554-bea1-faa7b9d5753e e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mDetermining version of request: POST /v1/images Accept: /e[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:47e[00m
2017-02-15 14:24:29.137 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-c5e53f94-19f5-4554-bea1-faa7b9d5753e e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mUsing url versioninge[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:60e[00m
2017-02-15 14:24:29.137 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-c5e53f94-19f5-4554-bea1-faa7b9d5753e e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mMatched version: v1e[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:72e[00m
2017-02-15 14:24:29.137 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[01;36mreq-c5e53f94-19f5-4554-bea1-faa7b9d5753e e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mnew path /v1/imagese[00m e[00;33mfrom (pid=5906) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:73e[00m
2017-02-15 14:24:29.139 e[00;32mDEBUG oslo_policy.policy [e[01;36mreq-35282f95-e4ad-40f6-9213-6a56c61d6c3b e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mReloaded policy file: /etc/glance/policy.jsone[00m e[00;33mfrom (pid=5906) loadpolicy_file /usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py:425e[00m
2017-02-15 14:24:29.139 e[00;32mDEBUG oslo_policy.policy [e[01;36mreq-35282f95-e4ad-40f6-9213-6a56c61d6c3b e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mReloaded policy file: /etc/glance/policy.jsone[00m e[00;33mfrom (pid=5906) loadpolicy_file /usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py:425e[00m
2017-02-15 14:24:29.139 e[00;32mDEBUG glance.registry.client.v1.api [e[01;36mreq-35282f95-e4ad-40f6-9213-6a56c61d6c3b e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mAdding image metadata...e[00m e[00;33mfrom (pid=5906) add_image_metadata /opt/stack/glance/glance/registry/client/v1/api.py:161e[00m
2017-02-15 14:24:29.140 e[00;32mDEBUG glance.common.client [e[01;36mreq-35282f95-e4ad-40f6-9213-6a56c61d6c3b e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mConstructed URL: http://0.0.0.0:9191/imagese[00m e[00;33mfrom (pid=5906) constructurl /opt/stack/glance/glance/common/client.py:401e[00m
2017-02-15 14:24:29.189 e[00;32mDEBUG glance.registry.client.v1.client [e[01;36mreq-35282f95-e4ad-40f6-9213-6a56c61d6c3b e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mRegistry request POST /images HTTP 200 request id req-1620bb84-58a4-499a-b54a-4eefb9c5059de[00m e[00;33mfrom (pid=5906) do_request /opt/stack/glance/glance/registry/client/v1/client.py:124e[00m
2017-02-15 14:24:29.189 e[00;32mDEBUG oslo_messaging.drivers.amqp [e[01;36mreq-35282f95-e4ad-40f6-9213-6a56c61d6c3b e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mUNIQUEID is 6ceecfde2dcf41748fbdc94edacc3d5f.e[00m e[00;33mfrom (pid=5906) addunique_id /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:258e[00m
2017-02-15 14:24:29.195 e[00;36mINFO eventlet.wsgi.server [e[01;36mreq-35282f95-e4ad-40f6-9213-6a56c61d6c3b e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;36m] e[01;35me[00;36m10.0.32.60 - - [15/Feb/2017 14:24:29] "POST /v1/images HTTP/1.1" 201 744 0.058661e[00m
2017-02-15 14:24:30.710 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[00;36m-e[00;32m] e[01;35me[00;32mDetermining version of request: HEAD /v1/images/cirros-0.3.4-x86_64-uec Accept: /e[00m e[00;33mfrom (pid=5907) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:47e[00m
2017-02-15 14:24:30.710 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[00;36m-e[00;32m] e[01;35me[00;32mUsing url versioninge[00m e[00;33mfrom (pid=5907) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:60e[00m
2017-02-15 14:24:30.710 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[00;36m-e[00;32m] e[01;35me[00;32mMatched version: v1e[00m e[00;33mfrom (pid=5907) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:72e[00m
2017-02-15 14:24:30.711 e[00;32mDEBUG glance.api.middleware.version_negotiation [e[00;36m-e[00;32m] e[01;35me[00;32mnew path /v1/images/cirros-0.3.4-x86_64-uece[00m e[00;33mfrom (pid=5907) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:73e[00m
2017-02-15 14:24:30.711 e[00;32mDEBUG keystoneclient.session [e[00;36m-e[00;32m] e[01;35me[00;32mREQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X GET http://10.0.32.60:35357 -H "Accept: application/json" -H "User-Agent: python-keystoneclient"e[00m e[00;33mfrom (pid=5907) httplog_request /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:196e[00m
2017-02-15 14:24:30.721 e[00;32mDEBUG keystoneclient.session [e[00;36m-e[00;32m] e[01;35me[00;32mRESP: [300] date: Wed, 15 Feb 2017 14:24:30 GMT vary: X-Auth-Token content-length: 591 content-type: application/json server: Apache/2.4.7 (Ubuntu)
RESP BODY: {"versions": {"values": [{"status": "stable", "updated": "2015-03-30T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.4", "links": [{"href": "http://10.0.32.60:35357/v3/", "rel": "self"}]}, {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://10.0.32.60:35357/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}]}}
e[00m e[00;33mfrom (pid=5907) httplog_response /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:226e[00m
2017-02-15 14:24:30.721 e[00;32mDEBUG keystoneclient.auth.identity.v3.base [e[00;36m-e[00;32m] e[01;35me[00;32mMaking authentication request to http://10.0.32.60:35357/v3/auth/tokense[00m e[00;33mfrom (pid=5907) get_auth_ref /usr/local/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py:171e[00m
2017-02-15 14:24:30.860 e[00;32mDEBUG keystoneclient.session [e[00;36m-e[00;32m] e[01;35me[00;32mREQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X GET http://10.0.32.60:35357/ -H "Accept: application/json" -H "User-Agent: python-keystoneclient"e[00m e[00;33mfrom (pid=5907) httplog_request /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:196e[00m
2017-02-15 14:24:30.864 e[00;32mDEBUG keystoneclient.session [e[00;36m-e[00;32m] e[01;35me[00;32mRESP: [300] date: Wed, 15 Feb 2017 14:24:30 GMT vary: X-Auth-Token content-length: 591 content-type: application/json server: Apache/2.4.7 (Ubuntu)
RESP BODY: {"versions": {"values": [{"status": "stable", "updated": "2015-03-30T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.4", "links": [{"href": "http://10.0.32.60:35357/v3/", "rel": "self"}]}, {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://10.0.32.60:35357/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}]}}
e[00m e[00;33mfrom (pid=5907) httplog_response /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:226e[00m
2017-02-15 14:24:30.866 e[00;32mDEBUG keystoneclient.session [e[00;36m-e[00;32m] e[01;35me[00;32mREQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X GET http://10.0.32.60:35357/v3/auth/tokens -H "X-Subject-Token: {SHA1}f394770ee7ff323e14661deb937fe46d831143e8" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}bab24e4a04d1d8b0883646a8f54bdd556e51623c"e[00m e[00;33mfrom (pid=5907) httplog_request /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:196e[00m
2017-02-15 14:24:30.913 e[00;32mDEBUG keystoneclient.session [e[00;36m-e[00;32m] e[01;35me[00;32mRESP: [200] content-length: 4773 x-subject-token: {SHA1}f394770ee7ff323e14661deb937fe46d831143e8 vary: X-Auth-Token server: Apache/2.4.7 (Ubuntu) date: Wed, 15 Feb 2017 14:24:30 GMT content-type: application/json x-openstack-request-id: req-3bb08a77-2e2f-4ca5-8e05-df579789a36a
RESP BODY: {"token": {"methods": ["password", "token"], "roles": [{"id": "544b6a735c914c3b988b066489acf613", "name": "admin"}], "expires_at": "2017-02-15T15:23:37.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "4fae5bfb06f249eaa48fb16dd09ea0bb", "name": "admin"}, "catalog": "", "extras": {}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "478c8491baa149968138a0c9faf4ea43", "name": "admin"}, "audit_ids": ["Ql8mGpt4TLu2z966qzx3kA"], "issued_at": "2017-02-15T14:23:37.535632"}}
e[00m e[00;33mfrom (pid=5907) httplog_response /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:226e[00m
2017-02-15 14:24:30.921 e[00;32mDEBUG oslo_policy.openstack.common.fileutils [e[01;36mreq-043b48a1-6a9e-4fb2-9b80-a35830336121 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mReloading cached file /etc/glance/policy.jsone[00m e[00;33mfrom (pid=5907) read_cached_file /usr/local/lib/python2.7/dist-packages/oslo_policy/openstack/common/fileutils.py:64e[00m
2017-02-15 14:24:30.922 e[00;32mDEBUG oslo_policy.policy [e[01;36mreq-043b48a1-6a9e-4fb2-9b80-a35830336121 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mReloaded policy file: /etc/glance/policy.jsone[00m e[00;33mfrom (pid=5907) loadpolicy_file /usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py:425e[00m
2017-02-15 14:24:30.923 e[00;32mDEBUG glance.common.client [e[01;36mreq-043b48a1-6a9e-4fb2-9b80-a35830336121 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[00;32m] e[01;35me[00;32mConstructed URL: http://0.0.0.0:9191/images/cirros-0.3.4-x86_64-uece[00m e[00;33mfrom (pid=5907) constructurl /opt/stack/glance/glance/common/client.py:401e[00m
2017-02-15 14:24:30.933 e[01;31mERROR glance.registry.client.v1.client [e[01;36mreq-043b48a1-6a9e-4fb2-9b80-a35830336121 e[00;36m478c8491baa149968138a0c9faf4ea43 4fae5bfb06f249eaa48fb16dd09ea0bbe[01;31m] e[01;35me[01;31mRegistry client request GET /images/cirros-0.3.4-x86_64-uec raised NotFounde[00m
e[01;31m2017-02-15 14:24:30.933 TRACE glance.registry.client.v1.client e[01;35me[00mTraceback (most recent call last):
e[01;31m2017-02-15 14:24:30.933 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/registry/client/v1/client.py", line 117, in do_request

Is there any update in github for elijah devstack? It could explain what is going on?
Please Tom, could you point me in the right direction here, I would be extremely grateful. Thanks very much in advance.

All the best,

Micael

[teiszler]

Micael,

A few things. It seems suspect to me that there needs to be images during the installation of DevStack. Indeed, I am a little confused how you would already have 3 CirrOS images that Glance would even be trying to handle. Is it possible that these images were left around from a previous DevStack installation that was not cleaned up? The DevStack code is separate from the cloudlet library (elijah-provisioning) and the OpenStack++ extension (elijah-openstack). That being said, I am working on replacing the DevStack portion of this setup with an Ansible script that will install OpenStack Kilo without using DevStack, though I am still working some issues out and not ready to push it to the elijah-openstack repository.

If you can get into the Horizon dashboard, it would seem that the DevStack installation was successful. At least partially. If you go into the dashboard, and goto the Admin->System->System Information section in the menu at the left, does it indicate that glance is enabled? Can you remove the Cirros images that are queued under the Images section? To me it would seem that some things were not cleaned up from a previous installation perhaps.

Regards,
Tom

[MSoares]

Dear Tom,

I have install devstack on fresh Ubuntu in my desktop installation. I have also to say that in my laptop, when I installed sucessfully openstack++, it came also with these 3 CirrOs images, I think that these images come with openstack for user to test dashboard and console.

I am able to login into dashboard, and it appears the 3 images queued in the System Images. I was able to sucessfully delete them. Regarding the System Configuration, all services are enabled.

So I have eliminate 3 CIrrOs images, logoff and login into Dashboard, and the System Images appears blank. I tryied to import Base Vm, and it starts working, and it appears Queued and I can let for 1h,2h and 3h and nothing changes.

It seems it get stucks somewhere, do you have some clue?

Bests regards,

Micael

[teiszler]

Hmm. This is odd. it seems that glance is misbehaving. Unfortunately the way DevStack works, there isn't a simple way just to reinstall that component. I would suggest trying to use glance commands to see if they can help point you in the right direction. Similar to how you issued the nova command to see if the services were running, you can also run glance image-list or even attempt to create an image from the command-line with glance image-create. I would try doing some of these operations and subsequently looking at the glance-api log to see if anything of interest shows up in the log or on stdout when you try them.

[MSoares]

Dear Tom, I have been trying to see what is happenning during an image creation.

Strangely, i suppose that the command line result in an ordinary image creation saving, something normal. Now on, I do not know, how many time I will have to wait to consider that the test is not correct, I have been waiting for 2:30.

Bests regards,
Micael

[MSoares]

Dear Tom,

after a better look, I had to come back again in devstack installation because the previous errors start here first.

It seems that during installation, it try to get CirrOS images but the result is that it does not found them because of an authentication required on 0.0.0.0:9191, I find it in g-api.log

DEBUG glance.common.client [e[01;36mreq-89d29dcd-f06a-4069-a9e1-34dcc7ac70b0 e[00;36mf47588354ae8454092cab6bc73d21344 7577d3e984814e969f3f30bfdc6f4515e[00;32m] e[01;35me[00;32mConstructed URL: http://0.0.0.0:9191/images/cirros-0.3.4-x86_64-uec-ramdiske[00m e[00;33mfrom (pid=29539) constructurl /opt/stack/glance/glance/common/client.py:401e[00m
2017-02-20 00:16:44.422 e[01;31mERROR glance.registry.client.v1.client [e[01;36mreq-89d29dcd-f06a-4069-a9e1-34dcc7ac70b0 e[00;36mf47588354ae8454092cab6bc73d21344 7577d3e984814e969f3f30bfdc6f4515e[01;31m] e[01;35me[01;31mRegistry client request GET /images/cirros-0.3.4-x86_64-uec-ramdisk raised NotFounde[00m
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00mTraceback (most recent call last):
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/registry/client/v1/client.py", line 117, in do_request
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m **kwargs)
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/common/client.py", line 71, in wrapped
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m return func(self, *args, **kwargs)
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/common/client.py", line 376, in do_request
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m headers=copy.deepcopy(headers))
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/common/client.py", line 88, in wrapped
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m return func(self, method, url, body, headers)
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m File "/opt/stack/glance/glance/common/client.py", line 523, in dorequest
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m raise exception.NotFound(res.read())
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00mNotFound: 404 Not Found
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00mThe resource could not be found.
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m
e[01;31m2017-02-20 00:16:44.422 TRACE glance.registry.client.v1.client e[01;35me[00m

Please help me with this, I am really stuck here and no idea how to proceed.

Bests regards,
Micael

[teiszler]

Micael,

It is odd that it would be trying to get to 0.0.0.0. I would expect it to be something like 127.0.0.1 or 10.0.32.60 like the keystone authentication service is listening on.

If you do the following, what output do you get?

openstack endpoint list

You should get a list of URLs that correspond to all the endpoints for the various OpenStack components. What is the URLs listed for glance?

[MSoares]

Tom,

yes I know, that is what i thought also.

stack@mickael-HP-ProDesk-600-G1-TWR:~$ openstack endpoint list
ERROR: openstack Authorization Failed: Cannot authenticate without an auth_url

I tryed to put some --os-auth-url=OS_AUTH_URL=http://controller:35357/v2.0,but no result.

It is very tricky,

[teiszler]

There should be a shell script called something like admin-openrc.sh. If you do source admin-openrc.sh it will setup environment variables that will eliminate the need for you to authenticate with each command.

If you can't find it, then you could always create one. It should look something like this:

export OS_PROJECT_DOMAIN_ID=default
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_NAME=admin
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=<your admin pw>
export OS_AUTH_URL=http://<ip of control node>:35357/v3
export OS_IDENTITY_API_VERSION=3

After you have created and sourced this script, try running the endpoint-list command again.

[MSoares]

I had to create the file admin-openrc.sh.

I source it but it maintain with error:

stack@mickael-HP-ProDesk-600-G1-TWR:~/devstack$ chmod +x admin-openrc.sh
stack@mickael-HP-ProDesk-600-G1-TWR:~/devstack$ source admin-openrc.sh
stack@mickael-HP-ProDesk-600-G1-TWR:~/devstack$ openstack endpoint list
ERROR: openstack Unable to establish connection to http://10.0.32.60:35357/v3/auth/tokens

[teiszler]

If you change the OS_IDENTITY_API_VERSION to 2 and also use v2.0 instead of v3 in the OS_AUTH_URL does it work?

[MSoares]

This is the admin-openrc.sh:

/bin/bash

export OS_PROJECT_DOMAIN_ID=default
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_NAME=admin
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=nomoresecrete
export OS_AUTH_URL=http://10.0.32.60:35357/v2
export OS_IDENTITY_API_VERSION=2

and the result in command line is:

stack@mickael-HP-ProDesk-600-G1-TWR:~/devstack$ gedit admin-openrc.sh
stack@mickael-HP-ProDesk-600-G1-TWR:~/devstack$ source admin-openrc.sh
stack@mickael-HP-ProDesk-600-G1-TWR:~/devstack$ openstack endpoint list
WARNING: keystoneclient.auth.identity.generic.base Discovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL.
WARNING: keystoneclient.auth.identity.generic.base Discovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL.
ERROR: openstack Could not determine a suitable URL for the plugin
stack@mickael-HP-ProDesk-600-G1-TWR:~/devstack$

[teiszler]

I believe it needs to be v2.0 in the URL. So http://10.0.32.60:35357/v2.0

[MSoares]

dear Tom,

the result for http://10.0.32.60:35357/v2.0, is:

stack@mickael-HP-ProDesk-600-G1-TWR:~/devstack$ openstack endpoint list
WARNING: keystoneclient.auth.identity.generic.base Discovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL.
WARNING: keystoneclient.auth.identity.generic.base Discovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL.
ERROR: openstack Could not determine a suitable URL for the plugin
stack@mickael-HP-ProDesk-600-G1-TWR:~/devstack$

If I join also 2.0 to export OS_IDENTITY_API_VERSION=2.0, the result is:

ERROR: openstack Unable to establish connection to http://10.0.32.60:35357/v2.0/tokens

[teiszler]

I am at a loss. At this point it may be best to start from the top. Have you considered beginning again with the installation of DevStack? It does not appear that we are making much progress by trying to troubleshoot the existing installation. I would recommend using the ./clean.sh script in the devstack directory to clean up the current installation beforehand.

[MSoares]

Dear Tom,

so am I. If I do ./unstack.sh, then ./clean.sh and ./stack.sh

I will have error:

2017-02-20 20:17:02.359 | You are using pip version 7.1.2, however version 9.0.1 is available.
2017-02-20 20:17:02.359 | You should consider upgrading via the 'pip install --upgrade pip' command.
2017-02-20 20:17:02.379 | ++ local 'test_req=testrepository>=0.0.18/test-requirements.txt'
2017-02-20 20:17:02.379 | ++ [[ -e testrepository>=0.0.18/test-requirements.txt ]]
2017-02-20 20:17:02.379 | ++ local image_lines
2017-02-20 20:17:02.379 | ++ local images
2017-02-20 20:17:02.379 | ++ local num_images
2017-02-20 20:17:02.379 | ++ local image_uuid
2017-02-20 20:17:02.379 | ++ local image_uuid_alt
2017-02-20 20:17:02.379 | ++ local password
2017-02-20 20:17:02.379 | ++ local line
2017-02-20 20:17:02.379 | ++ local flavors
2017-02-20 20:17:02.379 | ++ local available_flavors
2017-02-20 20:17:02.379 | ++ local flavors_ref
2017-02-20 20:17:02.379 | ++ local flavor_lines
2017-02-20 20:17:02.379 | ++ local public_network_id
2017-02-20 20:17:02.379 | ++ local public_router_id
2017-02-20 20:17:02.379 | ++ local tenant_networks_reachable
2017-02-20 20:17:02.379 | ++ local boto_instance_type=m1.tiny
2017-02-20 20:17:02.379 | ++ local ssh_connect_method=fixed
2017-02-20 20:17:02.379 | ++ ifs='
2017-02-20 20:17:02.379 | '
2017-02-20 20:17:02.379 | ++ declare -a images
2017-02-20 20:17:02.379 | ++ is_service_enabled glance
2017-02-20 20:17:02.381 | ++ return 0
2017-02-20 20:17:02.381 | ++ read -r IMAGE_NAME IMAGE_UUID
2017-02-20 20:17:02.382 | +++ glance image-list --status=active
2017-02-20 20:17:02.382 | +++ awk '-F|' '!/^(+--)|ID|aki|ari/ { print $3,$2 }'
2017-02-20 20:17:03.163 | An auth plugin is required to fetch a token
2017-02-20 20:17:03.206 | ++ case "${#images[*]}" in
2017-02-20 20:17:03.206 | ++ echo 'Found no valid images to use!'
2017-02-20 20:17:03.206 | Found no valid images to use!
2017-02-20 20:17:03.206 | ++ exit 1
2017-02-20 20:17:03.206 | + exit_trap
2017-02-20 20:17:03.206 | + local r=1
2017-02-20 20:17:03.207 | ++ jobs -p
2017-02-20 20:17:03.208 | + jobs=
2017-02-20 20:17:03.208 | + [[ -n '' ]]
2017-02-20 20:17:03.208 | + kill_spinner
2017-02-20 20:17:03.208 | + '[' '!' -z '' ']'
2017-02-20 20:17:03.208 | + [[ 1 -ne 0 ]]
2017-02-20 20:17:03.208 | + echo 'Error on exit'
2017-02-20 20:17:03.208 | Error on exit
2017-02-20 20:17:03.208 | + generate-subunit 1487621147 676 fail
2017-02-20 20:17:03.827 | + [[ -z /opt/stack/logs ]]
2017-02-20 20:17:03.827 | + /home/stack/devstack/tools/worlddump.py -d /opt/stack/logs
2017-02-20 20:17:03.913 | + exit 1

it is so tricky
this is the link for 2 files of stack.log
https://drive.google.com/open?id=0B2_7RViS1PuGZVB0RDFmVlNDY3M

Now on, if I change admin-openrc file and put:

export OS_AUTH_URL=http://10.0.32.60:35357/v2
export OS_IDENTITY_API_VERSION=2.0

and after that I make unstack, clean and stack, it will return the next error:

2017-02-21 01:12:28.784 | + [[ -r /home/stack/devstack/local.conf ]]
2017-02-21 01:12:28.784 | + local configfile group
2017-02-21 01:12:28.784 | + for group in '$matchgroups'
2017-02-21 01:12:28.784 | ++ get_meta_section_files /home/stack/devstack/local.conf post-config
2017-02-21 01:12:28.784 | ++ local file=/home/stack/devstack/local.conf
2017-02-21 01:12:28.785 | ++ local matchgroup=post-config
2017-02-21 01:12:28.785 | ++ [[ -r /home/stack/devstack/local.conf ]]
2017-02-21 01:12:28.785 | ++ awk -v matchgroup=post-config '
2017-02-21 01:12:28.785 | /^[[.+\|.*]]/ {
2017-02-21 01:12:28.785 | gsub("[][]", "", $1);
2017-02-21 01:12:28.785 | split($1, a, "|");
2017-02-21 01:12:28.785 | if (a[1] == matchgroup)
2017-02-21 01:12:28.785 | print a[2]
2017-02-21 01:12:28.785 | }
2017-02-21 01:12:28.785 | ' /home/stack/devstack/local.conf
2017-02-21 01:12:28.788 | + is_service_enabled s-proxy
2017-02-21 01:12:28.791 | + return 1
2017-02-21 01:12:28.791 | + is_service_enabled glance
2017-02-21 01:12:28.793 | + return 0
2017-02-21 01:12:28.793 | + echo_summary 'Starting Glance'
2017-02-21 01:12:28.793 | + [[ -t 3 ]]
2017-02-21 01:12:28.793 | + [[ True != \T\r\u\e ]]
2017-02-21 01:12:28.793 | + echo -e Starting Glance
2017-02-21 01:12:28.793 | + start_glance
2017-02-21 01:12:28.793 | + local service_protocol=http
2017-02-21 01:12:28.793 | + is_service_enabled tls-proxy
2017-02-21 01:12:28.796 | + return 1
2017-02-21 01:12:28.796 | + run_process g-reg '/usr/local/bin/glance-registry --config-file=/etc/glance/glance-registry.conf'
2017-02-21 01:12:28.796 | + local service=g-reg
2017-02-21 01:12:28.796 | + local 'command=/usr/local/bin/glance-registry --config-file=/etc/glance/glance-registry.conf'
2017-02-21 01:12:28.796 | + local group=
2017-02-21 01:12:28.796 | + is_service_enabled g-reg
2017-02-21 01:12:28.799 | + return 0
2017-02-21 01:12:28.799 | + [[ True = \T\r\u\e ]]
2017-02-21 01:12:28.799 | + screen_process g-reg '/usr/local/bin/glance-registry --config-file=/etc/glance/glance-registry.conf' ''
2017-02-21 01:12:28.799 | + local name=g-reg
2017-02-21 01:12:28.799 | + local 'command=/usr/local/bin/glance-registry --config-file=/etc/glance/glance-registry.conf'
2017-02-21 01:12:28.799 | + local group=
2017-02-21 01:12:28.799 | + SCREEN_NAME=stack
2017-02-21 01:12:28.799 | + SERVICE_DIR=/opt/stack/status
2017-02-21 01:12:28.800 | ++ trueorfalse True USE_SCREEN
2017-02-21 01:12:28.802 | + USE_SCREEN=True
2017-02-21 01:12:28.802 | + screen -S stack -X screen -t g-reg
2017-02-21 01:12:28.803 | + local real_logfile=/opt/stack/logs/g-reg.log.2017-02-21-010519
2017-02-21 01:12:28.803 | + echo 'LOGDIR: /opt/stack/logs'
2017-02-21 01:12:28.803 | LOGDIR: /opt/stack/logs
2017-02-21 01:12:28.803 | + echo 'SCREEN_LOGDIR: '
2017-02-21 01:12:28.803 | SCREEN_LOGDIR:
2017-02-21 01:12:28.805 | + echo 'log: /opt/stack/logs/g-reg.log.2017-02-21-010519'
2017-02-21 01:12:28.805 | log: /opt/stack/logs/g-reg.log.2017-02-21-010519
2017-02-21 01:12:28.805 | + [[ -n /opt/stack/logs ]]
2017-02-21 01:12:28.805 | + screen -S stack -p g-reg -X logfile /opt/stack/logs/g-reg.log.2017-02-21-010519
2017-02-21 01:12:28.806 | + screen -S stack -p g-reg -X log on
2017-02-21 01:12:28.811 | + ln -sf /opt/stack/logs/g-reg.log.2017-02-21-010519 /opt/stack/logs/g-reg.log
2017-02-21 01:12:28.811 | + [[ -n '' ]]
2017-02-21 01:12:28.811 | + sleep 3
2017-02-21 01:12:31.810 | ++ echo -ne '\015'
2017-02-21 01:12:31.811 | + NL=$'\r'
2017-02-21 01:12:31.811 | + [[ -n '' ]]
2017-02-21 01:12:31.811 | + screen_rc g-reg '/usr/local/bin/glance-registry --config-file=/etc/glance/glance-registry.conf'
2017-02-21 01:12:31.811 | + SCREEN_NAME=stack
2017-02-21 01:12:31.811 | + SCREENRC=/home/stack/devstack/stack-screenrc
2017-02-21 01:12:31.811 | + [[ ! -e /home/stack/devstack/stack-screenrc ]]
2017-02-21 01:12:31.811 | + grep g-reg /home/stack/devstack/stack-screenrc
2017-02-21 01:12:31.812 | ++ echo -ne '\015'
2017-02-21 01:12:31.812 | + NL=$'\r'
2017-02-21 01:12:31.812 | + echo 'screen -t g-reg bash'
"'17-02-21 01:12:31.812 | + echo 'stuff "/usr/local/bin/glance-registry --config-file=/etc/glance/glance-registry.conf
2017-02-21 01:12:31.812 | + [[ -n /opt/stack/logs ]]
2017-02-21 01:12:31.812 | + echo 'logfile /opt/stack/logs/g-reg.log.2017-02-21-010519'
2017-02-21 01:12:31.812 | + echo 'log on'
2017-02-21 01:12:31.812 | + screen -S stack -p g-reg -X stuff '/usr/local/bin/glance-registry --config-file=/etc/glance/glance-registr'.conf & echo $! >/opt/stack/status/stack/g-reg.pid; fg || echo "g-reg failed to start" | tee "/opt/stack/status/stack/g-reg.failure"
2017-02-21 01:12:31.818 | + run_process g-api '/usr/local/bin/glance-api --config-file=/etc/glance/glance-api.conf'
2017-02-21 01:12:31.818 | + local service=g-api
2017-02-21 01:12:31.818 | + local 'command=/usr/local/bin/glance-api --config-file=/etc/glance/glance-api.conf'
2017-02-21 01:12:31.818 | + local group=
2017-02-21 01:12:31.818 | + is_service_enabled g-api
2017-02-21 01:12:31.820 | + return 0
2017-02-21 01:12:31.820 | + [[ True = \T\r\u\e ]]
2017-02-21 01:12:31.820 | + screen_process g-api '/usr/local/bin/glance-api --config-file=/etc/glance/glance-api.conf' ''
2017-02-21 01:12:31.820 | + local name=g-api
2017-02-21 01:12:31.820 | + local 'command=/usr/local/bin/glance-api --config-file=/etc/glance/glance-api.conf'
2017-02-21 01:12:31.820 | + local group=
2017-02-21 01:12:31.820 | + SCREEN_NAME=stack
2017-02-21 01:12:31.820 | + SERVICE_DIR=/opt/stack/status
2017-02-21 01:12:31.820 | ++ trueorfalse True USE_SCREEN
2017-02-21 01:12:31.823 | + USE_SCREEN=True
2017-02-21 01:12:31.823 | + screen -S stack -X screen -t g-api
2017-02-21 01:12:31.827 | + local real_logfile=/opt/stack/logs/g-api.log.2017-02-21-010519
2017-02-21 01:12:31.827 | + echo 'LOGDIR: /opt/stack/logs'
2017-02-21 01:12:31.827 | LOGDIR: /opt/stack/logs
2017-02-21 01:12:31.827 | + echo 'SCREEN_LOGDIR: '
2017-02-21 01:12:31.827 | SCREEN_LOGDIR:
2017-02-21 01:12:31.827 | + echo 'log: /opt/stack/logs/g-api.log.2017-02-21-010519'
2017-02-21 01:12:31.827 | log: /opt/stack/logs/g-api.log.2017-02-21-010519
2017-02-21 01:12:31.827 | + [[ -n /opt/stack/logs ]]
2017-02-21 01:12:31.827 | + screen -S stack -p g-api -X logfile /opt/stack/logs/g-api.log.2017-02-21-010519
2017-02-21 01:12:31.828 | + screen -S stack -p g-api -X log on
2017-02-21 01:12:31.829 | + ln -sf /opt/stack/logs/g-api.log.2017-02-21-010519 /opt/stack/logs/g-api.log
2017-02-21 01:12:31.831 | + [[ -n '' ]]
2017-02-21 01:12:31.831 | + sleep 3
2017-02-21 01:12:34.832 | ++ echo -ne '\015'
2017-02-21 01:12:34.832 | + NL=$'\r'
2017-02-21 01:12:34.832 | + [[ -n '' ]]
2017-02-21 01:12:34.832 | + screen_rc g-api '/usr/local/bin/glance-api --config-file=/etc/glance/glance-api.conf'
2017-02-21 01:12:34.832 | + SCREEN_NAME=stack
2017-02-21 01:12:34.832 | + SCREENRC=/home/stack/devstack/stack-screenrc
2017-02-21 01:12:34.832 | + [[ ! -e /home/stack/devstack/stack-screenrc ]]
2017-02-21 01:12:34.832 | + grep g-api /home/stack/devstack/stack-screenrc
2017-02-21 01:12:34.833 | ++ echo -ne '\015'
2017-02-21 01:12:34.833 | + NL=$'\r'
2017-02-21 01:12:34.833 | + echo 'screen -t g-api bash'
"'17-02-21 01:12:34.833 | + echo 'stuff "/usr/local/bin/glance-api --config-file=/etc/glance/glance-api.conf
2017-02-21 01:12:34.833 | + [[ -n /opt/stack/logs ]]
2017-02-21 01:12:34.833 | + echo 'logfile /opt/stack/logs/g-api.log.2017-02-21-010519'
2017-02-21 01:12:34.833 | + echo 'log on'
2017-02-21 01:12:34.833 | + screen -S stack -p g-api -X stuff '/usr/local/bin/glance-api --config-file=/etc/glance/glance-api.conf & e'ho $! >/opt/stack/status/stack/g-api.pid; fg || echo "g-api failed to start" | tee "/opt/stack/status/stack/g-api.failure"
2017-02-21 01:12:34.836 | + echo 'Waiting for g-api (10.0.32.60:9292) to start...'
2017-02-21 01:12:34.836 | Waiting for g-api (10.0.32.60:9292) to start...
2017-02-21 01:12:34.836 | + wait_for_service 60 http://10.0.32.60:9292
2017-02-21 01:12:34.836 | + local timeout=60
2017-02-21 01:12:34.836 | + local url=http://10.0.32.60:9292
2017-02-21 01:12:34.836 | + timeout 60 sh -c 'while ! curl -g -k --noproxy '\''*'\'' -s http://10.0.32.60:9292 >/dev/null; do sleep 1; done'
2017-02-21 01:12:34.963 | + is_service_enabled g-search
2017-02-21 01:12:34.970 | + return 1
2017-02-21 01:12:34.970 | + is_service_enabled g-reg
2017-02-21 01:12:34.971 | + return 0
2017-02-21 01:12:34.971 | ++ openstack token issue -c id -f value
2017-02-21 01:12:37.004 | + TOKEN=21a3e4b1fd9e40b6a0f0077d6b5f61a6
2017-02-21 01:12:37.004 | + die_if_not_set 1205 TOKEN 'Keystone fail to get token'
2017-02-21 01:12:37.004 | + local exitcode=0
2017-02-21 01:12:37.006 | + echo_summary 'Uploading images'
2017-02-21 01:12:37.006 | + [[ -t 3 ]]
2017-02-21 01:12:37.006 | + [[ True != \T\r\u\e ]]
2017-02-21 01:12:37.006 | + echo -e Uploading images
2017-02-21 01:12:37.006 | + [[ -n '' ]]
2017-02-21 01:12:37.006 | + for image_url in '${IMAGE_URLS//,/ }'
2017-02-21 01:12:37.006 | + upload_image http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-uec.tar.gz 21a3e4b1fd9e40b6a0f0077d6b5f61a6
2017-02-21 01:12:37.006 | + local image_url=http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-uec.tar.gz
2017-02-21 01:12:37.006 | + local token=21a3e4b1fd9e40b6a0f0077d6b5f61a6
2017-02-21 01:12:37.006 | + local image image_fname image_name
2017-02-21 01:12:37.006 | + mkdir -p /home/stack/devstack/files/images
2017-02-21 01:12:37.007 | ++ basename http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-uec.tar.gz
2017-02-21 01:12:37.008 | + image_fname=cirros-0.3.4-x86_64-uec.tar.gz
2017-02-21 01:12:37.008 | + [[ http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-uec.tar.gz != file* ]]
2017-02-21 01:12:37.008 | + [[ ! -f /home/stack/devstack/files/cirros-0.3.4-x86_64-uec.tar.gz ]]
2017-02-21 01:12:37.008 | ++ stat -c %s /home/stack/devstack/files/cirros-0.3.4-x86_64-uec.tar.gz
2017-02-21 01:12:37.009 | + [[ 8683894 = \0 ]]
2017-02-21 01:12:37.009 | + image=/home/stack/devstack/files/cirros-0.3.4-x86_64-uec.tar.gz
2017-02-21 01:12:37.009 | + [[ http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-uec.tar.gz =~ openvz ]]
2017-02-21 01:12:37.009 | + [[ http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-uec.tar.gz =~ .vmdk ]]
2017-02-21 01:12:37.009 | + [[ http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-uec.tar.gz =~ .vhd.tgz ]]
2017-02-21 01:12:37.009 | + [[ http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-uec.tar.gz =~ .xen-raw.tgz ]]
2017-02-21 01:12:37.009 | + local kernel=
2017-02-21 01:12:37.009 | + local ramdisk=
2017-02-21 01:12:37.009 | + local disk_format=
2017-02-21 01:12:37.009 | + local container_format=
2017-02-21 01:12:37.009 | + local unpack=
2017-02-21 01:12:37.009 | + local img_property=
2017-02-21 01:12:37.009 | + case "$image_fname" in
2017-02-21 01:12:37.009 | + '[' cirros-0.3.4-x86_64-uec '!=' cirros-0.3.4-x86_64-uec.tar.gz ']'
2017-02-21 01:12:37.009 | + image_name=cirros-0.3.4-x86_64-uec
2017-02-21 01:12:37.009 | + local xdir=/home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec
2017-02-21 01:12:37.009 | + rm -Rf /home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec
2017-02-21 01:12:37.013 | + mkdir /home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec
2017-02-21 01:12:37.013 | + tar -zxf /home/stack/devstack/files/cirros-0.3.4-x86_64-uec.tar.gz -C /home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec
2017-02-21 01:12:37.206 | ++ for f in '"$xdir/"-vmlinuz' '"$xdir/"aki-*/image'
2017-02-21 01:12:37.207 | ++ '[' -f /home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec/cirros-0.3.4-x86_64-vmlinuz ']'
2017-02-21 01:12:37.207 | ++ echo /home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec/cirros-0.3.4-x86_64-vmlinuz
2017-02-21 01:12:37.207 | ++ break
2017-02-21 01:12:37.207 | ++ true
2017-02-21 01:12:37.207 | + kernel=/home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec/cirros-0.3.4-x86_64-vmlinuz
2017-02-21 01:12:37.207 | ++ for f in '"$xdir/"-initrd' '"$xdir/"ari-*/image'
2017-02-21 01:12:37.207 | ++ '[' -f /home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec/cirros-0.3.4-x86_64-initrd ']'
2017-02-21 01:12:37.207 | ++ echo /home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec/cirros-0.3.4-x86_64-initrd
2017-02-21 01:12:37.207 | ++ break
2017-02-21 01:12:37.207 | ++ true
2017-02-21 01:12:37.208 | + ramdisk=/home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec/cirros-0.3.4-x86_64-initrd
2017-02-21 01:12:37.208 | ++ for f in '"$xdir/".img' '"$xdir/"ami-/image'
2017-02-21 01:12:37.208 | ++ '[' -f /home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec/cirros-0.3.4-x86_64-blank.img ']'
2017-02-21 01:12:37.208 | ++ echo /home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec/cirros-0.3.4-x86_64-blank.img
2017-02-21 01:12:37.208 | ++ break
2017-02-21 01:12:37.208 | ++ true
2017-02-21 01:12:37.208 | + image=/home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec/cirros-0.3.4-x86_64-blank.img
2017-02-21 01:12:37.208 | + [[ -z cirros-0.3.4-x86_64-uec ]]
2017-02-21 01:12:37.208 | + is_arch ppc64
2017-02-21 01:12:37.209 | ++ uname -m
2017-02-21 01:12:37.209 | + [[ x86_64 == \p\p\c\6\4 ]]
2017-02-21 01:12:37.209 | + is_arch aarch64
2017-02-21 01:12:37.209 | ++ uname -m
2017-02-21 01:12:37.210 | + [[ x86_64 == \a\a\r\c\h\6\4 ]]
2017-02-21 01:12:37.210 | + '[' '' = bare ']'
2017-02-21 01:12:37.210 | + local kernel_id= ramdisk_id=
2017-02-21 01:12:37.210 | + '[' -n /home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec/cirros-0.3.4-x86_64-vmlinuz ']'
2017-02-21 01:12:37.211 | ++ openstack --os-token 21a3e4b1fd9e40b6a0f0077d6b5f61a6 --os-url http://10.0.32.60:9292 image create cirros-0.3.4-x86_64-uec-kernel --public --container-format aki --disk-format aki
2017-02-21 01:12:37.211 | ++ get_field 2
2017-02-21 01:12:37.211 | ++ local data field
2017-02-21 01:12:37.211 | ++ read data
2017-02-21 01:12:37.215 | ++ grep ' id '
2017-02-21 01:12:39.564 | WARNING: urllib3.connectionpool Connection pool is full, discarding connection: 10.0.32.60
2017-02-21 01:12:39.625 | ++ '[' 2 -lt 0 ']'
2017-02-21 01:12:39.625 | ++ field='$3'
2017-02-21 01:12:39.625 | ++ echo '| id | b123f18c-cb82-4294-9f84-9a7154554447 |'
2017-02-21 01:12:39.625 | ++ awk '-F[ \t]\|[ \t]' '{print $3}'
2017-02-21 01:12:39.627 | ++ read data
2017-02-21 01:12:39.628 | + kernel_id=b123f18c-cb82-4294-9f84-9a7154554447
2017-02-21 01:12:39.628 | + '[' -n /home/stack/devstack/files/images/cirros-0.3.4-x86_64-uec/cirros-0.3.4-x86_64-initrd ']'
2017-02-21 01:12:39.628 | ++ openstack --os-token 21a3e4b1fd9e40b6a0f0077d6b5f61a6 --os-url http://10.0.32.60:9292 image create cirros-0.3.4-x86_64-uec-ramdisk --public --container-format ari --disk-format ari
2017-02-21 01:12:39.632 | ++ get_field 2
2017-02-21 01:12:39.636 | ++ local data field
2017-02-21 01:12:39.636 | ++ read data
2017-02-21 01:12:39.637 | ++ grep ' id '
2017-02-21 01:12:41.494 | WARNING: urllib3.connectionpool Connection pool is full, discarding connection: 10.0.32.60
2017-02-21 01:12:41.566 | ++ '[' 2 -lt 0 ']'
2017-02-21 01:12:41.566 | ++ field='$3'
2017-02-21 01:12:41.566 | ++ awk '-F[ \t]\|[ \t]' '{print $3}'
2017-02-21 01:12:41.567 | ++ echo '| id | a9ff3099-e30d-48f1-bd5d-50e6861862da |'
2017-02-21 01:12:41.568 | ++ read data
2017-02-21 01:12:41.568 | + ramdisk_id=a9ff3099-e30d-48f1-bd5d-50e6861862da
2017-02-21 01:12:41.568 | + openstack --os-token 21a3e4b1fd9e40b6a0f0077d6b5f61a6 --os-url http://10.0.32.60:9292 image create cirros-0.3.4-x86_64-uec --public --container-format ami --disk-format ami --property kernel_id=b123f18c-cb82-4294-9f84-9a7154554447 --property ramdisk_id=a9ff3099-e30d-48f1-bd5d-50e6861862da
2017-02-21 01:12:43.343 | ERROR: openstack 401 Unauthorized: This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. (HTTP 401)
2017-02-21 01:12:43.402 | + exit_trap
2017-02-21 01:12:43.402 | + local r=1
2017-02-21 01:12:43.403 | ++ jobs -p
2017-02-21 01:12:43.403 | + jobs=
2017-02-21 01:12:43.403 | + [[ -n '' ]]
2017-02-21 01:12:43.404 | + kill_spinner
2017-02-21 01:12:43.404 | + '[' '!' -z '' ']'
2017-02-21 01:12:43.404 | + [[ 1 -ne 0 ]]
2017-02-21 01:12:43.404 | + echo 'Error on exit'
2017-02-21 01:12:43.404 | Error on exit
2017-02-21 01:12:43.404 | + generate-subunit 1487639119 444 fail
2017-02-21 01:12:43.822 | + [[ -z /opt/stack/logs ]]
2017-02-21 01:12:43.822 | + /home/stack/devstack/tools/worlddump.py -d /opt/stack/logs
2017-02-21 01:12:43.892 | + exit 1
stack@mickael-HP-ProDesk-600-G1-TWR:~/devstack$

[teiszler]

Micael,

If it is possible, it may be best to start with a clean install of 14.04. DevStack makes many modifications to the system and it though clean.sh should revert things, it may not be cleaning everything up.

One thing that did give me pause, is that you are executing stack.sh as the user 'stack'. Is this user in the /etc/sudoers file? If not, it might explain why some of the steps in the process are not working properly.

[MSoares]

Tom,

I had add a new user stack with privleges before starting elijah devstack installation, after that I have login stack and start installation.

As you say the better is make a clean install of Ubuntu 14.04. Tomorrow, I will start early and give you some feedback.

Thanks for your help