Cloudlet does not create overlay

[dewanta87]

Hi Thom,

Thanks for providing new stackrc file. I just installed cloudlet & openstack successfully. I could login to horizon dashboard. Then, came another issues, I couldn't import base VM and also create image from the horizon dashboard. The got stuck on never ending queue (I waited for 1 - 2 hours hoping it will be successful, haha..). But when I tried cloudlet command tool, I can import base VM as given in this link https://github.com/cmusatyalab/elijah-provisioning

Anyway, I found something suspicious, when I checked g-api.log

2017-02-24 09:33:37.370 DEBUG glance.common.client [req-24922081-f425-455e-b751-03c8b781b28c 9502c8fdb4bf458296bf37937b8a775d 2bd257f8e8334fec9c3682ac03d74642] Constructed URL: http://0.0.0.0:9191/images/ff8a16a8-9e65-4cb7-9fac-97be9f89a8ce from (pid=12540) constructurl /opt/stack/glance/glance/common/client.py:401

I am not sure whether this issue is caused by failure in parsing ip/url to upload / create image or any other issues.
I will update if I found any other progress.

[teiszler]

Favian,

Did you try to import the base VM from this link https://storage.cmusatyalab.org/cloudlet-vm/precise-hotplug.zip? Under Admin( or Compute) -> Images, what is the status of these images?

And when you say you cannot create an image was this from the Instances pane in the horizon dashboard? You could also try creating an image from the command-line with glance create-image.

[teiszler]

I just tried it on the DevStack instance I setup yesterday and mine are all stuck in the Queued state. This is reminiscent of what Micael had been seeing also. My plan is to reinstitute the changes I made changing all the repository branches to tags/kilo-eol and seeing if this fixes the problems with Glance. I will try that this morning and let you know. If it works out, I will again update the stackrc on elijah-openstack so you can try it too.

Thanks,
Tom

[MSoares]

Hi guys,

sorry but today It will not possible to test it, but tomorrow I will try it and give you some feedback.

[dewanta87]

Tom,

I download precise-hotplug.zip from that site.

Anyway, I used glance create-image and the result was same. It got stuck on never-ending queue.

[MSoares]

Dear Tom,

I have successfully install devstack and openstack++.
I open dashboard and I am starting uploading Base VM from https://storage.cmusatyalab.org/cloudlet-vm/precise-hotplug.zip in order to see if I will be stuck also.

[MSoares]

It is still in queued state.

[teiszler]

@MSoares, @dewanta87
As I suspected. It is frustrating. I am unsure what has caused this as it used to work. My goal today is to get the Ansible solution to a point where I can release it is a branch of the elijah-openstack repository and then you and Favian can try it out. I need to work on the directions a bit to explain the parts of the script that might need modified and go through it step by step. I will let you know when this is up.

Thanks,
Tom

[teiszler]

Micael/Favian,

I just created an ansible branch in elijah-openstack that has new instructions on how to use the Ansible script I created in order to install everything. I have tested this on a single node with a single NIC, though the script is flexible enough to install to a multi-node cluster with several compute nodes and/or machines that have two NICs. The README.md has step by step instructions on how to install ansible, configure the variables, and then run the script. If you run into trouble let me know, but hopefully this will make things easier. This ansible script will install OpenStack Kilo, the cloudlet library from elijah-provisioning (though you have to make sure you setup the local user and password in roles/openstack-compute/vars/main.yml so that the fabric script in elijah-provisioning can run otherwise that step will be stuck), and also the OpenStack++ extensions from elijah-openstack.

Make sure that when you clone elijah-openstack you include '-b ansible' since it is still in a branch and not under master yet. Let me know if you have questions.

https://github.com/cmusatyalab/elijah-openstack/tree/ansible

Thanks,
Tom

[MSoares]

Dear Tom,

I have try your repo and I get the next error:

PLAY [controller, compute] *****************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [openstack-common : (OS-COMMON) modify grub config to show text on boot] **
changed: [localhost]

TASK [openstack-common : (OS-COMMON) install Ubuntu-Cloud-Keyring] *************
ok: [localhost]

TASK [openstack-common : (OS-COMMON) add Kilo repo] ****************************
ok: [localhost]

TASK [openstack-common : (OS-COMMON) update cache and perform upgrade] *********
ok: [localhost]

TASK [openstack-common : (OS-COMMON) install python openstack client] **********
ok: [localhost] => (item=[u'python-openstackclient'])

PLAY [controller] **************************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [openstack-controller : (CONTROLLER) install mariadb and python lib] ******
ok: [localhost] => (item=[u'mariadb-server', u'python-mysqldb', u'python-pip'])

TASK [openstack-controller : pip] **********************************************
ok: [localhost]

TASK [openstack-controller : (CONTROLLER) run mysql installation] **************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "No remaining responses for 'Enter current password for root', output was ' (enter for none): \r\nERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)\r\nEnter current password for root'"}
to retry, use: --limit @/root/elijah-openstack/ansible/openstack-kilo.retry

PLAY RECAP *********************************************************************
localhost : ok=9 changed=1 unreachable=0 failed=1

~/elijah-openstack/ansible/roles/openstack-compute/vars/main.yml
rabbit_pw: mickael
mysql_pw: mickael
db_pw: mickael
keystone_admin_pw: mickael
admin_token: mickael
metadata_secret: mickael

openstack_controller_node: localhost

controller_ip: 127.0.0.1

openstack_controller_node: 10.0.32.60

controller_ip: 10.0.32.60

pub_iface: eth0
flat_iface: eth10

required to run the fabric script from https://github.com/cmusatyalab/elijah-provisioning

local_user: root
local_user_pw:

~/elijah-openstack/ansible/roles/openstack-controller/vars/main.yml

rabbit_pw: mickael
mysql_pw: mickael
db_pw: mickael
keystone_admin_pw: mickael
admin_token: mickael
metadata_secret: mickael

openstack_controller_node: localhost

controller_ip: 127.0.0.1

openstack_controller_node: 10.0.32.60
controller_ip: 10.0.32.60
pub_iface: eth0
flat_iface: eth10

What is your opinion?

[teiszler]

Micael,

What is the username you log into the machine with? This is the username and password that should be specified as the local_user and local_user_pw in openstack-compute/vars/main.yml. This will be used later in the script to install elijah-provisioning.

There are a few things you could try to troubleshoot the problem with the mysql installation. First, you could run it manually (the command executed in the tasks is 'mysql_secure_installation') and see if you can enter the correct password at the prompt. The ansible script will try both an empty password and the value of mysql_db var for this prompt. It appears neither worked and so the script stopped.

If entering the password manually gets you to the next step in the installation script, you can reference the task for the rest of the answers:

- name: (CONTROLLER) run mysql installation
  expect:
    command: mysql_secure_installation
    echo: yes
    responses:
      Enter current password for root: ["\n", "{{ mysql_pw }}"]
      Change the root password: "n"
      Set root password: "Y"
      New password: "{{ mysql_pw }}"
      Re-enter new password: "{{ mysql_pw }}"
      Remove anonymous users: "Y"
      Disallow root login remotely: "Y"
      Remove test database and access to it: "Y"
Reload privilege tables now: "Y"

If you figure out what the correct response is to the password prompt, you can ensure that it is one of the responses to the 'Enter current password for root' prompt in the ansible script. Also make sure that the prompt is the same, since ansible will do regex matching to find the responses to the prompts.

If you can't get this to automate via ansible, but you can successfully finish mysql_secure_installation manually, then you can just comment out this part of the ansible script by putting # in front of each line of that task and then rerun the ansible script from the top.

[MSoares]

Dear Tom,

I had to remove all mysql directory and files and reinstall it and after that the problem was solve. Then I tried to relaunch again the ansible install and it goes running until:

root@mickael-HP-ProDesk-600-G1-TWR:~/elijah-openstack/ansible# ansible-playbook -i ./hosts openstack-kilo.yaml --limit @/root/elijah-openstack/ansible/openstack-kilo.retry

PLAY [controller, compute] *****************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [openstack-common : (OS-COMMON) modify grub config to show text on boot] **
changed: [localhost]

TASK [openstack-common : (OS-COMMON) install Ubuntu-Cloud-Keyring] *************
ok: [localhost]

TASK [openstack-common : (OS-COMMON) add Kilo repo] ****************************
ok: [localhost]

TASK [openstack-common : (OS-COMMON) update cache and perform upgrade] *********
ok: [localhost]

TASK [openstack-common : (OS-COMMON) install python openstack client] **********
ok: [localhost] => (item=[u'python-openstackclient'])

PLAY [controller] **************************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [openstack-controller : (CONTROLLER) install mariadb and python lib] ******
changed: [localhost] => (item=[u'mariadb-server', u'python-mysqldb', u'python-pip'])

TASK [openstack-controller : pip] **********************************************
ok: [localhost]

TASK [openstack-controller : (CONTROLLER) run mysql installation] **************
changed: [localhost]

TASK [openstack-controller : (CONTROLLER) create mysql conf] *******************
changed: [localhost]

RUNNING HANDLER [openstack-controller : restart mysql] *************************
changed: [localhost]

TASK [openstack-controller : (CONTROLLER) install RabbitMQ] ********************
ok: [localhost]

TASK [openstack-controller : (CONTROLLER) check if openstack user already exists] ***
changed: [localhost]

TASK [openstack-controller : (CONTROLLER) create openstack rabbitmq user] ******
changed: [localhost]

TASK [openstack-controller : (CONTROLLER) set user perms] **********************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) transfer keystonedb SQL script] ********
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) check if keystone db already exists] ***
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) create keystone db] ********************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) disable automatic startup of keystone] *
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) install keystone...] *******************
changed: [localhost] => (item=[u'keystone'])

TASK [openstack-controller : (KEYSTONE) install apache/memcached] *************
changed: [localhost] => (item=[u'apache2', u'libapache2-mod-wsgi', u'memcached', u'python-memcache'])

TASK [openstack-controller : (KEYSTONE) replace keystone conf] *****************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) populate identity service database] ****
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) replace servername in apache conf] *****
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) create wsgi-keystone.conf] *************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) enable identity service virtual hosts] *
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) create wsgi dir] ***********************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) copy keystone wsgi components] *********
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) copy keystone wsgi components] *********
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) stop keystone service if running (it will restart with apache2)] ***
ok: [localhost]

TASK [openstack-controller : (KEYSTONE) remove SQLite] *************************
ok: [localhost]

RUNNING HANDLER [openstack-controller : restart apache] ************************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) wait 5s for keystone to come back up] **
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) create identity service] ***************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) create identity endpoint] **************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) create admin project] ******************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) create service project] ****************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) create admin user] *********************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) create admin role] *********************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) create user role] **********************
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) add admin role to admin user] **********
changed: [localhost]

TASK [openstack-controller : (KEYSTONE) create OpenStack client script] ********
changed: [localhost]

TASK [openstack-controller : (GLANCE) transfer glancedb SQL script] ************
changed: [localhost]

TASK [openstack-controller : (GLANCE) check if glance db already exists] *******
changed: [localhost]

TASK [openstack-controller : (GLANCE) create glance db] ************************
changed: [localhost]

TASK [openstack-controller : (GLANCE) check if glance user already exists] *****
changed: [localhost]

TASK [openstack-controller : (GLANCE) create glance user] **********************
changed: [localhost]

TASK [openstack-controller : (GLANCE) check if glance user is admin in service project] ***
changed: [localhost]

TASK [openstack-controller : (GLANCE) add glance user to service project] ******
changed: [localhost]

TASK [openstack-controller : (GLANCE) check if glance service exists] **********
changed: [localhost]

TASK [openstack-controller : (GLANCE) create glance service entity] ************
changed: [localhost]

TASK [openstack-controller : (GLANCE) check if glance endpoint exists] *********
changed: [localhost]

TASK [openstack-controller : (GLANCE) create api endpoint] *********************
changed: [localhost]

TASK [openstack-controller : (GLANCE) install glance and associated packages] **
changed: [localhost] => (item=[u'glance', u'python-glanceclient'])

TASK [openstack-controller : (GLANCE) replace glance-api.conf] *****************
changed: [localhost]

TASK [openstack-controller : (GLANCE) replace glance-registry.conf] ************
changed: [localhost]

TASK [openstack-controller : (GLANCE) populate image service database] *********
changed: [localhost]

RUNNING HANDLER [openstack-controller : restart glance] ************************
changed: [localhost] => (item=glance-registry)
changed: [localhost] => (item=glance-api)

TASK [openstack-controller : (GLANCE) remove SQLite] ***************************
ok: [localhost]

TASK [openstack-controller : (NOVA) transfer novadb SQL script] ****************
changed: [localhost]

TASK [openstack-controller : (NOVA) check if nova db already exists] ***********
changed: [localhost]

TASK [openstack-controller : (NOVA) create nova db] ****************************
changed: [localhost]

TASK [openstack-controller : (NOVA) check if nova user already exists] *********
changed: [localhost]

TASK [openstack-controller : (NOVA) create nova user] **************************
changed: [localhost]

TASK [openstack-controller : (NOVA) check if nova user is admin in service project] ***
changed: [localhost]

TASK [openstack-controller : (NOVA) add nova user to service project] **********
changed: [localhost]

TASK [openstack-controller : (NOVA) check if nova service exists] **************
changed: [localhost]

TASK [openstack-controller : (NOVA) create nova service entity] ****************
changed: [localhost]

TASK [openstack-controller : (NOVA) check if nova endpoint exists] *************
changed: [localhost]

TASK [openstack-controller : (NOVA) create nova api endpoint] ******************
changed: [localhost]

TASK [openstack-controller : (NOVA) install nova-api and associated packages] **
changed: [localhost] => (item=[u'nova-api', u'nova-cert', u'nova-conductor', u'nova-consoleauth', u'nova-novncproxy', u'nova-scheduler', u'python-novaclient'])

TASK [openstack-controller : (NOVA) install nova-network] **********************
changed: [localhost] => (item=[u'nova-network'])

TASK [openstack-controller : (NOVA) replace nova.conf] *************************
changed: [localhost]

TASK [openstack-controller : (NOVA) replace api-paste.ini] *********************
changed: [localhost]

TASK [openstack-controller : (NOVA) replace rootwrap.conf] *********************
changed: [localhost]

TASK [openstack-controller : (NOVA) replace policy.json] ***********************
changed: [localhost]

TASK [openstack-controller : (NOVA) populate nova service database] ************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "/bin/sh -c 'nova-manage db sync' nova", "delta": "0:00:43.436821", "end": "2017-03-07 23:01:57.914250", "failed": true, "rc": 1, "start": "2017-03-07 23:01:14.477429", "stderr": "No handlers could be found for logger \"oslo_config.cfg\"\n2017-03-07 23:01:56.725 19712 CRITICAL nova [-] DBDuplicateEntry: (IntegrityError) (1062, \"Duplicate entry 'mickael-HP-ProDesk-600-G1-TWR-conductor-0' for key 'uniq_services0host0topic0deleted'\") 'ALTER TABLE services ADD CONSTRAINT uniq_services0host0topic0deleted UNIQUE (host, topic, deleted)' ()\n2017-03-07 23:01:56.725 19712 TRACE nova Traceback (most recent call last):\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/bin/nova-manage\", line 10, in \n2017-03-07 23:01:56.725 19712 TRACE nova sys.exit(main())\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/opt/stack/nova/nova/cmd/manage.py\", line 1382, in main\n2017-03-07 23:01:56.725 19712 TRACE nova ret = fn(fn_args, **fn_kwargs)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/opt/stack/nova/nova/cmd/manage.py\", line 912, in sync\n2017-03-07 23:01:56.725 19712 TRACE nova return migration.db_sync(version)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/opt/stack/nova/nova/db/migration.py\", line 26, in db_sync\n2017-03-07 23:01:56.725 19712 TRACE nova return IMPL.db_sync(version=version, database=database)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/opt/stack/nova/nova/db/sqlalchemy/migration.py\", line 57, in db_sync\n2017-03-07 23:01:56.725 19712 TRACE nova version)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/migrate/versioning/api.py\", line 186, in upgrade\n2017-03-07 23:01:56.725 19712 TRACE nova return migrate(url, repository, version, upgrade=True, err=err, **opts)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"\", line 2, in _migrate\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py\", line 160, in withengine\n2017-03-07 23:01:56.725 19712 TRACE nova return f(a, *kw)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/migrate/versioning/api.py\", line 366, in migrate\n2017-03-07 23:01:56.725 19712 TRACE nova schema.runchange(ver, change, changeset.step)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/migrate/versioning/schema.py\", line 93, in runchange\n2017-03-07 23:01:56.725 19712 TRACE nova change.run(self.engine, step)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/migrate/versioning/script/py.py\", line 148, in run\n2017-03-07 23:01:56.725 19712 TRACE nova scriptfunc(engine)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/opt/stack/nova/nova/db/sqlalchemy/migrate_repo/versions/216_havana.py\", line 1173, in upgrade\n2017-03-07 23:01:56.725 19712 TRACE nova name='uniq_services0host0topic0deleted').create()\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/migrate/changeset/constraint.py\", line 44, in create\n2017-03-07 23:01:56.725 19712 TRACE nova self._doimports('constraintgenerator', *a, **kw)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/migrate/changeset/constraint.py\", line 32, in _doimports\n2017-03-07 23:01:56.725 19712 TRACE nova run_single_visitor(engine, visitorcallable, self, *a, **kw)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/migrate/changeset/databases/visitor.py\", line 85, in run_single_visitor\n2017-03-07 23:01:56.725 19712 TRACE nova fn(element, **kwargs)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/migrate/changeset/ansisql.py\", line 294, in visit_migrate_unique_constraint\n2017-03-07 23:01:56.725 19712 TRACE nova self.visitconstraint(p, **k)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/migrate/changeset/ansisql.py\", line 300, in visitconstraint\n2017-03-07 23:01:56.725 19712 TRACE nova self.execute()\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/migrate/changeset/ansisql.py\", line 44, in execute\n2017-03-07 23:01:56.725 19712 TRACE nova return self.connection.execute(self.buffer.getvalue())\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 721, in execute\n2017-03-07 23:01:56.725 19712 TRACE nova return self.executetext(object, multiparams, params)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 870, in executetext\n2017-03-07 23:01:56.725 19712 TRACE nova statement, parameters\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 958, in executecontext\n2017-03-07 23:01:56.725 19712 TRACE nova context)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/compat/handle_error.py\", line 261, in handledbapi_exception\n2017-03-07 23:01:56.725 19712 TRACE nova e, statement, parameters, cursor, context)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 1156, in handledbapi_exception\n2017-03-07 23:01:56.725 19712 TRACE nova util.raise_from_cause(newraise, exc_info)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py\", line 199, in raise_from_cause\n2017-03-07 23:01:56.725 19712 TRACE nova reraise(type(exception), exception, tb=exc_tb)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 951, in executecontext\n2017-03-07 23:01:56.725 19712 TRACE nova context)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py\", line 436, in do_execute\n2017-03-07 23:01:56.725 19712 TRACE nova cursor.execute(statement, parameters)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py\", line 174, in execute\n2017-03-07 23:01:56.725 19712 TRACE nova self.errorhandler(self, exc, value)\n2017-03-07 23:01:56.725 19712 TRACE nova File \"/usr/lib/python2.7/dist-packages/MySQLdb/connections.py\", line 36, in defaulterrorhandler\n2017-03-07 23:01:56.725 19712 TRACE nova raise errorclass, errorvalue\n2017-03-07 23:01:56.725 19712 TRACE nova DBDuplicateEntry: (IntegrityError) (1062, \"Duplicate entry 'mickael-HP-ProDesk-600-G1-TWR-conductor-0' for key 'uniq_services0host0topic0deleted'\") 'ALTER TABLE services ADD CONSTRAINT uniq_services0host0topic0deleted UNIQUE (host, topic, deleted)' ()\n2017-03-07 23:01:56.725 19712 TRACE nova ", "stdout": "Command failed, please check log for more info", "stdout_lines": ["Command failed, please check log for more info"], "warnings": []}
to retry, use: --limit @/root/elijah-openstack/ansible/openstack-kilo.retry

PLAY RECAP *********************************************************************
localhost : ok=77 changed=66 unreachable=0 failed=1

Do you have some clue?

[teiszler]

Though I haven't run into this issue myself, it seems that the problem lies when updating the nova database. What I would try is to log in manually via mysql --user root --password=<your password> and then drop the nova database with DROP DATABASE IF EXISTS nova; One nice thing about Ansible and this solution is that you can run any section that is tagged and since I have tagged each component of script, you can just add a --tags nova to your ansible-playbook command and it will only run the tasks associated with the nova component and will not redo keystone, glance,etc. If nova successfully completes, then you can run the rest of the tasks by specifying their tags (--tags heat,horizon,openstack-ext,compute) when you launch ansible-playbook.

[MSoares]

--tags heat,horizon,openstack-ext,compute

@teiszler,

I run your command and it solve nova problem, I have install after that heat, but it give new error on dashboard, as you can see below.

TASK [openstack-controller : (HORIZON) install dashboard] **********************
failed: [localhost] (item=[u'openstack-dashboard']) => {"cache_update_time": 1488959603, "cache_updated": false, "failed": true, "item": ["openstack-dashboard"], "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\" install 'openstack-dashboard' -t 'trusty-updates/kilo'' failed: dpkg: error processing package openstack-dashboard (--configure):\n subprocess installed post-installation script returned error exit status 1\ndpkg: dependency problems prevent configuration of openstack-dashboard-ubuntu-theme:\n openstack-dashboard-ubuntu-theme depends on openstack-dashboard (= 1:2015.1.4-0ubuntu2); however:\n Package openstack-dashboard is not configured yet.\n\ndpkg: error processing package openstack-dashboard-ubuntu-theme (--configure):\n dependency problems - leaving unconfigured\nErrors were encountered while processing:\n openstack-dashboard\n openstack-dashboard-ubuntu-theme\nE: Sub-process /usr/bin/dpkg returned an error code (1)\n", "stderr": "dpkg: error processing package openstack-dashboard (--configure):\n subprocess installed post-installation script returned error exit status 1\ndpkg: dependency problems prevent configuration of openstack-dashboard-ubuntu-theme:\n openstack-dashboard-ubuntu-theme depends on openstack-dashboard (= 1:2015.1.4-0ubuntu2); however:\n Package openstack-dashboard is not configured yet.\n\ndpkg: error processing package openstack-dashboard-ubuntu-theme (--configure):\n dependency problems - leaving unconfigured\nErrors were encountered while processing:\n openstack-dashboard\n openstack-dashboard-ubuntu-theme\nE: Sub-process /usr/bin/dpkg returned an error code (1)\n", "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following extra packages will be installed:\n openstack-dashboard-ubuntu-theme python-appconf python-compressor\n python-django python-django-common python-django-horizon\n python-django-pyscss python-openstack-auth python-pint python-pyscss\nSuggested packages:\n python-flup python-bcrypt geoip-database-extra geoip-database-contrib\n python-django-doc ipython bpython libgdal1\nThe following NEW packages will be installed:\n openstack-dashboard openstack-dashboard-ubuntu-theme python-appconf\n python-compressor python-django python-django-common python-django-horizon\n python-django-pyscss python-openstack-auth python-pint python-pyscss\n0 upgraded, 11 newly installed, 0 to remove and 4 not upgraded.\nNeed to get 6098 kB of archives.\nAfter this operation, 75.1 MB of additional disk space will be used.\nGet:1 http://pt.archive.ubuntu.com/ubuntu/ trusty/main python-appconf all 0.5-2build1 [7696 B]\nGet:2 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-django-common all 1.7.6-1ubuntu2.3~cloud0 [1474 kB]\nGet:3 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-django all 1.7.6-1ubuntu2.3~cloud0 [974 kB]\nGet:4 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-compressor all 1.4-2ubuntu3~cloud0 [65.7 kB]\nGet:5 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-pyscss amd64 1.2.1-0ubuntu2~cloud1 [83.5 kB]\nGet:6 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-django-pyscss all 1.0.3-3ubuntu1~cloud0 [8722 B]\nGet:7 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-openstack-auth all 1.1.9-0ubuntu0.15.04.1~cloud0 [51.1 kB]\nGet:8 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-pint all 0.6-0ubuntu1~cloud0 [67.7 kB]\nGet:9 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-django-horizon all 1:2015.1.4-0ubuntu2 [1754 kB]\nGet:10 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main openstack-dashboard all 1:2015.1.4-0ubuntu2 [1541 kB]\nGet:11 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main openstack-dashboard-ubuntu-theme all 1:2015.1.4-0ubuntu2 [71.0 kB]\nFetched 6098 kB in 11s (516 kB/s)\nSelecting previously unselected package python-django-common.\n(Reading database ... 236794 files and directories currently installed.)\nPreparing to unpack .../python-django-common_1.7.6-1ubuntu2.3~cloud0_all.deb ...\nUnpacking python-django-common (1.7.6-1ubuntu2.3~cloud0) ...\nSelecting previously unselected package python-django.\nPreparing to unpack .../python-django_1.7.6-1ubuntu2.3~cloud0_all.deb ...\nUnpacking python-django (1.7.6-1ubuntu2.3~cloud0) ...\nSelecting previously unselected package python-appconf.\nPreparing to unpack .../python-appconf_0.5-2build1_all.deb ...\nUnpacking python-appconf (0.5-2build1) ...\nSelecting previously unselected package python-compressor.\nPreparing to unpack .../python-compressor_1.4-2ubuntu3~cloud0_all.deb ...\nUnpacking python-compressor (1.4-2ubuntu3~cloud0) ...\nSelecting previously unselected package python-pyscss.\nPreparing to unpack .../python-pyscss_1.2.1-0ubuntu2~cloud1_amd64.deb ...\nUnpacking python-pyscss (1.2.1-0ubuntu2~cloud1) ...\nSelecting previously unselected package python-django-pyscss.\nPreparing to unpack .../python-django-pyscss_1.0.3-3ubuntu1~cloud0_all.deb ...\nUnpacking python-django-pyscss (1.0.3-3ubuntu1~cloud0) ...\nSelecting previously unselected package python-openstack-auth.\nPreparing to unpack .../python-openstack-auth_1.1.9-0ubuntu0.15.04.1~cloud0_all.deb ...\nUnpacking python-openstack-auth (1.1.9-0ubuntu0.15.04.1~cloud0) ...\nSelecting previously unselected package python-pint.\nPreparing to unpack .../python-pint_0.6-0ubuntu1~cloud0_all.deb ...\nUnpacking python-pint (0.6-0ubuntu1~cloud0) ...\nSelecting previously unselected package python-django-horizon.\nPreparing to unpack .../python-django-horizon_1%3a2015.1.4-0ubuntu2_all.deb ...\nUnpacking python-django-horizon (1:2015.1.4-0ubuntu2) ...\nSelecting previously unselected package openstack-dashboard.\nPreparing to unpack .../openstack-dashboard_1%3a2015.1.4-0ubuntu2_all.deb ...\nUnpacking openstack-dashboard (1:2015.1.4-0ubuntu2) ...\nSelecting previously unselected package openstack-dashboard-ubuntu-theme.\nPreparing to unpack .../openstack-dashboard-ubuntu-theme_1%3a2015.1.4-0ubuntu2_all.deb ...\nUnpacking openstack-dashboard-ubuntu-theme (1:2015.1.4-0ubuntu2) ...\nProcessing triggers for man-db (2.6.7.1-1ubuntu1) ...\nSetting up python-django-common (1.7.6-1ubuntu2.3~cloud0) ...\nSetting up python-django (1.7.6-1ubuntu2.3~cloud0) ...\nSetting up python-appconf (0.5-2build1) ...\nSetting up python-compressor (1.4-2ubuntu3~cloud0) ...\nSetting up python-pyscss (1.2.1-0ubuntu2~cloud1) ...\nupdate-alternatives: using /usr/bin/python2-pyscss to provide /usr/bin/pyscss (pyscss) in auto mode\nSetting up python-django-pyscss (1.0.3-3ubuntu1~cloud0) ...\nSetting up python-openstack-auth (1.1.9-0ubuntu0.15.04.1~cloud0) ...\nSetting up python-pint (0.6-0ubuntu1~cloud0) ...\nSetting up python-django-horizon (1:2015.1.4-0ubuntu2) ...\nSetting up openstack-dashboard (1:2015.1.4-0ubuntu2) ...\nAdding system user horizon' (UID 126) ...\nAdding new userhorizon' (UID 126) with group horizon' ...\nNot creating home directory/usr/share/openstack-dashboard/'.\nCollecting and compressing static assets...\nTraceback (most recent call last):\n File \"manage.py\", line 25, in \n execute_from_command_line(sys.argv)\n File \"/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py\", line 385, in execute_from_command_line\n utility.execute()\n File \"/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py\", line 345, in execute\n settings.INSTALLED_APPS\n File \"/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py\", line 46, in getattr\n self.setup(name)\n File \"/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py\", line 42, in _setup\n self.wrapped = Settings(settings_module)\n File \"/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py\", line 98, in init\n % (self.SETTINGS_MODULE, e)\nImportError: Could not import settings 'openstack_dashboard.settings' (Is it on sys.path? Is there an import error in the settings file?): No module named xstatic.main\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following extra packages will be installed:", " openstack-dashboard-ubuntu-theme python-appconf python-compressor", " python-django python-django-common python-django-horizon", " python-django-pyscss python-openstack-auth python-pint python-pyscss", "Suggested packages:", " python-flup python-bcrypt geoip-database-extra geoip-database-contrib", " python-django-doc ipython bpython libgdal1", "The following NEW packages will be installed:", " openstack-dashboard openstack-dashboard-ubuntu-theme python-appconf", " python-compressor python-django python-django-common python-django-horizon", " python-django-pyscss python-openstack-auth python-pint python-pyscss", "0 upgraded, 11 newly installed, 0 to remove and 4 not upgraded.", "Need to get 6098 kB of archives.", "After this operation, 75.1 MB of additional disk space will be used.", "Get:1 http://pt.archive.ubuntu.com/ubuntu/ trusty/main python-appconf all 0.5-2build1 [7696 B]", "Get:2 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-django-common all 1.7.6-1ubuntu2.3~cloud0 [1474 kB]", "Get:3 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-django all 1.7.6-1ubuntu2.3~cloud0 [974 kB]", "Get:4 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-compressor all 1.4-2ubuntu3~cloud0 [65.7 kB]", "Get:5 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-pyscss amd64 1.2.1-0ubuntu2~cloud1 [83.5 kB]", "Get:6 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-django-pyscss all 1.0.3-3ubuntu1~cloud0 [8722 B]", "Get:7 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-openstack-auth all 1.1.9-0ubuntu0.15.04.1~cloud0 [51.1 kB]", "Get:8 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-pint all 0.6-0ubuntu1~cloud0 [67.7 kB]", "Get:9 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main python-django-horizon all 1:2015.1.4-0ubuntu2 [1754 kB]", "Get:10 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main openstack-dashboard all 1:2015.1.4-0ubuntu2 [1541 kB]", "Get:11 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main openstack-dashboard-ubuntu-theme all 1:2015.1.4-0ubuntu2 [71.0 kB]", "Fetched 6098 kB in 11s (516 kB/s)", "Selecting previously unselected package python-django-common.", "(Reading database ... 236794 files and directories currently installed.)", "Preparing to unpack .../python-django-common_1.7.6-1ubuntu2.3~cloud0_all.deb ...", "Unpacking python-django-common (1.7.6-1ubuntu2.3~cloud0) ...", "Selecting previously unselected package python-django.", "Preparing to unpack .../python-django_1.7.6-1ubuntu2.3~cloud0_all.deb ...", "Unpacking python-django (1.7.6-1ubuntu2.3~cloud0) ...", "Selecting previously unselected package python-appconf.", "Preparing to unpack .../python-appconf_0.5-2build1_all.deb ...", "Unpacking python-appconf (0.5-2build1) ...", "Selecting previously unselected package python-compressor.", "Preparing to unpack .../python-compressor_1.4-2ubuntu3~cloud0_all.deb ...", "Unpacking python-compressor (1.4-2ubuntu3~cloud0) ...", "Selecting previously unselected package python-pyscss.", "Preparing to unpack .../python-pyscss_1.2.1-0ubuntu2~cloud1_amd64.deb ...", "Unpacking python-pyscss (1.2.1-0ubuntu2~cloud1) ...", "Selecting previously unselected package python-django-pyscss.", "Preparing to unpack .../python-django-pyscss_1.0.3-3ubuntu1~cloud0_all.deb ...", "Unpacking python-django-pyscss (1.0.3-3ubuntu1~cloud0) ...", "Selecting previously unselected package python-openstack-auth.", "Preparing to unpack .../python-openstack-auth_1.1.9-0ubuntu0.15.04.1~cloud0_all.deb ...", "Unpacking python-openstack-auth (1.1.9-0ubuntu0.15.04.1~cloud0) ...", "Selecting previously unselected package python-pint.", "Preparing to unpack .../python-pint_0.6-0ubuntu1~cloud0_all.deb ...", "Unpacking python-pint (0.6-0ubuntu1~cloud0) ...", "Selecting previously unselected package python-django-horizon.", "Preparing to unpack .../python-django-horizon_1%3a2015.1.4-0ubuntu2_all.deb ...", "Unpacking python-django-horizon (1:2015.1.4-0ubuntu2) ...", "Selecting previously unselected package openstack-dashboard.", "Preparing to unpack .../openstack-dashboard_1%3a2015.1.4-0ubuntu2_all.deb ...", "Unpacking openstack-dashboard (1:2015.1.4-0ubuntu2) ...", "Selecting previously unselected package openstack-dashboard-ubuntu-theme.", "Preparing to unpack .../openstack-dashboard-ubuntu-theme_1%3a2015.1.4-0ubuntu2_all.deb ...", "Unpacking openstack-dashboard-ubuntu-theme (1:2015.1.4-0ubuntu2) ...", "Processing triggers for man-db (2.6.7.1-1ubuntu1) ...", "Setting up python-django-common (1.7.6-1ubuntu2.3~cloud0) ...", "Setting up python-django (1.7.6-1ubuntu2.3~cloud0) ...", "Setting up python-appconf (0.5-2build1) ...", "Setting up python-compressor (1.4-2ubuntu3~cloud0) ...", "Setting up python-pyscss (1.2.1-0ubuntu2~cloud1) ...", "update-alternatives: using /usr/bin/python2-pyscss to provide /usr/bin/pyscss (pyscss) in auto mode", "Setting up python-django-pyscss (1.0.3-3ubuntu1~cloud0) ...", "Setting up python-openstack-auth (1.1.9-0ubuntu0.15.04.1~cloud0) ...", "Setting up python-pint (0.6-0ubuntu1~cloud0) ...", "Setting up python-django-horizon (1:2015.1.4-0ubuntu2) ...", "Setting up openstack-dashboard (1:2015.1.4-0ubuntu2) ...", "Adding system user horizon' (UID 126) ...", "Adding new userhorizon' (UID 126) with group horizon' ...", "Not creating home directory/usr/share/openstack-dashboard/'.", "Collecting and compressing static assets...", "Traceback (most recent call last):", " File \"manage.py\", line 25, in ", " execute_from_command_line(sys.argv)", " File \"/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py\", line 385, in execute_from_command_line", " utility.execute()", " File \"/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py\", line 345, in execute", " settings.INSTALLED_APPS", " File \"/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py\", line 46, in getattr", " self.setup(name)", " File \"/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py\", line 42, in _setup", " self.wrapped = Settings(settings_module)", " File \"/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py\", line 98, in init", " % (self.SETTINGS_MODULE, e)", "ImportError: Could not import settings 'openstack_dashboard.settings' (Is it on sys.path? Is there an import error in the settings file?): No module named xstatic.main"]}
to retry, use: --limit @/root/elijah-openstack/ansible/openstack-kilo.retry

PLAY RECAP *********************************************************************
localhost : ok=32 changed=29 unreachable=0 failed=1

How can we solve problem of dependencies for Horizon install?

[teiszler]

Micael,

Was this a clean install of Ubuntu 14.04? My suspicion is that dashboard component of openstack was left around after your attempts at installing things via DevStack.

You can try to entirely remove Horizon first with apt-get purge openstack-dashboard. This should uninstall the package and remove any config files. Then try to run the playbook again for the Horizon tag.

Regards,
Tom

[MSoares]

Dear Tom,

I had to clean up my computer, and made successfully a fresh install of Ansible.
I do not understand how to lunch Openstack Dashboard, since when I open browser and for localhost it appears Apache index page.

I have seen some logs into /var/log/nova but I do not have seen anything.

Do you have some clue Tom?

[teiszler]

@MSoares,

The dashboard is at http://localhost/horizon. You should then be able to log in with the admin user and password you specified in the Ansible variables.

Regards,
Tom

[MSoares]

@teiszler,

sorry for that, indeed after a reboot of my browser and with http://localhost/horizon, is it possible to enter Dashboard. So, I have follow https://github.com/cmusatyalab/elijah-openstack/tree/ansible and I have sucessfully Import Base Vm from Sample Base VM . I have starting resume Base VM and I will give you feedback asap.

[MSoares]

Dear @teiszler,

Sorry for my late reply but I have Been out of scope due to an health problem and Ir will be out for more 2 erros.

I try @ellis first install and it worked great, I was able to make a cloudlet and run a VM on it.
Right now, I have to figure out to criate a Little APP and make prove of mobile Edge computing theory In order to achieve my milestones my thesys. Do you have some ideas, in which i can prove concept.

Bests regards,
Micael Soares

[teiszler]

@MSoares,

I am glad to hear you are up and running; sorry to hear about your health issues.

As far as what app to create, you might have a look at the following publications and demos we have from our Elijah project at CMU. They should give you some insight into the types of applications that can really showcase the benefits of edge computing.

http://elijah.cs.cmu.edu/publications.html
http://elijah.cs.cmu.edu/videos.html

Regards,
Tom