Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1692685 - Bad/missing home directory for user vdsm causes a failure
Summary: Bad/missing home directory for user vdsm causes a failure
Keywords:
Status: NEW
Alias: None
Product: vdsm
Classification: oVirt
Component: Packaging.rpm
Version: 4.30.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified vote
Target Milestone: ---
: ---
Assignee: Marcin Sobczyk
QA Contact: Lukas Svaty
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-26 08:56 UTC by Yedidyah Bar David
Modified: 2019-03-26 09:58 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
oVirt Team: Infra


Attachments (Terms of Use)

Description Yedidyah Bar David 2019-03-26 08:56:31 UTC
Description of problem:

Not sure what the flow was, that lead to current state, but we have:

$ grep vdsm /etc/passwd
vdsm:x:36:36:Node Virtualization Manager:/:/sbin/nologin

With this, many things worked well (including e.g. creating a VM), but ovirt-engine-metrics installation failed. Didn't look exactly where, but it seems to try to create a template or something like that. vdsm log has:

2019-03-26 09:04:56,397+0200 INFO  (jsonrpc/5) [api.host] FINISH getJobs return={'status': {'message': 'Done', 'code': 0}, 'jobs': {u'61c75a07-ba9a-43da-9743-d7aca66b8a11': {'status': 'failed', 'error': {'message': 'General Exception: (\'Command [\\\'/usr/bin/virt-sysprep\\\', \\\'-a\\\', u\\\'/rhev/data-center/mnt/vserver-spider.eng.lab.tlv.redhat.com:_pub_eslutsky_rhev-setup-01_rhev/1ac36da8-d085-468c-9341-0805dc9863fe/images/3e20471d-8f03-4308-a889-8d325481b294/2c5041db-2200-48cc-945f-1fe1a88528a3\\\'] failed with rc=1 out=[\\\'[   0.0] Examining the guest ...\\\'] err=["libvirt: XML-RPC error : Cannot create user runtime directory \\\'//.cache/libvirt\\\': Permission denied", \\\'virt-sysprep: error: libguestfs error: could not connect to libvirt (URI = \\\', "qemu:///session): Cannot create user runtime directory \\\'//.cache/libvirt\\\': ", \\\'Permission denied [code=38 int1=13]\\\', \\\'\\\', \\\'If reporting bugs, run virt-sysprep with debugging enabled and include the \\\', \\\'complete output:\\\', \\\'\\\', \\\'  virt-sysprep -v -x [...]\\\']\',)', 'code': 100}, 'job_type': 'virt', 'id': u'61c75a07-ba9a-43da-9743-d7aca66b8a11', 'description': 'seal_vm'}}} from=::ffff:10.35.17.99,34428, flow_id=ed1fcca8-ed54-4305-9368-8fb25116f7a1 (api:52)

Not sure what's the best solution, if at all - perhaps check (where? in rpm scripts? service start?) that user has writable home directory or something like that. Or perhaps explicitly set libvirt cache dir, etc.

Version-Release number of selected component (if applicable):

4.3, but I think much longer before that as well

How reproducible:
Always

Steps to Reproduce:
1. Create a user vdsm with non-writable (by it) home directory (e.g. '/')
2. Add the machine as a host to an engine
3. Try to deploy metrics. Perhaps it's enough to try to create a template.

Actual results:
Fails as above

Expected results:
Should fail much earlier, perhaps on vdsm package installation (an error, at least) or on vdsm service start

Additional info:

Comment 1 Martin Perina 2019-03-26 09:52:19 UTC
AFAIK vdsm user is created during vdsm package installation:

https://github.com/oVirt/vdsm/blob/master/vdsm.spec.in#L756

So are you sure that vdsm user was not manually altered afterwards? Is this really always reproducible?

Comment 2 Yedidyah Bar David 2019-03-26 09:55:25 UTC
(In reply to Martin Perina from comment #1)
> AFAIK vdsm user is created during vdsm package installation:
> 
> https://github.com/oVirt/vdsm/blob/master/vdsm.spec.in#L756

Previous line is:
/usr/bin/getent passwd %{vdsm_user} >/dev/null || \

So if you pre-create it, it's not created again.

> 
> So are you sure that vdsm user was not manually altered afterwards? Is this
> really always reproducible?

I didn't try to reproduce myself, but I think it is, based on above.

Comment 3 Yedidyah Bar David 2019-03-26 09:58:19 UTC
BTW, on the specific machine where this happened, I failed to find who/what created the user. It might have been edited manually, or a result of a series of attempts to install/reinstall/upgrade RHEL/RHVH.


Note You need to log in before you can comment on or make changes to this bug.