Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1511967 - Console.log file does not get updated across instance reboots due to permission issues [NEEDINFO]
Summary: Console.log file does not get updated across instance reboots due to permissi...
Keywords:
Status: NEW
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: 10.0 (Newton)
Assignee: nova-maint
QA Contact: nova-maint
URL:
Whiteboard:
Depends On:
Blocks: 1381612
TreeView+ depends on / blocked
 
Reported: 2017-11-10 14:20 UTC by Punit Kundal
Modified: 2019-04-11 12:06 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
kchamart: needinfo? (pkundal)


Attachments (Terms of Use)

Description Punit Kundal 2017-11-10 14:20:50 UTC
Description of problem:

When Using NFS as a backend for nova (mounting NFS shares on /var/lib/nova/instances); the openstack console log show <instance_UUID> command get's stuck and does not get refreshed with the latest information

Example:

rebooting the instance:

root@cfs3tnc02 b234f7e0-a276-4276-84c2-f4f0d89567c0(keystone_admin_v3)]# nova reboot test-cloud-init_rhel74
Request to reboot server <Server: test-cloud-init_rhel74> has been accepted.

checking the output of openstack console log show:

[root@cfs3tnc02 b234f7e0-a276-4276-84c2-f4f0d89567c0(keystone_admin_v3)]# openstack console log show b234f7e0-a276-4276-84c2-f4f0d89567c0 | tail -10
[    0.984735] emc: device handler registered
[    0.985620] alua: device handler registered
[    0.986501] libphy: Fixed MDIO Bus: probed
[    0.987392] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.988475] ehci-pci: EHCI PCI platform driver
[    0.989384] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.990484] ohci-pci: OHCI PCI platform driver
[    0.991414] uhci_hcd: USB Universal Host Controller Interface driver
[    1.013453] uhci_hcd 0000:00:01.2: UHCI Host Controller

Additionally the console.log file is not refreshed with the latest information either:

root@cfs3tnc02 b234f7e0-a276-4276-84c2-f4f0d89567c0(keystone_admin_v3)]# pwd
/var/lib/nova/instances/b234f7e0-a276-4276-84c2-f4f0d89567c0
[root@cfs3tnc02 b234f7e0-a276-4276-84c2-f4f0d89567c0(keystone_admin_v3)]# cat console.log | tail -10
[    0.983874] hp_sw: device handler registered
[    0.984735] emc: device handler registered
[    0.985620] alua: device handler registered
[    0.986501] libphy: Fixed MDIO Bus: probed
[    0.987392] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.988475] ehci-pci: EHCI PCI platform driver
[    0.989384] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.990484] ohci-pci: OHCI PCI platform driver
[    0.991414] uhci_hcd: USB Universal Host Controller Interface driver
[    1.013453] uhci_hcd 0000:00:01.2: UHCI Host Controller

When observing the file size; it keeps growing (keeps writing to the file descriptor):

[root@cfs3tnc02 b234f7e0-a276-4276-84c2-f4f0d89567c0(keystone_admin_v3)]# ls -ltra
total 52660
-rw-r--r--.   1 nova nova       79 Oct 26 13:37 disk.info
drwxr-xr-x.   2 nova nova     4096 Oct 26 13:37 .
drwxr-xr-x. 580 nova nova    65536 Oct 26 16:26 ..
-rw-r--r--.   1 nova qemu    16271 Oct 26 16:32 console.log
-rw-r--r--.   1 qemu qemu 53673984 Oct 26 16:34 disk

[root@cfs3tnc02 b234f7e0-a276-4276-84c2-f4f0d89567c0(keystone_admin_v3)]# ls -ltra
total 52660
-rw-r--r--.   1 nova nova       79 Oct 26 13:37 disk.info
drwxr-xr-x.   2 nova nova     4096 Oct 26 13:37 .
drwxr-xr-x. 580 nova nova    65536 Oct 26 16:26 ..
-rw-r--r--.   1 nova qemu    53749 Oct 26 16:32 console.log
-rw-r--r--.   1 qemu qemu 53673984 Oct 26 16:34 disk


When the permissions are manually changed to qemu:qemu; things appear to work fine:

[root@cfs3tnc02 b234f7e0-a276-4276-84c2-f4f0d89567c0(keystone_admin_v3)]# chown qemu:qemu console.log

[root@cfs3tnc02 b234f7e0-a276-4276-84c2-f4f0d89567c0(keystone_admin_v3)]# tail console.log
[    2.157854] type=1404 audit(1509028618.617:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295
[    2.379473] type=1403 audit(1509028618.839:3): policy loaded auid=4294967295 ses=4294967295
[    2.385291] systemd[1]: Successfully loaded SELinux policy in 227.462ms.
[    2.429790] ip_tables: (C) 2000-2006 Netfilter Core Team
[    2.430901] systemd[1]: Inserted module 'ip_tables'
[    2.444714] systemd[1]: Relabelled /dev and /run in 11.564ms.

Welcome to Red Hat Enterprise Linux Server 7.4 (Maipo)!


And after this the openstack console log <instance_UUID> will show the last data. 



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Adding additional information in a private comment.

Comment 3 Kashyap Chamarthy 2017-11-17 13:38:53 UTC
This seems to be fixed with the latest 'openstack-selinux' package:

    openstack-selinux-0.8.11-1.el7

Can you please try with it and report back?.  I've read some confirmation from RDO lists that the above fixes the issue.

The temporary workaround is to restore the SELinux context on that file:

    `restorecon -F console.log`


Note You need to log in before you can comment on or make changes to this bug.