Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1685578 - Bad file descriptor when restoring from Samba
Summary: Bad file descriptor when restoring from Samba
Keywords:
Status: NEW
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Appliance
Version: 5.10.1
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: GA
: 5.10.z
Assignee: Nick LaMuro
QA Contact: Jaroslav Henner
Red Hat CloudForms Documentation
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-05 14:28 UTC by Jaroslav Henner
Modified: 2019-04-11 13:03 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Category: Bug
Cloudforms Team: CFME Core
Target Upstream Version:


Attachments (Terms of Use)

Description Jaroslav Henner 2019-03-05 14:28:58 UTC
Description of problem:
When restoring the db from samba using appliance_console, the command fails and there is 

Errno::EBADF: Bad file descriptor - sendfile

in the appliance_console log


Version-Release number of selected component (if applicable):
cfme-5.10.1.2-2.el7cf.x86_64

How reproducible:
2/2

Steps to Reproduce:
0. backup to samba using the appliance console
1. ap
2. 6) Restore Database From Backup
3. 3) Samba (SMB)


Actual results:
Restore Database From Backup


Note: A database restore cannot be undone.  The restore will use the file: smb://X.Y.Z.Ž/public/db_backup/tmp/evm_db.backup.
Are you sure you would like to restore the database? (Y/N): y

Restoring the database...

Database restore failed. Check the logs for more information

Press any key to continue.



E, [2019-03-05T09:01:03.890151 #8758] ERROR -- : rake aborted!
Errno::EBADF: Bad file descriptor - sendfile
/opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-71d6e5aabd0b/lib/gems/pending/util/mount/miq_generic_mount_session.rb:304:in `copy_stream'
/opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-71d6e5aabd0b/lib/gems/pending/util/mount/miq_generic_mount_session.rb:304:in `download_single'
/opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-71d6e5aabd0b/lib/gems/pending/util/miq_file_storage.rb:145:in `download'
/var/www/miq/vmdb/lib/evm_database_ops.rb:166:in `block in with_file_storage'
/opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-71d6e5aabd0b/lib/gems/pending/util/miq_file_storage.rb:31:in `with_interface_class'
/var/www/miq/vmdb/lib/evm_database_ops.rb:139:in `with_file_storage'
/var/www/miq/vmdb/lib/evm_database_ops.rb:92:in `restore'
/var/www/miq/vmdb/lib/tasks/evm_dba.rake:263:in `block (4 levels) in <top (required)>'
/opt/rh/cfme-gemset/gems/rake-12.3.2/exe/rake:27:in `<top (required)>'
Tasks: TOP => evm:db:restore:remote
(See full trace by running task with --trace)



Expected results:
DB restored fine

Additional info:
The file was there. The /srv/samba is "aliased" to [public]
[root@utility-vm ~]# ls /srv/samba/db_backup/tmp/evm_db.backup 
/srv/samba/db_backup/tmp/evm_db.backup

Comment 2 Jaroslav Henner 2019-03-05 14:36:15 UTC
It is fine when backup file is local and evmserverd is off

Comment 3 Jaroslav Henner 2019-03-05 15:46:54 UTC
I am not sure what did I do wrong, but on cfme-rhevm-5.10.0.33-1.x86_64.qcow2 it worked fine and now I cannot reproduce with cfme-rhevm-5.10.1.2-2.x86_64.qcow2 either. Maybe the file was wrong.

I had some problems with the backup/restore process hanging when evmserverd was running. Should it be off for backup? Should it be off for restore?

Comment 4 Nick LaMuro 2019-04-10 22:13:40 UTC
Sorry for the delay on this, missed it coming my way.

I think when you are restoring from a "backup" versus a "dump", you want to shut down the server (and postgres), since it is effectively pulling the rug from under the postgres process and completely replacing the data dir (effectively a `rm -rf ...` and a `tar -xzvf ...`).  With a dump, it is just doing some SQL commands to add the tables and data, so that should be fine to run in place with `postgres` still running.

The `evmserverd` process should probably always be turned off when trying to restore since it stores the server processes and worker process as database records, so it is eventually going to get in a bad state if you drop and reload the database it is running from.

Carboni, do you have anything to add, or does that sound correct?


* * *

If there is a way to reproduce the still, I can look into this, but from the sounds of things, it seems like this was just weird fluke with a specific build, and now it is working just fine?

Happy to take a look if there is a problem though, and again, sorry for not getting to this sooner.


-Nick

Comment 5 Nick Carboni 2019-04-11 13:03:18 UTC
That sounds about right.

I also thought we had a check to ensure that there were no connections to the database before running a restore so at the least I would think evmserverd would have to be stopped.
Specifically this one: https://github.com/ManageIQ/manageiq/blob/0557115ad89e278d334f9f942cccc7f1417975b0/lib/evm_database_ops.rb#L182-L186


Note You need to log in before you can comment on or make changes to this bug.