Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1515074 - can not new storage domain with iscsi lun
Summary: can not new storage domain with iscsi lun
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: 4.1.7.6
Hardware: x86_64
OS: Linux
unspecified
medium vote
Target Milestone: ovirt-4.2.1
: ---
Assignee: shani
QA Contact: Raz Tamir
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-20 06:13 UTC by alex
Modified: 2017-11-26 12:44 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-11-26 12:44:33 UTC
oVirt Team: Storage
rule-engine: ovirt-4.2+


Attachments (Terms of Use)
vdsm.log. (deleted)
2017-11-21 04:27 UTC, alex
no flags Details
whole day log (deleted)
2017-11-21 08:32 UTC, alex
no flags Details
reproduce issue (deleted)
2017-11-23 02:12 UTC, alex
no flags Details
reproduce issue (deleted)
2017-11-23 02:12 UTC, alex
no flags Details
screenshot of issue (deleted)
2017-11-23 02:13 UTC, alex
no flags Details
screenshot of issue (deleted)
2017-11-23 02:14 UTC, alex
no flags Details
reproduce issue (deleted)
2017-11-23 02:35 UTC, alex
no flags Details

Description alex 2017-11-20 06:13:27 UTC
Description of problem:
When I new a 'data' storage domain with iscsi lun, i get a error dialog.
Error while executing action New SAN Storage Domain: Internal Engine Error

Version-Release number of selected component (if applicable):
[root@cloudstack01 ~]# rpm -aq|grep ovirt
libgovirt-0.3.3-5.el7.x86_64
ovirt-web-ui-1.2.1-1.el7.centos.noarch
ovirt-engine-dbscripts-4.1.7.6-1.el7.centos.noarch
ovirt-host-deploy-1.6.7-1.el7.centos.noarch
ovirt-engine-tools-backup-4.1.7.6-1.el7.centos.noarch
ovirt-engine-setup-plugin-ovirt-engine-4.1.7.6-1.el7.centos.noarch
ovirt-engine-wildfly-overlay-10.0.0-1.el7.noarch
ovirt-engine-setup-base-4.1.7.6-1.el7.centos.noarch
ovirt-release41-4.1.7-1.el7.centos.noarch
ovirt-imageio-common-1.0.0-1.el7.noarch
ovirt-vmconsole-1.0.4-1.el7.centos.noarch
ovirt-engine-setup-plugin-websocket-proxy-4.1.7.6-1.el7.centos.noarch
ovirt-engine-websocket-proxy-4.1.7.6-1.el7.centos.noarch
ovirt-engine-4.1.7.6-1.el7.centos.noarch
ovirt-setup-lib-1.1.4-1.el7.centos.noarch
ovirt-iso-uploader-4.0.2-1.el7.centos.noarch
ovirt-engine-dwh-setup-4.1.8-1.el7.centos.noarch
ovirt-engine-extensions-api-impl-4.1.7.6-1.el7.centos.noarch
ovirt-engine-cli-3.6.9.2-1.el7.centos.noarch
ovirt-engine-userportal-4.1.7.6-1.el7.centos.noarch
ovirt-engine-lib-4.1.7.6-1.el7.centos.noarch
ovirt-engine-setup-4.1.7.6-1.el7.centos.noarch
ovirt-engine-sdk-python-3.6.9.1-1.el7.centos.noarch
ovirt-engine-restapi-4.1.7.6-1.el7.centos.noarch
ovirt-imageio-proxy-setup-1.0.0-0.201701151456.git89ae3b4.el7.centos.noarch
ovirt-engine-dwh-4.1.8-1.el7.centos.noarch
ovirt-engine-extension-aaa-jdbc-1.1.6-1.el7.centos.noarch
ovirt-vmconsole-host-1.0.4-1.el7.centos.noarch
ovirt-engine-setup-plugin-vmconsole-proxy-helper-4.1.7.6-1.el7.centos.noarch
ovirt-host-deploy-java-1.6.7-1.el7.centos.noarch
python-ovirt-engine-sdk4-4.1.7-2.el7.centos.x86_64
ovirt-engine-webadmin-portal-4.1.7.6-1.el7.centos.noarch
ovirt-imageio-proxy-1.0.0-0.201701151456.git89ae3b4.el7.centos.noarch
ovirt-engine-wildfly-10.1.0-1.el7.x86_64
ovirt-engine-setup-plugin-ovirt-engine-common-4.1.7.6-1.el7.centos.noarch
ovirt-engine-vmconsole-proxy-helper-4.1.7.6-1.el7.centos.noarch
ovirt-engine-tools-4.1.7.6-1.el7.centos.noarch
ovirt-engine-metrics-1.0.7.1-1.el7.centos.noarch
ovirt-engine-dashboard-1.1.7-1.el7.centos.noarch
ovirt-imageio-daemon-1.0.0-1.el7.noarch
ovirt-vmconsole-proxy-1.0.4-1.el7.centos.noarch
ovirt-engine-backend-4.1.7.6-1.el7.centos.noarch



How reproducible:


Steps to Reproduce:
1.iscsi/multipath ready. i can do pvcreate/vgcreate/lvcreate on /dev/mapper/lun1 manually.
2.new storage domain with iscis, the other parameters are remain default.
3.

Actual results:


Expected results:


Additional info:

i took a look on engine.log(on the engine side) and vdsm.log(on the host side). it looks like a timer out breaking the createstoragedomain command.
i don't know why create lvm procedure is so slow on my host, so that 
(vdsm.Scheduler) [Executor] Worker blocked: <Worker name=jsonrpc/0 running <Task <JsonRpcTask {'params': {u'name': u'ab', u'domainType': 3, u'domainClass': 1, u'typeArgs': u'nqLWk5-wyfU-gvDF-Iadc-5g7M-5uyr-57dq78', u'version': u'4', u'storagedomainID': u'a78e0bc2-b52a-4709-913b-d17d93edf6f5'}, 'jsonrpc': '2.0', 'method': u'StorageDomain.create', 'id': u'3899b5b0-1758-41a6-8d80-f20ca96089f7'} at 0x2f82710> timeout=60, duration=180 at 0x2f828d0> task#=1486 at 0x2edd6d0> (executor:351)
2017-11-20 12:16:32,202+0800 INFO  (periodic/0) [vdsm.api] START repoStats(options=None) from=internal, task_id=c621bbcc-56cf-40f5-97fa-1eb326c2337f (api:46)



engine.log
2017-11-20 12:11:28,315+08 INFO  [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (default task-198) [f833f6ef-85ac-4db8-a917-deae91de5c9f] Running command: AddSANStorageDomainCommand internal: false. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN
2017-11-20 12:11:28,355+08 INFO  [org.ovirt.engine.core.bll.profiles.AddDiskProfileCommand] (default task-198) [42655719] Running command: AddDiskProfileCommand internal: true. Entities affected :  ID: a78e0bc2-b52a-4709-913b-d17d93edf6f5 Type: StorageAction group CREATE_STORAGE_DISK_PROFILE with role type ADMIN
2017-11-20 12:11:28,365+08 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-198) [42655719] EVENT_ID: USER_ADDED_DISK_PROFILE(10,120), Correlation ID: 42655719, Job ID: ab6bac8d-5fd9-4450-b4a3-fd5685e62096, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Disk Profile ab was successfully added (User: admin@internal-authz).
2017-11-20 12:11:28,372+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (default task-198) [42655719] START, CreateVGVDSCommand(HostName = cloudstack02, CreateVGVDSCommandParameters:{runAsync='true', hostId='7296e4c9-8a12-4280-a3e5-96c5a80b33c5', storageDomainId='a78e0bc2-b52a-4709-913b-d17d93edf6f5', deviceList='[lun1]', force='true'}), log id: 374819d9
2017-11-20 12:13:30,661+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (default task-198) [42655719] FINISH, CreateVGVDSCommand, return: nqLWk5-wyfU-gvDF-Iadc-5g7M-5uyr-57dq78, log id: 374819d9
2017-11-20 12:13:30,663+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-198) [42655719] START, CreateStorageDomainVDSCommand(HostName = cloudstack02, CreateStorageDomainVDSCommandParameters:{runAsync='true', hostId='7296e4c9-8a12-4280-a3e5-96c5a80b33c5', storageDomain='StorageDomainStatic:{name='ab', id='a78e0bc2-b52a-4709-913b-d17d93edf6f5'}', args='nqLWk5-wyfU-gvDF-Iadc-5g7M-5uyr-57dq78'}), log id: 7722eb3d
2017-11-20 12:16:28,318+08 ERROR [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (Transaction Reaper Worker 0) [] Transaction rolled-back for command 'org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand'.
2017-11-20 12:16:31,022+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-198) [42655719] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand' return value 'StatusOnlyReturn [status=Status [code=5022, message=Message timeout which can be caused by communication issues]]'
2017-11-20 12:16:31,022+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-198) [42655719] HostName = cloudstack02
2017-11-20 12:16:31,022+08 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-198) [42655719] Failed in 'CreateStorageDomainVDS' method, for vds: 'cloudstack02'; host: '10.100.254.9': Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000460: Error checking for a transaction
2017-11-20 12:16:31,023+08 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-198) [42655719] Command 'CreateStorageDomainVDSCommand(HostName = cloudstack02, CreateStorageDomainVDSCommandParameters:{runAsync='true', hostId='7296e4c9-8a12-4280-a3e5-96c5a80b33c5', storageDomain='StorageDomainStatic:{name='ab', id='a78e0bc2-b52a-4709-913b-d17d93edf6f5'}', args='nqLWk5-wyfU-gvDF-Iadc-5g7M-5uyr-57dq78'})' execution failed: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000460: Error checking for a transaction
2017-11-20 12:16:31,023+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-198) [42655719] FINISH, CreateStorageDomainVDSCommand, log id: 7722eb3d
2017-11-20 12:16:31,023+08 ERROR [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (default task-198) [42655719] Command 'org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand' failed: EngineException: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000460: Error checking for a transaction (Failed with error ENGINE and code 5001)
2017-11-20 12:16:31,025+08 INFO  [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (default task-198) [42655719] Command [id=398a5d92-b1ff-4abe-92c9-65641ee55668]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.profiles.DiskProfile; snapshot: 5b81caa1-902e-48d1-b76f-0be5f77ab716.
2017-11-20 12:16:31,026+08 INFO  [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (default task-198) [42655719] Command [id=398a5d92-b1ff-4abe-92c9-65641ee55668]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.StorageDomainDynamic; snapshot: a78e0bc2-b52a-4709-913b-d17d93edf6f5.
2017-11-20 12:16:31,027+08 INFO  [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (default task-198) [42655719] Command [id=398a5d92-b1ff-4abe-92c9-65641ee55668]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.StorageDomainStatic; snapshot: a78e0bc2-b52a-4709-913b-d17d93edf6f5.
2017-11-20 12:16:31,066+08 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-198) [42655719] EVENT_ID: USER_ADD_STORAGE_DOMAIN_FAILED(957), Correlation ID: f833f6ef-85ac-4db8-a917-deae91de5c9f, Job ID: ab6bac8d-5fd9-4450-b4a3-fd5685e62096, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Failed to add Storage Domain ab. (User: admin@internal-authz)




vdsm.log
2017-11-20 12:13:28,530+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call LVMVolumeGroup.create succeeded in 120.28 seconds (__init__:539)
2017-11-20 12:13:30,539+0800 INFO  (jsonrpc/0) [vdsm.api] START createStorageDomain(storageType=3, sdUUID=u'a78e0bc2-b52a-4709-913b-d17d93edf6f5', domainName=u'ab', typeSpecificArg=u'nqLWk5-wyfU-gvDF-Iadc-5g7M-5uyr-57dq78', domClass=1, domVersion=u'4', options=None) from=::ffff:10.100.254.17,35478, flow_id=42655719, task_id=bab3ee3f-415b-4389-ae9b-7fc15b54cb3b (api:46)
2017-11-20 12:13:30,540+0800 INFO  (itmap/0) [IOProcessClient] Starting client ioprocess-211 (__init__:330)
2017-11-20 12:13:30,553+0800 INFO  (ioprocess/48783) [IOProcess] Starting ioprocess (__init__:452)
2017-11-20 12:13:30,555+0800 INFO  (jsonrpc/0) [storage.StorageDomain] sdUUID=a78e0bc2-b52a-4709-913b-d17d93edf6f5 domainName=ab domClass=1 vgUUID=nqLWk5-wyfU-gvDF-Iadc-5g7M-5uyr-57dq78 storageType=3 version=4 (blockSD:1008)
2017-11-20 12:13:31,756+0800 INFO  (periodic/0) [vdsm.api] START repoStats(options=None) from=internal, task_id=a2d0c6d8-ae6c-499e-9a5d-9e74a61d2bde (api:46)
2017-11-20 12:13:31,757+0800 INFO  (periodic/0) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000521038', 'lastCheck': '1.8', 'valid': True}} from=internal, task_id=a2d0c6d8-ae6c-499e-9a5d-9e74a61d2bde (api:52)
2017-11-20 12:13:32,667+0800 INFO  (jsonrpc/5) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,35478, flow_id=20b02bd1, task_id=9266236a-e7be-4f2c-96b0-31a57b95c9a6 (api:46)
2017-11-20 12:13:32,668+0800 INFO  (jsonrpc/5) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000521038', 'lastCheck': '2.7', 'valid': True}} from=::ffff:10.100.254.17,35478, flow_id=20b02bd1, task_id=9266236a-e7be-4f2c-96b0-31a57b95c9a6 (api:52)
2017-11-20 12:13:32,679+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:13:33,693+0800 INFO  (jsonrpc/0) [storage.StorageDomainManifest] size 512 MB (metaratio 262144) (blockSD:595)
2017-11-20 12:13:33,693+0800 INFO  (jsonrpc/0) [storage.LVM] Creating LV (vg=a78e0bc2-b52a-4709-913b-d17d93edf6f5, lv=metadata, size=512m, activate=True, contiguous=False, initialTags=()) (lvm:1147)
2017-11-20 12:13:34,043+0800 DEBUG (periodic/2) [virt.sampling.VMBulkSampler] sampled timestamp 4529185.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:13:35,118+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:13:43,536+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:13:46,795+0800 INFO  (periodic/0) [vdsm.api] START repoStats(options=None) from=internal, task_id=32cd0352-1c82-45a3-b074-e697443c09c3 (api:46)
2017-11-20 12:13:46,795+0800 INFO  (periodic/0) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000807372', 'lastCheck': '6.8', 'valid': True}} from=internal, task_id=32cd0352-1c82-45a3-b074-e697443c09c3 (api:52)
2017-11-20 12:13:48,722+0800 INFO  (jsonrpc/3) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,35478, flow_id=7ff9d97b, task_id=94598c82-20dc-43d8-a159-771cba738a91 (api:46)
2017-11-20 12:13:48,722+0800 INFO  (jsonrpc/3) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00100907', 'lastCheck': '1.3', 'valid': True}} from=::ffff:10.100.254.17,35478, flow_id=7ff9d97b, task_id=94598c82-20dc-43d8-a159-771cba738a91 (api:52)
2017-11-20 12:13:48,733+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.02 seconds (__init__:539)
2017-11-20 12:13:49,042+0800 DEBUG (periodic/2) [virt.sampling.VMBulkSampler] sampled timestamp 4529200.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:13:50,131+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:13:59,543+0800 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:14:01,834+0800 INFO  (periodic/2) [vdsm.api] START repoStats(options=None) from=internal, task_id=6c5b715f-9513-487a-b21c-849b402116c8 (api:46)
2017-11-20 12:14:01,834+0800 INFO  (periodic/2) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000815108', 'lastCheck': '1.8', 'valid': True}} from=internal, task_id=6c5b715f-9513-487a-b21c-849b402116c8 (api:52)
2017-11-20 12:14:04,050+0800 DEBUG (periodic/0) [virt.sampling.VMBulkSampler] sampled timestamp 4529215.16 elapsed 0.010 acquired True domains all (sampling:508)
2017-11-20 12:14:04,782+0800 INFO  (jsonrpc/6) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,35478, flow_id=680491d, task_id=fcfe35c3-70ae-459f-a9aa-bbc7a38ebe41 (api:46)
2017-11-20 12:14:04,783+0800 INFO  (jsonrpc/6) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000815108', 'lastCheck': '4.8', 'valid': True}} from=::ffff:10.100.254.17,35478, flow_id=680491d, task_id=fcfe35c3-70ae-459f-a9aa-bbc7a38ebe41 (api:52)
2017-11-20 12:14:04,793+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:14:05,154+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:14:15,554+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:14:16,872+0800 INFO  (periodic/2) [vdsm.api] START repoStats(options=None) from=internal, task_id=a4dfc3be-7329-4aca-a9ad-8518719343f3 (api:46)
2017-11-20 12:14:16,872+0800 INFO  (periodic/2) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00114324', 'lastCheck': '6.9', 'valid': True}} from=internal, task_id=a4dfc3be-7329-4aca-a9ad-8518719343f3 (api:52)
2017-11-20 12:14:19,050+0800 DEBUG (periodic/3) [virt.sampling.VMBulkSampler] sampled timestamp 4529230.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:14:20,174+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:14:20,832+0800 INFO  (jsonrpc/3) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,35478, flow_id=4790178e, task_id=76b28e39-5a6f-4eb8-a118-ae91c34fab1e (api:46)
2017-11-20 12:14:20,832+0800 INFO  (jsonrpc/3) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000951949', 'lastCheck': '0.8', 'valid': True}} from=::ffff:10.100.254.17,35478, flow_id=4790178e, task_id=76b28e39-5a6f-4eb8-a118-ae91c34fab1e (api:52)
2017-11-20 12:14:20,843+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.02 seconds (__init__:539)
2017-11-20 12:14:30,534+0800 WARN  (vdsm.Scheduler) [Executor] Worker blocked: <Worker name=jsonrpc/0 running <Task <JsonRpcTask {'params': {u'name': u'ab', u'domainType': 3, u'domainClass': 1, u'typeArgs': u'nqLWk5-wyfU-gvDF-Iadc-5g7M-5uyr-57dq78', u'version': u'4', u'storagedomainID': u'a78e0bc2-b52a-4709-913b-d17d93edf6f5'}, 'jsonrpc': '2.0', 'method': u'StorageDomain.create', 'id': u'3899b5b0-1758-41a6-8d80-f20ca96089f7'} at 0x2f82710> timeout=60, duration=60 at 0x2f828d0> task#=1486 at 0x2edd6d0> (executor:351)
2017-11-20 12:14:31,561+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:14:31,913+0800 INFO  (periodic/3) [vdsm.api] START repoStats(options=None) from=internal, task_id=84ab7652-fa8a-4162-a656-f4130304e35f (api:46)
2017-11-20 12:14:31,913+0800 INFO  (periodic/3) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000843371', 'lastCheck': '1.9', 'valid': True}} from=internal, task_id=84ab7652-fa8a-4162-a656-f4130304e35f (api:52)
2017-11-20 12:14:34,049+0800 DEBUG (periodic/2) [virt.sampling.VMBulkSampler] sampled timestamp 4529245.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:14:35,197+0800 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:14:36,877+0800 INFO  (jsonrpc/6) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,35478, flow_id=752444c, task_id=7bc0a479-3329-44fe-94b3-afc4b3906685 (api:46)
2017-11-20 12:14:36,878+0800 INFO  (jsonrpc/6) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000843371', 'lastCheck': '6.9', 'valid': True}} from=::ffff:10.100.254.17,35478, flow_id=752444c, task_id=7bc0a479-3329-44fe-94b3-afc4b3906685 (api:52)
2017-11-20 12:14:36,889+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:14:38,887+0800 INFO  (jsonrpc/0) [storage.StorageDomainManifest] Create: SORT MAPPING: ['/dev/mapper/lun1'] (blockSD:523)
2017-11-20 12:14:40,022+0800 INFO  (monitor/2898fb9) [IOProcessClient] Closing client ioprocess-211 (__init__:598)
2017-11-20 12:14:42,271+0800 INFO  (jsonrpc/0) [storage.LVM] Creating LV (vg=a78e0bc2-b52a-4709-913b-d17d93edf6f5, lv=outbox, size=16m, activate=True, contiguous=False, initialTags=()) (lvm:1147)
2017-11-20 12:14:46,951+0800 INFO  (periodic/3) [vdsm.api] START repoStats(options=None) from=internal, task_id=4b24b171-38a4-42ce-9b8c-f784c8f43058 (api:46)
2017-11-20 12:14:46,952+0800 INFO  (periodic/3) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000962885', 'lastCheck': '6.9', 'valid': True}} from=internal, task_id=4b24b171-38a4-42ce-9b8c-f784c8f43058 (api:52)
2017-11-20 12:14:47,569+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:14:49,049+0800 DEBUG (periodic/2) [virt.sampling.VMBulkSampler] sampled timestamp 4529260.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:14:50,031+0800 INFO  (itmap/0) [IOProcessClient] Starting client ioprocess-212 (__init__:330)
2017-11-20 12:14:50,044+0800 INFO  (ioprocess/48850) [IOProcess] Starting ioprocess (__init__:452)
2017-11-20 12:14:50,065+0800 INFO  (monitor/2898fb9) [storage.StorageDomain] Resource namespace 01_img_2898fb98-76d5-451c-bde8-1c9faedea45e already registered (sd:727)
2017-11-20 12:14:50,066+0800 INFO  (monitor/2898fb9) [storage.StorageDomain] Resource namespace 02_vol_2898fb98-76d5-451c-bde8-1c9faedea45e already registered (sd:736)
2017-11-20 12:14:50,217+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:14:52,656+0800 INFO  (jsonrpc/1) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,35478, flow_id=1300b9d8, task_id=57375eb1-f97b-453a-b69e-3d2b0a534019 (api:46)
2017-11-20 12:14:52,656+0800 INFO  (jsonrpc/1) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000906721', 'lastCheck': '2.6', 'valid': True}} from=::ffff:10.100.254.17,35478, flow_id=1300b9d8, task_id=57375eb1-f97b-453a-b69e-3d2b0a534019 (api:52)
2017-11-20 12:14:52,667+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:15:01,991+0800 INFO  (periodic/2) [vdsm.api] START repoStats(options=None) from=internal, task_id=2f832fea-134b-436c-b311-bc5a1cf231a5 (api:46)
2017-11-20 12:15:01,991+0800 INFO  (periodic/2) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00105722', 'lastCheck': '1.9', 'valid': True}} from=internal, task_id=2f832fea-134b-436c-b311-bc5a1cf231a5 (api:52)
2017-11-20 12:15:03,575+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:15:04,047+0800 DEBUG (periodic/3) [virt.sampling.VMBulkSampler] sampled timestamp 4529275.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:15:05,239+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:15:07,085+0800 INFO  (jsonrpc/0) [storage.LVM] Creating LV (vg=a78e0bc2-b52a-4709-913b-d17d93edf6f5, lv=xleases, size=1024m, activate=True, contiguous=False, initialTags=()) (lvm:1147)
2017-11-20 12:15:08,718+0800 INFO  (jsonrpc/7) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,35478, flow_id=1d7c3b0f, task_id=f0ada105-e733-4458-84fa-f55b0369f4cc (api:46)
2017-11-20 12:15:08,718+0800 INFO  (jsonrpc/7) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000928911', 'lastCheck': '1.3', 'valid': True}} from=::ffff:10.100.254.17,35478, flow_id=1d7c3b0f, task_id=f0ada105-e733-4458-84fa-f55b0369f4cc (api:52)
2017-11-20 12:15:08,730+0800 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:15:17,028+0800 INFO  (periodic/2) [vdsm.api] START repoStats(options=None) from=internal, task_id=c189ed6f-de9d-4ba3-b8f3-ec2462a80cf6 (api:46)
2017-11-20 12:15:17,028+0800 INFO  (periodic/2) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000928911', 'lastCheck': '6.9', 'valid': True}} from=internal, task_id=c189ed6f-de9d-4ba3-b8f3-ec2462a80cf6 (api:52)
2017-11-20 12:15:19,046+0800 DEBUG (periodic/1) [virt.sampling.VMBulkSampler] sampled timestamp 4529290.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:15:19,584+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:15:20,258+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:15:24,767+0800 INFO  (jsonrpc/4) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,35478, flow_id=2d351c63, task_id=7da1cc67-8776-4146-a7fe-a8debedb660e (api:46)
2017-11-20 12:15:24,767+0800 INFO  (jsonrpc/4) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000834676', 'lastCheck': '4.7', 'valid': True}} from=::ffff:10.100.254.17,35478, flow_id=2d351c63, task_id=7da1cc67-8776-4146-a7fe-a8debedb660e (api:52)
2017-11-20 12:15:24,778+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:15:30,531+0800 WARN  (vdsm.Scheduler) [Executor] Worker blocked: <Worker name=jsonrpc/0 running <Task <JsonRpcTask {'params': {u'name': u'ab', u'domainType': 3, u'domainClass': 1, u'typeArgs': u'nqLWk5-wyfU-gvDF-Iadc-5g7M-5uyr-57dq78', u'version': u'4', u'storagedomainID': u'a78e0bc2-b52a-4709-913b-d17d93edf6f5'}, 'jsonrpc': '2.0', 'method': u'StorageDomain.create', 'id': u'3899b5b0-1758-41a6-8d80-f20ca96089f7'} at 0x2f82710> timeout=60, duration=120 at 0x2f828d0> task#=1486 at 0x2edd6d0> (executor:351)
2017-11-20 12:15:32,059+0800 INFO  (periodic/1) [vdsm.api] START repoStats(options=None) from=internal, task_id=68067d31-0f02-45a8-8135-6676380ddb87 (api:46)
2017-11-20 12:15:32,059+0800 INFO  (periodic/1) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000877707', 'lastCheck': '1.9', 'valid': True}} from=internal, task_id=68067d31-0f02-45a8-8135-6676380ddb87 (api:52)
2017-11-20 12:15:34,045+0800 DEBUG (periodic/2) [virt.sampling.VMBulkSampler] sampled timestamp 4529305.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:15:35,278+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:15:35,590+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:15:40,774+0800 INFO  (jsonrpc/2) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,35478, flow_id=698cb7b3, task_id=752e8d3b-886c-4b23-b567-56062cdf940d (api:46)
2017-11-20 12:15:40,775+0800 INFO  (jsonrpc/2) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00100253', 'lastCheck': '0.7', 'valid': True}} from=::ffff:10.100.254.17,35478, flow_id=698cb7b3, task_id=752e8d3b-886c-4b23-b567-56062cdf940d (api:52)
2017-11-20 12:15:40,786+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:15:47,086+0800 INFO  (periodic/1) [vdsm.api] START repoStats(options=None) from=internal, task_id=a53494d0-562b-4c4f-9d5f-cbac4ea59dbf (api:46)
2017-11-20 12:15:47,086+0800 INFO  (periodic/1) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00100253', 'lastCheck': '7.0', 'valid': True}} from=internal, task_id=a53494d0-562b-4c4f-9d5f-cbac4ea59dbf (api:52)
2017-11-20 12:15:49,044+0800 DEBUG (periodic/2) [virt.sampling.VMBulkSampler] sampled timestamp 4529320.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:15:50,122+0800 INFO  (monitor/2898fb9) [IOProcessClient] Closing client ioprocess-212 (__init__:598)
2017-11-20 12:15:50,298+0800 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:15:51,595+0800 INFO  (jsonrpc/6) [throttled] Current getAllVmStats: {} (throttledlog:105)
2017-11-20 12:15:51,596+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:15:56,821+0800 INFO  (jsonrpc/5) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,35478, flow_id=257edb78, task_id=5a29c50b-144a-4fc7-83a8-d3fea0264290 (api:46)
2017-11-20 12:15:56,821+0800 INFO  (jsonrpc/5) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000992501', 'lastCheck': '6.7', 'valid': True}} from=::ffff:10.100.254.17,35478, flow_id=257edb78, task_id=5a29c50b-144a-4fc7-83a8-d3fea0264290 (api:52)
2017-11-20 12:15:56,832+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.02 seconds (__init__:539)
2017-11-20 12:16:00,884+0800 INFO  (jsonrpc/0) [storage.LVM] Creating LV (vg=a78e0bc2-b52a-4709-913b-d17d93edf6f5, lv=leases, size=2048m, activate=True, contiguous=False, initialTags=()) (lvm:1147)
2017-11-20 12:16:02,125+0800 INFO  (periodic/2) [vdsm.api] START repoStats(options=None) from=internal, task_id=1cc5eac0-f12b-4f8d-81b8-27d852f4a460 (api:46)
2017-11-20 12:16:02,126+0800 INFO  (periodic/2) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000896727', 'lastCheck': '2.0', 'valid': True}} from=internal, task_id=1cc5eac0-f12b-4f8d-81b8-27d852f4a460 (api:52)
2017-11-20 12:16:04,042+0800 DEBUG (periodic/1) [virt.sampling.VMBulkSampler] sampled timestamp 4529335.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:16:05,321+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:16:07,602+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:16:12,873+0800 INFO  (jsonrpc/3) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,35478, flow_id=28d72569, task_id=5ff79c64-8f50-45ef-84d8-1f7113f66892 (api:46)
2017-11-20 12:16:12,874+0800 INFO  (jsonrpc/3) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00123235', 'lastCheck': '2.7', 'valid': True}} from=::ffff:10.100.254.17,35478, flow_id=28d72569, task_id=5ff79c64-8f50-45ef-84d8-1f7113f66892 (api:52)
2017-11-20 12:16:12,885+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:16:17,163+0800 INFO  (periodic/2) [vdsm.api] START repoStats(options=None) from=internal, task_id=f3858d92-90f4-48b9-ab04-d44558d4004f (api:46)
2017-11-20 12:16:17,163+0800 INFO  (periodic/2) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00123235', 'lastCheck': '7.0', 'valid': True}} from=internal, task_id=f3858d92-90f4-48b9-ab04-d44558d4004f (api:52)
2017-11-20 12:16:19,051+0800 DEBUG (periodic/0) [virt.sampling.VMBulkSampler] sampled timestamp 4529350.16 elapsed 0.010 acquired True domains all (sampling:508)
2017-11-20 12:16:20,344+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:16:23,611+0800 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:16:28,712+0800 INFO  (jsonrpc/6) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,35478, flow_id=14aa0ba8, task_id=8943b2ae-6908-4105-a7fe-17c7ca04e983 (api:46)
2017-11-20 12:16:28,713+0800 INFO  (jsonrpc/6) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00094238', 'lastCheck': '1.3', 'valid': True}} from=::ffff:10.100.254.17,35478, flow_id=14aa0ba8, task_id=8943b2ae-6908-4105-a7fe-17c7ca04e983 (api:52)
2017-11-20 12:16:28,725+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.02 seconds (__init__:539)
2017-11-20 12:16:30,535+0800 WARN  (vdsm.Scheduler) [Executor] Worker blocked: <Worker name=jsonrpc/0 running <Task <JsonRpcTask {'params': {u'name': u'ab', u'domainType': 3, u'domainClass': 1, u'typeArgs': u'nqLWk5-wyfU-gvDF-Iadc-5g7M-5uyr-57dq78', u'version': u'4', u'storagedomainID': u'a78e0bc2-b52a-4709-913b-d17d93edf6f5'}, 'jsonrpc': '2.0', 'method': u'StorageDomain.create', 'id': u'3899b5b0-1758-41a6-8d80-f20ca96089f7'} at 0x2f82710> timeout=60, duration=180 at 0x2f828d0> task#=1486 at 0x2edd6d0> (executor:351)
2017-11-20 12:16:32,202+0800 INFO  (periodic/0) [vdsm.api] START repoStats(options=None) from=internal, task_id=c621bbcc-56cf-40f5-97fa-1eb326c2337f (api:46)
2017-11-20 12:16:32,202+0800 INFO  (periodic/0) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00094238', 'lastCheck': '2.0', 'valid': True}} from=internal, task_id=c621bbcc-56cf-40f5-97fa-1eb326c2337f (api:52)
2017-11-20 12:16:34,049+0800 DEBUG (periodic/2) [virt.sampling.VMBulkSampler] sampled timestamp 4529365.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:16:35,367+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:16:39,616+0800 INFO  (Reactor thread) [ProtocolDetector.AcceptorImpl] Accepted connection from ::ffff:10.100.254.17:45990 (protocoldetector:72)
2017-11-20 12:16:39,641+0800 INFO  (Reactor thread) [ProtocolDetector.Detector] Detected protocol stomp from ::ffff:10.100.254.17:45990 (protocoldetector:127)
2017-11-20 12:16:39,642+0800 INFO  (Reactor thread) [Broker.StompAdapter] Processing CONNECT request (stompreactor:103)
2017-11-20 12:16:39,643+0800 INFO  (JsonRpc (StompReactor)) [Broker.StompAdapter] Subscribe command received (stompreactor:132)
2017-11-20 12:16:39,644+0800 INFO  (JsonRpc (StompReactor)) [Broker.StompAdapter] Subscribe command received (stompreactor:132)
2017-11-20 12:16:40,646+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:16:44,761+0800 INFO  (jsonrpc/1) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,45990, flow_id=7b34dba4, task_id=b32f224f-c1fc-499f-80a0-dcf0285d9074 (api:46)
2017-11-20 12:16:44,761+0800 INFO  (jsonrpc/1) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000872837', 'lastCheck': '4.6', 'valid': True}} from=::ffff:10.100.254.17,45990, flow_id=7b34dba4, task_id=b32f224f-c1fc-499f-80a0-dcf0285d9074 (api:52)
2017-11-20 12:16:44,772+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.02 seconds (__init__:539)
2017-11-20 12:16:47,240+0800 INFO  (periodic/0) [vdsm.api] START repoStats(options=None) from=internal, task_id=f5552b87-645c-4f6f-8b9e-5a82f70fdebc (api:46)
2017-11-20 12:16:47,240+0800 INFO  (periodic/0) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000872837', 'lastCheck': '7.1', 'valid': True}} from=internal, task_id=f5552b87-645c-4f6f-8b9e-5a82f70fdebc (api:52)
2017-11-20 12:16:49,048+0800 DEBUG (periodic/0) [virt.sampling.VMBulkSampler] sampled timestamp 4529380.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:16:50,387+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:16:56,652+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:16:59,595+0800 INFO  (jsonrpc/0) [storage.LVM] Creating LV (vg=a78e0bc2-b52a-4709-913b-d17d93edf6f5, lv=ids, size=8m, activate=True, contiguous=False, initialTags=()) (lvm:1147)
2017-11-20 12:17:00,862+0800 INFO  (jsonrpc/7) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,45990, flow_id=581c2067, task_id=893b2884-a684-4461-9c43-53b66bf5e6f3 (api:46)
2017-11-20 12:17:00,863+0800 INFO  (jsonrpc/7) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000957625', 'lastCheck': '0.7', 'valid': True}} from=::ffff:10.100.254.17,45990, flow_id=581c2067, task_id=893b2884-a684-4461-9c43-53b66bf5e6f3 (api:52)
2017-11-20 12:17:00,874+0800 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.02 seconds (__init__:539)
2017-11-20 12:17:02,268+0800 INFO  (periodic/0) [vdsm.api] START repoStats(options=None) from=internal, task_id=cb58f773-7c42-4203-968b-1f09965b7ff2 (api:46)
2017-11-20 12:17:02,268+0800 INFO  (periodic/0) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000957625', 'lastCheck': '2.1', 'valid': True}} from=internal, task_id=cb58f773-7c42-4203-968b-1f09965b7ff2 (api:52)
2017-11-20 12:17:04,046+0800 DEBUG (periodic/2) [virt.sampling.VMBulkSampler] sampled timestamp 4529395.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:17:05,410+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:17:12,660+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:17:16,784+0800 INFO  (jsonrpc/4) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,45990, flow_id=70768dc9, task_id=d65f622f-021f-4680-8bec-6ef2115b3705 (api:46)
2017-11-20 12:17:16,784+0800 INFO  (jsonrpc/4) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000896266', 'lastCheck': '6.6', 'valid': True}} from=::ffff:10.100.254.17,45990, flow_id=70768dc9, task_id=d65f622f-021f-4680-8bec-6ef2115b3705 (api:52)
2017-11-20 12:17:16,796+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:17:17,296+0800 INFO  (periodic/0) [vdsm.api] START repoStats(options=None) from=internal, task_id=f82502c0-00bb-40fd-b17b-57c43cab3670 (api:46)
2017-11-20 12:17:17,296+0800 INFO  (periodic/0) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000896266', 'lastCheck': '7.1', 'valid': True}} from=internal, task_id=f82502c0-00bb-40fd-b17b-57c43cab3670 (api:52)
2017-11-20 12:17:19,044+0800 DEBUG (periodic/3) [virt.sampling.VMBulkSampler] sampled timestamp 4529410.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:17:20,430+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:17:28,668+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:17:30,538+0800 WARN  (vdsm.Scheduler) [Executor] Worker blocked: <Worker name=jsonrpc/0 running <Task <JsonRpcTask {'params': {u'name': u'ab', u'domainType': 3, u'domainClass': 1, u'typeArgs': u'nqLWk5-wyfU-gvDF-Iadc-5g7M-5uyr-57dq78', u'version': u'4', u'storagedomainID': u'a78e0bc2-b52a-4709-913b-d17d93edf6f5'}, 'jsonrpc': '2.0', 'method': u'StorageDomain.create', 'id': u'3899b5b0-1758-41a6-8d80-f20ca96089f7'} at 0x2f82710> timeout=60, duration=240 at 0x2f828d0> task#=1486 at 0x2edd6d0> (executor:351)
2017-11-20 12:17:32,335+0800 INFO  (periodic/3) [vdsm.api] START repoStats(options=None) from=internal, task_id=ec403ecd-2b46-4430-a58d-9a5eb4451ed4 (api:46)
2017-11-20 12:17:32,336+0800 INFO  (periodic/3) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000916356', 'lastCheck': '2.1', 'valid': True}} from=internal, task_id=ec403ecd-2b46-4430-a58d-9a5eb4451ed4 (api:52)
2017-11-20 12:17:32,831+0800 INFO  (jsonrpc/2) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,45990, flow_id=2637c15, task_id=4b60a34a-4cd8-4a72-b2e0-4a65e8a53520 (api:46)
2017-11-20 12:17:32,831+0800 INFO  (jsonrpc/2) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000916356', 'lastCheck': '2.6', 'valid': True}} from=::ffff:10.100.254.17,45990, flow_id=2637c15, task_id=4b60a34a-4cd8-4a72-b2e0-4a65e8a53520 (api:52)
2017-11-20 12:17:32,841+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.02 seconds (__init__:539)
2017-11-20 12:17:34,042+0800 DEBUG (periodic/0) [virt.sampling.VMBulkSampler] sampled timestamp 4529425.16 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:17:35,453+0800 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:17:44,674+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:17:47,374+0800 INFO  (periodic/3) [vdsm.api] START repoStats(options=None) from=internal, task_id=1867c3c4-08d5-4a27-9b0e-0e91610297ab (api:46)
2017-11-20 12:17:47,375+0800 INFO  (periodic/3) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000932822', 'lastCheck': '7.1', 'valid': True}} from=internal, task_id=1867c3c4-08d5-4a27-9b0e-0e91610297ab (api:52)
2017-11-20 12:17:48,876+0800 INFO  (jsonrpc/5) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,45990, flow_id=10c224e1, task_id=f3345d09-5feb-4110-b8b4-ed50791bdcd4 (api:46)
2017-11-20 12:17:48,877+0800 INFO  (jsonrpc/5) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000800942', 'lastCheck': '1.5', 'valid': True}} from=::ffff:10.100.254.17,45990, flow_id=10c224e1, task_id=f3345d09-5feb-4110-b8b4-ed50791bdcd4 (api:52)
2017-11-20 12:17:48,888+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:17:49,052+0800 DEBUG (periodic/3) [virt.sampling.VMBulkSampler] sampled timestamp 4529440.17 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:17:50,476+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2017-11-20 12:17:55,803+0800 INFO  (jsonrpc/0) [storage.LVM] Creating LV (vg=a78e0bc2-b52a-4709-913b-d17d93edf6f5, lv=inbox, size=16m, activate=True, contiguous=False, initialTags=()) (lvm:1147)
2017-11-20 12:18:00,680+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01 seconds (__init__:539)
2017-11-20 12:18:02,412+0800 INFO  (periodic/3) [vdsm.api] START repoStats(options=None) from=internal, task_id=7d179931-2131-46cb-90ea-a1f540d2a289 (api:46)
2017-11-20 12:18:02,413+0800 INFO  (periodic/3) [vdsm.api] FINISH repoStats return={u'2898fb98-76d5-451c-bde8-1c9faedea45e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000796331', 'lastCheck': '2.2', 'valid': True}} from=internal, task_id=7d179931-2131-46cb-90ea-a1f540d2a289 (api:52)
2017-11-20 12:18:04,060+0800 DEBUG (periodic/0) [virt.sampling.VMBulkSampler] sampled timestamp 4529455.17 elapsed 0.000 acquired True domains all (sampling:508)
2017-11-20 12:18:04,975+0800 INFO  (jsonrpc/3) [vdsm.api] START repoStats(options=None) from=::ffff:10.100.254.17,45990, flow_id=26b559d3, task_id=0c4ae29b-d669-411a-9228-e3c31a596790 (api:46)

Comment 1 Yaniv Kaul 2017-11-20 20:02:42 UTC
Can you please attach the output of the ovirt log collector? The VDSM log from cloudstack02'; host: '10.100.254.9' might be useful.

Comment 2 alex 2017-11-21 04:27:24 UTC
Created attachment 1356359 [details]
vdsm.log.

 i short it because the file too big.

Comment 3 Yaniv Kaul 2017-11-21 07:50:43 UTC
(In reply to alex from comment #2)
> Created attachment 1356359 [details]
> vdsm.log.
> 
>  i short it because the file too big.

So place it in some share. We can't help without the logs.

Comment 4 alex 2017-11-21 08:32:24 UTC
Created attachment 1356452 [details]
whole day log

Comment 5 alex 2017-11-21 08:34:55 UTC
i'm sure that the plane text log includes the printout when the 'problem' happens.

i also attach an xz vdsm.log.

please help me out, thanks.

Comment 6 Allon Mureinik 2017-11-21 10:21:05 UTC
Neither of the logs contains the vgcreate call.

Can you please retry the scenario and attach fresh vdsm.log and engine.log?
Thanks!

Comment 7 alex 2017-11-23 02:12:17 UTC
Created attachment 1357952 [details]
reproduce issue

reproduce issue.

Comment 8 alex 2017-11-23 02:12:49 UTC
Created attachment 1357954 [details]
reproduce issue

Comment 9 alex 2017-11-23 02:13:46 UTC
Created attachment 1357955 [details]
screenshot of issue

Comment 10 alex 2017-11-23 02:14:13 UTC
Created attachment 1357956 [details]
screenshot of issue

Comment 11 alex 2017-11-23 02:34:38 UTC
Dear Allon,

First, i reproduct issue again and collect vdsm.log/engine.log and two screenshots. Then uploaded them already.

I check the vdsm.log, you are right. there isn't any pvcreate and vgcreate clue.
After i take a look into /usr/share/vdsm/storage/lvm.py, here is pieces of code:

def createLV(vgName, lvName, size, activate=True, contiguous=False,
             initialTags=()):
    """
    Size units: MB (1024 ** 2 = 2 ** 20)B.
    """
    # WARNING! From man vgs:
    # All sizes are output in these units: (h)uman-readable, (b)ytes,
    # (s)ectors, (k)ilobytes, (m)egabytes, (g)igabytes, (t)erabytes,
    # (p)etabytes, (e)xabytes.
    # Capitalise to use multiples of 1000 (S.I.) instead of 1024.

    log.info("Creating LV (vg=%s, lv=%s, size=%sm, activate=%s, "
             "contiguous=%s, initialTags=%s)",
             vgName, lvName, size, activate, contiguous, initialTags)
    cont = {True: "y", False: "n"}[contiguous]
    cmd = ["lvcreate"]
    cmd.extend(LVM_NOBACKUP)

log.info func is there

But
def createVG(vgName, devices, initialTag, metadataSize, force=False):
    pvs = [_fqpvname(pdev) for pdev in _normalizeargs(devices)]
    _checkpvsblksize(pvs)

    _initpvs(pvs, metadataSize, force)
    # Activate the 1st PV metadata areas
    cmd = ["pvchange", "--metadataignore", "n"]
    cmd.append(pvs[0])
    rc, out, err = _lvminfo.cmd(cmd, tuple(pvs))
    if rc != 0:
        raise se.PhysDevInitializationError(pvs[0])

    options = ["--physicalextentsize", "%dm" % VG_EXTENT_SIZE_MB]
    if initialTag:
        options.extend(("--addtag", initialTag))
    cmd = ["vgcreate"] + options + [vgName] + pvs
    rc, out, err = _lvminfo.cmd(cmd, tuple(pvs))
    if rc == 0:
        _lvminfo._invalidatepvs(pvs)
        _lvminfo._invalidatevgs(vgName)
        log.debug("Cache after createvg %s", _lvminfo._vgs)
    else:
        raise se.VolumeGroupCreateError(vgName, pvs)

it seems no loggin while execute createVG.

[root@cloudstack01 storage]# rpm -qa|grep vdsm
vdsm-cli-4.19.37-1.el7.centos.noarch
vdsm-jsonrpc-4.19.37-1.el7.centos.noarch
vdsm-hook-floppy-4.19.37-1.el7.centos.noarch
vdsm-python-4.19.37-1.el7.centos.noarch
vdsm-yajsonrpc-4.19.37-1.el7.centos.noarch
vdsm-api-4.19.37-1.el7.centos.noarch
vdsm-hook-vmfex-dev-4.19.37-1.el7.centos.noarch
vdsm-xmlrpc-4.19.37-1.el7.centos.noarch
vdsm-jsonrpc-java-1.3.13-2.el7.centos.noarch
vdsm-4.19.37-1.el7.centos.x86_64

Also, after alert dialog popup. i use pvscan/vgscan/lvscan, i get these:
[root@cloudstack01 storage]# pvs
  PV               VG                                   Fmt  Attr PSize  PFree
  /dev/mapper/lun1 ebc70330-dffc-411e-8086-65a81cad6442 lvm2 a--  <2.00t 1.99t
[root@cloudstack01 storage]# vgs
  VG                                   #PV #LV #SN Attr   VSize  VFree
  ebc70330-dffc-411e-8086-65a81cad6442   1   7   0 wz--n- <2.00t 1.99t
[root@cloudstack01 storage]# lvs
  LV       VG                                   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  ids      ebc70330-dffc-411e-8086-65a81cad6442 -wi-a----- 128.00m                                                    
  inbox    ebc70330-dffc-411e-8086-65a81cad6442 -wi-a----- 128.00m                                                    
  leases   ebc70330-dffc-411e-8086-65a81cad6442 -wi-a-----   2.00g                                                    
  master   ebc70330-dffc-411e-8086-65a81cad6442 -wi-a-----   1.00g                                                    
  metadata ebc70330-dffc-411e-8086-65a81cad6442 -wi-a----- 512.00m                                                    
  outbox   ebc70330-dffc-411e-8086-65a81cad6442 -wi-a----- 128.00m                                                    
  xleases  ebc70330-dffc-411e-8086-65a81cad6442 -wi-a-----   1.00g  


BTW, i joined ovirt IRC, registed to nickserv, verified and identified. When i join the #ovirt channel, i got set mode +R. But it seems nobody speak in the channel, and nobody response to my hello. Anything wrong with me?

Thanks.

Comment 12 alex 2017-11-23 02:35:13 UTC
Created attachment 1357957 [details]
reproduce issue

Comment 13 shani 2017-11-23 11:30:50 UTC
Hi Alex,

After a deep look on your logs, we suspect the problem is related to a slow network connection on your storage domain:

+ Taken from engine.log:
2017-11-23 09:34:45,331+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-137) [4313b70f] START, CreateStorageDomainVDSCommand(...), log id: 5d2883a4

2017-11-23 09:37:45,398+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-137) [4313b70f] FINISH, CreateStorageDomainVDSCommand, log id: 5d2883a4

2017-11-23 09:37:45,399+08 ERROR [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (default task-137) [4313b70f] Command 'org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues (Failed with error VDS_NETWORK_ERROR and code 5022)

Which can be inferred from your log that after a "3 minutes connection", the engine reached to a timeout.  
While:

+ Taken from vdsm log:
2017-11-23 09:43:21,752+0800 INFO  (jsonrpc/0) [vdsm.api] FINISH createStorageDomain return=None from=::ffff:10.100.254.17,43746, flow_id=4313b70f, task_id=a155275c-76da-49e8-87b1-a81043d06c45 (api:52)

Which means that the storage domain's creation was finished by the VDSM only ~6 minutes later and therefore the storage domain couldn't be created.

We recommended you to check the cause for a slow transaction between engine and vdsm.

Also, in order to try creating that storage domain, you can increase the VDStimeout by exceuting those command from your ovirt-engine's directory:
(time unit is seconds)

$ ./bin/engine-config -g vdsTimeout
> vdsTimeout: 180 version: general

$ ./bin/engine-config -s vdsTimeout=600
$ ./bin/engine-config -g vdsTimeout
> vdsTimeout: 600 version: general

Comment 14 alex 2017-11-25 16:22:30 UTC
(In reply to shani from comment #13)
> Hi Alex,
> 
> After a deep look on your logs, we suspect the problem is related to a slow
> network connection on your storage domain:
> 
> + Taken from engine.log:
> 2017-11-23 09:34:45,331+08 INFO 
> [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand]
> (default task-137) [4313b70f] START, CreateStorageDomainVDSCommand(...), log
> id: 5d2883a4
> 
> 2017-11-23 09:37:45,398+08 INFO 
> [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand]
> (default task-137) [4313b70f] FINISH, CreateStorageDomainVDSCommand, log id:
> 5d2883a4
> 
> 2017-11-23 09:37:45,399+08 ERROR
> [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand]
> (default task-137) [4313b70f] Command
> 'org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand'
> failed: EngineException:
> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException:
> VDSGenericException: VDSNetworkException: Message timeout which can be
> caused by communication issues (Failed with error VDS_NETWORK_ERROR and code
> 5022)
> 
> Which can be inferred from your log that after a "3 minutes connection", the
> engine reached to a timeout.  
> While:
> 
> + Taken from vdsm log:
> 2017-11-23 09:43:21,752+0800 INFO  (jsonrpc/0) [vdsm.api] FINISH
> createStorageDomain return=None from=::ffff:10.100.254.17,43746,
> flow_id=4313b70f, task_id=a155275c-76da-49e8-87b1-a81043d06c45 (api:52)
> 
> Which means that the storage domain's creation was finished by the VDSM only
> ~6 minutes later and therefore the storage domain couldn't be created.
> 
> We recommended you to check the cause for a slow transaction between engine
> and vdsm.
> 
> Also, in order to try creating that storage domain, you can increase the
> VDStimeout by exceuting those command from your ovirt-engine's directory:
> (time unit is seconds)
> 
> $ ./bin/engine-config -g vdsTimeout
> > vdsTimeout: 180 version: general
> 
> $ ./bin/engine-config -s vdsTimeout=600
> $ ./bin/engine-config -g vdsTimeout
> > vdsTimeout: 600 version: general

Hi, Shani

I followed your instruction, the timer expired after 600s, and problem changed to another one, internal error.

It seems i have to fix my slow storage problems. I reinstall the host from scrach and reconfig my dell IPSAN. Now, pvs/vgs/lvs commands running very fast. 

I reinstall ovirt-engine and ovirt-host again, create data storage domain with iscsi. Great, very thing smooth. Meanwhile, ovirt-iso-uploader is faster than before, 12MB vs 100Kb. Although 12MB not good, but it's not a big deal so far, because i'm just play ovirt in the testbed instead of production environment.

Thanks alot, i think this tichet should be closed.


Note You need to log in before you can comment on or make changes to this bug.