Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.

Bug 593620

Summary: performance is not improved with x2apic
Product: Red Hat Enterprise Linux 6 Reporter: Suqin Huang <shuang>
Component: qemu-kvmAssignee: Virtualization Maintenance <virt-maint>
Status: CLOSED NOTABUG QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: low    
Version: 6.0CC: gleb, john.cooper, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-05-25 04:49:49 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Description Flags
script for pipe test none

Description Suqin Huang 2010-05-19 10:15:59 UTC
Created attachment 415074 [details]
script for pipe test

Description of problem:

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. boot guest with x2apic (at least 2 vcpus, and enable ept in host)
2. command
/usr/libexec/qemu-kvm  -smp 2 -m 2G -drive file=/root/images/rhel6-32.qcow2,if=virtio,boot=on -net nic,vlan=0,macaddr=00:90:4a:01:00:37,model=virtio -net tap,vlan=0,script=/etc/qemu-ifup  -uuid `uuidgen` -no-hpet -usbdevice tablet  -rtc-td-hack  -startdate now -cpu qemu64,+x2apic -monitor stdio -vnc :1
3. run pipetest (attached), and get the result
4. boot guest without x2apic (same vcpu num as step1)
5. run pipetest, and get the result
6. compare the results.

Another testing:
run lmbench testing, and compare "Pipe latency"

Actual results:

with x2apic
56.19 usecs/loop.
56.45 usecs/loop.
57.05 usecs/loop.
56.48 usecs/loop.
55.45 usecs/loop.
55.13 usecs/loop.
53.89 usecs/loop.
54.55 usecs/loop.
55.93 usecs/loop.
57.98 usecs/loop.
56.59 usecs/loop.
54.89 usecs/loop.
54.25 usecs/loop.
54.25 usecs/loop.
56.51 usecs/loop.

without x2apic:
56.46 usecs/loop.
57.22 usecs/loop.
54.03 usecs/loop.
56.65 usecs/loop.
56.33 usecs/loop.
54.25 usecs/loop.
54.51 usecs/loop.
56.18 usecs/loop.
56.07 usecs/loop.
54.83 usecs/loop.
55.41 usecs/loop.
56.31 usecs/loop.
56.48 usecs/loop.
56.30 usecs/loop.
56.12 usecs/loop.
56.16 usecs/loop.

lmbench testing:

with x2apic:
Pipe latency: 72.7915 microseconds

without x2apic:
Pipe latency: 71.0693 microseconds 

Expected results:

Additional info:

1. host:
tree : RHEL6.0-20100511.3/  

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
stepping	: 4

flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid

2. guest: rhel6-32

Comment 2 Gleb Natapov 2010-05-22 18:40:35 UTC
As far as I see x2apic is available only on x86-64, so please retest with rhel6-64. Have you checked in the guest dmesg that x2apic was actually enabled prior to running benchmarks?

Comment 3 Suqin Huang 2010-05-24 03:21:46 UTC
retest with guest rhel6-64

1. with x2apic
[root@x64 ~]# dmesg | grep x2apic
Enabling x2apic
Enabled x2apic
Setting APIC routing to physical x2apic

50.93 usecs/loop.
44.41 usecs/loop.
50.23 usecs/loop.
50.34 usecs/loop.
50.56 usecs/loop.
49.80 usecs/loop.
49.93 usecs/loop.
48.71 usecs/loop.
49.02 usecs/loop.
51.95 usecs/loop.

2. without x2apic

78.32 usecs/loop.
75.71 usecs/loop.
79.79 usecs/loop.
79.01 usecs/loop.
78.26 usecs/loop.
80.96 usecs/loop.
78.66 usecs/loop.
78.80 usecs/loop.

Comment 4 Gleb Natapov 2010-05-24 05:22:21 UTC
So, I guess, we can close this one?

Comment 5 Suqin Huang 2010-05-25 04:49:49 UTC
close this bug as "NOT A BUG" according to comment 2 and comment 3

Comment 6 john cooper 2010-07-14 16:41:22 UTC
See also: