Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1516115 - Test OSP13 with OVS-DPDK + BCM57414 NetXtreme-E NIC
Summary: Test OSP13 with OVS-DPDK + BCM57414 NetXtreme-E NIC
Keywords:
Status: MODIFIED
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: openvswitch
Version: 7.4
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: pre-dev-freeze
: ---
Assignee: Christian Trautman
QA Contact: ovs-qe@redhat.com
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-22 05:25 UTC by Robin Cernin
Modified: 2018-11-02 02:01 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:


Attachments (Terms of Use)

Description Robin Cernin 2017-11-22 05:25:31 UTC
As per the https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html-single/network_functions_virtualization_planning_guide/#tested_nics the Broadcom NICs are note tested.

And as per http://dpdk.org/doc/nics it is supported.

Environment:
* ProLiant DL380 Gen10
* RHEL 7.4 (3.10.0-693.1.1.el7.x86_64)
* NIC: Broadcom Limited BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller
       - Subsystem: Hewlett Packard Enterprise Ethernet 25Gb 2-port 631SFP28 Adapter


# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.4 (Maipo)

# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-693.1.1.el7.x86_64 root=/dev/mapper/rhel_example-root ro crashkernel=auto rd.lvm.lv=rhel_example/root rd.lvm.lv=rhel_example/swap rhgb default_hugepagesz=1GB hugepagesz=1GB hugepages=128 hugepagesz=2M
hugepages=4096 intel_iommu=on iommu=pt nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce idle=poll isolcpus=16-23 nohz_full=16-23 rcu_nocbs=16-23
console=ttyS0,115200n8 selinux=0 enforcing=0


=> Red Hat provided DPDK RPM packages:
# rpm -qa | grep dpdk
dpdk-16.11-3.el7.x86_64
dpdk-tools-16.11-3.el7.x86_64


# lsmod | grep vfio
vfio_pci               41267  0
vfio_iommu_type1       22300  0
vfio                   32367  2 vfio_iommu_type1,vfio_pci irqbypass              13503  2 kvm,vfio_pci


# dpdk-devbind --status

Network devices using DPDK-compatible driver ============================================
0000:86:00.0 'BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller' drv=vfio-pci unused=bnxt_en
0000:86:00.1 'BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller' drv=vfio-pci unused=bnxt_en


# lspci -vvs 0000:86:00.0 | grep -i numa
	NUMA node: 1

# taskset 0x10000 /usr/bin/testpmd -c 0xe0000  -n 4 -w 0000:86:00.0 -w 0000:86:00.1 -- -i --rss-ip --nb-cores=2
 --nb-ports=2 --forward-mode=mac --eth-peer=0,00:10:94:10:00:01
 --eth-peer=1,00:10:94:10:00:02 --rxd=4096 --txd=4096 --rxq=1 --txq=1
EAL: Detected 32 lcore(s)
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: No probed ethernet devices
Interactive-mode selected
EAL: Error - exiting with code: 1
  Cause: Invalid port 2



Using upstream dpdk-17.08
=========================

# usertools/dpdk-devbind.py --bind vfio-pci 0000:86:00.0 0000:86:00.1
# usertools/dpdk-devbind.py --status

Network devices using DPDK-compatible driver ============================================
0000:86:00.0 'BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller 16d7' drv=vfio-pci unused=
0000:86:00.1 'BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller 16d7' drv=vfio-pci unused=


# taskset 0x10000 ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xe0000  -n 4 -w 0000:86:00.0 -w 0000:86:00.1 -- -i --rss-ip --nb-cores=2
 --nb-ports=2 --forward-mode=mac --eth-peer=0,00:10:94:10:00:01
 --eth-peer=1,00:10:94:10:00:02 --rxd=4096 --txd=4096 --rxq=1 --txq=1
EAL: Detected 32 lcore(s)
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:86:00.0 on NUMA socket 1
EAL:   probe driver: 14e4:16d7 net_bnxt
EAL:   using IOMMU type 1 (Type 1)
PMD: Broadcom Cumulus driver bnxt

PMD: 1.8.0:20.6.135
PMD: Driver HWRM version: 1.7.7
PMD: BNXT Driver/HWRM API mismatch.
PMD: Firmware API version is newer than driver.
PMD: The driver may be missing features.
PMD: bnxt found at mem ef610000, node addr 0x7f65b3000000M
EAL: PCI device 0000:86:00.1 on NUMA socket 1
EAL:   probe driver: 14e4:16d7 net_bnxt
PMD: 1.8.0:20.6.135
PMD: Driver HWRM version: 1.7.7
PMD: BNXT Driver/HWRM API mismatch.
PMD: Firmware API version is newer than driver.
PMD: The driver may be missing features.
PMD: bnxt found at mem ef600000, node addr 0x7f65b3112000M Interactive-mode selected Set mac packet forwarding mode
USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=163456, size=2176, socket=0
USER1: create a new mbuf pool <mbuf_pool_socket_1>: n=163456, size=2176, socket=1 Configuring Port 0 (socket 1)
PMD: Port 0 Link Up - speed 25000 Mbps - full-duplex
PMD: VLAN Filtering: 1
PMD: VLAN Strip Offload: 1
Port 0: 9C:DC:71:B3:B8:A0
Configuring Port 1 (socket 1)
PMD: Port 1 Link Up - speed 25000 Mbps - full-duplex
PMD: VLAN Filtering: 1
PMD: VLAN Strip Offload: 1
Port 1: 9C:DC:71:B3:B8:A8
Checking link statuses...
Done
testpmd>

Comment 2 Timothy Redaelli 2017-11-27 15:54:57 UTC
The proper component for this bug should be "dpdk", but unlucky the project "OpenStack" doesn't have it.

Comment 3 Franck Baudin 2017-11-30 13:25:10 UTC
re-routing the BZ to the proper component, NIC certification is a RHEL followed/driven activity, and certification has to be done by the NIC vendor.


Note You need to log in before you can comment on or make changes to this bug.