# Nic pci passthrough acts differently on ONe 5.12

**URL:** <https://forum.opennebula.io/t/nic-pci-passthrough-acts-differently-on-one-5-12/8875>\
**Category:** Installation & Configuration\
**Created:** [October 12, 2020, 7:51am UTC](https://forum.opennebula.io/t/nic-pci-passthrough-acts-differently-on-one-5-12/8875 "2020-10-12T07:51:47Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![phil](https://avatars.discourse-cdn.com/v4/letter/p/e95f7d/32.png) [@phil](https://forum.opennebula.io/u/phil)\
**Post date:** [October 12, 2020, 7:51am UTC](https://forum.opennebula.io/t/nic-pci-passthrough-acts-differently-on-one-5-12/8875/1 "2020-10-12T07:51:47Z")

</div>

Hi all,

some of my virtual machines use a sriov virtual function as network interface. This vf is added to the vm using pci passthrough.

The network template looks like this:

```
  <TEMPLATE>
      <BRIDGE><![CDATA[sriov]]></BRIDGE>
      <BRIDGE_TYPE><![CDATA[linux]]></BRIDGE_TYPE>
      <DESCRIPTION><![CDATA[sriov virtual function]]></DESCRIPTION>
      <LABELS><![CDATA[mylabel]]></LABELS>
      <OUTER_VLAN_ID><![CDATA[]]></OUTER_VLAN_ID>
      <PHYDEV><![CDATA[]]></PHYDEV>
      <SECURITY_GROUPS><![CDATA[0]]></SECURITY_GROUPS>
      <VLAN_ID><![CDATA[]]></VLAN_ID>
      <VN_MAD><![CDATA[dummy]]></VN_MAD>
    </TEMPLATE>

```

and the (relevant part of the) vm template like this:

```
    <TEMPLATE>
      <PCI>
        <CLASS><![CDATA[0200]]></CLASS>
        <DEVICE><![CDATA[8090]]></DEVICE>
        <NETWORK><![CDATA[sriov-vf-network]]></NETWORK>
        <NETWORK_UNAME><![CDATA[oneadmin]]></NETWORK_UNAME>
        <SECURITY_GROUPS><![CDATA[0]]></SECURITY_GROUPS>
        <TYPE><![CDATA[NIC]]></TYPE>
        <VENDOR><![CDATA[1077]]></VENDOR>
      </PCI>
    </TEMPLATE>

```

With ONe 5.10 the resulting definition is:

```
  <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
    <devices>
      <hostdev mode='subsystem' type='pci' managed='yes'>
        <source>
          <address domain='0x0000' bus='0x43' slot='0x14' function='0x0'/>
        </source>
        <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0'/>
      </hostdev>
    </devices>
  </domain>

```

With ONe 5.12:

```
  <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
    <devices>
      <interface type='bridge'>
        <source bridge='sriov'/>
        <mac address='aa:05:ca:ba:98:00'/>
        <target dev='one-65-1'/>
        <model type='virtio'/>
      </interface>
    </devices>
  </domain>

```

Upon deployment virsh tries to create the pci-passtrough interface as a regular bridge, failing to find the “sriov” interface thus bailing out with

```
error: Cannot get interface MTU on 'sriov': No such device

```

Anyone got a clue on how to make pci passthrough work again?

---

<div class="post-metadata">

**Author:** ![phil](https://avatars.discourse-cdn.com/v4/letter/p/e95f7d/32.png) [@phil](https://forum.opennebula.io/u/phil)\
**Post date:** [October 12, 2020, 1:09pm UTC](https://forum.opennebula.io/t/nic-pci-passthrough-acts-differently-on-one-5-12/8875/2 "2020-10-12T13:09:35Z")

</div>

fwiw – passing the device without `type=nic` and `network=...` works fine even though the device doesn’t show up as network device in sunstone. I can live with that but I liked it better by seeing the device as nic.

---

<div class="post-metadata">

**Author:** ![rdiaz](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/rdiaz/32/6416_2.png) [@rdiaz](https://forum.opennebula.io/u/rdiaz)\
**Post date:** [October 20, 2020, 7:12pm UTC](https://forum.opennebula.io/t/nic-pci-passthrough-acts-differently-on-one-5-12/8875/3 "2020-10-20T19:12:05Z")

</div>

Hello @phil,

I think what is happening is this:

1. When you updated from 5.10 to 5.12 the configuration files: ` /var/lib/one/remotes/etc/im/kvm-probes.d/pci.conf` and `/etc/libvirt/qemu.conf` were overwritten with the default values.
2. As a result the `PCI DEVICES` are not monitored. You can check it with `onehost show <HOST_ID>`.
3. If you have configured a PCI passthrough NIC, and the PCI device is not available, OpenNebula configures an interface of type `bridge` and model `virtio` because the network driver used is ‘dummy’. Then, as the ‘srviov’ does not exist and driver is ‘dummy’ (OpenNebula does not create the bridge if not exist), you get the error: `error: Cannot get interface MTU on 'sriov': No such device`.

To solve this issue, restore your previous configuration in `/etc/libvirt/qemu.conf` [0] (restart livirtd service), and `/var/lib/one/remotes/etc/im/kvm-probes.d/pci.conf` [1] (restart opennebula and run `onehost forceupdate`). After that, try to instantiate the template again.

Hope this helps.

Cheers.

[0] [http://docs.opennebula.io/5.12/deployment/open\_cloud\_host\_setup/pci\_passthrough.html?highlight=passthrough#qemu-configuration](http://docs.opennebula.io/5.12/deployment/open_cloud_host_setup/pci_passthrough.html?highlight=passthrough#qemu-configuration)  
[1] [http://docs.opennebula.io/5.12/deployment/open\_cloud\_host\_setup/pci\_passthrough.html?highlight=passthrough#driver-configuration](http://docs.opennebula.io/5.12/deployment/open_cloud_host_setup/pci_passthrough.html?highlight=passthrough#driver-configuration)

---

<div class="post-metadata">

**Author:** ![phil](https://avatars.discourse-cdn.com/v4/letter/p/e95f7d/32.png) [@phil](https://forum.opennebula.io/u/phil)\
**Post date:** [October 21, 2020, 11:27am UTC](https://forum.opennebula.io/t/nic-pci-passthrough-acts-differently-on-one-5-12/8875/4 "2020-10-21T11:27:03Z")

</div>

Hi @rdiaz,

thanks for your answer. The `kvm-probes.d/pci.conf` is still the previous “working” one, it has not been overwritten by the package (the files are marked `%config(noreplace)` in the rpm) and would have been restored on my next ansible run.  
I see all pci devices with `onehost show` and I can assign them directly to the virtual machines, simply not as `TYPE = "NIC"`:

```auto
# pci.conf
:filter:
  - '1077:8070' # QLogic FastLinQ QL41000
  - '1077:8090' # QLogic FastLinQ QL41000 SRIOV VF

:short_address: []
:device_name: []

```

```auto
PCI DEVICES

   VM ADDR TYPE NAME
      03:00.0 1077:8070:0200 FastLinQ QL41000 Series 10/25/40/50GbE Controller
      03:00.1 1077:8070:0200 FastLinQ QL41000 Series 10/25/40/50GbE Controller
      85:00.0 1077:8070:0200 FastLinQ QL41000 Series 10/25/40/50GbE Controller
      85:00.1 1077:8070:0200 FastLinQ QL41000 Series 10/25/40/50GbE Controller
      85:00.2 1077:8070:0200 FastLinQ QL41000 Series 10/25/40/50GbE Controller
      85:00.3 1077:8070:0200 FastLinQ QL41000 Series 10/25/40/50GbE Controller
   84 85:02.0 1077:8090:0200 FastLinQ QL41000 Series Gigabit Ethernet Controlle
   84 85:02.1 1077:8090:0200 FastLinQ QL41000 Series Gigabit Ethernet Controlle
   84 85:02.2 1077:8090:0200 FastLinQ QL41000 Series Gigabit Ethernet Controlle
      85:02.3 1077:8090:0200 FastLinQ QL41000 Series Gigabit Ethernet Controlle

```

I see the passedthough nics in the VM as interface device and they’re working fine.  
FWIW: I did **not** blacklist the corresponding modules and I did **not** explicitly bind the pci devices to vfio-pci because it’s not needed:

```auto
$ ls -l /sys/bus/pci/drivers/vfio-pci/*:*
lrwxrwxrwx 1 root root 0 21. Okt 13:17 /sys/bus/pci/drivers/vfio-pci/0000:85:02.0 -> ../../../../devices/pci0000:80/0000:80:03.1/0000:85:02.0
lrwxrwxrwx 1 root root 0 21. Okt 13:17 /sys/bus/pci/drivers/vfio-pci/0000:85:02.1 -> ../../../../devices/pci0000:80/0000:80:03.1/0000:85:02.1
lrwxrwxrwx 1 root root 0 21. Okt 13:17 /sys/bus/pci/drivers/vfio-pci/0000:85:02.2 -> ../../../../devices/pci0000:80/0000:80:03.1/0000:85:02.2

```

This should not be relevant to OpenNebula, though.

¯\_(ツ)_/¯

---

<div class="post-metadata">

**Author:** ![rdiaz](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/rdiaz/32/6416_2.png) [@rdiaz](https://forum.opennebula.io/u/rdiaz)\
**Post date:** [October 22, 2020, 4:04pm UTC](https://forum.opennebula.io/t/nic-pci-passthrough-acts-differently-on-one-5-12/8875/5 "2020-10-22T16:04:17Z")

</div>

Hi @phil,

Please send the following information:

`onetemplate show -x <TEMPLATE_ID>`  
`onevm show -x <VM_ID`

For both cases: device with and without `type=nic` and `network=...`

Thanks.