Introduction

SmartIO enables applications to access PCIe devices located anywhere in a PCIe network. It’s implemented in such a way that the data path is purely hardware based and as such has very little overhead.

The SmartIO core provides the base for two user ended functionalities: Device lending and the SmartIO SISCI API Extension. Device lending allows unmodified kernel device drivers to access remote devices. For example, using device lending, an NVMe drive installed in node 4 can be borrowed and mounted on node 8. On the other hand, the SmartIO SISCI extensions enables the creation of custom drivers running in user space. For instance, the smartio_nvme example is a SmartIO SISCI example that implements a tiny part of an NVMe driver and talks directly with an NVMe drive.

Comparison of SmartIO SISCI and Device lending

Device lending:

• Unmodified device drivers (proprietary and open source)

• Device can only be used by single node at the same time

SmartIO SISCI:

• Multiple concurrent users

• Requires driver development or porting

• Can be used with other SISCI functionalities such as multicast or DMA

Core Concepts

In SmartIO, each Cluster Node can operate as a lender, borrower or both. A lender exports local PCIe devices to the cluster and enables them to be used by borrowers. When acting as a borrower, a node may borrow devices exported by lenders.