?Porting virtio to PowerVM Hypervisors KVM Forum 2010
Ricardo Marin Matinata – [email protected]
LTC Brazil
IBM Linux Technology Center
August 2010
Linux is a registered trademark of Linus Torvalds.
© 2010 IBM Corporation
?? ? ? ? ? ?
Background
Motivations
Virtual I/O on POWERVM Implementation Strategy Device Configuration Virtqueues
Expectations
© 2010 IBM Corporation
??
IBM Brazil entitled to incentive grants in Brazil, related to manufacturing of POWER Systems locally
Has to be POWER Systems related
Strong research “appeal”
?
?
Execution under responsibility of IBM LTC Brazil (arquitecture & PM), in partnership with Flextronics Institute.
Currently active
– –
Background
© 2010 IBM Corporation
?Porting virtIO to POWERVM
• Adds value to the platform by bringing interesting new devices, like viftFS
• Evaluates how well virtio maps to different virtualization models
• Builds team skills around virtualization -> give back to the ecosystem
Virtual Ethernet
Processor Resources
CPUs PPPP
Linux/AIX/IBMi
Power Hypervisor Memory
Virtual Storage
Resources
Memory
I/O Resources
I/O
Virtual TTY
MMMM AAAAA Service MMMM AAAAA Processor
© 2010 IBM Corporation
?Virtual FC
Virtual SCSI
Virtual Ethernet FC
Ethernet
Disk
Ethernet
Virtual I/O on POWERVM
VIOS #1
Virtual Virtual Ethernet SCSI
Linux – RHEL 5 dm-multipath
Linux – SLES 11 dm-multipath
VIOS #2
? Virtual I/O Architecture
? Mix of virtualized and/or real devices ? Multiple VIO Servers* supported
? Virtual SCSI
? ? ?
SAN
? Benefits
? Fewer adapters, I/O drawers, and
ports
? Improved speed to deployment
? Virtual Fibre Channel
? Utilizes N-Port ID Virtualization ? Simplifies storage managment
Virtual Ethernet
Virtual Virtual SCSI SCSI
Virtual Ethernet FC
Virtual FC
Virtual
POWER Hypervisor
Virtual SCSI, Fibre Channel, and DVD ? Virtual Ethernet
? VLAN and link aggregation support
Logical and physical volume virtual disks
Multi-path and redundancy options ? LPAR-to-LPAR virtual LANs
Virtual
? Shared Ethernet adapter failover
© 2010 IBM Corporation
?© 2010 IBM Corporation
•A Command/Response Queue (CRQ) facility which provides a pipe between partitions.
•An extended TCE table called the RTCE table which allows a partition to provide “windows” into the memory of its partition to its partner partition
•Remote DMA services that allow a server partition to transfer data to a partner partition’s memory via the RTCE table window panes.
?Implementation Details
VIO LPAR
Guest LPAR
block net
virtios.c
virtio_crq.c
virtio_ring.c virtio_vio.c
User Space
Kernel
© 2010 IBM Corporation
?VIO LPAR
Device Configuration
Guest LPAR
RDMA write to guest:
type
nvqs device_features config_size vqs_size
3
2
PROBE: TCE of guest table
1 Allocates Device Header and
RDMA write to guest: 6 config
5 DEVICE_ACKNOWLEDGE
7 DRIVER
9 DRIVER_OK
TCE map it:
u8 type, u8 num_vqs, u8 vqs_size, u32 device_features, u32 guest_features, u8 config_len, u8 device_status, u8 config[0]
Allocate config space 4 register_virtio_device()
virtio_dev_probe() finalize_features
10 RDMA copy from guest
guest_features
© 2010 IBM Corporation
virtio_vio_probe()
8
??
?
find_vqs
–
Expose TCEs for Descriptor Table, Available Ring and Used Ring
Virtqueues (plan)
Re-use vring
Hook HCALL_SEND_CRQ to vq.notify() – which is called by virtqueue_kick
? Should cause the host to RDMA copy-in Descriptor Table and Available Ring
–
–
–
–
vring_desc.addr should hold TCEs, not Guest Physicals (u64 is fine, changing semantics only)
vring’s add_buff should replace sg_phys() to sg_dma_address()
vring’s detach should dma_unmap_sg() on each freed descriptor
© 2010 IBM Corporation
??
?
?
?
Power Architecture Platform Requirements (PAPR)
www.power.org
–
References
“virtio: Towards a De-Facto Standard For Virtual I/O Devices”, Rusty Russel
Virtio PCI Card Specication v0.8.8 DRAFT, Rusty Russel
Kernel source tree
© 2010 IBM Corporation
Categories: News