1Introduction 2============ 3 4The device-mapper "unstriped" target provides a transparent mechanism to 5unstripe a device-mapper "striped" target to access the underlying disks 6without having to touch the true backing block-device. It can also be 7used to unstripe a hardware RAID-0 to access backing disks. 8 9Parameters: 10<number of stripes> <chunk size> <stripe #> <dev_path> <offset> 11 12<number of stripes> 13 The number of stripes in the RAID 0. 14 15<chunk size> 16 The amount of 512B sectors in the chunk striping. 17 18<dev_path> 19 The block device you wish to unstripe. 20 21<stripe #> 22 The stripe number within the device that corresponds to physical 23 drive you wish to unstripe. This must be 0 indexed. 24 25 26Why use this module? 27==================== 28 29An example of undoing an existing dm-stripe 30------------------------------------------- 31 32This small bash script will setup 4 loop devices and use the existing 33striped target to combine the 4 devices into one. It then will use 34the unstriped target ontop of the striped device to access the 35individual backing loop devices. We write data to the newly exposed 36unstriped devices and verify the data written matches the correct 37underlying device on the striped array. 38 39#!/bin/bash 40 41MEMBER_SIZE=$((128 * 1024 * 1024)) 42NUM=4 43SEQ_END=$((${NUM}-1)) 44CHUNK=256 45BS=4096 46 47RAID_SIZE=$((${MEMBER_SIZE}*${NUM}/512)) 48DM_PARMS="0 ${RAID_SIZE} striped ${NUM} ${CHUNK}" 49COUNT=$((${MEMBER_SIZE} / ${BS})) 50 51for i in $(seq 0 ${SEQ_END}); do 52 dd if=/dev/zero of=member-${i} bs=${MEMBER_SIZE} count=1 oflag=direct 53 losetup /dev/loop${i} member-${i} 54 DM_PARMS+=" /dev/loop${i} 0" 55done 56 57echo $DM_PARMS | dmsetup create raid0 58for i in $(seq 0 ${SEQ_END}); do 59 echo "0 1 unstriped ${NUM} ${CHUNK} ${i} /dev/mapper/raid0 0" | dmsetup create set-${i} 60done; 61 62for i in $(seq 0 ${SEQ_END}); do 63 dd if=/dev/urandom of=/dev/mapper/set-${i} bs=${BS} count=${COUNT} oflag=direct 64 diff /dev/mapper/set-${i} member-${i} 65done; 66 67for i in $(seq 0 ${SEQ_END}); do 68 dmsetup remove set-${i} 69done 70 71dmsetup remove raid0 72 73for i in $(seq 0 ${SEQ_END}); do 74 losetup -d /dev/loop${i} 75 rm -f member-${i} 76done 77 78Another example 79--------------- 80 81Intel NVMe drives contain two cores on the physical device. 82Each core of the drive has segregated access to its LBA range. 83The current LBA model has a RAID 0 128k chunk on each core, resulting 84in a 256k stripe across the two cores: 85 86 Core 0: Core 1: 87 __________ __________ 88 | LBA 512| | LBA 768| 89 | LBA 0 | | LBA 256| 90 ---------- ---------- 91 92The purpose of this unstriping is to provide better QoS in noisy 93neighbor environments. When two partitions are created on the 94aggregate drive without this unstriping, reads on one partition 95can affect writes on another partition. This is because the partitions 96are striped across the two cores. When we unstripe this hardware RAID 0 97and make partitions on each new exposed device the two partitions are now 98physically separated. 99 100With the dm-unstriped target we're able to segregate an fio script that 101has read and write jobs that are independent of each other. Compared to 102when we run the test on a combined drive with partitions, we were able 103to get a 92% reduction in read latency using this device mapper target. 104 105 106Example dmsetup usage 107===================== 108 109unstriped ontop of Intel NVMe device that has 2 cores 110----------------------------------------------------- 111dmsetup create nvmset0 --table '0 512 unstriped 2 256 0 /dev/nvme0n1 0' 112dmsetup create nvmset1 --table '0 512 unstriped 2 256 1 /dev/nvme0n1 0' 113 114There will now be two devices that expose Intel NVMe core 0 and 1 115respectively: 116/dev/mapper/nvmset0 117/dev/mapper/nvmset1 118 119unstriped ontop of striped with 4 drives using 128K chunk size 120-------------------------------------------------------------- 121dmsetup create raid_disk0 --table '0 512 unstriped 4 256 0 /dev/mapper/striped 0' 122dmsetup create raid_disk1 --table '0 512 unstriped 4 256 1 /dev/mapper/striped 0' 123dmsetup create raid_disk2 --table '0 512 unstriped 4 256 2 /dev/mapper/striped 0' 124dmsetup create raid_disk3 --table '0 512 unstriped 4 256 3 /dev/mapper/striped 0' 125