DM devices
Device Mapper (abbreviated DM) is the universal technology to access block devices by Linux. For a good overview over DM devices in relation to multipath see here. Ubuntu specific information on DM can be found here. For Debian see here.
All devices described so far are DM devices. Some of the mentioned devices have to be accessed as DM devices only in exceptional cases.
Warning
Especial care must be taken when directly manipulating DM devices. Direct modifications of DM devices are access on lowest level. Devices or information can easily be destroyed by such access. This is especially true when snapshots exist because the creation of snapshots adds further DM devices.
The only program to handle DM devices directly is dmsetup from the software package of the same name. It has quite a few sub-commands. For the best overview see the man page. Here are only the most important sub-commands described.
All DM devices can be displayed with the sub-command ls as follows:
root@kvm55:~# dmsetup ls 350002ac21d622071 (252, 4) 350002ac21d632071 (252, 2) vg-images (252, 0) vg3_nas-lv_nas5501 (252, 5) vg3_dbserver-lv_dbserver5501 (252, 3) vg-lv_dbslave5501 (252, 1)
The first column delivers the name of the device. The type of device can be derived from it:
350002ac21d622071, 350002ac21d632071: Multipath device
vg-images, vg3_nas-lv_nas5501 …: LVM logical volumes
The numbers in brackets are the major device number and the minor device number. The major device number of DM devices is always 252, while the minor device number represents the DM device number. The multipath device '350002ac21d622071' in the example above could be for instance accessed as DM device '/dev/dm-4'.
More detailed information on DM devices can be displayed by the sub-command info as follows:
root@kvm55:~# dmsetup info /dev/dm-2 Name: 350002ac21d632071 State: ACTIVE Read Ahead: 256 Tables present: LIVE Open count: 1 Event number: 1 Major, minor: 252, 2 Number of targets: 1 UUID: mpath-350002ac21d632071
For a detailed description of all displayed information see the man page of dmsetup. The most important information is 'Open count' - the number of open connection. When the number displayed here is greater than '0', the device is in use. Extreme care has to be taken when accessing DM devices in use by the means of dmsetup.
The assignment of DM devices to his connections to physical devices can be made visible with the help of the following commands and their options:
root@kvm55:~# lsscsi --device [4:2:0:0] disk LSI MR9240-4i 2.13 /dev/sda[8:0] [5:0:0:0] disk 3PARdata VV 3112 /dev/sdc[8:32] [5:0:0:1] disk 3PARdata VV 3112 /dev/sde[8:64] [5:0:0:254] enclosu 3PARdata SES 3112 - [6:0:0:0] disk 3PARdata VV 3112 /dev/sdb[8:16] [6:0:0:1] disk 3PARdata VV 3112 /dev/sdd[8:48] [6:0:0:254] enclosu 3PARdata SES 3112 -
root@kvm55:~# dmsetup ls --tree
350002ac21d622071 (252:4)
├─ (8:32)
└─ (8:16)
vg-images (252:0)
└─ (8:4)
vg3_nas-lv_nas5501 (252:5)
└─350002ac21d632071 (252:2)
├─ (8:64)
└─ (8:48)
vg3_dbserver-lv_dbserver5501 (252:3)
└─ (8:32)
vg-lv_dbslave5501 (252:1)
└─ (8:4)
The device '8:4' is a local hard disk. The other disks are iSCSI devices. Identical device labels like e.g. '8:32' in the output of both commands point to the same device.
Software RAID
Other DM devices are software RAID devices like e.g. '/dev/md0'. For unknown reasons the commands lsscsi or dmsetup don't display software raid devices while the command dmsetup ls –tree does.
This is an example from another KVM host with configured software RAID:
root@kvm62:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4]
md0 : active raid1 dm-4[1] dm-1[0]
976760768 blocks [2/2] [UU]
unused devices: <none>
root@kvm62:~# lsscsi --device [0:0:0:0] disk ATA M4-CT128M4SSD2 010G /dev/sda[8:0] [6:0:0:0] disk TOSHIBA MK1001TRKB 0106 /dev/sdb[8:16] [6:0:1:0] disk TOSHIBA MK1001TRKB 0106 /dev/sdc[8:32] [8:0:0:1] disk 3PARdata VV 3112 /dev/sdd[8:48] [8:0:0:254] enclosu 3PARdata SES 3112 - [9:0:0:1] disk 3PARdata VV 3112 /dev/sde[8:64] [9:0:0:254] enclosu 3PARdata SES 3112 -
root@kvm62:~# dmsetup ls --tree
350000393f8ca90dc-part1 (252:4)
└─350000393f8ca90dc (252:3)
└─ (8:32)
350000393f8ca9334-part1 (252:1)
└─350000393f8ca9334 (252:0)
└─ (8:16)
350002ac314162071 (252:8)
├─ (8:64)
└─ (8:48)
vg-lv_admin6201 (252:5)
└─ (9:0)
vg-images (252:2)
└─ (9:0)
vg-lv_backup6201 (252:6)
└─ (9:0)
vg3_lb-lv_lb6101 (252:7)
└─ (8:64)
Note the following interesting details of the example output:
- Partitions can also be DM devices. In the example above the DM devices 'dm-1' and 'dm-4' are partitions
- The DM devices of host 'kvm62' are not in perfect order. The device 'vg3_lb-lv_lb6101' should be bound to the multipath device '350002ac314162071'.
- The software RAID device '9:0' is displayed neither by the command
lsscsinor by the commanddmsetup ls.