Skip to main content

Check and list luns attached to HBA in RHEL6

This article will show you the mapping from physical HBA card to luns, I use SAN as example below, in general, it's also applys to any other devices whichever use sysfs, for example direct sas connect.

First, use lspci get HBA card  info installed on the host

# lspci | grep Fibre
15:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
15:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)

HBA detail info

# lspci -v -s 15:00.0
15:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
    Subsystem: QLogic Corp. Device 015d
    Physical Slot: 2
    Flags: bus master, fast devsel, latency 0, IRQ 24
    I/O ports at 2200 [size=256]
    Memory at 97b00000 (64-bit, non-prefetchable) [size=16K]
    Expansion ROM at 90000000 [disabled] [size=256K]
    Capabilities: [44] Power Management version 3
    Capabilities: [4c] Express Endpoint, MSI 00
    Capabilities: [88] MSI: Enable- Count=1/32 Maskable- 64bit+
    Capabilities: [98] Vital Product Data
    Capabilities: [a0] MSI-X: Enable+ Count=2 Masked-
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [138] Power Budgeting <?>
    Kernel driver in use: qla2xxx
    Kernel modules: qla2xxx
It tells you that one HBA card in PCI slot2, two FC ports

Find pci slot and scsi_host mapping

# ls -l /sys/class/scsi_host
total 0
...
lrwxrwxrwx 1 root root 0 Oct  9 12:58 host4 -> ../../devices/pci0000:00/0000:00:1f.5/host4/scsi_host/host4
lrwxrwxrwx 1 root root 0 Oct  9 12:58 host5 -> ../../devices/pci0000:00/0000:00:03.0/0000:15:00.0/host5/scsi_host/host5
lrwxrwxrwx 1 root root 0 Oct  9 12:58 host6 -> ../../devices/pci0000:00/0000:00:03.0/0000:15:00.1/host6/scsi_host/host6
You can easily tell that the first port of pcs slot 2(15:00.0) mapped to host5, the other mapped to host6

Find target luns by HBA port

Once you know the pci info of a HBA card, then you can find its port0 target luns or SAN devices.
Note 15:00.0 is used in this case
#find /sys/class/pci_bus/0000\:15/device/0000\:15\:00.0/host*/rport-*/target*/*/state | awk -F'/' '{print $11}' | sort
...
5:0:0:0
5:0:0:1
5:0:0:10
5:0:0:11
5:0:0:2
5:0:0:3
5:0:0:31
5:0:0:4
5:0:0:5
...
It should be consistent with the devices in /proc/scsi/scsi
#cat /proc/scsi/scsi | grep scsi5
...
Host: scsi5 Channel: 00 Id: 04 Lun: 04
Host: scsi5 Channel: 00 Id: 04 Lun: 05
Host: scsi5 Channel: 00 Id: 04 Lun: 06
Host: scsi5 Channel: 00 Id: 04 Lun: 07
Host: scsi5 Channel: 00 Id: 04 Lun: 08
Host: scsi5 Channel: 00 Id: 04 Lun: 09
Host: scsi5 Channel: 00 Id: 04 Lun: 10
Host: scsi5 Channel: 00 Id: 04 Lun: 11
Host: scsi5 Channel: 00 Id: 04 Lun: 31
...
Note: if use the command for sas direct attached devices, change 'rport' to 'port', same applies to the example below.

Find block devices

If you are only interested in block devices, like tape drive, disk lun or cd rom, here is a way similar.
# find   /sys/class/pci_bus/0000\:15/device/0000\:15\:00.0/host*/rport-*/target*/*/block/*/stat | awk -F'/' '{print $11,$13}'
5:0:0:0 sdb
5:0:0:1 sdc
5:0:0:10 sdl
5:0:0:11 sdm
5:0:0:2 sdd
5:0:0:3 sde
5:0:0:4 sdf
5:0:0:5 sdg
5:0:0:6 sdh

Reverse search, find the physical port that a lun connected to

/proc/scsi/scsi doesn't tell you which physical port target luns are connected to In the reverse look, for a given device name, for example /dev/sdd, how do I know which hba port it connected to?
# udevadm info --query=path --name /dev/sdd
/devices/pci0000:00/0000:00:03.0/0000:15:00.0/host5/rport-5:0-0/target5:0:0/5:0:0:2/block/sdd
Is is clear?
Or
Multipath also can tell you some hint
multipath -ll | grep sdd
  `- 5:0:0:2  sdd  8:48    active ready running

Or, look into /dev/disk/by-path/ tree
...
lrwxrwxrwx 1 root root 10 Aug 15 16:49 /dev/disk/by-path/pci-0000:15:00.1-fc-0x22430080e524ebac-lun-4 -> ../../sdcx
lrwxrwxrwx 1 root root 10 Aug 15 16:49 /dev/disk/by-path/pci-0000:15:00.1-fc-0x22430080e524ebac-lun-5 -> ../../sdcy
...

Get HBA WWNA info:

# for port in /sys/class/fc_host/host[0-9]/port_name; { echo -n "$port : "; cat $port; }
/sys/class/fc_host/host5/port_name : 0x21000024ff3434e4
/sys/class/fc_host/host6/port_name : 0x21000024ff3434e5

Dynamically insert and remove SCSI devices

If a newer kernel and the /proc file system is running, a non-busy device can be removed and installed 'on the fly'.

To hot remove a SCSI device:

    echo 1 > /sys/class/scsi_device/h:c:t:l/device/delete
    or
    echo 1 > /sys/block/<dev>/device/delete
    where <dev> is like sda or sdb etc..
    old way
    echo "scsi remove-single-device a b c d" > /proc/scsi/scsi
and similar, to hot add a SCSI device, do
    echo "c t l" >  /sys/class/scsi_host/host<h>/scan
    or use wildcard like below
    echo "- - -" > /sys/class/scsi_host/host<h>/scan

    old way
    echo "scsi add-single-device h c t l" > /proc/scsi/scsi
where
          h == hostadapter id (first one being 0)
          c == SCSI channel on hostadapter (first one being 0)
          t == ID
          l == LUN (first one being 0)

Comments

Post a Comment

Popular posts from this blog

Interpreting the output of lspci

On Linux, the lspci command lists all PCI devices connected to a host (a computer). Modern computers and PCI devices communicate with each other via PCI Express buses instead of the older Conventional PCI and PCI-X buses since the former buses offer many advantages such as higher throughput rates, smaller physical footprint and native hot plugging functionality. The high performance of the PCI Express bus has also led it to take over the role of other buses such as AGP ; it is also expected that SATA buses too will be replaced by PCI Express buses in the future as solid-state drives become faster and therefore demand higher throughputs from the bus they are attached to (see this article for more on this topic). As a first step, open a terminal and run lspci without any flags (note: lspci may show more information if executed with root privileges): lspci   This is the output I get on my laptop: 00:00.0 Host bridge: Intel Corporation Haswell-ULT DRA...
Post a Comment
Read more

How to Remove a Storage Device (LUN)

Before removing access to the storage device itself, it is advisable to back up data from the device first. Afterwards, flush I/O and remove all operating system references to the device. Stop all access to the device that has to be removed. Unmount the device. Remove the device from any md and LVM volume that is using it. If a multipath device is being removed, run  multipath -l  and take note of all the paths to the device. When this has been done, remove the multipath device: # multipath -f device   Use the following command to flush any outstanding I/O to all paths to the device: # blockdev –flushbufs device   Remove any reference to the device's path-based name, like  /dev/sd  or  /dev/disk/by-path  or the major:minor number, in applications, scripts, or utilities on the system. This is important to ensure that a different device, when added in the future, will not be mistaken for the current device. The fi...
Post a Comment
Read more

UNIX/Linux Advanced File Permissions - SUID,SGID and Sticky Bit

UNIX/Linux Advanced File Permissions - SUID,SGID and Sticky Bit After you have worked for a while with Linux you discover probably that there is much more to file permissions than just the "rwx" bits. When you look around in your file system you will see "s" and "t" $ ls -ld /tmp drwxrwxrwt 29 root root 36864 Mar 21 19:49 /tmp $ which passwd /usr/bin/passwd $ ls -l /usr/bin/passwd -rwsr-xr-x 1 root root 22984 Jan 6 2007 /usr/bin/passwd What is this "s" and "t" bit? The vector of permission bits is really 4 * 3 bits long. Yes there are 12 permission bits,not just 9.The first three bits are special and are frequently zero. And you almost always learn about the trailing 9 bits first.Some people stop there and never learn those first three bits. The forth permission bit is used only when a special mode of a file needs to be set. It has the value 4 for SUID, 2 for SGID and 1 for the sticky bit. The other 3 bits have ...
Post a Comment
Read more