Skip to main content

Check and list luns attached to HBA in RHEL6

This article will show you the mapping from physical HBA card to luns, I use SAN as example below, in general, it's also applys to any other devices whichever use sysfs, for example direct sas connect.

First, use lspci get HBA card  info installed on the host

# lspci | grep Fibre
15:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
15:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)

HBA detail info

# lspci -v -s 15:00.0
15:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
    Subsystem: QLogic Corp. Device 015d
    Physical Slot: 2
    Flags: bus master, fast devsel, latency 0, IRQ 24
    I/O ports at 2200 [size=256]
    Memory at 97b00000 (64-bit, non-prefetchable) [size=16K]
    Expansion ROM at 90000000 [disabled] [size=256K]
    Capabilities: [44] Power Management version 3
    Capabilities: [4c] Express Endpoint, MSI 00
    Capabilities: [88] MSI: Enable- Count=1/32 Maskable- 64bit+
    Capabilities: [98] Vital Product Data
    Capabilities: [a0] MSI-X: Enable+ Count=2 Masked-
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [138] Power Budgeting <?>
    Kernel driver in use: qla2xxx
    Kernel modules: qla2xxx
It tells you that one HBA card in PCI slot2, two FC ports

Find pci slot and scsi_host mapping

# ls -l /sys/class/scsi_host
total 0
...
lrwxrwxrwx 1 root root 0 Oct  9 12:58 host4 -> ../../devices/pci0000:00/0000:00:1f.5/host4/scsi_host/host4
lrwxrwxrwx 1 root root 0 Oct  9 12:58 host5 -> ../../devices/pci0000:00/0000:00:03.0/0000:15:00.0/host5/scsi_host/host5
lrwxrwxrwx 1 root root 0 Oct  9 12:58 host6 -> ../../devices/pci0000:00/0000:00:03.0/0000:15:00.1/host6/scsi_host/host6
You can easily tell that the first port of pcs slot 2(15:00.0) mapped to host5, the other mapped to host6

Find target luns by HBA port

Once you know the pci info of a HBA card, then you can find its port0 target luns or SAN devices.
Note 15:00.0 is used in this case
#find /sys/class/pci_bus/0000\:15/device/0000\:15\:00.0/host*/rport-*/target*/*/state | awk -F'/' '{print $11}' | sort
...
5:0:0:0
5:0:0:1
5:0:0:10
5:0:0:11
5:0:0:2
5:0:0:3
5:0:0:31
5:0:0:4
5:0:0:5
...
It should be consistent with the devices in /proc/scsi/scsi
#cat /proc/scsi/scsi | grep scsi5
...
Host: scsi5 Channel: 00 Id: 04 Lun: 04
Host: scsi5 Channel: 00 Id: 04 Lun: 05
Host: scsi5 Channel: 00 Id: 04 Lun: 06
Host: scsi5 Channel: 00 Id: 04 Lun: 07
Host: scsi5 Channel: 00 Id: 04 Lun: 08
Host: scsi5 Channel: 00 Id: 04 Lun: 09
Host: scsi5 Channel: 00 Id: 04 Lun: 10
Host: scsi5 Channel: 00 Id: 04 Lun: 11
Host: scsi5 Channel: 00 Id: 04 Lun: 31
...
Note: if use the command for sas direct attached devices, change 'rport' to 'port', same applies to the example below.

Find block devices

If you are only interested in block devices, like tape drive, disk lun or cd rom, here is a way similar.
# find   /sys/class/pci_bus/0000\:15/device/0000\:15\:00.0/host*/rport-*/target*/*/block/*/stat | awk -F'/' '{print $11,$13}'
5:0:0:0 sdb
5:0:0:1 sdc
5:0:0:10 sdl
5:0:0:11 sdm
5:0:0:2 sdd
5:0:0:3 sde
5:0:0:4 sdf
5:0:0:5 sdg
5:0:0:6 sdh

Reverse search, find the physical port that a lun connected to

/proc/scsi/scsi doesn't tell you which physical port target luns are connected to In the reverse look, for a given device name, for example /dev/sdd, how do I know which hba port it connected to?
# udevadm info --query=path --name /dev/sdd
/devices/pci0000:00/0000:00:03.0/0000:15:00.0/host5/rport-5:0-0/target5:0:0/5:0:0:2/block/sdd
Is is clear?
Or
Multipath also can tell you some hint
multipath -ll | grep sdd
  `- 5:0:0:2  sdd  8:48    active ready running

Or, look into /dev/disk/by-path/ tree
...
lrwxrwxrwx 1 root root 10 Aug 15 16:49 /dev/disk/by-path/pci-0000:15:00.1-fc-0x22430080e524ebac-lun-4 -> ../../sdcx
lrwxrwxrwx 1 root root 10 Aug 15 16:49 /dev/disk/by-path/pci-0000:15:00.1-fc-0x22430080e524ebac-lun-5 -> ../../sdcy
...

Get HBA WWNA info:

# for port in /sys/class/fc_host/host[0-9]/port_name; { echo -n "$port : "; cat $port; }
/sys/class/fc_host/host5/port_name : 0x21000024ff3434e4
/sys/class/fc_host/host6/port_name : 0x21000024ff3434e5

Dynamically insert and remove SCSI devices

If a newer kernel and the /proc file system is running, a non-busy device can be removed and installed 'on the fly'.

To hot remove a SCSI device:

    echo 1 > /sys/class/scsi_device/h:c:t:l/device/delete
    or
    echo 1 > /sys/block/<dev>/device/delete
    where <dev> is like sda or sdb etc..
    old way
    echo "scsi remove-single-device a b c d" > /proc/scsi/scsi
and similar, to hot add a SCSI device, do
    echo "c t l" >  /sys/class/scsi_host/host<h>/scan
    or use wildcard like below
    echo "- - -" > /sys/class/scsi_host/host<h>/scan

    old way
    echo "scsi add-single-device h c t l" > /proc/scsi/scsi
where
          h == hostadapter id (first one being 0)
          c == SCSI channel on hostadapter (first one being 0)
          t == ID
          l == LUN (first one being 0)

Comments

Post a Comment

Popular posts from this blog

Boot process hangs at dracut: Switching root

Environment Red Hat Enterprise Linux 6 Issue When server is booting the boot process hangs at  dracut: Switching root , and never displays anything else. Raw device-mapper: ioctl: 4.33.1-ioctl (2015-8-18) initialised: xx-xxxx@redhat.com udev: starting version 147 dracut: Starting plymouth daemon dracut: rd_NO_DM: removing DM RAID activation dracut: rd_NO_MD: removing MD RAID activation scsi0 : ata_piix scsi1 : ata_piix ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 Refined TSC clocksource calibration: 2599.999 MHz. virtio-pci 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 11 (level, high) -> IRQ 11 virtio-pci 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10 virtio-pci 0000:00:07.0: PCI INT A -> Link[LNKC] -> GSI 11 (level, high) -> IRQ 11 virtio-pci 0000:00:08.0: PCI INT A -> Link[LNKD] -> GSI 11 (level, high) -> IRQ 11 input: ImExPS/2 Gener
Post a Comment
Read more

Interpreting the output of lspci

On Linux, the lspci command lists all PCI devices connected to a host (a computer). Modern computers and PCI devices communicate with each other via PCI Express buses instead of the older Conventional PCI and PCI-X buses since the former buses offer many advantages such as higher throughput rates, smaller physical footprint and native hot plugging functionality. The high performance of the PCI Express bus has also led it to take over the role of other buses such as AGP ; it is also expected that SATA buses too will be replaced by PCI Express buses in the future as solid-state drives become faster and therefore demand higher throughputs from the bus they are attached to (see this article for more on this topic). As a first step, open a terminal and run lspci without any flags (note: lspci may show more information if executed with root privileges): lspci   This is the output I get on my laptop: 00:00.0 Host bridge: Intel Corporation Haswell-ULT DRAM Co
Post a Comment
Read more

How to get the SAN environment information and statistics on AIX, HP-UX, Linux, Solaris, and Windows

How to get the SAN environment information and statistics on AIX, HP-UX, Linux, Solaris, and Windows Description NetBackup SAN Client is supported on the Linux , Solaris, Windows, HP-UX and AIX operating systems.  These environments provide the initiator device driver which can login to the SAN client media server and mount an pseudo   target device “ARCHIVE PYTHON” so that the backup or restore can be use the fiber transport (FT).  If there is an issue in the SAN environment, it is necessary to get the information/statistics from the SAN fabric for analysis.  The commands below can be used, on the respective operating system, to gather the necessary information. If the outputs show many or steadily increasing error counts, that indicates one or more issues with  the fabric  infrastructure. The issue(s) can be caused by cabling, SFP, san switch, DWDM, HBA or ISL and those components will need to be analyzed and evaluated.  Linux Get the hardware information fo
Post a Comment
Read more