From: Alex Moskalenko <mav@elserv.msk.su> To: hardware@lists.altlinux.org Subject: [Hardware] Работа с iSCSI HBA ServerEngines 10Gbe (be3) в HP ProLiant BL460c G7 Date: Sat, 07 Apr 2012 21:13:12 +0400 Message-ID: <4F8075A8.1070402@elserv.msk.su> (raw) Здравствуйте! Подскажите пожалуйста, получалось ли у кого-нибудь работать с iSCSI-инициатором из HP Proliant BL460c G7 на наших дистрибутивах? Интересует работа именно через be2iscsi, так как через стандартный iscsi-tcp все работает. На данный момент у меня на всех испробованных ядрах (kernel-image-ovz-el-2.6.32-alt57, kernel-image-std-def-3.0.26-alt0.M60P.1, kernel-image-std-def-3.2.13-alt1) получается одна и та же картина: Устройство опознается, загружается модуль be2iscsi; iscsiadm -m iface устройства be2iscsi видит; Попытка подключиться к таргету (как IET, так и LIO) заканчивается отказом сетевого интерфейса (ethX) этого адаптера и подвисанием iscsi-интерфейса (процессы /lib/udev/scsi_id висят в uninterruptible sleep). Соответственно, доступ к серверу по сети пропадает. То же самое более подробно: root@blade2 log]# uname -a Linux blade2.office 3.2.13-std-def-alt1 #1 SMP Tue Mar 27 09:33:17 UTC 2012 x86_64 GNU/Linux [root@blade2 log]# lspci -v 02:00.0 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 01) Subsystem: Hewlett-Packard Company NC553i 10Gb 2-port FlexFabric Converged Network Adapter Flags: bus master, fast devsel, latency 0, IRQ 24 Memory at fbff0000 (64-bit, non-prefetchable) [size=16K] Memory at fbfc0000 (64-bit, non-prefetchable) [size=128K] Memory at fbfa0000 (64-bit, non-prefetchable) [size=128K] [virtual] Expansion ROM at f8000000 [disabled] [size=256K] Capabilities: [40] Power Management version 3 Capabilities: [48] MSI-X: Enable+ Count=32 Masked- Capabilities: [c0] Express Endpoint, MSI 00 Capabilities: [b8] Vital Product Data Capabilities: [100] Advanced Error Reporting Capabilities: [180] Single Root I/O Virtualization (SR-IOV) Capabilities: [160] Alternative Routing-ID Interpretation (ARI) Capabilities: [168] Device Serial Number 44-1e-a1-ff-fe-52-11-00 Capabilities: [12c] #17 Kernel driver in use: be2net Kernel modules: be2net 02:00.1 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 01) Subsystem: Hewlett-Packard Company NC553i 10Gb 2-port FlexFabric Converged Network Adapter Flags: bus master, fast devsel, latency 0, IRQ 34 Memory at fbf90000 (64-bit, non-prefetchable) [size=16K] Memory at fbf60000 (64-bit, non-prefetchable) [size=128K] Memory at fbf40000 (64-bit, non-prefetchable) [size=128K] [virtual] Expansion ROM at f8040000 [disabled] [size=256K] Capabilities: [40] Power Management version 3 Capabilities: [48] MSI-X: Enable+ Count=32 Masked- Capabilities: [c0] Express Endpoint, MSI 00 Capabilities: [b8] Vital Product Data Capabilities: [100] Advanced Error Reporting Capabilities: [180] Single Root I/O Virtualization (SR-IOV) Capabilities: [160] Alternative Routing-ID Interpretation (ARI) Capabilities: [168] Device Serial Number 44-1e-a1-ff-fe-52-11-00 Capabilities: [12c] #17 Kernel driver in use: be2net Kernel modules: be2net 02:00.2 Mass storage controller: Emulex Corporation OneConnect 10Gb iSCSI Initiator (be3) (rev 01) Subsystem: Hewlett-Packard Company Device 3315 Flags: bus master, fast devsel, latency 0, IRQ 35 Memory at fbf30000 (64-bit, non-prefetchable) [size=16K] Memory at fbf00000 (64-bit, non-prefetchable) [size=128K] Memory at fbee0000 (64-bit, non-prefetchable) [size=128K] [virtual] Expansion ROM at f8080000 [disabled] [size=256K] Capabilities: [40] Power Management version 3 Capabilities: [48] MSI-X: Enable+ Count=32 Masked- Capabilities: [c0] Express Endpoint, MSI 00 Capabilities: [b8] Vital Product Data Capabilities: [100] Advanced Error Reporting Capabilities: [180] Single Root I/O Virtualization (SR-IOV) Capabilities: [160] Alternative Routing-ID Interpretation (ARI) Capabilities: [168] Device Serial Number 44-1e-a1-ff-fe-52-11-00 Capabilities: [12c] #17 Kernel driver in use: be2iscsi Kernel modules: be2iscsi 02:00.3 Mass storage controller: Emulex Corporation OneConnect 10Gb iSCSI Initiator (be3) (rev 01) Subsystem: Hewlett-Packard Company Device 3315 Flags: bus master, fast devsel, latency 0, IRQ 36 Memory at fbed0000 (64-bit, non-prefetchable) [size=16K] Memory at fbea0000 (64-bit, non-prefetchable) [size=128K] Memory at fbe80000 (64-bit, non-prefetchable) [size=128K] [virtual] Expansion ROM at f80c0000 [disabled] [size=256K] Capabilities: [40] Power Management version 3 Capabilities: [48] MSI-X: Enable+ Count=32 Masked- Capabilities: [c0] Express Endpoint, MSI 00 Capabilities: [b8] Vital Product Data Capabilities: [100] Advanced Error Reporting Capabilities: [180] Single Root I/O Virtualization (SR-IOV) Capabilities: [160] Alternative Routing-ID Interpretation (ARI) Capabilities: [168] Device Serial Number 44-1e-a1-ff-fe-52-11-00 Capabilities: [12c] #17 Kernel driver in use: be2iscsi Kernel modules: be2iscsi [root@blade2 ~]# iscsiadm --version iscsiadm version 2.0-872 [root@blade2 ~]# iscsiadm -m iface -I be2iscsi.44:1e:a1:52:11:01 # BEGIN RECORD 2.0-872 iface.iscsi_ifacename = be2iscsi.44:1e:a1:52:11:01 iface.net_ifacename = <empty> iface.ipaddress = 10.10.10.130 iface.hwaddress = 44:1e:a1:52:11:01 iface.transport_name = be2iscsi iface.initiatorname = iqn.2012-04.local.iscsi:blade2 # END RECORD iscsiadm -m node -I be2iscsi.44:1e:a1:52:11:01 --targetname iqn.2012-04.local.iscsi.test:test -p 10.10.10.1 --login После этой команды сеть обычно отваливается. С консоли видно, что команда с задержкой секунд 20 отрабатывает. В системе появляются диски соответственно LUNам на таргете. Любая попытка обращения к ним приводит к отправке процесса в uninterruptible sleep. В messages после попытки подключения таргета появляются следующие сообщения: Apr 7 18:58:02 blade2 kernel: [ 136.525847] (beiscsi_process_cq():1896):CQ Error notification for cmd.. code 18 cid 0x0 Apr 7 18:58:03 blade2 iscsid: Connection1:0 to [target: iqn.2012-04.local.iscsi.test:test, portal: 10.10.10.1,3260] through [ifac e: be2iscsi.44:1e:a1:52:11:01] is operational now Apr 7 18:58:23 blade2 kernel: [ 157.379351] (beiscsi_process_cq():1896):CQ Error notification for cmd.. code 18 cid 0x0 Apr 7 18:58:44 blade2 kernel: [ 178.402364] (beiscsi_process_cq():1896):CQ Error notification for cmd.. code 18 cid 0x0 Apr 7 18:58:59 blade2 kernel: [ 193.411235] connection1:0: detected conn error (1021) Apr 7 18:59:00 blade2 iscsid: Kernel reported iSCSI connection 1:0 error (1021) state (3) Apr 7 18:59:02 blade2 kernel: [ 196.475407] scsi 1:0:0:0: Direct-Access LIO-ORG FILEIO 4.0 PQ: 0 ANSI: 5 Apr 7 18:59:02 blade2 kernel: [ 196.475663] sd 1:0:0:0: Attached scsi generic sg1 type 0 Apr 7 18:59:02 blade2 kernel: [ 196.476116] sd 1:0:0:0: [sdb] 2097153 512-byte logical blocks: (1.07 GB/1.00 GiB) Apr 7 18:59:02 blade2 kernel: [ 196.476460] scsi 1:0:0:1: Direct-Access LIO-ORG FILEIO 4.0 PQ: 0 ANSI: 5 Apr 7 18:59:02 blade2 kernel: [ 196.476688] sd 1:0:0:1: Attached scsi generic sg2 type 0 Apr 7 18:59:02 blade2 kernel: [ 196.477255] sd 1:0:0:1: [sdc] 2097153 512-byte logical blocks: (1.07 GB/1.00 GiB) Apr 7 18:59:02 blade2 kernel: [ 196.477672] sd 1:0:0:0: [sdb] Write Protect is off Apr 7 18:59:02 blade2 kernel: [ 196.478294] sd 1:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA Apr 7 18:59:02 blade2 kernel: [ 196.478554] sd 1:0:0:1: [sdc] Write Protect is off Apr 7 18:59:02 blade2 kernel: [ 196.479093] sd 1:0:0:1: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA Apr 7 18:59:02 blade2 kernel: [ 196.481398] sdb: sdb1 Apr 7 18:59:02 blade2 kernel: [ 196.482642] sdc: sdc1 Apr 7 18:59:02 blade2 kernel: [ 196.484413] sd 1:0:0:0: [sdb] Attached SCSI disk Apr 7 18:59:02 blade2 kernel: [ 196.484797] sd 1:0:0:1: [sdc] Attached SCSI disk Apr 7 18:59:03 blade2 iscsid: connection1:0 is operational after recovery (1 attempts) Apr 7 18:59:03 blade2 kernel: [ 197.347283] be2net 0000:02:00.1: UE Detected!! Apr 7 18:59:04 blade2 kernel: [ 197.347289] be2net 0000:02:00.1: UE: RXULP2 bit set Apr 7 18:59:04 blade2 kernel: [ 197.347293] be2net 0000:02:00.1: Error in Card Detected! Cannot issue commands Apr 7 18:59:04 blade2 kernel: [ 197.451236] be2net 0000:02:00.0: UE Detected!! Apr 7 18:59:04 blade2 kernel: [ 197.451240] be2net 0000:02:00.0: UE: RXULP2 bit set Apr 7 18:59:04 blade2 kernel: [ 197.451244] be2net 0000:02:00.0: Error in Card Detected! Cannot issue commands Apr 7 18:59:12 blade2 kernel: [ 206.499211] connection1:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4294940032, last ping 4294 941282, now 4294942536 Apr 7 18:59:12 blade2 kernel: [ 206.499224] connection1:0: detected conn error (1011) Apr 7 18:59:13 blade2 iscsid: Kernel reported iSCSI connection 1:0 error (1011) state (3) При появлении сообщений от be2net перестает работать сетевой интерфейс. Ядро kernel-image-std-def-3.0.26-alt0.M60P.1 в дополнение к этому еще пишет следующие сообщения от be2net Apr 7 20:47:13 blade2 kernel: [ 76.027096] be2net 0000:02:00.0: Error in cmd completion - opcode 121, compl 2, extd 30 Apr 7 20:47:14 blade2 kernel: [ 76.679082] be2net 0000:02:00.1: Error in cmd completion - opcode 121, compl 2, extd 30 Из собственного BIOS контроллер видит таргеты и LUNы в них. Резюмируя написанное выше, хочу попросить помощи в запуске этой железяки в наших дистрибутивах. Заранее спасибо. PS iopen-iscsi версии 2.0-872 собрал сам. Наш 2.0-871 не работает с be2iscsi вообще. Дистрибутив - Server Light на p6, ядра из сизифа. -- WBR, Alex moskalenko
reply other threads:[~2012-04-07 17:13 UTC|newest] Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=4F8075A8.1070402@elserv.msk.su \ --to=mav@elserv.msk.su \ --cc=hardware@lists.altlinux.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
ALT Linux hardware support This inbox may be cloned and mirrored by anyone: git clone --mirror http://lore.altlinux.org/hardware/0 hardware/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 hardware hardware/ http://lore.altlinux.org/hardware \ hardware@altlinux.ru hardware@lists.altlinux.org hardware@lists.altlinux.ru hardware@lists.altlinux.com hardware@altlinux.org public-inbox-index hardware Example config snippet for mirrors. Newsgroup available over NNTP: nntp://lore.altlinux.org/org.altlinux.lists.hardware AGPL code for this site: git clone https://public-inbox.org/public-inbox.git