summaryrefslogtreecommitdiff
path: root/drivers/infiniband
AgeCommit message (Collapse)Author
2012-05-22Merge tag 'rdma-for-3.5' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband Pull InfiniBand/RDMA changes from Roland Dreier: - Add ocrdma hardware driver for Emulex IB-over-Ethernet adapters - Add generic and mlx4 support for "raw" QPs: allow suitably privileged applications to send and receive arbitrary packets directly to/from the hardware - Add "doorbell drop" handling to the cxgb4 driver - A fairly large batch of qib hardware driver changes - A few fixes for lockdep-detected issues - A few other miscellaneous fixes and cleanups Fix up trivial conflict in drivers/net/ethernet/emulex/benet/be.h. * tag 'rdma-for-3.5' of git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband: (53 commits) RDMA/cxgb4: Include vmalloc.h for vmalloc and vfree IB/mlx4: Fix mlx4_ib_add() error flow IB/core: Fix IB_SA_COMP_MASK macro IB/iser: Fix error flow in iser ep connection establishment IB/mlx4: Increase the number of vectors (EQs) available for ULPs RDMA/cxgb4: Add query_qp support RDMA/cxgb4: Remove kfifo usage RDMA/cxgb4: Use vmalloc() for debugfs QP dump RDMA/cxgb4: DB Drop Recovery for RDMA and LLD queues RDMA/cxgb4: Disable interrupts in c4iw_ev_dispatch() RDMA/cxgb4: Add DB Overflow Avoidance RDMA/cxgb4: Add debugfs RDMA memory stats cxgb4: DB Drop Recovery for RDMA and LLD queues cxgb4: Common platform specific changes for DB Drop Recovery cxgb4: Detect DB FULL events and notify RDMA ULD RDMA/cxgb4: Drop peer_abort when no endpoint found RDMA/cxgb4: Always wake up waiters in c4iw_peer_abort_intr() mlx4_core: Change bitmap allocator to work in round-robin fashion RDMA/nes: Don't call event handler if pointer is NULL RDMA/nes: Fix for the ORD value of the connecting peer ...
2012-05-22Merge branch 'for-next' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/nab/target-pending Pull scsi-target changes from Nicholas Bellinger: "There has been lots of work in existing code in a number of areas this past cycle. The major highlights have been: * Removal of transport_do_task_sg_chain() from core + fabrics (Roland) * target-core: Removal of se_task abstraction from target-core and enforce hw_max_sectors for pSCSI backends (hch) * Re-factoring of iscsi-target tx immediate/response queues (agrover) * Conversion of iscsi-target back to using target core memory allocation logic (agrover) We've had one last minute iscsi-target patch go into for-next to address a nasty regression bug related to the target core allocation logic conversion from agrover that is not included in friday's linux-next build, but has been included in this series. On the new fabric module code front for-3.5, here is a brief status update for the three currently in flight this round: * usb-gadget target driver: Sebastian Siewior's driver for supporting usb-gadget target mode operation. This will be going out as a separate PULL request from target-pending/usb-target-merge with subsystem maintainer ACKs. There is one minor target-core patch in this series required to function. * sbp ieee-1394/firewire target driver: Chris Boot's driver for supportting the Serial Block Protocol (SBP) across IEEE-1394 Firewire hardware. This will be going out as a separate PULL request from target-pending/sbp-target-merge with two additional drivers/firewire/ patches w/ subsystem maintainer ACKs. * qla2xxx LLD target mode infrastructure changes + tcm_qla2xxx: The Qlogic >= 24xx series HW target mode LLD infrastructure patch-set and tcm_qla2xxx fabric driver. Support for FC target mode using qla2xxx LLD code has been officially submitted by Qlogic to James below, and is currently outstanding but not yet merged into scsi.git/for-next.. [PATCH 00/22] qla2xxx: Updates for scsi "misc" branch http://www.spinics.net/lists/linux-scsi/msg59350.html Note there are *zero* direct dependencies upon this for-next series for the qla2xxx LLD target + tcm_qla2xxx patches submitted above, and over the last days the target mode team has been tracking down an tcm_qla2xxx specific active I/O shutdown bug that appears to now be almost squashed for 3.5-rc-fixes." * 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/nab/target-pending: (47 commits) iscsi-target: Fix iov_count calculation bug in iscsit_allocate_iovecs iscsi-target: remove dead code in iscsi_check_valuelist_for_support target: Handle ATA_16 passthrough for pSCSI backend devices target: Add MI_REPORT_TARGET_PGS ext. header + implict_trans_secs attribute target: Fix MAINTENANCE_IN service action CDB checks to use lower 5 bits target: add support for the WRITE_VERIFY command target: make target_put_session void target: cleanup transport_execute_tasks() target: Remove max_sectors device attribute for modern se_task less code target: lock => unlock typo in transport_lun_wait_for_tasks target: Enforce hw_max_sectors for SCF_SCSI_DATA_SG_IO_CDB target: remove the t_se_count field in struct se_cmd target: remove the t_task_cdbs_ex_left field in struct se_cmd target: remove the t_task_cdbs_left field in struct se_cmd target: remove struct se_task target: move the state and execute lists to the command target: simplify command to task linkage target: always allocate a single task target: replace ->execute_task with ->execute_cmd target: remove the task_sectors field in struct se_task ...
2012-05-21Merge branches 'core', 'cxgb4', 'ipath', 'iser', 'lockdep', 'mlx4', 'nes', ↵Roland Dreier
'ocrdma', 'qib' and 'raw-qp' into for-linus
2012-05-21RDMA/cxgb4: Include vmalloc.h for vmalloc and vfreeVipul Pandya
Signed-off-by: Vipul Pandya <vipul@chelsio.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-19IB/mlx4: Fix mlx4_ib_add() error flowJack Morgenstein
We need to use a different loop index for mlx4_counter_alloc() and for device_create_file() iterations: the mlx4_counter_alloc() loop index is used in the error flow to free counters. If the same loop index is used for device_create_file() and, say, the device_create_file() loop fails on the first iteration, the allocated counters will not be freed. Signed-off-by: Jack Morgenstein <jackm@dev.mellanox.co.il> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-19IB/iser: Fix error flow in iser ep connection establishmentOr Gerlitz
The current error flow code was releasing the IB connection object and calling iscsi_destroy_endpoint() directly without going through the reference counting mechanism introduced in commit 39ff05d ("IB/iser: Enhance disconnection logic for multi-pathing"). This resulted in a double free of the iscsi endpoint object, which causes a kernel NULL pointer dereference. Fix that by plugging into the IB conn reference counting correctly. Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-19IB/mlx4: Increase the number of vectors (EQs) available for ULPsShlomo Pongratz
Enable IB ULPs to use a larger portion of the device EQs (which map to IRQs). The mlx4_ib driver follows the mlx4_core framework of the EQs to be divided among the device ports. In this scheme, for each IB port, the number of allocated EQs follows the number of cores, subject to other system constraints, such as number available MSI-X vectors. Signed-off-by: Shlomo Pongratz <shlomop@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-18RDMA/cxgb4: Add query_qp supportVipul Pandya
This allows querying the QP state before flushing. Signed-off-by: Vipul Pandya <vipul@chelsio.com> Signed-off-by: Steve Wise <swise@opengridcomputing.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-18RDMA/cxgb4: Remove kfifo usageVipul Pandya
Using kfifos for ID management was limiting the number of QPs and preventing NP384 MPI jobs. So replace it with a simple bitmap allocator. Remove IDs from the IDR tables before deallocating them. This bug was causing the BUG_ON() in insert_handle() to fire because the ID was getting reused before being removed from the IDR table. Signed-off-by: Vipul Pandya <vipul@chelsio.com> Signed-off-by: Steve Wise <swise@opengridcomputing.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-18RDMA/cxgb4: Use vmalloc() for debugfs QP dumpVipul Pandya
This allows dumping thousands of QPs. Log active open failures of interest. Signed-off-by: Vipul Pandya <vipul@chelsio.com> Signed-off-by: Steve Wise <swise@opengridcomputing.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-18RDMA/cxgb4: DB Drop Recovery for RDMA and LLD queuesVipul Pandya
Add module option db_fc_threshold which is the count of active QPs that trigger automatic db flow control mode. Automatically transition to/from flow control mode when the active qp count crosses db_fc_theshold. Add more db debugfs stats On DB DROP event from the LLD, recover all the iwarp queues. Signed-off-by: Vipul Pandya <vipul@chelsio.com> Signed-off-by: Steve Wise <swise@opengridcomputing.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-18RDMA/cxgb4: Disable interrupts in c4iw_ev_dispatch()Vipul Pandya
Use GFP_ATOMIC in _insert_handle() if ints are disabled. Don't panic if we get an abort with no endpoint found. Just log a warning. Signed-off-by: Vipul Pandya <vipul@chelsio.com> Signed-off-by: Steve Wise <swise@opengridcomputing.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-18RDMA/cxgb4: Add DB Overflow AvoidanceVipul Pandya
Get FULL/EMPTY/DROP events from LLD. On FULL event, disable normal user mode DB rings. Add modify_qp semantics to allow user processes to call into the kernel to ring doobells without overflowing. Add DB Full/Empty/Drop stats. Mark queues when created indicating the doorbell state. If we're in the middle of db overflow avoidance, then newly created queues should start out in this mode. Bump the C4IW_UVERBS_ABI_VERSION to 2 so the user mode library can know if the driver supports the kernel mode db ringing. Signed-off-by: Vipul Pandya <vipul@chelsio.com> Signed-off-by: Steve Wise <swise@opengridcomputing.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-18RDMA/cxgb4: Add debugfs RDMA memory statsVipul Pandya
Signed-off-by: Vipul Pandya <vipul@chelsio.com> Signed-off-by: Steve Wise <swise@opengridcomputing.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-15RDMA/cxgb4: Drop peer_abort when no endpoint foundSteve Wise
Log a warning and drop the abort message. Otherwise we will do a bogus wake_up() and crash. Signed-off-by: Steve Wise <swise@opengridcomputing.com> Cc: <stable@vger.kernel.org> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-15RDMA/cxgb4: Always wake up waiters in c4iw_peer_abort_intr()Steve Wise
This fixes a race where an ingress abort fails to wake up the thread blocked in rdma_init() causing the app to hang. Signed-off-by: Steve Wise <swise@opengridcomputing.com> Cc: <stable@vger.kernel.org> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-14RDMA/nes: Don't call event handler if pointer is NULLTatyana Nikolova
Don't call the ibqp event_handler pointer in the case it wasn't initialized. Signed-off-by: Tatyana Nikolova <Tatyana.E.Nikolova@intel.com> Signed-off-by: Donald Wood <Donald.E.Wood@intel.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-14RDMA/nes: Fix for the ORD value of the connecting peerTatyana Nikolova
Set ORD value of the connecting peer to be at least one in order to accommodate an RDMA READ Request message. Signed-off-by: Tatyana Nikolova <Tatyana.E.Nikolova@intel.com> Signed-off-by: Donald Wood <Donald.E.Wood@intel.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-14IB/qib: Add cache line awareness to qib_qp and qib_devdata structuresMike Marciniszyn
This patch reorganizes the QP and devdata files to be more cache line aware. qib_qp fields in particular are split into read-mostly, send, and receive fields. qib_devdata fields are split into read-mostly and read/write fields Testing has show that bidirectional tests improve by as much as 100% with this patch. Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-14IB/qib: MADs with misset M_Keys should return failureJim Foraker
If a MAD is sent directly to the local HCA rather than placed on a QP and the MAD fails M_Key checks, there is no means to generate a timeout for the client, which may hang. Instead we report IB_MAD_RESULT_FAILURE, which operates the same for on-the-wire packets, but will generate a send failure back to the client. Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com> Signed-off-by: Jim Foraker <foraker1@llnl.gov> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-14IB/qib: Fix M_Key lease timeout handlingJim Foraker
If a port has an M_Key lease set, the M_Key protect bits set to 1, and a SubnSet arrives with an invalid M_Key, an M_Key mismatch trap is generated, the lease timer begins as expected, and eventually the M_Key protect bits will be set back to 0 as per the spec. However, if any other SMP with an invalid M_Key arrives, the lease timer is expired and the M_Key protect bits remain in force. This is not according to to spec. In particular, C14-17 says that a lease timer that is underway is not affected by protection level checks (ie, at protection level 1, a SubnGet with a bad M_Key may be successful, but does not stop the timer), and C14-19 says that the timer shall stop when a valid M_Key has been received. C14-19 is the only compliance statement that specifies a stopping condition for the timer. This behavior is magnified if the port's Master SM LID attribute points at itself. In that case, the M_Key mismatch trap is sufficient to expire the timer, and the mkey lease attribute is rendered useless. Reviewed-by: Ram Vepa <ram.vepa@qlogic.com> Signed-off-by: Jim Foraker <foraker1@llnl.gov> Signed-off-by: Mike Marciniszyn <mike.marciniszyn@qlogic.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-14IB/qib: Fix QLE734X link cyclingMitko Haralanov
The SERDES was using the incorrect Frequency Loop Bandwidth setting causing the link to cycle through the Physical link negotiation state machine. Fixing the Frequency Loop Bandwidth setting in the SERDES helps the link come up faster and more reliably. Signed-off-by: Mitko Haralanov <mitko.haralanov@intel.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-14IB/qib: Display correct value for number of contextsMitko Haralanov
A "fix" for a bug with the number of contexts on a single-port board caused the calculation to be off by one, which causes problems with the upper layers. The same problem exists for number of free contexts, which is also fixed here. Signed-off-by: Mitko Haralanov <mitko.haralanov@intel.com> Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-14IB/qib: Correct ordering of reregister vs. port active eventsTodd Rimmer
When a port first goes active with SMA Set(PortInfo) and reregister bit set, the driver sends up the reregister event followed by a port active event. The problem is that in response to reregister event most apps try to issue a SA query of some sort, but that fails because port is not active. The qib driver needs to a trivial change to correct this behavior. This issue has been there for a while; however the recent serdes work has probably made the delay between the reregister event and the active event larger and hence opened the race far enough so that its being seen more often. The patch also changes the clientrereg local to a u8 and saves off the rereg bit into it. The code following the nested subn_get_portinfo() now restores that bit per o14-12.2.1 with a logical OR from that copy. Reviewed-by: Ram Vepa <ram.vepa@qlogic.com> Signed-off-by: Mike Marciniszyn <mike.marciniszyn@qlogic.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-14IB/qib: Optimize pio ack buffer allocationMike Marciniszyn
This patch optimizes pio buffer allocation in the kernel. For qib, kernel pio buffers are used for sending acks. The code to allocate the buffer would always start at 0 until it found a buffer. This means that an average of 64 comparisions were done on each allocate, since the busy bit won't be cleared until the bits are refreshed when buffers are exhausted. This patch adds two new fields in the devdata struct, last_pio and min_kernel_pio. last_pio is the last buffer that was allocated. min_kernel_pio is the lowest potential available buffer. min_kernel_pio is modifed as contexts are allocated and deallocted. Reviewed-by: Ramkrishna Vepa <ramkrishna.vepa@intel.com> Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-14IB/qib: Add prefetch for eager buffersMike Marciniszyn
Add a prefetch call when a packet has been stored. The nature of the prefetch is correctly determined by the alternatives mechanism. Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-11IB/core: Fix mismatch between locked and pinned pagesYishai Hadas
Commit bc3e53f682d9 ("mm: distinguish between mlocked and pinned pages") introduced a separate counter for pinned pages and used it in the IB stack. However, in ib_umem_get() the pinned counter is incremented, but ib_umem_release() wrongly decrements the locked counter. Fix this. Signed-off-by: Yishai Hadas <yishaih@mellanox.com> Reviewed-by: Christoph Lameter <cl@linux.com> Cc: <stable@vger.kernel.org> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08IB/mlx4: Replace printk(KERN_yyy...) with pr_yyy(...)Shlomo Pongratz
Signed-off-by: Shlomo Pongratz <shlomop@mellanox.com> [ Replace one more printk_once() with pr_info_once(). - Roland ] Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08IB/mlx4: Put priority bits in WQE of IBoE MLX QPOren Duer
Otherwise CM packets going over MLX QP1 get fixed scheduling priority 0. We want CM packets to get the same scheduling priority, and therefore map to the same SQ (Schedule Queue) and eventually TC (Traffic Class), as the application requested for the actual QP used for the connection. Signed-off-by: Oren Duer <oren@mellanox.co.il> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08IB/mlx4: Add raw packet QP supportOr Gerlitz
Implement raw packet QPs for Ethernet ports using the MLX transport (as done by the mlx4_en Ethernet netdevice driver). Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08IB/core: Add raw packet QP typeOr Gerlitz
IB_QPT_RAW_PACKET allows applications to build a complete packet, including L2 headers, when sending; on the receive side, the HW will not strip any headers. This QP type is designed for userspace direct access to Ethernet; for example by applications that do TCP/IP themselves. Only processes with the NET_RAW capability are allowed to create raw packet QPs (the name "raw packet QP" is supposed to suggest an analogy to AF_PACKET / SOL_RAW sockets). Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Reviewed-by: Sean Hefty <sean.hefty@intel.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08RDMA/ocrdma: Fix build with IPV6=nRoland Dreier
When IPV6 is not enabled: ERROR: "register_inet6addr_notifier" [drivers/infiniband/hw/ocrdma/ocrdma.ko] undefined! ERROR: "unregister_inet6addr_notifier" [drivers/infiniband/hw/ocrdma/ocrdma.ko] undefined! Fix this by wrapping the inet6 calls in #ifdef IPV6. Also make the ocrdma module depend on (IPV6 || IPV6=n) to forbid the case of modular ipv6 but built-in ocrdma (which can't work, because ocrdma calls ipv6 functions). Reported-by: Randy Dunlap <rdunlap@xenotime.net> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08RDMA/ocrdma: Tiny locking cleanupDan Carpenter
We only need to disable the IRQs one time. Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> [ Rename "wq_flags" to more conventional "flags." - Roland ] Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08RDMA/ocrdma: Fix check for NULL instead of IS_ERRDan Carpenter
The ocrdma_alloc_lkey() function never returns NULL pointers -- it returns ERR_PTRs. Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08RDMA/ocrdma: Don't sleep in atomic notifier handlerSasha Levin
Events sent to ocrdma_inet6addr_event() are sent from an atomic context, therefore we can't try to lock a mutex within the notifier callback. We could just switch the mutex to a spinlock since all it does it protect a list, but I've gone ahead and switched the list to use RCU instead. I couldn't fully test it since I don't have IB hardware, so if it doesn't fully work for some reason let me know and I'll switch it back to using a spinlock. Signed-off-by: Sasha Levin <levinsasha928@gmail.com> [ Fixed locking in ocrdma_add(). - Roland ] Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08RDMA/ocrdma: Remove write-only variablesRoland Dreier
Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08RDMA/ocrdma: Set event's device member in ocrdma_dispatch_ibevent()Roland Dreier
We need to set ib_evt.device, or else ib_dispatch_event() will crash when we call it for unaffiliated events (and consumers may get confused in their QP/CQ/SRQ event handler for affiliated events). Also fix sparse warning: drivers/infiniband/hw/ocrdma/ocrdma_hw.c:678:36: warning: Using plain integer as NULL pointer There's no need to clear ib_evt, since every member is initialized. Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08RDMA/ocrdma: Make needlessly global functions/structs staticRoland Dreier
Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08RDMA/ocrdma: Fix warnings about uninitialized variablesRoland Dreier
First, fix drivers/infiniband/hw/ocrdma/ocrdma_verbs.c: In function 'ocrdma_alloc_pd': drivers/infiniband/hw/ocrdma/ocrdma_verbs.c:371:17: warning: 'dpp_page_addr' may be used uninitialized in this function [-Wuninitialized] drivers/infiniband/hw/ocrdma/ocrdma_verbs.c:337:6: note: 'dpp_page_addr' was declared here which seems that it may border on a bug (the call to ocrdma_del_mmap() might conceivably do bad things if pd->dpp_enabled is not set and dpp_page_addr ends up with just the wrong value). Also take care of: drivers/infiniband/hw/ocrdma/ocrdma_hw.c: In function 'ocrdma_init_hw': drivers/infiniband/hw/ocrdma/ocrdma_hw.c:2587:5: warning: 'status' may be used uninitialized in this function [-Wuninitialized] drivers/infiniband/hw/ocrdma/ocrdma_hw.c:2549:17: note: 'status' was declared here which is only real if num_eq == 0, which should be impossible. Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08RDMA/ocrdma: Add driver for Emulex OneConnect IBoE RDMA adapterParav Pandit
Signed-off-by: Parav Pandit <parav.pandit@emulex.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08RDMA/cma: Fix lockdep false positive recursive lockingSean Hefty
The following lockdep problem was reported by Or Gerlitz <ogerlitz@mellanox.com>: [ INFO: possible recursive locking detected ] 3.3.0-32035-g1b2649e-dirty #4 Not tainted --------------------------------------------- kworker/5:1/418 is trying to acquire lock: (&id_priv->handler_mutex){+.+.+.}, at: [<ffffffffa0138a41>] rdma_destroy_i d+0x33/0x1f0 [rdma_cm] but task is already holding lock: (&id_priv->handler_mutex){+.+.+.}, at: [<ffffffffa0135130>] cma_disable_ca llback+0x24/0x45 [rdma_cm] other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(&id_priv->handler_mutex); lock(&id_priv->handler_mutex); *** DEADLOCK *** May be due to missing lock nesting notation 3 locks held by kworker/5:1/418: #0: (ib_cm){.+.+.+}, at: [<ffffffff81042ac1>] process_one_work+0x210/0x4a 6 #1: ((&(&work->work)->work)){+.+.+.}, at: [<ffffffff81042ac1>] process_on e_work+0x210/0x4a6 #2: (&id_priv->handler_mutex){+.+.+.}, at: [<ffffffffa0135130>] cma_disab le_callback+0x24/0x45 [rdma_cm] stack backtrace: Pid: 418, comm: kworker/5:1 Not tainted 3.3.0-32035-g1b2649e-dirty #4 Call Trace: [<ffffffff8102b0fb>] ? console_unlock+0x1f4/0x204 [<ffffffff81068771>] __lock_acquire+0x16b5/0x174e [<ffffffff8106461f>] ? save_trace+0x3f/0xb3 [<ffffffff810688fa>] lock_acquire+0xf0/0x116 [<ffffffffa0138a41>] ? rdma_destroy_id+0x33/0x1f0 [rdma_cm] [<ffffffff81364351>] mutex_lock_nested+0x64/0x2ce [<ffffffffa0138a41>] ? rdma_destroy_id+0x33/0x1f0 [rdma_cm] [<ffffffff81065a78>] ? trace_hardirqs_on_caller+0x11e/0x155 [<ffffffff81065abc>] ? trace_hardirqs_on+0xd/0xf [<ffffffffa0138a41>] rdma_destroy_id+0x33/0x1f0 [rdma_cm] [<ffffffffa0139c02>] cma_req_handler+0x418/0x644 [rdma_cm] [<ffffffffa012ee88>] cm_process_work+0x32/0x119 [ib_cm] [<ffffffffa0130299>] cm_req_handler+0x928/0x982 [ib_cm] [<ffffffffa01302f3>] ? cm_req_handler+0x982/0x982 [ib_cm] [<ffffffffa0130326>] cm_work_handler+0x33/0xfe5 [ib_cm] [<ffffffff81065a78>] ? trace_hardirqs_on_caller+0x11e/0x155 [<ffffffffa01302f3>] ? cm_req_handler+0x982/0x982 [ib_cm] [<ffffffff81042b6e>] process_one_work+0x2bd/0x4a6 [<ffffffff81042ac1>] ? process_one_work+0x210/0x4a6 [<ffffffff813669f3>] ? _raw_spin_unlock_irq+0x2b/0x40 [<ffffffff8104316e>] worker_thread+0x1d6/0x350 [<ffffffff81042f98>] ? rescuer_thread+0x241/0x241 [<ffffffff81046a32>] kthread+0x84/0x8c [<ffffffff8136e854>] kernel_thread_helper+0x4/0x10 [<ffffffff81366d59>] ? retint_restore_args+0xe/0xe [<ffffffff810469ae>] ? __init_kthread_worker+0x56/0x56 [<ffffffff8136e850>] ? gs_change+0xb/0xb The actual locking is fine, since we're dealing with different locks, but from the same lock class. cma_disable_callback() acquires the listening id mutex, whereas rdma_destroy_id() acquires the mutex for the new connection id. To fix this, delay the call to rdma_destroy_id() until we've released the listening id mutex. Signed-off-by: Sean Hefty <sean.hefty@intel.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08IB/uverbs: Lock SRQ / CQ / PD objects in a consistent orderRoland Dreier
Since XRC support was added, the uverbs code has locked SRQ, CQ and PD objects needed during QP and SRQ creation in different orders depending on the the code path. This leads to the (at least theoretical) possibility of deadlock, and triggers the lockdep splat below. Fix this by making sure we always lock the SRQ first, then CQs and finally the PD. ====================================================== [ INFO: possible circular locking dependency detected ] 3.4.0-rc5+ #34 Not tainted ------------------------------------------------------- ibv_srq_pingpon/2484 is trying to acquire lock: (SRQ-uobj){+++++.}, at: [<ffffffffa00af51b>] idr_read_uobj+0x2f/0x4d [ib_uverbs] but task is already holding lock: (CQ-uobj){+++++.}, at: [<ffffffffa00af51b>] idr_read_uobj+0x2f/0x4d [ib_uverbs] which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #2 (CQ-uobj){+++++.}: [<ffffffff81070fd0>] lock_acquire+0xbf/0xfe [<ffffffff81384f28>] down_read+0x34/0x43 [<ffffffffa00af51b>] idr_read_uobj+0x2f/0x4d [ib_uverbs] [<ffffffffa00af542>] idr_read_obj+0x9/0x19 [ib_uverbs] [<ffffffffa00b16c3>] ib_uverbs_create_qp+0x180/0x684 [ib_uverbs] [<ffffffffa00ae3dd>] ib_uverbs_write+0xb7/0xc2 [ib_uverbs] [<ffffffff810fe47f>] vfs_write+0xa7/0xee [<ffffffff810fe65f>] sys_write+0x45/0x69 [<ffffffff8138cdf9>] system_call_fastpath+0x16/0x1b -> #1 (PD-uobj){++++++}: [<ffffffff81070fd0>] lock_acquire+0xbf/0xfe [<ffffffff81384f28>] down_read+0x34/0x43 [<ffffffffa00af51b>] idr_read_uobj+0x2f/0x4d [ib_uverbs] [<ffffffffa00af542>] idr_read_obj+0x9/0x19 [ib_uverbs] [<ffffffffa00af8ad>] __uverbs_create_xsrq+0x96/0x386 [ib_uverbs] [<ffffffffa00b31b9>] ib_uverbs_detach_mcast+0x1cd/0x1e6 [ib_uverbs] [<ffffffffa00ae3dd>] ib_uverbs_write+0xb7/0xc2 [ib_uverbs] [<ffffffff810fe47f>] vfs_write+0xa7/0xee [<ffffffff810fe65f>] sys_write+0x45/0x69 [<ffffffff8138cdf9>] system_call_fastpath+0x16/0x1b -> #0 (SRQ-uobj){+++++.}: [<ffffffff81070898>] __lock_acquire+0xa29/0xd06 [<ffffffff81070fd0>] lock_acquire+0xbf/0xfe [<ffffffff81384f28>] down_read+0x34/0x43 [<ffffffffa00af51b>] idr_read_uobj+0x2f/0x4d [ib_uverbs] [<ffffffffa00af542>] idr_read_obj+0x9/0x19 [ib_uverbs] [<ffffffffa00b1728>] ib_uverbs_create_qp+0x1e5/0x684 [ib_uverbs] [<ffffffffa00ae3dd>] ib_uverbs_write+0xb7/0xc2 [ib_uverbs] [<ffffffff810fe47f>] vfs_write+0xa7/0xee [<ffffffff810fe65f>] sys_write+0x45/0x69 [<ffffffff8138cdf9>] system_call_fastpath+0x16/0x1b other info that might help us debug this: Chain exists of: SRQ-uobj --> PD-uobj --> CQ-uobj Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(CQ-uobj); lock(PD-uobj); lock(CQ-uobj); lock(SRQ-uobj); *** DEADLOCK *** 3 locks held by ibv_srq_pingpon/2484: #0: (QP-uobj){+.+...}, at: [<ffffffffa00b162c>] ib_uverbs_create_qp+0xe9/0x684 [ib_uverbs] #1: (PD-uobj){++++++}, at: [<ffffffffa00af51b>] idr_read_uobj+0x2f/0x4d [ib_uverbs] #2: (CQ-uobj){+++++.}, at: [<ffffffffa00af51b>] idr_read_uobj+0x2f/0x4d [ib_uverbs] stack backtrace: Pid: 2484, comm: ibv_srq_pingpon Not tainted 3.4.0-rc5+ #34 Call Trace: [<ffffffff8137eff0>] print_circular_bug+0x1f8/0x209 [<ffffffff81070898>] __lock_acquire+0xa29/0xd06 [<ffffffffa00af37c>] ? __idr_get_uobj+0x20/0x5e [ib_uverbs] [<ffffffffa00af51b>] ? idr_read_uobj+0x2f/0x4d [ib_uverbs] [<ffffffff81070fd0>] lock_acquire+0xbf/0xfe [<ffffffffa00af51b>] ? idr_read_uobj+0x2f/0x4d [ib_uverbs] [<ffffffff81070eee>] ? lock_release+0x166/0x189 [<ffffffff81384f28>] down_read+0x34/0x43 [<ffffffffa00af51b>] ? idr_read_uobj+0x2f/0x4d [ib_uverbs] [<ffffffffa00af51b>] idr_read_uobj+0x2f/0x4d [ib_uverbs] [<ffffffffa00af542>] idr_read_obj+0x9/0x19 [ib_uverbs] [<ffffffffa00b1728>] ib_uverbs_create_qp+0x1e5/0x684 [ib_uverbs] [<ffffffff81070fec>] ? lock_acquire+0xdb/0xfe [<ffffffff81070c09>] ? lock_release_non_nested+0x94/0x213 [<ffffffff810d470f>] ? might_fault+0x40/0x90 [<ffffffff810d470f>] ? might_fault+0x40/0x90 [<ffffffffa00ae3dd>] ib_uverbs_write+0xb7/0xc2 [ib_uverbs] [<ffffffff810fe47f>] vfs_write+0xa7/0xee [<ffffffff810ff736>] ? fget_light+0x3b/0x99 [<ffffffff810fe65f>] sys_write+0x45/0x69 [<ffffffff8138cdf9>] system_call_fastpath+0x16/0x1b Reported-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08IB/uverbs: Make lockdep output more readableRoland Dreier
Add names for our lockdep classes, so instead of having to decipher lockdep output with mysterious names: Chain exists of: key#14 --> key#11 --> key#13 lockdep will give us something nicer: Chain exists of: SRQ-uobj --> PD-uobj --> CQ-uobj Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08IB/ipath: Replace open-coded ARRAY_SIZE with macroMike Marciniszyn
Change sizeof(array)/sizeof(array[0]) to ARRAY_SIZE. Signed-off-by: Mike Marciniszyn <mike.marciniszyn@qlogic.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08IB/ipath: Replace open-coded ARRAY_SIZE with macroJim Cromie
Signed-off-by: Jim Cromie <jim.cromie@gmail.com> Acked-by: Mike Marciniszyn <mike.marciniszyn@intel.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08RDMA/cxgb4: Use dst parameter in import_ep()Steve Wise
Function import_ep() is incorrectly using ep->dst instead of the dst ptr passed in. This causes a crash when accepting new rdma connections becase ep->dst is not initialized yet. Signed-off-by: Steve Wise <swise@opengridcomputing.com> Cc: <stable@vger.kernel.org> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08IB/core: Use qp->usecnt to track multicast attach/detachOr Gerlitz
Just as we don't allow PDs, CQs, etc. to be destroyed if there are QPs that are attached to them, don't let a QP be destroyed if there are multicast group(s) attached to it. Use the existing usecnt field of struct ib_qp which was added by commit 0e0ec7e ("RDMA/core: Export ib_open_qp() to share XRC TGT QPs") to track this. Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2012-05-08Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/netDavid S. Miller
Conflicts: drivers/net/ethernet/intel/e1000e/param.c drivers/net/wireless/iwlwifi/iwl-agn-rx.c drivers/net/wireless/iwlwifi/iwl-trans-pcie-rx.c drivers/net/wireless/iwlwifi/iwl-trans.h Resolved the iwlwifi conflict with mainline using 3-way diff posted by John Linville and Stephen Rothwell. In 'net' we added a bug fix to make iwlwifi report a more accurate skb->truesize but this conflicted with RX path changes that happened meanwhile in net-next. In e1000e a conflict arose in the validation code for settings of adapter->itr. 'net-next' had more sophisticated logic so that logic was used. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-04-26Merge tag 'ib-fixes' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband Pull infiniband fixes from Roland Dreier: "A few fixes for regressions introduced in 3.4-rc1: - fix memory leak in mlx4 - fix two problems with new MAD response generation code" * tag 'ib-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband: IB/mlx4: Fix memory leaks in ib_link_query_port() IB/mad: Don't send response for failed MADs IB/mad: Set 'D' bit in response for unhandled MADs
2012-04-24Merge branches 'mad-response' and 'mlx4' into fixesRoland Dreier