Introduce a new lock to protect the list of fc_fcp_pkt structs in libfc
instead of using the host lock. This reduces the contention of this heavily
used lock, and I see up to a 25% performance gain in CPU bound small I/O
tests when scaling out across multiple quad-core CPUs.
The big win is in removing the host lock from the completion path
completely, as it does not need to be held around the call to scsi_done.
Signed-off-by: Chris Leech <christopher.leech@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
There are cases outside of our control that may result in a transmit
skb being linearized in dev_queue_xmit. There are a couple of bugs
in libfc/fcoe that can result in a panic at that point. This patch
contains two fixes to prevent those panics.
1) use fast cloning instead of shared skbs with dev_queue_xmit
dev_queue_xmit doen't want shared skbuffs being passed in, and
__skb_linearize will BUG if the skb is shared. FCoE is holding an extra
reference around the call to dev_queue_xmit, so that when it returns an
error code indicating the frame has been dropped it can maintain it's
own backlog and retransmit. Switch to using fast skb cloning for this
instead.
2) don't append compound pages as > PAGE_SIZE skb fragments
fc_fcp_send_data will append pages from a scatterlist to the nr_frags[]
if the netdev supports it. But, it's using > PAGE_SIZE compound pages
as a single skb_frag. In the highmem linearize case that page will be
passed to kmap_atomic to get a mapping to copy out of, but
kmap_atomic will only allow access to the first PAGE_SIZE part.
The memcpy will keep going and cause a page fault once is crosses the
first boundary.
If fc_fcp_send_data uses linear buffers from the start, it calls
kmap_atomic one PAGE_SIZE at a time. That same logic needs to be
applied when setting up skb_frags.
Signed-off-by: Chris Leech <christopher.leech@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
DID_NO_CONNECT is not a nice value to use for pkt alloc failures,
because you can probably retry and IO will become available again.
For the device reset callout, we do not want to set the scsi command
result for the above reason, and because we do not need to set
the scsi_cmd->result in this path. We and other drivers do not set it
for success for example, and we do not set it for other failure.
And scsi-ml does not send every command through this path, and it is
not expecting us to use the scsi_cmnd struct like a cmd coming thruogh
queuecommand. I think it is more for storage in case we need a cmd
struct for a tmf and to give us certain params like the LUN.
Patch was made over scsi-misc today.
Signed-off-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Adds last_can_queue_ramp_down_time and updates this on every
ramp down. If last_can_queue_ramp_down_time is not zero then
do ramp up on any IO completion in added fc_fcp_can_queue_ramp_up.
Reset last_can_queue_ramp_down_time to zero once can_queue
is ramped up to added max_can_queue limit, this is to avoid any
more ramp up attempts on subsequent IO completion.
The ramp down and up are skipped for FC_CAN_QUEUE_PERIOD
to avoid infrequent changes to can_queue, this required
keeping track of ramp up time also in last_can_queue_ramp_up_time.
Adds code to ramp down can_queue if lp->qfull is set, with added
new ramp up code the can_queue will be increased after
FC_CAN_QUEUE_PERIOD, therefore it is safe to do ramp down
without fsp in this case and will avoid thrash. This required
fc_fcp_can_queue_ramp_down locking change so that it can be
called with Scsi_Host lock held.
Removes si->throttled and fsp state FC_SRB_NOMEM, not needed with
added ramp up code.
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Currently can_queue is reduced only if frame alloc fails
during fc_fcp_send_data but frame alloc can fail at several
other places in FCP data path and can_queue needs to be
reduced for any FCP frame alloc failure.
This patch adds fc_fcp_frame_alloc for all FCP frame allocations
and if fc_frame_alloc fails in fc_fcp_frame_alloc then reduce
can_queue in fc_fcp_frame_alloc, this will reduce can_queue for
all FCP frame alloc failures.
This required moving fc_fcp_reduce_can_queue up, to build without
adding its prototype. Also renamed fc_fcp_reduce_can_queue to
fc_fcp_can_queue_ramp_down.
Removes fc_fcp_reduce_can_queue calling from fc_fcp_recv since
not needed with added fc_fcp_frame_alloc reducing can_queue.
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Cleans up frame allocation APIs to have just single fc_frame_alloc API.
Removes _fc_frame_alloc, renames __fc_frame_alloc to _fc_frame_alloc.
Modifies fc_fcp_send_data for removed _fc_frame_alloc, fc_fcp_send_data
was the only user of removed _fc_frame_alloc.
Also Adds check in fc_frame_alloc to do mod by 4 for only non-zero
len value.
This patch is prep work to fix can_queue reducing in next patch.
Single fc_frame_alloc API helps in fixing can_queue reducing in
next patch.
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
This patch makes a variety of cleanup changes to all libfc files.
This patch adds kernel-doc headers to all functions lacking them
and attempts to better format existing headers. It also add kernel-doc
headers to structures.
This patch ensures that the current naming conventions for local ports,
remote ports and remote port private data is upheld in the following
manner.
struct instance (i.e. variable name)
--------------------------------------------------
fc_lport lport
fc_rport rport
fc_rport_libfc_priv rpriv
fc_rport_priv rdata
I also renamed dns_rp and ptp_rp to dns_rdata and ptp_rdata
respectively.
I used emacs 'indent-region' and 'tabify' on all libfc files
to correct spacing alignments.
I feel sorry for anyone attempting to review this patch.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
When handling the multi-frame responses of fc pass-thru requests,
a code segment similar to fc_fcp_recv_data (routine to receive
inbound SCSI data) is used in the response handler. This patch
is to add a routine, called fc_copy_buffer_to_sglist(), to handle
the common function of copying data from a buffer to a scatter-
gather list in order to avoid code duplication.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
These routines are for the libfc kernel module and should be in
the libfc .c file.
Moving the libfc __init routine into fc_libfc.c caused the creation
of the fc_setup_fcp() and fc_destroy_fcp() routines so that
scsi_pkt_cachep was not exposed outside of fc_fcp.c.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
include/scsi/libfc.h is currently loaded with common code
shared between libfc's sub-modules as well as shared between
libfc and fcoe. Previous patches attempted to move out
non-common code. This patch creates two files for common
libfc routines that will not be shared with fcoe, fnic or
any other LLDs.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
This function is never used, let's remove it.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Adjust queue_depth on fc_change_queue_depth call back
with reason SCSI_QDEPTH_RAMP_UP, no additional resource
adjustments necessary for libfc.
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Acked-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
The cmd_per_lun value is used by scsi-ml as fall back lowest
queue_depth value but in case of libfc cmd_per_lun is set to
same value as max queue_depth = 32.
So this patch reduces cmd_per_lun value to 3 and configures
each lun with default max queue_depth 32 in fc_slave_alloc.
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Acked-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
This converts the libfc using scsi_track_queue_full to
track the queue full from the change_queue_depth callback.
Signed-off-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Acked-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
This patch modifies scsi_host_template->change_queue_depth so that
it takes an argument indicating why it is being called. This will be
used so that if a LLD needs to do some extra processing when
handling queue fulls or later ramp ups, it can do so.
This is a simple port of the drivers setting a change_queue_depth
callback. In the patch I just have these LLDs adjust the queue depth
if the user was requesting it.
Signed-off-by: Mike Christie <michaelc@cs.wisc.edu>
[Vasu.Dev: v2
Also converted pmcraid_change_queue_depth and then verified
all modules compile using "make allmodconfig" for any new build
warnings on X86_64.
Updated original description after combing two original
patches from Mike to make this patch git bisectable.]
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
[jejb: fixed up 53c700]
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
I was running into several different panics under stress, which I traced down
to a few different possible slab corruption issues in error handling paths.
I have not yet looked into why these exchange sends fail, but with these
fixes my test system is much more stable under stress than before.
fc_elsct_send() could fail and either leave the passed in frame intact
(failure in fc_ct/els_fill) or the frame could have been freed if the
failure was is fc_exch_seq_send(). The caller had no way of knowing, and
there was a potential double free in the error handling in fc_fcp_rec().
Make fc_elsct_send() always free the frame before returning, and remove the
fc_frame_free() call in fc_fcp_rec().
While fc_exch_seq_send() did always consume the frame, there were double free
bugs in the error handling of fc_fcp_cmd_send() and fc_fcp_srr() as well.
Numerous calls to error handling routines (fc_disc_error(),
fc_lport_error(), fc_rport_error_retry() ) were passing in a frame pointer that
had already been freed in the case of an error. I have changed the call
sites to pass in a NULL pointer, but there may be more appropriate error
codes to use.
Question: Why do these error routines take a frame pointer anyway? I
understand passing in a pointer encoded error to the response handlers, but
the error routines take no action on a valid pointer and should never be
called that way.
Signed-off-by: Chris Leech <christopher.leech@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
In case of sequence offload, in fc_fcp_send_data(), the skb_fill_page_info()
called may end up adding more frags to the skb_shinfo(fp_skb(fp))->frags[],
exceeding SKB_MAX_FRAGS, this eventually corrupts the memory. I am adding the
FR_FRAME_SG_LEN back, but as SKB_MAX_FRAGS -1, leaving 1 for our fcoe_eof_crc
page. And send will be broken into multiple large sends if the frame already
contains more frags than skb handle.
Signed-off-by: Yi Zou <yi.zou@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
This bug is exposed when there is a link flap in LLD. Particularly, when it
happens right after a SCSI write command is sent out, no FCP_DATA is sent,
causing fsp->status_code to be set as FC_DATA_UNDRUN in fc_fcp_complete_locked
even no SCSI status is received. Consequently, fc_io_compl treats this as DID_OK.
This results in SCSI returning successful to the initial I/O request even
there is no DATA actually sent. Particularly, if you run an I/O tool w/ data
verification on, the read back for verification is gonna fail.
This is fixed here by checking when FC_DATA_UNDRUN happens, SCSI status is
received w/ FC_SRB_RCV_STATUS set in fsp->state.
Signed-off-by: Yi Zou <yi.zou@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
This argument isn't used, let's not pass it into the routine.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
These are a few functions that were not used by other
modules. They did not need to be exported so this patch
removes the EXPORT_SYMBOLS call for each.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
xid 0 was used as an indication of invalid xid before but now xid 0
can be used as a valid exchange i. This patch fixes the ddp completion
in fcp layer, i.e., in fc_fcp.c:fc_fcp_ddp_done() function, to make sure it
does not use xid 0 for indication of an invalid xid, instead, it now
uses use FC_XID_UNKNOWN for such indication.
Signed-off-by: Yi Zou <yi.zou@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
tt.elsct_send is used by both FCP and by the rport state machine.
After further patches, these two modules will use different
structures for the remote port.
So, change elsct_send to use the FC_ID instead of the fc_rport_priv
as its argument. It currently only uses the FC_ID anyway.
For CT requests the destination FC_ID is still implicitly 0xfffffc.
After further patches the did arg on CT requests will be used to
specify the FC_ID being inquired about for GPN_ID or other queries.
Signed-off-by: Joe Eykholt <jeykholt@cisco.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
The rport and discovery modules deal with remote ports
before fc_remote_port_add() can be done, because the
full set of rport identifiers is not known at early stages.
In preparation for splitting the fc_rport/fc_rport_priv allocation,
make fc_rport_priv the primary interface for the remote port and
discovery engines.
The FCP / SCSI layers still deal with fc_rport and
fc_rport_libfc_priv, however.
Signed-off-by: Joe Eykholt <jeykholt@cisco.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
I don't believe this check is needed any more in the current kernel, which,
if I understand correctly, is for compound page where only the first page
is supposed to get ref-counted.
Signed-off-by: Yi Zou <yi.zou@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
FC_FRAME_SG_LEN is 4 which is too small when offload is enabled. Actually, the
WARN_ON() in fc_fcp_send_data() should be:
WARN_ON(skb_shinfo(fp_skb(fp))->nr_frags > MAX_SKB_FRAGS);
But since we will not get anything more than 64K anyway, so there is no need
to do this anyway here. Therefore, I am getting rid of FC_FRAME_SG_LEN here
and the WARN_ON here.
Signed-off-by: Yi Zou <yi.zou@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
This patch adds the /sys/module/libfc/parameters/debug_logging
file to sysfs as a module parameter. It accepts an integer
bitmask for logging. Currently it supports:
bit
LSB 0 = general libfc debugging
1 = lport debugging
2 = disc debugging
3 = rport debugging
4 = fcp debugging
5 = EM debugging
6 = exch/seq debugging
7 = scsi logging (mostly error handling)
the other bits are not used at this time.
The patch converts all of the libfc source files to use
these new macros and removes the old FC_DBG macro.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
If we aborted a command, because it timed out we should not use
DID_ABORT. It will fail the command right away back to the upper
layer. We want to use something that indicated that the problem
did not complete normally, but it was not a fatal problem.
Signed-off-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
When building with a .config generated from 'make allmodconfig'
some build warnings are generated. This patch corrects the warnings,
adds a FC_FID_NONE (= 0) enumeration for FC-IDs and cleans up one
variable naming to meet our variable naming conventions. For example,
fc_lport's should be named "lport," not "lp."
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
Just sets up build environment for libfcoe module towards a
libfcoe library for libfc LLDs using FCoE as libfc transport.
Common library code to libfcoe is added in next patch.
Also, updated MODULE_LICENSE from "GPL" string to "GPL v2" for
libfc, libfcoe and fcoe modules to accurately match the licenses.
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
Remove the hotplug creation of dev_stats, we allocate for all possible CPUs
now when we allocate the lport.
v2: Durring the 2.6.30 merge window, before these patches were comitted,
'percpu_ptr' was renamed 'per_cpu_ptr'. This latest update updates this
patch for the name change.
Signed-off-by: Yi Zou <yi.zou@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
When LLD supports direct data placement (ddp) for large receive of an scsi
i/o coming into fc_fcp, we call into libfc_function_template's ddp_setup()
to prepare for a ddp of large receive for this read I/O. When I/O is complete,
we call the corresponding ddp_done() to get the length of data ddped as well
as to let LLD do clean up.
fc_fcp_ddp_setup()/fc_fcp_ddp_done() are added to setup and complete a ddped
read I/O described by the given fc_fcp_pkt. They would call into corresponding
ddp_setup/ddp_done implemented by the fcoe layer. Eventually, fcoe layer calls
into LLD's ddp_setup/ddp_done provided through net_device
Signed-off-by: Yi Zou <yi.zou@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
Make sure for large send is supported by LLD in outgoing FCP data, we are only
sending the lso_max a time in one single large send, since that is what
supported by LLD.
Signed-off-by: Yi Zou <yi.zou@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
The SUGGEST_* flags in the SCSI command result have been out of fashion
for a while and we don't actually use them in the error handling.
Remove the remaining occurrences.
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
1) Added '()' for function names in kerneldoc comments
2) Changed comment bookends from '**/' to '*/'. The comment on the the
mailing list was that '**/' "is consistently unconventional. Not
wrong, just odd." The Documentation/kernel-doc-nano-HOWTO.txt
states that kerneldoc comment blocks should end with '**/' but most
(if not all) instance I found under drivers/scsi/ were only using
the '*/' so I converted to that style.
3) Removed incorrect linebreaks in kerneldoc comments where found
4) Removed a few unnecessary blank comment lines in kerneldoc comment
blocks
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
The fc_fcp_complete_locked detected data underrun in this case and set
the FC_DATA_UNDRUN but that was ignored by fc_io_compl for all cases
including read underrun.
Added code to not to ignore FC_DATA_UNDRUN for read IO and instead
suggested scsi-ml to retry cmd to recover from lost data frame.
Not sure if it is okay to ignore FC_DATA_UNDRUN for other case, so let
code as is for other cases but removed or-ing with zero valued fsp->cdb_status
for those cases.
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
The fcoe_xmit could call fc_pause in case the pending skb queue len is larger
than FCOE_MAX_QUEUE_DEPTH, the fc_pause was trying to grab lport->lp_muex to
change lport->link_status and that had these issues :-
1. The fcoe_xmit was getting called with bh disabled, thus causing
"BUG: scheduling while atomic" when grabbing lport->lp_muex with bh disabled.
2. fc_linkup and fc_linkdown function calls lport_enter function with
lport->lp_mutex held and these enter function in turn calls fcoe_xmit to send
lport related FC frame, e.g. fc_linkup => fc_lport_enter_flogi to send flogi
req. In this case grabbing the same lport->lp_mutex again in fc_puase from
fcoe_xmit would cause deadlock.
The lport->lp_mutex was used for setting FC_PAUSE in fcoe_xmit path but
FC_PAUSE bit was not used anywhere beside just setting and clear this
bit in lport->link_status, instead used a separate field qfull in fc_lport
to eliminate need for lport->lp_mutex to track pending queue full condition
and in turn avoid above described two locking issues.
Also added check for lp->qfull in fc_fcp_lport_queue_ready to trigger
SCSI_MLQUEUE_HOST_BUSY when lp->qfull is set to prevent more scsi-ml cmds
while lp->qfull is set.
This patch eliminated FC_LINK_UP and FC_PAUSE and instead used dedicated
fields in fc_lport for this, this simplified all related conditional
code.
Also removed fc_pause and fc_unpause functions and instead used newly added
lport->qfull directly in fcoe.
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
libFC is composed of 4 blocks supported by an exchange manager
and a framing library. The upper 4 layers are fc_lport, fc_disc,
fc_rport and fc_fcp. A LLD that uses libfc could choose to
either use libfc's block, or using the transport template
defined in libfc.h, override one or more blocks with its own
implementation.
The EM (Exchange Manager) manages exhcanges/sequences for all
commands- ELS, CT and FCP.
The framing library frames ELS and CT commands.
The fc_lport block manages the library's representation of the
host's FC enabled ports.
The fc_disc block manages discovery of targets as well as
handling changes that occur in the FC fabric (via. RSCN events).
The fc_rport block manages the library's representation of other
entities in the FC fabric. Currently the library uses this block
for targets, its peer when in point-to-point mode and the
directory server, but can be extended for other entities if
needed.
The fc_fcp block interacts with the scsi-ml and handles all
I/O.
Signed-off-by: Robert Love <robert.w.love@intel.com>
[jejb: added include of delay.h to fix ppc64 compile prob spotted by sfr]
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
