Johannes Weiner 777c6c5f1f wait: prevent exclusive waiter starvation ... With exclusive waiters, every process woken up through the wait queue must ensure that the next waiter down the line is woken when it has finished. Interruptible waiters don't do that when aborting due to a signal. And if an aborting waiter is concurrently woken up through the waitqueue, noone will ever wake up the next waiter. This has been observed with __wait_on_bit_lock() used by lock_page_killable(): the first contender on the queue was aborting when the actual lock holder woke it up concurrently. The aborted contender didn't acquire the lock and therefor never did an unlock followed by waking up the next waiter. Add abort_exclusive_wait() which removes the process' wait descriptor from the waitqueue, iff still queued, or wakes up the next waiter otherwise. It does so under the waitqueue lock. Racing with a wake up means the aborting process is either already woken (removed from the queue) and will wake up the next waiter, or it will remove itself from the queue and the concurrent wake up will apply to the next waiter after it. Use abort_exclusive_wait() in __wait_event_interruptible_exclusive() and __wait_on_bit_lock() when they were interrupted by other means than a wake up through the queue. [akpm@linux-foundation.org: coding-style fixes] Reported-by: Chris Mason <chris.mason@oracle.com> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org> Mentored-by: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Matthew Wilcox <matthew@wil.cx> Cc: Chuck Lever <cel@citi.umich.edu> Cc: Nick Piggin <nickpiggin@yahoo.com.au> Cc: Ingo Molnar <mingo@elte.hu> Cc: <stable@kernel.org> ["after some testing"] Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>		2009-02-05 12:56:48 -08:00
..
irq	Merge branch 'core/xen' into x86/urgent	2009-02-04 14:54:56 +01:00
power	Hibernation: Introduce system_entering_hibernation	2009-01-27 02:15:45 -05:00
time	hrtimers: allow the hot-unplugging of all cpus	2009-01-30 22:35:29 +01:00
trace	ftrace: do_each_pid_task() needs rcu lock	2009-02-03 22:50:58 +01:00
.gitignore
acct.c	[CVE-2009-0029] System call wrappers part 04	2009-01-14 14:15:19 +01:00
async.c	kernel/async.c: fix printk warnings	2009-02-05 12:56:46 -08:00
audit.c	[PATCH] fix broken timestamps in AVC generated by kernel threads	2008-12-09 02:27:41 -05:00
audit.h	fixing audit rule ordering mess, part 1	2009-01-04 15:14:41 -05:00
audit_tree.c	audit: validate comparison operations, store them in sane form	2009-01-04 15:14:42 -05:00
auditfilter.c	audit: validate comparison operations, store them in sane form	2009-01-04 15:14:42 -05:00
auditsc.c	make sure that filterkey of task,always rules is reported	2009-01-04 15:14:42 -05:00
backtracetest.c
bounds.c
capability.c	[CVE-2009-0029] System call wrappers part 04	2009-01-14 14:15:19 +01:00
cgroup.c	cgroup: fix root_count when mount fails due to busy subsystem	2009-01-29 18:04:45 -08:00
cgroup_debug.c
cgroup_freezer.c	freezer_cg: disable writing freezer.state of root cgroup	2008-11-12 17:17:16 -08:00
compat.c	Allow times and time system calls to return small negative values	2009-01-06 15:59:13 -08:00
configs.c
cpu.c	stop_machine/cpu hotplug: fix disable_nonboot_cpus	2009-01-07 11:36:14 -08:00
cpuset.c	cpuset: fix possible deadlock in async_rebuild_sched_domains	2009-01-19 02:44:00 +01:00
cred-internals.h	CRED: Inaugurate COW credentials	2008-11-14 10:39:23 +11:00
cred.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/security-testing-2.6	2009-01-09 13:59:25 -08:00
delayacct.c	schedstat: consolidate per-task cpu runtime stats	2008-12-18 13:54:01 +01:00
dma-coherent.c	dma-coherent: Restore dma_alloc_from_coherent() large alloc fall back policy.	2009-01-21 18:51:53 +09:00
dma.c
exec_domain.c	[CVE-2009-0029] System call wrappers part 04	2009-01-14 14:15:19 +01:00
exit.c	[CVE-2009-0029] System call wrappers part 08	2009-01-14 14:15:21 +01:00
extable.c	Merge branch 'core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2008-12-30 16:10:19 -08:00
fork.c	Merge branch 'timers-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2009-01-26 09:47:43 -08:00
freezer.c
futex.c	[CVE-2009-0029] System call wrappers part 31	2009-01-14 14:15:31 +01:00
futex_compat.c	CRED: Use RCU to access another task's creds and to release a task's own creds	2008-11-14 10:39:19 +11:00
hrtimer.c	hrtimer: prevent negative expiry value after clock_was_set()	2009-01-30 22:35:34 +01:00
itimer.c	[CVE-2009-0029] System call wrappers part 05	2009-01-14 14:15:20 +01:00
kallsyms.c	Revert "kbuild: strip generated symbols from *.ko"	2009-01-14 21:38:20 +01:00
Kconfig.freezer
Kconfig.hz
Kconfig.preempt	rcu: provide RCU options on non-preempt architectures too	2008-12-25 09:31:28 +01:00
kexec.c	[CVE-2009-0029] System call wrappers part 07	2009-01-14 14:15:20 +01:00
kfifo.c
kgdb.c
kmod.c	kmod: fix varargs kernel-doc	2009-01-06 15:59:27 -08:00
kprobes.c	kprobes: check CONFIG_FREEZER instead of CONFIG_PM	2009-01-16 14:32:17 -05:00
ksysfs.c	kernel/ksysfs.c:fix dependence on CONFIG_NET	2009-01-06 10:44:31 -08:00
kthread.c	tracepoints: add DECLARE_TRACE() and DEFINE_TRACE()	2008-11-16 09:01:36 +01:00
latencytop.c	KSYM_SYMBOL_LEN fixes	2008-12-10 08:01:54 -08:00
lockdep.c	Merge branch 'core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2008-12-30 16:10:19 -08:00
lockdep_internals.h
lockdep_proc.c
Makefile	kernel/up.c: omit it if SMP=y, USE_GENERIC_SMP_HELPERS=n	2009-01-14 09:42:11 -08:00
marker.c	markers/tracpoints: fix non-modular build	2008-11-16 09:52:03 +01:00
module.c	modules: Use a better scheme for refcounting	2009-02-02 19:17:55 -08:00
mutex-debug.c
mutex-debug.h
mutex.c	mutex: __used is needed for function referenced only from inline asm	2008-11-24 10:00:28 +01:00
mutex.h
notifier.c	Merge commit 'v2.6.28-rc6' into core/debug	2008-11-26 08:22:50 +01:00
ns_cgroup.c	ns_cgroup: remove unused spinlock	2009-01-08 08:31:02 -08:00
nsproxy.c	User namespaces: set of cleanups (v2)	2008-11-24 18:57:41 -05:00
panic.c	oops: increment the oops UUID every time we oops	2009-01-06 15:59:12 -08:00
params.c
pid.c	pid: generalize task_active_pid_ns	2009-01-08 08:31:12 -08:00
pid_namespace.c
pm_qos_params.c
posix-cpu-timers.c	itimers: remove the per-cpu-ish-ness	2009-01-07 18:52:44 +01:00
posix-timers.c	[CVE-2009-0029] System call wrappers part 05	2009-01-14 14:15:20 +01:00
printk.c	[CVE-2009-0029] System call wrappers part 27	2009-01-14 14:15:29 +01:00
profile.c	profile: don't include <asm/ptrace.h> twice.	2009-01-06 15:59:14 -08:00
ptrace.c	[CVE-2009-0029] System call wrappers part 27	2009-01-14 14:15:29 +01:00
rcuclassic.c	rcu: add __cpuinit to rcu_init_percpu_data()	2009-01-14 11:26:40 +01:00
rcupdate.c	rcu: eliminate synchronize_rcu_xxx macro	2009-01-05 10:18:08 +01:00
rcupreempt.c	rcu: eliminate synchronize_rcu_xxx macro	2009-01-05 10:18:08 +01:00
rcupreempt_trace.c	"Tree RCU": scalable classic RCU implementation	2008-12-18 21:56:04 +01:00
rcutorture.c	rcu: fix bug in rcutorture system-shutdown code	2009-01-07 23:36:25 +01:00
rcutree.c	rcu: add __cpuinit to rcu_init_percpu_data()	2009-01-14 11:26:40 +01:00
rcutree_trace.c	"Tree RCU": scalable classic RCU implementation	2008-12-18 21:56:04 +01:00
relay.c	relay: fix lock imbalance in relay_late_setup_files	2009-01-18 20:29:35 +01:00
res_counter.c	memcg: memory cgroup resource counters for hierarchy	2009-01-08 08:31:05 -08:00
resource.c	resources: fix parameter name and kernel-doc	2009-01-15 16:39:38 -08:00
rtmutex-debug.c
rtmutex-debug.h
rtmutex-tester.c
rtmutex.c
rtmutex.h
rtmutex_common.h
rwsem.c
sched.c	wait: prevent exclusive waiter starvation	2009-02-05 12:56:48 -08:00
sched_clock.c	sched_clock: prevent scd->clock from moving backwards, take #2	2008-12-31 09:53:21 +01:00
sched_cpupri.c	sched: fix section mismatch	2009-01-06 11:07:15 +01:00
sched_cpupri.h	sched: convert struct cpupri_vec cpumask_var_t.	2008-11-24 17:52:22 +01:00
sched_debug.c	sched: partly revert "sched debug: remove NULL checking in print_cfs_rt_rq()"	2009-01-11 02:40:32 +01:00
sched_fair.c	sched: fix buddie group latency	2009-02-01 10:49:51 +01:00
sched_features.h	sched: backward looking buddy	2008-11-05 10:30:14 +01:00
sched_idletask.c
sched_rt.c	sched_rt: don't use first_cpu on cpumask created with cpumask_and	2009-02-01 10:49:52 +01:00
sched_stats.h	itimers: remove the per-cpu-ish-ness	2009-01-07 18:52:44 +01:00
seccomp.c
semaphore.c
signal.c	signals, debug: fix BUG: using smp_processor_id() in preemptible code in print_fatal_signal()	2009-01-27 00:36:19 +01:00
smp.c	generic-ipi: use per cpu data for single cpu ipi calls	2009-01-30 18:31:08 +01:00
softirq.c	Merge branch 'cpus4096-for-linus-3' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2009-01-03 12:04:39 -08:00
softlockup.c	softlock: fix false panic which can occur if softlockup_thresh is reduced	2009-01-14 11:48:07 +01:00
spinlock.c
srcu.c
stacktrace.c	stacktrace: provide save_stack_trace_tsk() weak alias	2008-12-25 11:44:43 +01:00
stop_machine.c	stop_machine: introduce stop_machine_create/destroy.	2009-01-05 08:40:14 +10:30
sys.c	revert "rlimit: permit setting RLIMIT_NOFILE to RLIM_INFINITY"	2009-02-05 12:56:47 -08:00
sys_ni.c	[CVE-2009-0029] Make sys_syslog a conditional system call	2009-01-14 14:15:16 +01:00
sysctl.c	Merge branch 'core/debugobjects' into core/urgent	2009-01-22 10:03:02 +01:00
sysctl_check.c
taskstats.c	cpumask: convert rest of files in kernel/	2009-01-01 10:12:28 +10:30
test_kprobes.c	kprobes: add tests for register_kprobes	2009-01-06 15:59:20 -08:00
time.c	[CVE-2009-0029] System call wrappers part 01	2009-01-14 14:15:18 +01:00
timeconst.pl
timer.c	[CVE-2009-0029] System call wrappers part 27	2009-01-14 14:15:29 +01:00
tracepoint.c	markers/tracpoints: fix non-modular build	2008-11-16 09:52:03 +01:00
tsacct.c	mm: introduce get_mm_hiwater_xxx(), fix taskstats->hiwater_xxx accounting	2009-01-06 15:59:09 -08:00
uid16.c	[CVE-2009-0029] System call wrappers part 19	2009-01-14 14:15:26 +01:00
up.c	smp_call_function_single(): be slightly less stupid, fix #2	2009-01-12 16:04:37 +01:00
user.c	Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2008-12-28 12:27:58 -08:00
user_namespace.c	User namespaces: set of cleanups (v2)	2008-11-24 18:57:41 -05:00
utsname.c
utsname_sysctl.c
wait.c	wait: prevent exclusive waiter starvation	2009-02-05 12:56:48 -08:00
workqueue.c	work_on_cpu: Use our own workqueue.	2009-01-19 22:36:07 +01:00