Peter Zijlstra 88ec22d3ed sched: Remove the cfs_rq dependency from set_task_cpu() ... In order to remove the cfs_rq dependency from set_task_cpu() we need to ensure the task is cfs_rq invariant for all callsites. The simple approach is to substract cfs_rq->min_vruntime from se->vruntime on dequeue, and add cfs_rq->min_vruntime on enqueue. However, this has the downside of breaking FAIR_SLEEPERS since we loose the old vruntime as we only maintain the relative position. To solve this, we observe that we only migrate runnable tasks, we do this using deactivate_task(.sleep=0) and activate_task(.wakeup=0), therefore we can restrain the min_vruntime invariance to that state. The only other case is wakeup balancing, since we want to maintain the old vruntime we cannot make it relative on dequeue, but since we don't migrate inactive tasks, we can do so right before we activate it again. This is where we need the new pre-wakeup hook, we need to call this while still holding the old rq->lock. We could fold it into ->select_task_rq(), but since that has multiple callsites and would obfuscate the locking requirements, that seems like a fudge. This leaves the fork() case, simply make sure that ->task_fork() leaves the ->vruntime in a relative state. This covers all cases where set_task_cpu() gets called, and ensures it sees a relative vruntime. Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Mike Galbraith <efault@gmx.de> LKML-Reference: <20091216170518.191697025@chello.nl> Signed-off-by: Ingo Molnar <mingo@elte.hu>		2009-12-16 19:01:58 +01:00
..
gcov	microblaze: Enable GCOV_PROFILE_ALL	2009-09-21 14:29:21 +02:00
irq	genirq: Convert irq_desc.lock to raw_spinlock	2009-12-14 23:55:33 +01:00
power	vt: introduce and use vt_kmsg_redirect() function	2009-12-15 08:53:28 -08:00
time	clockevents: Convert to raw_spinlock	2009-12-14 23:55:34 +01:00
trace	locking: Convert __raw_spin* functions to arch_spin*	2009-12-14 23:55:32 +01:00
.gitignore
acct.c	bsdacct: fix uid/gid misreporting	2009-12-15 08:53:10 -08:00
async.c
audit.c	Audit: send signal info if selinux is disabled	2009-09-24 03:50:26 -04:00
audit.h	Fix rule eviction order for AUDIT_DIR	2009-06-24 00:02:38 -04:00
audit_tree.c	Fix rule eviction order for AUDIT_DIR	2009-06-24 00:02:38 -04:00
audit_watch.c	Audit: reorganize struct audit_watch to save 8 bytes	2009-09-24 03:50:25 -04:00
auditfilter.c	Audit: clean up all op= output to include string quoting	2009-06-24 00:00:52 -04:00
auditsc.c	Audit: rearrange audit_context to save 16 bytes per struct	2009-09-24 03:50:26 -04:00
backtracetest.c
bounds.c
capability.c	remove CONFIG_SECURITY_FILE_CAPABILITIES compile option	2009-11-24 15:06:47 +11:00
cgroup.c	cgroup: fix strstrip() misuse	2009-10-29 07:39:25 -07:00
cgroup_freezer.c	cgroups: let ss->can_attach and ss->attach do whole threadgroups at a time	2009-09-24 07:20:58 -07:00
compat.c
configs.c
cpu.c	sched: Fix set_cpu_active() in cpu_down()	2009-12-16 19:01:53 +01:00
cpuset.c	sched: Fix balance vs hotplug race	2009-12-06 21:10:56 +01:00
cred-internals.h
cred.c	creds_are_invalid() needs to be exported for use by modules:	2009-09-23 11:02:26 -07:00
delayacct.c	headers: taskstats_kern.h trim	2009-09-18 09:48:52 -07:00
dma.c
exec_domain.c
exit.c	sched: Convert pi_lock to raw_spinlock	2009-12-14 23:55:33 +01:00
extable.c
fork.c	sched: Convert pi_lock to raw_spinlock	2009-12-14 23:55:33 +01:00
freezer.c	sched: fix nr_uninterruptible accounting of frozen tasks really	2009-07-18 14:19:53 +02:00
futex.c	rtmutes: Convert rtmutex.lock to raw_spinlock	2009-12-14 23:55:33 +01:00
futex_compat.c	futex: Fix compat_futex to be same as futex for REQUEUE_PI	2009-08-10 15:41:12 +02:00
groups.c	groups: move code to kernel/groups.c	2009-06-16 19:47:48 -07:00
hrtimer.c	hrtimers: Convert to raw_spinlocks	2009-12-14 23:55:34 +01:00
hung_task.c	softlockup: Fix hung_task_check_count sysctl	2009-11-27 06:21:57 +01:00
hw_breakpoint.c	perf_event: Convert to raw_spinlock	2009-12-14 23:55:34 +01:00
itimer.c	itimers: Fix racy writes to cpu_itimer fields	2009-11-18 16:32:12 +01:00
kallsyms.c	hw-breakpoints: Fix broken hw-breakpoint sample module	2009-11-10 11:23:29 +01:00
Kconfig.freezer
Kconfig.hz
Kconfig.locks	mutex: Better control mutex adaptive spinning config	2009-12-03 11:50:11 +01:00
Kconfig.preempt
kexec.c	kexec: fix omitting offset in extended crashkernel syntax	2009-07-29 19:10:34 -07:00
kfifo.c	kfifo: Use "const" definitions	2009-09-19 13:13:17 -07:00
kgdb.c	kgdb: Always process the whole breakpoint list on activate or deactivate	2009-12-11 08:43:20 -06:00
kmod.c	security: report the module name to security_module_request	2009-11-10 09:33:46 +11:00
kprobes.c	Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2009-12-05 15:30:21 -08:00
ksysfs.c
kthread.c	sched: Move kthread_bind() back to kthread.c	2009-12-16 19:01:57 +01:00
latencytop.c
lockdep.c	locking: Convert __raw_spin* functions to arch_spin*	2009-12-14 23:55:32 +01:00
lockdep_internals.h	lockdep: BFS cleanup	2009-07-24 10:53:29 +02:00
lockdep_proc.c	seq_file: constify seq_operations	2009-09-23 07:39:29 -07:00
lockdep_states.h
Makefile	Merge branch 'kvm-updates/2.6.33' of git://git.kernel.org/pub/scm/virt/kvm/kvm	2009-12-08 08:02:38 -08:00
module.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu	2009-12-14 09:58:24 -08:00
mutex-debug.c	headers: remove sched.h from interrupt.h	2009-10-11 11:20:58 -07:00
mutex-debug.h	locking: Implement new raw_spinlock	2009-12-14 23:55:32 +01:00
mutex.c	mutex: Better control mutex adaptive spinning config	2009-12-03 11:50:11 +01:00
mutex.h
notifier.c	kprobes: Fix to add __kprobes to notify_die	2009-08-30 03:08:26 +02:00
ns_cgroup.c	cgroups: let ss->can_attach and ss->attach do whole threadgroups at a time	2009-09-24 07:20:58 -07:00
nsproxy.c	nsproxy: extract create_nsproxy()	2009-06-18 13:03:56 -07:00
panic.c	Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2009-10-08 12:16:35 -07:00
params.c	tree-wide: convert open calls to remove spaces to skip_spaces() lib function	2009-12-15 08:53:32 -08:00
perf_event.c	perf_event: Convert to raw_spinlock	2009-12-14 23:55:34 +01:00
pid.c	mm: also use alloc_large_system_hash() for the PID hash table	2009-09-22 07:17:38 -07:00
pid_namespace.c	pidns: deny CLONE_PARENT|CLONE_NEWPID combination	2009-09-24 07:21:04 -07:00
pm_qos_params.c	pm_qos: clean up racy global "name" variable	2009-10-14 15:31:10 +02:00
posix-cpu-timers.c	posix-cpu-timers: optimize and document timer_create callback	2009-11-18 12:36:05 +01:00
posix-timers.c	time: Introduce CLOCK_REALTIME_COARSE	2009-08-21 21:43:46 +02:00
printk.c	Merge branch 'core-printk-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2009-12-05 09:50:22 -08:00
profile.c	kernel/profile.c: Switch /proc/irq/prof_cpu_mask to seq_file	2009-09-20 20:15:40 +02:00
ptrace.c	ptrace: __ptrace_detach: do __wake_up_parent() if we reap the tracee	2009-09-24 07:20:59 -07:00
rcupdate.c	rcu: Re-arrange code to reduce #ifdef pain	2009-11-22 18:58:16 +01:00
rcutiny.c	rcu: Eliminate unneeded function wrapping	2009-11-22 18:58:16 +01:00
rcutorture.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu	2009-12-14 09:58:24 -08:00
rcutree.c	rcu: Add expedited grace-period support for preemptible RCU	2009-12-03 11:35:25 +01:00
rcutree.h	rcu: Add expedited grace-period support for preemptible RCU	2009-12-03 11:35:25 +01:00
rcutree_plugin.h	rcu: Add expedited grace-period support for preemptible RCU	2009-12-03 11:35:25 +01:00
rcutree_trace.c	rcu: Add expedited grace-period support for preemptible RCU	2009-12-03 11:35:25 +01:00
relay.c	const: mark struct vm_struct_operations	2009-09-27 11:39:25 -07:00
res_counter.c	memcg: some modification to softlimit under hierarchical memory reclaim.	2009-10-01 16:11:13 -07:00
resource.c	resources: when allocate_resource() fails, leave resource untouched	2009-11-04 13:06:46 -08:00
rtmutex-debug.c	sched: Convert pi_lock to raw_spinlock	2009-12-14 23:55:33 +01:00
rtmutex-debug.h
rtmutex-tester.c
rtmutex.c	rtmutes: Convert rtmutex.lock to raw_spinlock	2009-12-14 23:55:33 +01:00
rtmutex.h
rtmutex_common.h
rwsem.c
sched.c	sched: Remove the cfs_rq dependency from set_task_cpu()	2009-12-16 19:01:58 +01:00
sched_clock.c	sched: Fix cpu_clock() in NMIs, on !CONFIG_HAVE_UNSTABLE_SCHED_CLOCK	2009-12-15 09:04:36 +01:00
sched_cpupri.c	sched: Convert cpupri lock to raw_spinlock	2009-12-14 23:55:33 +01:00
sched_cpupri.h	sched: Convert cpupri lock to raw_spinlock	2009-12-14 23:55:33 +01:00
sched_debug.c	sched: Convert rq->lock to raw_spinlock	2009-12-14 23:55:33 +01:00
sched_fair.c	sched: Remove the cfs_rq dependency from set_task_cpu()	2009-12-16 19:01:58 +01:00
sched_features.h	sched: Discard some old bits	2009-12-09 10:03:07 +01:00
sched_idletask.c	Merge branch 'linus' into sched/urgent	2009-12-16 18:33:49 +01:00
sched_rt.c	sched: Add pre and post wakeup hooks	2009-12-16 19:01:58 +01:00
sched_stats.h
seccomp.c
semaphore.c
signal.c	Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2009-12-05 15:30:21 -08:00
slow-work-debugfs.c	SLOW_WORK: Move slow_work's proc file to debugfs	2009-12-01 08:20:31 -08:00
slow-work.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/sysctl-2.6	2009-12-08 07:38:50 -08:00
slow-work.h	SLOW_WORK: Move slow_work's proc file to debugfs	2009-12-01 08:20:31 -08:00
smp.c	Merge branch 'core-locking-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2009-12-15 09:02:01 -08:00
softirq.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu	2009-12-14 09:58:24 -08:00
softlockup.c	percpu: make percpu symbols under kernel/ and mm/ unique	2009-10-29 22:34:13 +09:00
spinlock.c	locking: Cleanup the name space completely	2009-12-14 23:55:33 +01:00
srcu.c	rcu: Add synchronize_srcu_expedited()	2009-10-26 09:40:30 +01:00
stacktrace.c
stop_machine.c
sys.c	kernel/sys.c: fix "warning: do-while statement is not a compound statement" noise	2009-12-15 08:53:26 -08:00
sys_ni.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next-2.6	2009-12-08 07:55:01 -08:00
sysctl.c	'sysctl_max_map_count' should be non-negative	2009-12-15 08:53:23 -08:00
sysctl_binary.c	sysctl binary: Reorder the tests to process wild card entries first.	2009-11-12 01:42:31 -08:00
sysctl_check.c	ipv4 05/05: add sysctl to accept packets with local source addresses	2009-12-03 12:14:38 -08:00
taskstats.c	genetlink: make netns aware	2009-07-12 14:03:27 -07:00
test_kprobes.c
time.c	Merge branch 'timers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2009-12-08 19:27:08 -08:00
timeconst.pl
timer.c	Merge branch 'timers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2009-09-23 09:46:15 -07:00
tracepoint.c	trivial: fix typo "to to" in multiple files	2009-09-21 15:14:55 +02:00
tsacct.c
uid16.c	headers: utsname.h redux	2009-09-23 18:13:10 -07:00
up.c
user-return-notifier.c	core: Clean up user return notifers use of per_cpu	2009-12-02 10:22:59 +01:00
user.c	uids: Prevent tear down race	2009-11-02 16:02:39 +01:00
user_namespace.c
utsname.c	utsns: extract creeate_uts_ns()	2009-06-18 13:03:55 -07:00
utsname_sysctl.c	sysctl kernel: Remove binary sysctl logic	2009-11-12 02:04:55 -08:00
wait.c	locking, sched: Give waitqueue spinlocks their own lockdep classes	2009-08-10 14:43:09 +02:00
workqueue.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq	2009-12-10 09:35:44 -08:00