Skip to content

Commit b3fd4ea

Browse files
committed
Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull RCU updates from Ingo Molnar: "Main changes: - Torture-test changes, including refactoring of rcutorture and introduction of a vestigial locktorture. - Real-time latency fixes. - Documentation updates. - Miscellaneous fixes" * 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (77 commits) rcu: Provide grace-period piggybacking API rcu: Ensure kernel/rcu/rcu.h can be sourced/used stand-alone rcu: Fix sparse warning for rcu_expedited from kernel/ksysfs.c notifier: Substitute rcu_access_pointer() for rcu_dereference_raw() Documentation/memory-barriers.txt: Clarify release/acquire ordering rcutorture: Save kvm.sh output to log rcutorture: Add a lock_busted to test the test rcutorture: Place kvm-test-1-run.sh output into res directory rcutorture: Rename TREE_RCU-Kconfig.txt locktorture: Add kvm-recheck.sh plug-in for locktorture rcutorture: Gracefully handle NULL cleanup hooks locktorture: Add vestigial locktorture configuration rcutorture: Introduce "rcu" directory level underneath configs rcutorture: Rename kvm-test-1-rcu.sh rcutorture: Remove RCU dependencies from ver_functions.sh API rcutorture: Create CFcommon file for common Kconfig parameters rcutorture: Create config files for scripted test-the-test testing rcutorture: Add an rcu_busted to test the test locktorture: Add a lock-torture kernel module rcutorture: Abstract kvm-recheck.sh ...
2 parents 462bf23 + 7de700e commit b3fd4ea

137 files changed

Lines changed: 2454 additions & 1078 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

Documentation/RCU/RTFP.txt

Lines changed: 125 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,14 @@ has lapsed, so this approach may be used in non-GPL software, if desired.
3131
(In contrast, implementation of RCU is permitted only in software licensed
3232
under either GPL or LGPL. Sorry!!!)
3333

34+
In 1987, Rashid et al. described lazy TLB-flush [RichardRashid87a].
35+
At first glance, this has nothing to do with RCU, but nevertheless
36+
this paper helped inspire the update-side batching used in the later
37+
RCU implementation in DYNIX/ptx. In 1988, Barbara Liskov published
38+
a description of Argus that noted that use of out-of-date values can
39+
be tolerated in some situations. Thus, this paper provides some early
40+
theoretical justification for use of stale data.
41+
3442
In 1990, Pugh [Pugh90] noted that explicitly tracking which threads
3543
were reading a given data structure permitted deferred free to operate
3644
in the presence of non-terminating threads. However, this explicit
@@ -41,11 +49,11 @@ providing a fine-grained locking design, however, it would be interesting
4149
to see how much of the performance advantage reported in 1990 remains
4250
today.
4351

44-
At about this same time, Adams [Adams91] described ``chaotic relaxation'',
45-
where the normal barriers between successive iterations of convergent
46-
numerical algorithms are relaxed, so that iteration $n$ might use
47-
data from iteration $n-1$ or even $n-2$. This introduces error,
48-
which typically slows convergence and thus increases the number of
52+
At about this same time, Andrews [Andrews91textbook] described ``chaotic
53+
relaxation'', where the normal barriers between successive iterations
54+
of convergent numerical algorithms are relaxed, so that iteration $n$
55+
might use data from iteration $n-1$ or even $n-2$. This introduces
56+
error, which typically slows convergence and thus increases the number of
4957
iterations required. However, this increase is sometimes more than made
5058
up for by a reduction in the number of expensive barrier operations,
5159
which are otherwise required to synchronize the threads at the end
@@ -55,7 +63,8 @@ is thus inapplicable to most data structures in operating-system kernels.
5563

5664
In 1992, Henry (now Alexia) Massalin completed a dissertation advising
5765
parallel programmers to defer processing when feasible to simplify
58-
synchronization. RCU makes extremely heavy use of this advice.
66+
synchronization [HMassalinPhD]. RCU makes extremely heavy use of
67+
this advice.
5968

6069
In 1993, Jacobson [Jacobson93] verbally described what is perhaps the
6170
simplest deferred-free technique: simply waiting a fixed amount of time
@@ -90,27 +99,29 @@ mechanism, which is quite similar to RCU [Gamsa99]. These operating
9099
systems made pervasive use of RCU in place of "existence locks", which
91100
greatly simplifies locking hierarchies and helps avoid deadlocks.
92101

93-
2001 saw the first RCU presentation involving Linux [McKenney01a]
94-
at OLS. The resulting abundance of RCU patches was presented the
95-
following year [McKenney02a], and use of RCU in dcache was first
96-
described that same year [Linder02a].
102+
The year 2000 saw an email exchange that would likely have
103+
led to yet another independent invention of something like RCU
104+
[RustyRussell2000a,RustyRussell2000b]. Instead, 2001 saw the first
105+
RCU presentation involving Linux [McKenney01a] at OLS. The resulting
106+
abundance of RCU patches was presented the following year [McKenney02a],
107+
and use of RCU in dcache was first described that same year [Linder02a].
97108

98109
Also in 2002, Michael [Michael02b,Michael02a] presented "hazard-pointer"
99110
techniques that defer the destruction of data structures to simplify
100111
non-blocking synchronization (wait-free synchronization, lock-free
101112
synchronization, and obstruction-free synchronization are all examples of
102-
non-blocking synchronization). In particular, this technique eliminates
103-
locking, reduces contention, reduces memory latency for readers, and
104-
parallelizes pipeline stalls and memory latency for writers. However,
105-
these techniques still impose significant read-side overhead in the
106-
form of memory barriers. Researchers at Sun worked along similar lines
107-
in the same timeframe [HerlihyLM02]. These techniques can be thought
108-
of as inside-out reference counts, where the count is represented by the
109-
number of hazard pointers referencing a given data structure rather than
110-
the more conventional counter field within the data structure itself.
111-
The key advantage of inside-out reference counts is that they can be
112-
stored in immortal variables, thus allowing races between access and
113-
deletion to be avoided.
113+
non-blocking synchronization). The corresponding journal article appeared
114+
in 2004 [MagedMichael04a]. This technique eliminates locking, reduces
115+
contention, reduces memory latency for readers, and parallelizes pipeline
116+
stalls and memory latency for writers. However, these techniques still
117+
impose significant read-side overhead in the form of memory barriers.
118+
Researchers at Sun worked along similar lines in the same timeframe
119+
[HerlihyLM02]. These techniques can be thought of as inside-out reference
120+
counts, where the count is represented by the number of hazard pointers
121+
referencing a given data structure rather than the more conventional
122+
counter field within the data structure itself. The key advantage
123+
of inside-out reference counts is that they can be stored in immortal
124+
variables, thus allowing races between access and deletion to be avoided.
114125

115126
By the same token, RCU can be thought of as a "bulk reference count",
116127
where some form of reference counter covers all reference by a given CPU
@@ -123,8 +134,10 @@ can be thought of in other terms as well.
123134

124135
In 2003, the K42 group described how RCU could be used to create
125136
hot-pluggable implementations of operating-system functions [Appavoo03a].
126-
Later that year saw a paper describing an RCU implementation of System
127-
V IPC [Arcangeli03], and an introduction to RCU in Linux Journal
137+
Later that year saw a paper describing an RCU implementation
138+
of System V IPC [Arcangeli03] (following up on a suggestion by
139+
Hugh Dickins [Dickins02a] and an implementation by Mingming Cao
140+
[MingmingCao2002IPCRCU]), and an introduction to RCU in Linux Journal
128141
[McKenney03a].
129142

130143
2004 has seen a Linux-Journal article on use of RCU in dcache
@@ -383,6 +396,21 @@ for Programming Languages and Operating Systems}"
383396
}
384397
}
385398

399+
@phdthesis{HMassalinPhD
400+
,author="H. Massalin"
401+
,title="Synthesis: An Efficient Implementation of Fundamental Operating
402+
System Services"
403+
,school="Columbia University"
404+
,address="New York, NY"
405+
,year="1992"
406+
,annotation={
407+
Mondo optimizing compiler.
408+
Wait-free stuff.
409+
Good advice: defer work to avoid synchronization. See page 90
410+
(PDF page 106), Section 5.4, fourth bullet point.
411+
}
412+
}
413+
386414
@unpublished{Jacobson93
387415
,author="Van Jacobson"
388416
,title="Avoid Read-Side Locking Via Delayed Free"
@@ -671,6 +699,20 @@ Orran Krieger and Rusty Russell and Dipankar Sarma and Maneesh Soni"
671699
[Viewed October 18, 2004]"
672700
}
673701

702+
@conference{Michael02b
703+
,author="Maged M. Michael"
704+
,title="High Performance Dynamic Lock-Free Hash Tables and List-Based Sets"
705+
,Year="2002"
706+
,Month="August"
707+
,booktitle="{Proceedings of the 14\textsuperscript{th} Annual ACM
708+
Symposium on Parallel
709+
Algorithms and Architecture}"
710+
,pages="73-82"
711+
,annotation={
712+
Like the title says...
713+
}
714+
}
715+
674716
@Conference{Linder02a
675717
,Author="Hanna Linder and Dipankar Sarma and Maneesh Soni"
676718
,Title="Scalability of the Directory Entry Cache"
@@ -727,6 +769,24 @@ Andrea Arcangeli and Andi Kleen and Orran Krieger and Rusty Russell"
727769
}
728770
}
729771

772+
@conference{Michael02a
773+
,author="Maged M. Michael"
774+
,title="Safe Memory Reclamation for Dynamic Lock-Free Objects Using Atomic
775+
Reads and Writes"
776+
,Year="2002"
777+
,Month="August"
778+
,booktitle="{Proceedings of the 21\textsuperscript{st} Annual ACM
779+
Symposium on Principles of Distributed Computing}"
780+
,pages="21-30"
781+
,annotation={
782+
Each thread keeps an array of pointers to items that it is
783+
currently referencing. Sort of an inside-out garbage collection
784+
mechanism, but one that requires the accessing code to explicitly
785+
state its needs. Also requires read-side memory barriers on
786+
most architectures.
787+
}
788+
}
789+
730790
@unpublished{Dickins02a
731791
,author="Hugh Dickins"
732792
,title="Use RCU for System-V IPC"
@@ -735,6 +795,17 @@ Andrea Arcangeli and Andi Kleen and Orran Krieger and Rusty Russell"
735795
,note="private communication"
736796
}
737797

798+
@InProceedings{HerlihyLM02
799+
,author={Maurice Herlihy and Victor Luchangco and Mark Moir}
800+
,title="The Repeat Offender Problem: A Mechanism for Supporting Dynamic-Sized,
801+
Lock-Free Data Structures"
802+
,booktitle={Proceedings of 16\textsuperscript{th} International
803+
Symposium on Distributed Computing}
804+
,year=2002
805+
,month="October"
806+
,pages="339-353"
807+
}
808+
738809
@unpublished{Sarma02b
739810
,Author="Dipankar Sarma"
740811
,Title="Some dcache\_rcu benchmark numbers"
@@ -749,6 +820,19 @@ Andrea Arcangeli and Andi Kleen and Orran Krieger and Rusty Russell"
749820
}
750821
}
751822

823+
@unpublished{MingmingCao2002IPCRCU
824+
,Author="Mingming Cao"
825+
,Title="[PATCH]updated ipc lock patch"
826+
,month="October"
827+
,year="2002"
828+
,note="Available:
829+
\url{https://lkml.org/lkml/2002/10/24/262}
830+
[Viewed February 15, 2014]"
831+
,annotation={
832+
Mingming Cao's patch to introduce RCU to SysV IPC.
833+
}
834+
}
835+
752836
@unpublished{LinusTorvalds2003a
753837
,Author="Linus Torvalds"
754838
,Title="Re: {[PATCH]} small fixes in brlock.h"
@@ -982,6 +1066,23 @@ Realtime Applications"
9821066
}
9831067
}
9841068

1069+
@article{MagedMichael04a
1070+
,author="Maged M. Michael"
1071+
,title="Hazard Pointers: Safe Memory Reclamation for Lock-Free Objects"
1072+
,Year="2004"
1073+
,Month="June"
1074+
,journal="IEEE Transactions on Parallel and Distributed Systems"
1075+
,volume="15"
1076+
,number="6"
1077+
,pages="491-504"
1078+
,url="Available:
1079+
\url{http://www.research.ibm.com/people/m/michael/ieeetpds-2004.pdf}
1080+
[Viewed March 1, 2005]"
1081+
,annotation={
1082+
New canonical hazard-pointer citation.
1083+
}
1084+
}
1085+
9851086
@phdthesis{PaulEdwardMcKenneyPhD
9861087
,author="Paul E. McKenney"
9871088
,title="Exploiting Deferred Destruction:

Documentation/RCU/checklist.txt

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -256,10 +256,10 @@ over a rather long period of time, but improvements are always welcome!
256256
variations on this theme.
257257

258258
b. Limiting update rate. For example, if updates occur only
259-
once per hour, then no explicit rate limiting is required,
260-
unless your system is already badly broken. The dcache
261-
subsystem takes this approach -- updates are guarded
262-
by a global lock, limiting their rate.
259+
once per hour, then no explicit rate limiting is
260+
required, unless your system is already badly broken.
261+
Older versions of the dcache subsystem take this approach,
262+
guarding updates with a global lock, limiting their rate.
263263

264264
c. Trusted update -- if updates can only be done manually by
265265
superuser or some other trusted user, then it might not
@@ -268,14 +268,22 @@ over a rather long period of time, but improvements are always welcome!
268268
the machine.
269269

270270
d. Use call_rcu_bh() rather than call_rcu(), in order to take
271-
advantage of call_rcu_bh()'s faster grace periods.
271+
advantage of call_rcu_bh()'s faster grace periods. (This
272+
is only a partial solution, though.)
272273

273274
e. Periodically invoke synchronize_rcu(), permitting a limited
274275
number of updates per grace period.
275276

276277
The same cautions apply to call_rcu_bh(), call_rcu_sched(),
277278
call_srcu(), and kfree_rcu().
278279

280+
Note that although these primitives do take action to avoid memory
281+
exhaustion when any given CPU has too many callbacks, a determined
282+
user could still exhaust memory. This is especially the case
283+
if a system with a large number of CPUs has been configured to
284+
offload all of its RCU callbacks onto a single CPU, or if the
285+
system has relatively little free memory.
286+
279287
9. All RCU list-traversal primitives, which include
280288
rcu_dereference(), list_for_each_entry_rcu(), and
281289
list_for_each_safe_rcu(), must be either within an RCU read-side

Documentation/kernel-per-CPU-kthreads.txt

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -162,7 +162,18 @@ Purpose: Execute workqueue requests
162162
To reduce its OS jitter, do any of the following:
163163
1. Run your workload at a real-time priority, which will allow
164164
preempting the kworker daemons.
165-
2. Do any of the following needed to avoid jitter that your
165+
2. A given workqueue can be made visible in the sysfs filesystem
166+
by passing the WQ_SYSFS to that workqueue's alloc_workqueue().
167+
Such a workqueue can be confined to a given subset of the
168+
CPUs using the /sys/devices/virtual/workqueue/*/cpumask sysfs
169+
files. The set of WQ_SYSFS workqueues can be displayed using
170+
"ls sys/devices/virtual/workqueue". That said, the workqueues
171+
maintainer would like to caution people against indiscriminately
172+
sprinkling WQ_SYSFS across all the workqueues. The reason for
173+
caution is that it is easy to add WQ_SYSFS, but because sysfs is
174+
part of the formal user/kernel API, it can be nearly impossible
175+
to remove it, even if its addition was a mistake.
176+
3. Do any of the following needed to avoid jitter that your
166177
application cannot tolerate:
167178
a. Build your kernel with CONFIG_SLUB=y rather than
168179
CONFIG_SLAB=y, thus avoiding the slab allocator's periodic

0 commit comments

Comments
 (0)