@@ -8,10 +8,13 @@ sometimes also called dog-piling, cache miss storm, cache choking, or the
88thundering herd problem. Let's look at how that applies to landing page
99caching.
1010
11- >>> import time
12- >>> def generate_landing_page ():
13- ... time.sleep(0.2 ) # Work really hard.
14- ... # Return HTML response.
11+ .. code-block :: python
12+
13+ import time
14+
15+ def generate_landing_page ():
16+ time.sleep(0.2 ) # Work really hard.
17+ # Return HTML response.
1518
1619 Imagine a website under heavy load with a function used to generate the landing
1720page. There are five processes each with two threads for a total of ten
@@ -24,11 +27,16 @@ When we look at the number of concurrent workers and the latency with no
2427caching at all, the graph looks as above. Notice each worker constantly
2528regenerates the page with a consistently slow latency.
2629
27- >>> import diskcache as dc
28- >>> cache = dc.Cache()
29- >>> @ cache.memoize(expire = 1 )
30- ... def generate_landing_page ():
31- ... time.sleep(0.2 )
30+ .. code-block :: python
31+ :emphasize- lines: 5
32+
33+ import diskcache as dc
34+
35+ cache = dc.Cache()
36+
37+ @cache.memoize (expire = 1 )
38+ def generate_landing_page ():
39+ time.sleep(0.2 )
3240
3341 Assume the result of generating the landing page can be memoized for one
3442second. Memoization supports a traditional caching strategy. After each second,
@@ -43,13 +51,16 @@ the landing page requires significant resources then the spikes may be
4351prohibitive.
4452
4553To reduce the number of concurrent workers, a barrier can be used to
46- synchronize generating the landing page::
54+ synchronize generating the landing page.
4755
48- >>> @cache.memoize(expire=0)
49- ... @dc.barrier(cache, dc.Lock)
50- ... @cache.memoize(expire=1)
51- ... def generate_landing_page():
52- ... time.sleep(0.2)
56+ .. code-block :: python
57+ :emphasize- lines: 1 ,2 ,3
58+
59+ @cache.memoize (expire = 0 )
60+ @dc.barrier (cache, dc.Lock)
61+ @cache.memoize (expire = 1 )
62+ def generate_landing_page ():
63+ time.sleep(0.2 )
5364
5465 The double-checked locking uses two memoization decorators to optimistically
5566look up the cached result before locking. With `expire ` set to zero, the
@@ -69,9 +80,12 @@ recomputation would be a function of the number of workers, the expiration
6980time, and the duration of computation. Fortunately, Vattani, et al. published
7081the solution in "Optimal Probabilistic Cache Stampede Prevention" in 2015.
7182
72- >>> @ dc.memoize_stampede(cache, expire = 1 )
73- ... def generate_landing_page ():
74- ... time.sleep(0.2 )
83+ .. code-block :: python
84+ :emphasize- lines: 1
85+
86+ @dc.memoize_stampede (cache, expire = 1 )
87+ def generate_landing_page ():
88+ time.sleep(0.2 )
7589
7690 Early probabilistic recomputation uses a random number generator to simulate a
7791cache miss prior to expiration. The new result is then computed in a separate
@@ -86,9 +100,12 @@ cache. Behind the scenes, separate threads of execution are recomputing the
86100result of workers and updating the cache. The concurrency graph shows a nearly
87101constant stream of workers recomputing the function's result.
88102
89- >>> @ dc.memoize_stampede(cache, expire = 1 , beta = 0.5 )
90- ... def generate_landing_page ():
91- ... time.sleep(0.2 )
103+ .. code-block :: python
104+ :emphasize- lines: 1
105+
106+ @dc.memoize_stampede (cache, expire = 1 , beta = 0.5 )
107+ def generate_landing_page ():
108+ time.sleep(0.2 )
92109
93110 Vattani described an additional parameter, :math: `\beta `, which could be used
94111to tune the eagerness of recomputation. As the number and frequency of
@@ -102,9 +119,12 @@ Latency is now still its theoretical best while the worker load has decreased
102119significantly. The likelihood of simulated cache misses is now half what it was
103120before. The value was determined through experimentation.
104121
105- >>> @ dc.memoize_stampede(cache, expire = 1 , beta = 0.3 )
106- ... def generate_landing_page ():
107- ... time.sleep(0.2 )
122+ .. code-block :: python
123+ :emphasize- lines: 1
124+
125+ @dc.memoize_stampede (cache, expire = 1 , beta = 0.3 )
126+ def generate_landing_page ():
127+ time.sleep(0.2 )
108128
109129 Lets see what happens when :math: `\beta ` is set too low.
110130
0 commit comments