Skip to content

Commit 96fe42f

Browse files
author
Glyn Normington
committed
Refactor memory heuristics
Split memory heuristics into JRE specific strategies and general utilities including MemorySize, MemoryLimit, WeightBalancingMemoryHeuristic, MemoryBucket, and StackMemoryBucket. MemorySize encapsulates all use of memory sizes and arithmetic on memory sizes and enforces proper units and dimensions. MemoryLimit encapsulates converting $MEMORY_LIMIT into a MemorySize. WeightBalancingMemoryHeuristic, together with it helper classes MemoryBucket and StackMemoryBucket, encapsulate the algorithm by which unspecified memory sizes are defaulted and positive or negative excess (in specified memory sizes) is redistributed to unspecified, adjustable memory. StackMemoryBucket is a special kind of MemoryBucket which is not adjustable even it its value is not specified and which calculates its excess based on the default stack size rather than presuming a particular number of threads. Create separate strategies and configurations for OpenJDK before and after the replacement of permgen by metaspace. Move all the above memory classes to the jre/memory directory and this directory causing require logic to fail in buildpack.rb. [#49694523]
1 parent aba9b9b commit 96fe42f

19 files changed

Lines changed: 1403 additions & 417 deletions

‎config/memory_heuristics_openjdk.yml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Cloud Foundry Java Buildpack
2+
# Copyright (c) 2013 the original author or authors.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
# Configuration for WeightBalancingMemoryHeuristic.
17+
#
18+
# Each weighting is a positive decimal fraction less than 1 and is used to indicate the relative amount of memory that
19+
# should be allotted to the corresponding memory category. The sum of the weightings must not exceed 1.
20+
---
21+
heap: 0.5
22+
metaspace: 0.3
23+
stack: 0.1
24+
native: 0.1

config/memory_heuristics.yml renamed to config/memory_heuristics_openjdk_pre8.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
# See the License for the specific language governing permissions and
1414
# limitations under the License.
1515

16-
# Configuration for MemoryHeuristics.
16+
# Configuration for WeightBalancingMemoryHeuristic.
1717
#
1818
# Each weighting is a decimal fraction between 0 and 1 and is used, except in the case of 'native', to calculate the
1919
# corresponding default memory size. The sum of the weightings must not exceed 1.

‎lib/java_buildpack/buildpack.rb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ def self.require_component_files
109109
component_files.concat jre_directory.children
110110

111111
component_files.each do |file|
112-
require file.relative_path_from(root_directory)
112+
require file.relative_path_from(root_directory) unless file.directory?
113113
end
114114
end
115115

‎lib/java_buildpack/jre/memory/memory_bucket.rb‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Cloud Foundry Java Buildpack
2+
# Copyright (c) 2013 the original author or authors.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
require 'java_buildpack/jre/memory/memory_size'
17+
18+
module JavaBuildpack
19+
20+
# A MemoryBucket is used to calculate default sizes for various type of memory
21+
class MemoryBucket
22+
23+
# @!attribute [r] size
24+
# @return [Numeric, nil] the size of the memory bucket in KB or nil if this has not been specified by the user or
25+
# defaulted
26+
attr_reader :size
27+
28+
# Constructs a memory bucket.
29+
#
30+
# @param [String] name a non-empty, human-readable name for this memory bucket, used only in diagnostics
31+
# @param [Numeric] weighting a number between 0 and 1 corresponding to the proportion of total memory which this
32+
# memory bucket should consume by default
33+
# @param [Numeric, nil] size a user-specified size of the memory bucket in KB or nil if the user did not specify a
34+
# size
35+
# @param [Boolean] adjustable whether the size of this memory bucket can grow/shrink or is fixed. If the user
36+
# specified the size of the memory bucket, the size is fixed, regardless of the value of this parameter, although
37+
# the parameter value must still be valid.
38+
# @param [Numeric] total_memory the total virtual memory size of the operating system process in KB
39+
def initialize(name, weighting, size, adjustable, total_memory)
40+
@name = MemoryBucket.validate_name name
41+
@weighting = validate_weighting weighting
42+
@size_specified = size ? validate_memory_size(size, 'size') : nil
43+
@adjustable = (validate_adjustable adjustable) && !@size_specified
44+
@total_memory = validate_memory_size(total_memory, 'total_memory')
45+
@size = @size_specified || default_size
46+
end
47+
48+
# Returns the excess memory in this memory bucket.
49+
#
50+
# @return [Numeric] the excess memory in KB
51+
def excess
52+
@size_specified ? @size_specified - default_size : MemorySize.ZERO
53+
end
54+
55+
# Returns the adjustable weighting of this memory bucket.
56+
#
57+
# @return [Numeric] the adjustable weighting
58+
def adjustable_weighting
59+
@adjustable ? @weighting : 0
60+
end
61+
62+
# Adjusts the size by the appropriate proportion for this memory bucket.
63+
#
64+
# @param [Numeric] total_excess
65+
# @param [Numeric] total_adjustable_weighting
66+
def adjust(total_excess, total_adjustable_weighting)
67+
if @adjustable
68+
if total_adjustable_weighting == 0
69+
@size = MemorySize.ZERO
70+
else
71+
@size = default_size - (total_excess - excess) * @weighting / total_adjustable_weighting
72+
end
73+
end
74+
end
75+
76+
protected
77+
78+
def set_size(size)
79+
@size = size
80+
end
81+
82+
private
83+
84+
def default_size
85+
@total_memory * @weighting
86+
end
87+
88+
89+
def self.validate_name(name)
90+
raise "Invalid MemoryBucket name '#{name}'" if name.nil? || name.to_s.size == 0
91+
name
92+
end
93+
94+
def validate_weighting(weighting)
95+
raise diagnose_weighting(weighting, 'not numeric') unless MemoryBucket.is_numeric weighting
96+
raise diagnose_weighting(weighting, 'negative') if weighting < 0
97+
raise diagnose_weighting(weighting, 'greater than 1') if weighting > 1
98+
weighting
99+
end
100+
101+
def diagnose_weighting(weighting, reason)
102+
"Invalid weighting '#{@weighting}' for #{identify} : #{reason}"
103+
end
104+
105+
def self.is_numeric(w)
106+
Float(w) rescue false
107+
end
108+
109+
def identify
110+
"MemoryBucket #{@name}"
111+
end
112+
113+
def validate_memory_size(size, parameter_name)
114+
raise "Invalid '#{parameter_name}' parameter of class '#{size.class}' for #{identify} : not a MemorySize" unless size.is_a? MemorySize
115+
size
116+
end
117+
118+
def validate_adjustable(adjustable)
119+
raise "Invalid 'adjustable' parameter for #{identify} : not true or false" unless !!adjustable == adjustable
120+
adjustable
121+
end
122+
123+
end
124+
125+
end

‎lib/java_buildpack/jre/memory/memory_heuristics_openjdk.rb‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Cloud Foundry Java Buildpack
2+
# Copyright (c) 2013 the original author or authors.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
require 'java_buildpack/jre/memory/memory_bucket'
17+
require 'java_buildpack/jre/memory/stack_memory_bucket'
18+
require 'java_buildpack/jre/memory/weight_balancing_memory_heuristic'
19+
20+
module JavaBuildpack
21+
22+
# A utility for defaulting Java memory settings.
23+
class MemoryHeuristicsOpenJDK
24+
25+
# @!attribute [r] heap
26+
# @return [String] the maximum heap sizes, e.g. '1M'
27+
attr_reader :heap
28+
29+
# @!attribute [r] metaspace
30+
# @return [String, nil] the maximum metaspace size, e.g. '1M', or nil if the maximum metaspace size is not set
31+
attr_reader :metaspace
32+
33+
# @!attribute [r] stack
34+
# @return [String] the stack size, e.g. '1M'
35+
attr_reader :stack
36+
37+
# Creates an instance based on a hash containing memory settings, a configuration file containing weightings, and the application's memory size in $MEMORY_LIMIT.
38+
def initialize(args)
39+
40+
weight_balancing_memory_heuristic = WeightBalancingMemoryHeuristic.new(MEMORY_HEURISTICS_YAML_FILE, args)
41+
42+
@heap = weight_balancing_memory_heuristic.output['heap']
43+
@metaspace = weight_balancing_memory_heuristic.output['metaspace']
44+
@stack = weight_balancing_memory_heuristic.output['stack']
45+
end
46+
47+
private
48+
49+
MEMORY_HEURISTICS_YAML_FILE = 'memory_heuristics_openjdk.yml'
50+
51+
end
52+
end

‎lib/java_buildpack/jre/memory/memory_heuristics_openjdk_pre8.rb‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Cloud Foundry Java Buildpack
2+
# Copyright (c) 2013 the original author or authors.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
require 'java_buildpack/jre/memory/memory_bucket'
17+
require 'java_buildpack/jre/memory/stack_memory_bucket'
18+
require 'java_buildpack/jre/memory/weight_balancing_memory_heuristic'
19+
20+
module JavaBuildpack
21+
22+
# A utility for defaulting Java memory settings.
23+
class MemoryHeuristicsOpenJDKPre8
24+
25+
# @!attribute [r] heap
26+
# @return [String] the maximum heap size, e.g. '1M'
27+
attr_reader :heap
28+
29+
# @!attribute [r] permgen
30+
# @return [String, nil] the maximum permgen size, e.g. '1M', or nil if the maximum permgen size is not set
31+
attr_reader :permgen
32+
33+
# @!attribute [r] stack
34+
# @return [String] the stack size, e.g. '1M'
35+
attr_reader :stack
36+
37+
# Creates an instance based on a hash containing memory settings, a configuration file containing weightings, and the application's memory size in $MEMORY_LIMIT.
38+
def initialize(args)
39+
40+
weight_balancing_memory_heuristic = WeightBalancingMemoryHeuristic.new(MEMORY_HEURISTICS_YAML_FILE, args)
41+
42+
@heap = weight_balancing_memory_heuristic.output['heap']
43+
@permgen = weight_balancing_memory_heuristic.output['permgen']
44+
@stack = weight_balancing_memory_heuristic.output['stack']
45+
end
46+
47+
private
48+
49+
MEMORY_HEURISTICS_YAML_FILE = 'memory_heuristics_openjdk_pre8.yml'
50+
51+
end
52+
end

‎lib/java_buildpack/jre/memory/memory_limit.rb‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Cloud Foundry Java Buildpack
2+
# Copyright (c) 2013 the original author or authors.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
require 'java_buildpack/jre/memory/memory_size'
17+
18+
module JavaBuildpack
19+
20+
# A utility for handling Java memory settings.
21+
class MemoryLimit
22+
23+
# Returns the application's memory limit.
24+
# @return [MemorySize] the application's memory limit
25+
def self.memory_limit
26+
memory_limit = ENV['MEMORY_LIMIT']
27+
raise ":memory_limit not specified in $MEMORY_LIMIT" unless memory_limit
28+
memory_limit_size = MemorySize.new(memory_limit)
29+
raise "Invalid negative $MEMORY_LIMIT #{memory_limit}" if memory_limit_size < MemorySize.ZERO
30+
memory_limit_size
31+
end
32+
33+
end
34+
end

0 commit comments

Comments
 (0)