Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8324649: Shenandoah: replace implementation of free set #17561

Closed
wants to merge 126 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
126 commits
Select commit Hold shift + click to select a range
702710e
Improve documentation of how Evac-OOM Protocol works
kdnilsen Jan 12, 2024
61b575f
Merge branch 'openjdk:master' into master
kdnilsen Jan 17, 2024
51d056f
Revert "Improve documentation of how Evac-OOM Protocol works"
kdnilsen Jan 17, 2024
bb936f9
Replace ShenandoahFreeSet impl for better generality
kdnilsen Jan 23, 2024
ba98e42
Merge branch 'openjdk:master' into master
kdnilsen Jan 23, 2024
1dc47d6
Move collector free to mutator free at start of update refs
kdnilsen Jan 24, 2024
836e219
Clean up comments and remove debug instrumentation
kdnilsen Jan 24, 2024
bc3b7a9
Fix up white space and comments
kdnilsen Jan 24, 2024
441487c
Merge branch 'openjdk:master' into master
kdnilsen Jan 25, 2024
5bb309a
Remove unnecessary change related to debugging
kdnilsen Jan 25, 2024
90295ae
Remove comments regarding GenShen-specific behavior
kdnilsen Jan 26, 2024
ba8cd03
Remove unnecessary include
kdnilsen Jan 26, 2024
813ba6e
Fix typo in comment
kdnilsen Jan 27, 2024
0a66217
Replace ShenandoahFreeMemoryType with ShenandoahFreeSetRegionType
kdnilsen Jan 30, 2024
98ab312
Change ShenandoahSetsOfFreeRegions to ShenandoahRegionPartition
kdnilsen Jan 30, 2024
00e66e5
More consistent naming of partitions vs sets
kdnilsen Jan 31, 2024
3c06018
Clarify preconditions for allocate_single and allocate_contiguous
kdnilsen Jan 31, 2024
08f011a
Fix comment in allocate_single()
kdnilsen Jan 31, 2024
59c0602
Remove mention of vestigial evac_expended in comment
kdnilsen Jan 31, 2024
6aabfab
Change clear_internal() to make_all_regions_unavailable()
kdnilsen Jan 31, 2024
5f7e90a
Change shrink_bounds_if_touched to shrink_range_if_boundary_modified
kdnilsen Jan 31, 2024
758ecf9
Add comments to describe _leftmosts and _rightmosts and _empty ranges
kdnilsen Jan 31, 2024
7816f3a
Change names of shrink_bounds_if_touched and expand_bounds_maybe
kdnilsen Jan 31, 2024
fb1f5bf
Rename and comments for _capacity_of and _used_by
kdnilsen Jan 31, 2024
dafc363
Merge branch 'openjdk:master' into master
kdnilsen Feb 2, 2024
ae345f9
Replace line accidentally deleted
kdnilsen Feb 2, 2024
97d9e7d
Rename ShenandoahRegionPartition to ShenandoahRegionPartitions
kdnilsen Feb 2, 2024
f44ffb2
Fix comments re: _leftmosts_empty and _rightmosts_empty
kdnilsen Feb 2, 2024
5e9084f
In comments replace consumed with allocated
kdnilsen Feb 3, 2024
3cabc3c
Fix comment that describes empty free set condition
kdnilsen Feb 3, 2024
8bbbaf6
Adjust enum ShenandoahFreeSetPartitionId to clarify NotFree is not a …
kdnilsen Feb 3, 2024
9c9bcdc
Change loop iterator to ssize_t from int
kdnilsen Feb 3, 2024
48ba59a
Rename move_regions_from_collector_to_mutator_partition
kdnilsen Feb 3, 2024
1c9c764
Remove unhelpful comment that might cause undue concern to maintainers
kdnilsen Feb 3, 2024
3c666eb
Rename retire_within_partition
kdnilsen Feb 3, 2024
b40844d
Fix description of move_regions_from_collector_to_mutator
kdnilsen Feb 3, 2024
59fbd2b
Use ssize_t for iterating over partition regions
kdnilsen Feb 3, 2024
f754e46
Fix comment describing retirement of regions following failed allocation
kdnilsen Feb 3, 2024
96d9fa6
Remove extraneous assertion
kdnilsen Feb 3, 2024
5e27a58
Correct an invalid assertion
kdnilsen Feb 3, 2024
07fe812
Combine first two passes over freeset during rebuild
kdnilsen Feb 7, 2024
7d5c1fc
Fix whitespace
kdnilsen Feb 7, 2024
b2ba4cf
Respond to review feedback
kdnilsen Feb 7, 2024
655e30f
Experiment with proposed ShenandoahPackEvacTightly option
kdnilsen Feb 12, 2024
b34d03b
Revert "Experiment with proposed ShenandoahPackEvacTightly option"
kdnilsen Feb 13, 2024
daacbaa
Use bitmap to represent each freeset for improved performance
kdnilsen Feb 19, 2024
3ec905e
Fix an error in search for contiguous regions
kdnilsen Feb 20, 2024
c4c252e
Merge branch 'openjdk:master' into master
kdnilsen Feb 21, 2024
fb23e41
Merge remote-tracking branch 'origin/master' into restructure-free-se…
kdnilsen Feb 21, 2024
bb5390a
Refinements to improve performance
kdnilsen Feb 21, 2024
2ef2c08
Bug fixes and performance improvements
kdnilsen Feb 23, 2024
bd0e1e8
Fix whitespace
kdnilsen Feb 23, 2024
e692b27
Fix off-by-one error in is_forward_consecutive_ones()
kdnilsen Feb 27, 2024
85e76af
Two bug fixes for better performance
kdnilsen Feb 28, 2024
41ba86a
Merge branch 'openjdk:master' into master
kdnilsen Feb 28, 2024
3371e9d
Merge remote-tracking branch 'origin/master' into restructure-free-set
kdnilsen Feb 28, 2024
dc47f3f
Fix white space
kdnilsen Feb 28, 2024
44008aa
Fix 32-bit size formatting in log messages
kdnilsen Feb 29, 2024
5d9194e
Address 32-bit compile issues
kdnilsen Feb 29, 2024
1aa5a3e
Remove instrumentation and cleanup magic numbers
kdnilsen Feb 29, 2024
f215a70
Merge branch 'openjdk:master' into master
kdnilsen Mar 1, 2024
c2f9556
Simplify log_info and log_debug parameters
kdnilsen Mar 1, 2024
42af76b
Respond to reviewer feedback and improve comments
kdnilsen Mar 2, 2024
a45f976
Fix initialization order for ShenandoahFreeSet
kdnilsen Mar 2, 2024
2526384
Revert redundant change from https://github.com/openjdk/jdk/pull/18083
kdnilsen Mar 3, 2024
2b9a470
Add unit tests for ShenandoahSimpleBitMap
kdnilsen Mar 4, 2024
9be1a16
Correct errors in ShenandoahSimpleBitMap unit tests
kdnilsen Mar 4, 2024
d9359a3
Fix whitespace
kdnilsen Mar 4, 2024
8f3dbaf
Fix typo
kdnilsen Mar 4, 2024
12327c0
Fix another typo
kdnilsen Mar 4, 2024
4c1cf6a
Fix compilation errors for ShenandoahSimpleBitMap test
kdnilsen Mar 5, 2024
db7506e
Add new source file
kdnilsen Mar 5, 2024
f9d61f2
Fix windows compiler errors
kdnilsen Mar 5, 2024
a0b6635
Fix another compiler error
kdnilsen Mar 5, 2024
9b50307
Experiment: force gtest failure to confirm gtest is exercised
kdnilsen Mar 5, 2024
4c82947
Changes to Simple BitMap test
kdnilsen Mar 5, 2024
6d71c32
Fix gtest for ShenandoahSimpleBitMap and fix ShennadoahSimpleBitMap bugs
kdnilsen Mar 7, 2024
c045e9f
Use existing macros for bit twiddling
kdnilsen Mar 13, 2024
46989ac
Reduce strength of bit manipulation operations
kdnilsen Mar 13, 2024
baaca57
Move comments that describe functions into header file
kdnilsen Mar 13, 2024
f58ef72
Add comment to describe flip_to_gc()
kdnilsen Mar 13, 2024
94909f7
Revert accidental deletion of blank line
kdnilsen Mar 13, 2024
3f8749a
Revert another accidental deletion of a blank line
kdnilsen Mar 13, 2024
9cdb82e
Restore call to notify_Mutator_alloc_words() following humongous alloc
kdnilsen Mar 13, 2024
3ce9022
Rewrite comment describing rationale for Collector free set membership
kdnilsen Mar 13, 2024
57d2e71
Remove comment that pertains only to generational mode
kdnilsen Mar 13, 2024
eebb78a
Exploit count_<leading|trailing>_zeros instructions
kdnilsen Mar 21, 2024
d65c2ea
Give special handling to count-zeros when arg is 0
kdnilsen Mar 21, 2024
38faeea
Fix one bug and add disabled instrumentation for debugging
kdnilsen Mar 22, 2024
7ced517
Fix typo in log message for 32-bit platforms
kdnilsen Mar 22, 2024
b320a3a
Add a few more conditions to ShenandoahSimpleBitMapTest
kdnilsen Mar 22, 2024
04d76e8
Fix for certain compilers
kdnilsen Mar 26, 2024
4d6b5cd
Merge branch 'openjdk:master' into master
kdnilsen Mar 26, 2024
7bc418f
Merge remote-tracking branch 'origin/master' into restructure-free-set
kdnilsen Mar 26, 2024
84c5875
Remove debugging instrumentation
kdnilsen Mar 28, 2024
7fe605f
Merge branch 'openjdk:master' into master
kdnilsen Mar 28, 2024
e536520
Separate ShenandoahSimpleBitMap into distinct source files
kdnilsen Apr 13, 2024
b64739a
Make ShenadoahFreeSetPartitionId an enum class
kdnilsen Apr 13, 2024
0cdd774
Cosmetic reformat after edits that changed source line length
kdnilsen Apr 16, 2024
4a646ec
Do not treat integer values as booleans
kdnilsen Apr 16, 2024
43e851d
Replace tail recursion with iteration
kdnilsen Apr 16, 2024
4b1d6a3
Fix indentation of function declaration
kdnilsen Apr 16, 2024
8c2d801
Respond to reviewer suggestions
kdnilsen Apr 17, 2024
24ccf88
Fix typo
kdnilsen Apr 22, 2024
9890693
Add include for standard defs
kdnilsen Apr 29, 2024
b69a4bf
Fix whitespace
kdnilsen Apr 29, 2024
3bf3df2
Refinements to support zero-build compiles
kdnilsen Apr 30, 2024
06909ba
Fix NumPartition type
kdnilsen Apr 30, 2024
5c97323
Simplify by combining implemnetations of shrink_interval functions
kdnilsen Apr 30, 2024
2a83e0a
Simplify partition_membership_name by code reuse
kdnilsen Apr 30, 2024
d4c9072
Assert progress on find_next and find_prev
kdnilsen Apr 30, 2024
8f5ea03
Remove unnecessary call to update_watermark
kdnilsen Apr 30, 2024
2e224f6
Merge branch 'openjdk:master' into master
kdnilsen Apr 30, 2024
d6e3546
Merge remote-tracking branch 'origin/master' into restructure-free-set
kdnilsen Apr 30, 2024
46ad5c6
Merge branch 'openjdk:master' into master
kdnilsen May 3, 2024
4f09d33
Merge remote-tracking branch 'origin/master' into restructure-free-set
kdnilsen May 3, 2024
be3a76d
Respond to reviewer comments
kdnilsen May 7, 2024
0fe4392
Improve comment to clarify loop of find_first_consecutive_set_bits()
kdnilsen May 7, 2024
0bc2f9d
Fix typo in comment
kdnilsen May 7, 2024
bb2a5ac
Another attempt at improving clarity of comments
kdnilsen May 7, 2024
ced947f
Respond to reviewer feedback
kdnilsen May 13, 2024
572aa2b
Fix copyright notice on test file
kdnilsen May 13, 2024
3ed62eb
Directly address range arrays when adjusting intervals
kdnilsen May 15, 2024
fe76f49
Improvements to dump_bitmap_* services
kdnilsen May 15, 2024
93761ed
Rename raw_assign_membership()
kdnilsen May 15, 2024
3219119
Rename establish_mutator_intervals
kdnilsen May 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp
Original file line number Diff line number Diff line change
@@ -964,7 +964,7 @@ HeapWord* ShenandoahFreeSet::allocate_single(ShenandoahAllocRequest& req, bool&
flip_to_gc(r);
HeapWord *result = try_allocate_in(r, req, in_new_region);
if (result != nullptr) {
log_debug(gc, free)("Flipped region " SIZE_FORMAT " to gc for request: " PTR_FORMAT, idx, p2i(&req));
log_debug(gc)("Flipped region " SIZE_FORMAT " to gc for request: " PTR_FORMAT, idx, p2i(&req));
return result;
}
}
@@ -992,7 +992,7 @@ HeapWord* ShenandoahFreeSet::try_allocate_in(ShenandoahHeapRegion* r, Shenandoah
in_new_region = r->is_empty();

if (in_new_region) {
log_debug(gc, free)("Using new region (" SIZE_FORMAT ") for %s (" PTR_FORMAT ").",
log_debug(gc)("Using new region (" SIZE_FORMAT ") for %s (" PTR_FORMAT ").",
r->index(), ShenandoahAllocRequest::alloc_type_to_string(req.type()), p2i(&req));
}

@@ -1006,7 +1006,7 @@ HeapWord* ShenandoahFreeSet::try_allocate_in(ShenandoahHeapRegion* r, Shenandoah
}
if (adjusted_size >= req.min_size()) {
result = r->allocate(adjusted_size, req.type());
log_debug(gc, free)("Allocated " SIZE_FORMAT " words (adjusted from " SIZE_FORMAT ") for %s @" PTR_FORMAT
log_debug(gc)("Allocated " SIZE_FORMAT " words (adjusted from " SIZE_FORMAT ") for %s @" PTR_FORMAT
" from %s region " SIZE_FORMAT ", free bytes remaining: " SIZE_FORMAT,
adjusted_size, req.size(), ShenandoahAllocRequest::alloc_type_to_string(req.type()), p2i(result),
_partitions.partition_membership_name(r->index()), r->index(), r->free());
@@ -1021,7 +1021,7 @@ HeapWord* ShenandoahFreeSet::try_allocate_in(ShenandoahHeapRegion* r, Shenandoah
result = r->allocate(size, req.type());
if (result != nullptr) {
// Record actual allocation size
log_debug(gc, free)("Allocated " SIZE_FORMAT " words for %s @" PTR_FORMAT
log_debug(gc)("Allocated " SIZE_FORMAT " words for %s @" PTR_FORMAT
" from %s region " SIZE_FORMAT ", free bytes remaining: " SIZE_FORMAT,
size, ShenandoahAllocRequest::alloc_type_to_string(req.type()), p2i(result),
_partitions.partition_membership_name(r->index()), r->index(), r->free());
@@ -1250,7 +1250,7 @@ void ShenandoahFreeSet::find_regions_with_alloc_capacity(size_t &cset_regions) {
mutator_regions++;
mutator_used += (region_size_bytes - ac);

log_debug(gc, free)(
log_debug(gc)(
" Adding Region " SIZE_FORMAT " (Free: " SIZE_FORMAT "%s, Used: " SIZE_FORMAT "%s) to mutator partition",
idx, byte_size_in_proper_unit(region->free()), proper_unit_for_byte_size(region->free()),
byte_size_in_proper_unit(region->used()), proper_unit_for_byte_size(region->used()));
@@ -1305,16 +1305,16 @@ void ShenandoahFreeSet::move_regions_from_collector_to_mutator(size_t max_xfer_r
}

size_t collector_xfer = collector_empty_xfer + collector_not_empty_xfer;
log_info(gc, free)("At start of update refs, moving " SIZE_FORMAT "%s to Mutator free partition from Collector Reserve",
byte_size_in_proper_unit(collector_xfer), proper_unit_for_byte_size(collector_xfer));
log_info(gc)("At start of update refs, moving " SIZE_FORMAT "%s to Mutator free partition from Collector Reserve",
byte_size_in_proper_unit(collector_xfer), proper_unit_for_byte_size(collector_xfer));
}


// Overwrite arguments to represent the number of regions to be reclaimed from the cset
void ShenandoahFreeSet::prepare_to_rebuild(size_t &cset_regions) {
shenandoah_assert_heaplocked();

log_debug(gc, free)("Rebuilding FreeSet");
log_debug(gc)("Rebuilding FreeSet");

// This places regions that have alloc_capacity into the mutator partition.
find_regions_with_alloc_capacity(cset_regions);
@@ -1373,15 +1373,15 @@ void ShenandoahFreeSet::reserve_regions(size_t to_reserve) {
// tend to mix survivor objects with ephemeral objects, making it more difficult to reclaim the memory for the
// ephemeral objects.
_partitions.move_from_partition_to_partition(idx, Mutator, Collector, ac);
log_debug(gc,free)(" Shifting region " SIZE_FORMAT " from mutator_free to collector_free", idx);
log_debug(gc)(" Shifting region " SIZE_FORMAT " from mutator_free to collector_free", idx);
}
}

if (LogTarget(Info, gc, free)::is_enabled()) {
size_t reserve = _partitions.capacity_of(Collector);
if (reserve < to_reserve) {
log_info(gc, free)("Wanted " PROPERFMT " for young reserve, but only reserved: " PROPERFMT,
PROPERFMTARGS(to_reserve), PROPERFMTARGS(reserve));
log_debug(gc)("Wanted " PROPERFMT " for young reserve, but only reserved: " PROPERFMT,
PROPERFMTARGS(to_reserve), PROPERFMTARGS(reserve));
}
}
}
@@ -1403,9 +1403,9 @@ void ShenandoahFreeSet::log_status() {
for (uint i = 0; i < BUFFER_SIZE; i++) {
buffer[i] = '\0';
}
log_debug(gc, free)("FreeSet map legend:"
log_debug(gc)("FreeSet map legend:"
" M:mutator_free C:collector_free H:humongous _:retired");
log_debug(gc, free)(" mutator free range [" SIZE_FORMAT ".." SIZE_FORMAT "], "
log_debug(gc)(" mutator free range [" SIZE_FORMAT ".." SIZE_FORMAT "], "
" collector free range [" SIZE_FORMAT ".." SIZE_FORMAT "]",
_partitions.leftmost(Mutator), _partitions.rightmost(Mutator),
_partitions.leftmost(Collector), _partitions.rightmost(Collector));
@@ -1414,7 +1414,7 @@ void ShenandoahFreeSet::log_status() {
ShenandoahHeapRegion *r = _heap->get_region(i);
uint idx = i % 64;
if ((i != 0) && (idx == 0)) {
log_debug(gc, free)(" %6u: %s", i-64, buffer);
log_debug(gc)(" %6u: %s", i-64, buffer);
}
if (_partitions.in_free_set(Mutator, i)) {
size_t capacity = alloc_capacity(r);
@@ -1438,7 +1438,7 @@ void ShenandoahFreeSet::log_status() {
} else {
remnant = 64;
}
log_debug(gc, free)(" %6u: %s", (uint) (_heap->num_regions() - remnant), buffer);
log_debug(gc)(" %6u: %s", (uint) (_heap->num_regions() - remnant), buffer);
}
#endif