Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8291555: Implement alternative fast-locking scheme #10907

Closed
wants to merge 175 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
175 commits
Select commit Hold shift + click to select a range
5c0e46f
Initial stuff
rkennke Oct 24, 2022
84639ab
8295849: Consolidate Threads::owning_thread*
rkennke Oct 24, 2022
a0dc5f1
Merge branch 'JDK-8295849' into JDK-8291555-v2
rkennke Oct 25, 2022
ca225c9
Fix OM::has_owner()
Oct 25, 2022
04c780d
Improve condition in OM::has_owner()
rkennke Oct 25, 2022
37fa31b
Fix has_owner() condition
rkennke Oct 27, 2022
be0e9be
Merge branch 'JDK-8295849' into JDK-8291555-v2
rkennke Oct 28, 2022
1abb143
Implement most runtime and aarch64 parts
rkennke Oct 28, 2022
229a745
Couple of fixes
rkennke Oct 29, 2022
adcedbf
More fixes
rkennke Oct 29, 2022
7d30304
Implement SA parts, and more fixes
rkennke Oct 30, 2022
8d3a209
Implement x86_64 parts
rkennke Oct 31, 2022
535c776
More fixes
rkennke Oct 31, 2022
b28fca5
Merge remote-tracking branch 'upstream/master' into JDK-8291555-v2
rkennke Nov 8, 2022
7510c2d
Fix tabs
rkennke Nov 8, 2022
3430353
Fix x86_32
rkennke Nov 8, 2022
6876a3a
Start of optimization to avoid checking lock-stack size
Nov 15, 2022
c5749e3
RISCV parts
rkennke Nov 15, 2022
b0f86a8
8297036: Generalize C2 stub mechanism
Nov 16, 2022
e4ffc19
Merge branch 'JDK-8297036' into JDK-8291555-v2
Nov 16, 2022
5a04929
Running prototype of optimization
Nov 16, 2022
9e7acca
Re-init labels after code size estimate
Nov 16, 2022
7d3bc57
Re-init guard label in C2EntryBarrierStub
Nov 16, 2022
bd2403a
Rename file
Nov 16, 2022
c4c68ac
Extract C2CodeStubs into separate file
Nov 16, 2022
de65513
Don't expose stubs list
Nov 16, 2022
8325a01
x86_32 fixes
rkennke Nov 17, 2022
778b82b
Add missing include, minimal build could not find INCLUDE_RTM_OPTS
rkennke Nov 17, 2022
7b88d74
AArch64 parts
rkennke Nov 17, 2022
22359a6
Merge remote-tracking branch 'origin/JDK-8297036' into JDK-8297036
rkennke Nov 17, 2022
d186fa3
Rename files c2* to c2_*
rkennke Nov 17, 2022
f5b4dbd
Rename files in includes, too. Duh.
Nov 17, 2022
e5b1ab4
Merge remote-tracking branch 'origin/JDK-8291555-v2' into JDK-8291555-v2
Nov 17, 2022
ef996bf
Merge branch 'JDK-8297036' into JDK-8291555-v2
Nov 17, 2022
7f8199b
C1 parts of lock-stack checking optimization
Nov 17, 2022
c4caff1
Optimize fast-lock-enter, x86_64 parts
Nov 18, 2022
2473a52
Use compile arena to allocate stub list
Nov 18, 2022
c67fd49
PPC parts
Nov 18, 2022
51672ff
RISCV parts
Nov 18, 2022
38ae0bf
Add missing include (PPC)
Nov 18, 2022
85cd0f3
Some fixes (RISCV)
Nov 18, 2022
64ace9e
Add virtual destructor to C2CodeStub
Nov 18, 2022
ae0f230
Revert "Add virtual destructor to C2CodeStub"
Nov 18, 2022
86a70df
Simpler fast-paths in fast-locking
rkennke Nov 21, 2022
7ab52fc
Don't push/pop register around lock-stack-check fast-path
rkennke Nov 21, 2022
cc7ee49
Remove unneeded changes
rkennke Nov 21, 2022
355a186
Rename new platform files
Nov 22, 2022
30a2223
RISCV fixes
Nov 22, 2022
24518f4
x86_32 parts
rkennke Nov 22, 2022
9b2757b
AArch64 fixes
rkennke Nov 23, 2022
c33c0b2
AArch64: optimize away stack check before fast-locking
rkennke Nov 23, 2022
d4c2d6d
Merge branch 'master' into JDK-8291555-v2
rkennke Nov 23, 2022
83d6ab0
Avoid unnecessary branching after fast_unlock_impl in C2
rkennke Nov 23, 2022
802b5f8
Attempt at RISCV fixes
rkennke Nov 24, 2022
01fdaab
Revert dangling change in TestFloadSyncJNIArgs.java test
rkennke Nov 24, 2022
617a2c9
Move lock-stack to JavaThread
rkennke Nov 24, 2022
ed2ab01
Don't measure code size in advance, but verify it after emitting
Nov 29, 2022
1b4541f
AArch64 parts
rkennke Nov 29, 2022
604f2a4
x86_32 fix
rkennke Nov 29, 2022
438f00f
PPC fix
rkennke Nov 29, 2022
cdedf27
More RISCV fixes
rkennke Nov 30, 2022
e9a4461
Merge branch 'JDK-8297036' into JDK-8291555-v2
rkennke Dec 6, 2022
ed80b3f
Resolve merge conflicts
rkennke Dec 6, 2022
dd5454e
Merge remote-tracking branch 'origin/JDK-8291555-v2' into JDK-8291555-v2
rkennke Dec 6, 2022
e4bad77
Merge remote-tracking branch 'upstream/master' into JDK-8291555-v2
rkennke Dec 12, 2022
9c4045b
Merge remote-tracking branch 'origin/JDK-8291555-v2' into JDK-8291555-v2
rkennke Dec 12, 2022
44228a5
Adjust size for check-lock-stub
Dec 12, 2022
38909f2
Fix call to build_frame()
rkennke Dec 13, 2022
c6382ae
RISCV build fixes
rkennke Dec 13, 2022
964a2a0
Optimize fast-locking in aarch64
rkennke Dec 13, 2022
6aaec5a
Some refactorings (aarch64)
rkennke Dec 13, 2022
84dc812
Remove stray include line (PPC)
rkennke Dec 13, 2022
64e0457
Some refactorings (x86)
Dec 13, 2022
e8e82b5
Refactorings and cleanups (runtime)
Dec 13, 2022
1fbf2cc
Fix method signature of build_frame() on ppc, arm and s390
rkennke Dec 14, 2022
36eb62f
Remove unnecessary assert
Dec 15, 2022
5e1926a
Merge remote-tracking branch 'origin/JDK-8291555-v2' into JDK-8291555-v2
Dec 16, 2022
3431658
Optimize the check-for-anon-owner fast-path
rkennke Jan 12, 2023
9c30b95
x86 part of optimization to check for anon owner
Jan 12, 2023
cc93924
In x86_32 C2 fast_lock(), CAS thread directly, instead of CASing stac…
rkennke Jan 13, 2023
4a08ea6
Merge branch 'master' into JDK-8291555-v2
rkennke Jan 13, 2023
c474671
Use testb when testing for anon owner in fast-path
rkennke Jan 17, 2023
a270919
Properly set ZF on anon-check path; avoid some conditional branches
rkennke Jan 18, 2023
c35e5a3
Change log message inflate(locked) -> inflate(has_locker)
rkennke Jan 18, 2023
4504c1c
Revert UseFastLocking to default to off
rkennke Jan 18, 2023
784b361
Merge branch 'master' into JDK-8291556-v2
rkennke Jan 26, 2023
77c8a82
8291555-dcubed-editorial-1
dcubed-ojdk Jan 27, 2023
bbe8c18
Daniel's suggested changes
rkennke Jan 30, 2023
74f3974
Fix anon owner in fast-path, avoid runtime call (x86_64)
Feb 9, 2023
38dae00
Fix anon owner in fast-path, avoid runtime call (aarch64)
rkennke Feb 9, 2023
2af1ce3
Small fixes
rkennke Feb 9, 2023
8bdeda3
Merge branch 'master' into JDK-8291555-v2
rkennke Feb 14, 2023
ed611b0
Fix merge error (move done label into correct places)
Feb 15, 2023
9d4ca05
Fix interpreter asymmetric fast-locking
rkennke Mar 7, 2023
12c2b8c
Use realloc instead of malloc+copy when growing the lock-stack
Mar 8, 2023
3beabad
Inline initial LockStack stack
rkennke Mar 8, 2023
3c9d0d8
Merge remote-tracking branch 'origin/JDK-8291555-v2' into JDK-8291555-v2
rkennke Mar 8, 2023
34cedc2
Various small fixes and improvements
rkennke Mar 9, 2023
f9f93b3
Merge branch 'master' into JDK-8291555-v2
rkennke Mar 9, 2023
51a00e9
Fixes in response to Daniel's review
rkennke Mar 10, 2023
8ba676a
Fixes in response to Daniel's review
Mar 10, 2023
8aad280
Use nullptr instead of NULL in touched code
rkennke Mar 10, 2023
4800ac4
Use nullptr instead of NULL in touched code (riscv)
Mar 10, 2023
27a4b10
Use nullptr instead of NULL in touched code (x86)
Mar 10, 2023
b59b20a
Use nullptr instead of NULL in touched code (shared)
rkennke Mar 10, 2023
5fe2afc
Merge remote-tracking branch 'origin/JDK-8291555-v2' into JDK-8291555-v2
rkennke Mar 10, 2023
0b7be89
Re-design LockStack for faster lock-stack depth-check
rkennke Mar 13, 2023
75db4f0
X86 parts
Mar 13, 2023
87b95bf
Use -w instructions in new locking code stubs (aarch64)
rkennke Mar 14, 2023
2f57209
SA fixes related to latest changes in LockStack
Mar 14, 2023
b834f0c
Change log message when inflating fast-locked object
Mar 14, 2023
dc6e930
Increase stub size of C2HandleAnonOwnerStub to 18
rkennke Mar 15, 2023
90bde3d
Use -w instructions in fast_unlock()
rkennke Mar 15, 2023
0ad01c1
More RISCV changes (by Fei Yang)
rkennke Mar 15, 2023
2445a19
Several changes (mostly cosmetic) in response to reviews
Mar 16, 2023
d8dd329
Set condition flags correctly after fast-lock call on aarch64
rkennke Mar 16, 2023
37f061b
Merge remote-tracking branch 'origin/JDK-8291555-v2' into JDK-8291555-v2
rkennke Mar 16, 2023
32fdda2
Ensure safepoint when processing lock-stack
Mar 27, 2023
d4da05d
(x86_32) Use existing thread register in fast_unlock() instead of fet…
rkennke Mar 28, 2023
7ccc4b6
Use long branches at start of fast_lock() and fast_unlock() unconditi…
rkennke Mar 28, 2023
de45c08
Address some of @dholmes-ora' review comments
rkennke Mar 28, 2023
798615f
Address @shipilev review comments (x86)
Mar 28, 2023
aa24deb
Process lock-stack oops before inspecting them, when in foreign threa…
rkennke Mar 29, 2023
885eb61
Relax verification in oops_do(), put back UseFastLocking in managemen…
Mar 29, 2023
62298e4
Bounds check in lock-stack verification; only do watermark if we have…
rkennke Mar 29, 2023
e5afb43
More verification through zapping unused entries; get rid of _thread …
rkennke Mar 30, 2023
db856dc
More verification through zapping unused entries, x86 parts
Mar 30, 2023
5717188
Address more of @shipilev's review comments
Mar 30, 2023
a75de7e
Use ANONYMOUS_OWNER constant in SA
rkennke Mar 30, 2023
b0ffd9a
Splice entry points for InterpreterRuntime::monitorenter()
rkennke Mar 30, 2023
e0e019e
Don't retry on inflate; use aarch64 ZF from fast_lock(); bunch of sma…
rkennke Mar 30, 2023
0f9519c
Several small changes based on reviews
Mar 30, 2023
815a820
Merge branch 'master' into JDK-8291555-v2
Mar 30, 2023
6514f83
Allow old monitorenter entry point when using heavy monitors
Mar 30, 2023
eaf2286
Fix release build
Mar 30, 2023
adeccaa
Use int instead of size_t for cached offsets, to match the uncached o…
Mar 31, 2023
0835f48
Check underflow, top-of-stack and mark-bits for sanity, in fast_unloc…
rkennke Mar 31, 2023
0e36620
Merge remote-tracking branch 'origin/JDK-8291555-v2' into JDK-8291555-v2
rkennke Mar 31, 2023
4c6f5ab
Arm port
tstuefe Mar 31, 2023
1ad9585
Thomas' comments
rkennke Mar 31, 2023
298031b
Fix assert in lock-stack boundaries check
rkennke Apr 1, 2023
6f8ab5d
Merge remote-tracking branch 'tstuefe/ARM-port-8291555' into JDK-8291…
rkennke Apr 2, 2023
8ef7b1e
Reject +UseFastLocking on unsupported platforms
rkennke Apr 2, 2023
1348f3b
Replace UseFastLocking with LockingMode flag
rkennke Apr 2, 2023
13c84b5
Fix typo
rkennke Apr 3, 2023
a6f9a1a
Allow scanning lock-stack outside safepoint
Apr 4, 2023
839f350
Address David's review comments
Apr 4, 2023
baf7162
Named constants for LockingMode
rkennke Apr 5, 2023
963de0e
Put back thread type check in OS::is_lock_owned()
rkennke Apr 5, 2023
d1c8826
RISCV update
rkennke Apr 6, 2023
9ad1042
Don't use NativeAccess in LockStack::contains()
rkennke Apr 12, 2023
cb260c1
Bunch of comments and typos
rkennke Apr 12, 2023
e60699e
Prefix LockingMode constants with LM_*
Apr 12, 2023
f545194
Replace UseHeavyMonitor with LockingMode == LM_MONITOR
Apr 12, 2023
db4ca10
A few more LM_ prefixes in 32bit code
Apr 13, 2023
d0a448c
Merge remote-tracking branch 'upstream/master' into JDK-8291555-v2
rkennke Apr 14, 2023
c348672
Simple build fix for extra arches
Apr 17, 2023
5d0a045
Remove unnecessary comments
rkennke Apr 20, 2023
5f927f9
Suggested changes by @merykitty
rkennke Apr 26, 2023
1323e95
Suggestios by @dcubed-ojdk
rkennke Apr 26, 2023
56d7f37
Fix formatting
rkennke Apr 28, 2023
d90810f
Merge branch 'master' into JDK-8291555-v2
rkennke Apr 28, 2023
50f1369
Fix arm and ppcle builds
rkennke Apr 28, 2023
39b199b
Merge commit '452cb8432f4d45c3dacd4415bc9499ae73f7a17c' into JDK-8291…
rkennke Apr 28, 2023
a3e41c4
Address @coleenp's review
rkennke May 2, 2023
9b25681
Fix copyright on new files
rkennke May 2, 2023
423dbcd
Add missing new file
rkennke May 2, 2023
5d5a43d
Address @dholmes-ora's review comments
rkennke May 3, 2023
e06c5ef
Address @dcubed-ojdk review comments
rkennke May 4, 2023
43cdbb5
Relax zapped-entry test when calling thread is not owning thread
May 5, 2023
82b8b70
Disable new lightweight locking in Zero
May 5, 2023
a65b3ae
Merge branch 'master' into JDK-8291555-v2
May 5, 2023
171aced
Only do lock-stack consistency checks when called from owning thread
May 5, 2023
0da2b84
Fix null -> nullptr typo
May 5, 2023
66a87a0
Only allow lock-stack verification for owning Java threads or at safe…
May 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/hotspot/share/runtime/lockStack.cpp
Original file line number Diff line number Diff line change
@@ -58,10 +58,12 @@ uint32_t LockStack::end_offset() {
return static_cast<uint32_t>(offset);
}

#ifdef ASSERT
static bool is_stack_watermark_processing(JavaThread* thread) {
StackWatermark* watermark = StackWatermarkSet::get(thread, StackWatermarkKind::gc);
return watermark->processing_started() && !watermark->processing_completed();
}
#endif

#ifndef PRODUCT
void LockStack::verify(const char* msg) const {
2 changes: 1 addition & 1 deletion src/hotspot/share/runtime/objectMonitor.hpp
Original file line number Diff line number Diff line change
@@ -158,7 +158,7 @@ class ObjectMonitor : public CHeapObj<mtObjectMonitor> {
static const uintptr_t ANONYMOUS_OWNER = 1;

private:
static constexpr void* anon_owner_ptr() { return reinterpret_cast<void*>(ANONYMOUS_OWNER); }
static void* anon_owner_ptr() { return reinterpret_cast<void*>(ANONYMOUS_OWNER); }

void* volatile _owner; // pointer to owning thread OR BasicLock
volatile uint64_t _previous_owner_tid; // thread id of the previous owner of the monitor