Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8324641: [IR Framework] Add Setup method to provide custom arguments and set fields #17557

Closed
wants to merge 38 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
f8845ec
8324641
eme64 Jan 24, 2024
42205ee
example is working
eme64 Jan 24, 2024
3eb7c18
Arguments values works again as well
eme64 Jan 24, 2024
62d907b
remove printFixedRandomArguments
eme64 Jan 24, 2024
8f6fe27
Adjusted all @Arguments with value
eme64 Jan 24, 2024
f0c6cd8
improve some comments
eme64 Jan 24, 2024
2a57e52
improve formatting and comments
eme64 Jan 25, 2024
d4d15d8
index -> invocationCounter
eme64 Jan 25, 2024
461405d
SetupExample.java
eme64 Jan 25, 2024
08670a5
make invocationCounter optional for Setup method
eme64 Jan 29, 2024
b725b29
fix trailing whitespaces
eme64 Jan 29, 2024
13c6621
update readme
eme64 Jan 29, 2024
b367647
add SetupInfo class
eme64 Jan 29, 2024
1f8f57a
fix up some tests
eme64 Jan 29, 2024
24eb09d
fix add and comma
eme64 Feb 1, 2024
1237c02
for Review by Christian
eme64 Feb 5, 2024
852e6e1
more for Christian
eme64 Feb 5, 2024
66c3ef7
setup string not list, for Christian
eme64 Feb 5, 2024
fddcdfe
made ArgumentsProvider an interface
eme64 Feb 5, 2024
8ff7dfa
fix more comments
eme64 Feb 5, 2024
e9d0106
Apply suggestions from code review
eme64 Feb 6, 2024
164665f
fix whitespace from last commit
eme64 Feb 6, 2024
12c9a45
Apply suggestions from code review
eme64 Feb 6, 2024
a1dda56
Apply suggestions from code review
eme64 Feb 6, 2024
3206074
rename BaseTestExample -> NormalTestExample
eme64 Feb 6, 2024
af5e7dc
add paragraph for Christian
eme64 Feb 6, 2024
b37b748
wip more tests
eme64 Feb 6, 2024
6efd271
wip: more tests
eme64 Feb 7, 2024
252815e
wip: more tests and fixes
eme64 Feb 7, 2024
d10fba5
wip: test fields and invocationCounter
eme64 Feb 7, 2024
4a8f96b
wip: throw tests, array tests, random tests
eme64 Feb 7, 2024
1f84ef5
improved and simplified examples
eme64 Feb 7, 2024
33029f6
remove tabs
eme64 Feb 7, 2024
f03f3b8
Apply suggestions from code review
eme64 Feb 8, 2024
47b8d25
more for Christian
eme64 Feb 8, 2024
617e40a
fix whitespace
eme64 Feb 8, 2024
b62b7ef
Merge branch 'master' into JDK-8324641
eme64 Feb 8, 2024
1f7872a
Merge branch 'master' into JDK-8324641
eme64 Feb 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 16 additions & 16 deletions test/hotspot/jtreg/compiler/lib/ir_framework/README.md
Original file line number Diff line number Diff line change
@@ -33,33 +33,33 @@ There are various ways how to set up and run a test within the `main()` method o
## 2. Features
The framework offers various annotations and flags to control how your test code should be invoked and being checked. This section gives an overview over all these features.

### 2.1 Test Method
### 2.1 Different Tests
There are two ways a test can be formulated, depending on how much control is needed over the test invocation.

#### Normal Test Method
#### Normal Test
The normal and simplest form of testing provides a single `@Test` annotated method which the framework invokes directly as part of the testing. The test method either has no arguments, or they must be specified with an `@Arguments` annotation.

Arguments can be provided with `@Arguments(values = {...})` by providing well-specified inputs for each individual argument. Alternatively, a setup method can be chosen with `@Arguments(setup = {"setupMethodName"})`, which computes arguments and can also set fields.
Arguments can be provided with `@Arguments(values = {...})` by providing well-specified inputs for each individual argument. Alternatively, a setup method can be chosen with `@Arguments(setup = "setupMethodName")`, which computes arguments and can also set fields.

More information on normal test methods with a precise definition can be found in the Javadocs of [Test](./Test.java). Concrete examples on how to specify a normal test can be found in [BaseTestsExample](../../../testlibrary_tests/ir_framework/examples/BaseTestExample.java).

#### Custom Run Tests
The normal test method is invoked directly by the framework, and hence the user does not have control over the invokation.
A custom run test gives full control over the invocation of the `@Test` annotated method to the user. The framework calls a dedicated `@Run` annotated method from which the user can invoke the `@Test` method according to their needs.

More information on checked tests with a precise definition can be found in the Javadocs of [Run](./Run.java). Concrete examples on how to specify a custom run test can be found in [CustomRunTestsExample](../../../testlibrary_tests/ir_framework/examples/CustomRunTestExample.java).

### 2.2 Setup Method
A `@Setup` annotated method can provide custom arguments and set fields before a normal test is run. A normal test method can be annotated with `@Arguments(setup = {"setupMethodName"})` to specify which setup method is to be used.
##### Setup Method
A `@Setup` annotated method can provide custom arguments and set fields before a normal test is run. A normal test method can be annotated with `@Arguments(setup = "setupMethodName")` to specify which setup method is to be used.

More information on checked tests with a precise definition can be found in the Javadocs of [Setup](./Setup.java). Concrete examples on how to specify a setup method can be found in [SetupExample](../../../testlibrary_tests/ir_framework/examples/SetupExample.java).

### 2.3 Check Method
##### Check Method
A `@Check` annotated method is invoked directly after a corresponding normal `@Test` annotated method. The user can perform various checks, such as test method return value and field value verification.

More information on check methods with a precise definition can be found in the Javadocs of [Check](./Check.java). Concrete examples on how to specify check methods can be found in [CheckedTestsExample](../../../testlibrary_tests/ir_framework/examples/CheckedTestExample.java).

### 2.4 IR Verification
#### Custom Run Tests
The normal test method is invoked directly by the framework, and hence the user does not have control over the invokation.
A custom run test gives full control over the invocation of the `@Test` annotated method to the user. The framework calls a dedicated `@Run` annotated method from which the user can invoke the `@Test` method according to their needs.

More information on checked tests with a precise definition can be found in the Javadocs of [Run](./Run.java). Concrete examples on how to specify a custom run test can be found in [CustomRunTestsExample](../../../testlibrary_tests/ir_framework/examples/CustomRunTestExample.java).

### 2.2 IR Verification
The main feature of this framework is to perform a simple but yet powerful regex-based C2 IR matching on the output of `-XX:+PrintIdeal`, `-XX:+PrintOptoAssembly` and/or on specific compile phases emitted by the compile command `-XX:CompileCommand=PrintIdealPhase` which supports the same set of compile phases as the Ideal Graph Visualizer (IGV).

The user has the possibility to add one or more `@IR` annotations to any `@Test` annotated method (regardless of the kind of test mentioned in section 2.1) to specify regex constraints/rules on the compiled IR shape of any compile phase (for simplicity, the framework treats the output of `-XX:+PrintIdeal` and `-XX:+PrintOptoAssembly` as a separate compile phase next to the compile phases emitted by `-XX:CompileCommand=PrintIdealPhase`).
@@ -142,12 +142,12 @@ An IR verification cannot always be performed. Certain VM flags explicitly disab

More information about IR matching can be found in the Javadocs of [IR](./IR.java). Concrete examples on how to specify IR constraint/rules can be found in [IRExample](../../../testlibrary_tests/ir_framework/examples/IRExample.java), [TestIRMatching](../../../testlibrary_tests/ir_framework/tests/TestIRMatching.java) (internal framework test), and [TestPhaseIRMatching](../../../testlibrary_tests/ir_framework/tests/TestPhaseIRMatching.java) (internal framework test).

### 2.5 Test VM Flags and Scenarios
### 2.3 Test VM Flags and Scenarios
The recommended way to use the framework is by defining a single `@run driver` statement in the JTreg header which, however, does not allow the specification of additional test VM flags. Instead, the user has the possibility to provide VM flags by calling `TestFramework.runWithFlags()` or by creating a `TestFramework` builder object on which `addFlags()` can be called.

If a user wants to provide multiple flag combinations for a single test, he or she has the option to provide different scenarios. A scenario based flag will always have precedence over other user defined flags. More information about scenarios can be found in the Javadocs of [Scenario](./Scenario.java).

### 2.6 Compiler Controls
### 2.4 Compiler Controls
The framework allows the use of additional compiler control annotations for helper method and classes in the same fashion as JMH does. The following annotations are supported and described in the referenced Javadocs for the annotation class:

- [@DontInline](./DontInline.java)
@@ -156,7 +156,7 @@ The framework allows the use of additional compiler control annotations for help
- [@ForceCompile](./ForceCompile.java)
- [@ForceCompileClassInitializer](./ForceCompileClassInitializer.java)

### 2.7 Framework Debug and Stress Flags
### 2.5 Framework Debug and Stress Flags
The framework provides various stress and debug flags. They should mainly be used as JTreg VM and/or Javaoptions (apart from `VerifyIR`). The following (property) flags are supported:

- `-DVerifyIR=false`: Explicitly disable IR verification. This is useful, for example, if some scenarios use VM flags that let `@IR` annotation rules fail and the user does not want to provide separate IR rules or add flag preconditions to the already existing IR rules.
Original file line number Diff line number Diff line change
@@ -246,7 +246,7 @@ public boolean isFixedRandom() {
return isFixedRandom;
}

public Object getValue(int invocationCounter) {
public Object getValue() {
if (isRandomEach) {
return getRandom(randomClass);
} else {
@@ -318,7 +318,7 @@ class BooleanToggleValue extends ArgumentValue {
}

@Override
public Object getValue(int invocationCounter) {
public Object getValue() {
previousBoolean = !previousBoolean;
return previousBoolean;
}
Original file line number Diff line number Diff line change
@@ -106,12 +106,12 @@ final class ValueArgumentsProvider implements ArgumentsProvider {

@Override
public Object[] getArguments(Object invocationTarget, int invocationCounter) {
return Arrays.stream(argumentValues).map(v -> v.getValue(invocationCounter)).toArray();
return Arrays.stream(argumentValues).map(v -> v.getValue()).toArray();
}
}

/**
* Used for @Arguments(setup = {"setupMethodName"}) to specify a setup method to provide arguments
* Used for @Arguments(setup = "setupMethodName") to specify a setup method to provide arguments
* and possibly set fields.
*/
final class SetupArgumentsProvider implements ArgumentsProvider {
Original file line number Diff line number Diff line change
@@ -506,7 +506,7 @@ static void enqueueForCompilation(Executable ex, CompLevel requestedCompLevel) {

/**
* Collect all @Setup annotated methods and add them to setupMethodMap, for convenience to reference later from
* tests with @Arguments(setup = {"setupMethodName"}).
* tests with @Arguments(setup = "setupMethodName").
*/
private void collectSetupMethods() {
for (Method m : testClass.getDeclaredMethods()) {
Original file line number Diff line number Diff line change
@@ -35,7 +35,7 @@

/**
* This file shows some examples of how to use a setup method, annotated with {@link Setup}, and referenced by
* a test method with @Arguments(setup = {"setupMethodName"}).
* a test method with @Arguments(setup = "setupMethodName").
*
* @see Setup
* @see Arguments