8344036: Tests tools/jlink/runtimeImage fail on AIX after JDK-8311302

[MBaesken]

On AIX we run into errors below, for example in test tools/jlink/runtimeImage/AddOptionsTest.java , this happened with fastdebug binaries, opt worked :

jlink options: --output java-base-with-opts-jlink-tmp --add-modules jdk.jlink,java.base --generate-linkable-runtime --keep-packaged-modules=java-base-with-opts-jlink-tmp/jmods
Error: Java heap space
java.lang.OutOfMemoryError: Java heap space
at java.base/java.io.InputStream.readNBytes(InputStream.java:447)
at java.base/java.io.InputStream.readAllBytes(InputStream.java:348)
at jdk.jlink/jdk.tools.jlink.plugin.ResourcePoolEntry.contentBytes(ResourcePoolEntry.java:127)
at jdk.jlink/jdk.tools.jlink.internal.runtimelink.ResourcePoolReader.getResourceBytes(ResourcePoolReader.java:54)
at jdk.jlink/jdk.tools.jlink.internal.runtimelink.JimageDiffGenerator.generateDiff(JimageDiffGenerator.java:89)
at jdk.jlink/jdk.tools.jlink.internal.ImageFileCreator.addResourceDiffFiles(ImageFileCreator.java:357)
at jdk.jlink/jdk.tools.jlink.internal.ImageFileCreator.generateJImage(ImageFileCreator.java:264)
at jdk.jlink/jdk.tools.jlink.internal.ImageFileCreator.writeImage(ImageFileCreator.java:202)
at jdk.jlink/jdk.tools.jlink.internal.ImageFileCreator.create(ImageFileCreator.java:131)
at jdk.jlink/jdk.tools.jlink.internal.JlinkTask$ImageHelper.retrieve(JlinkTask.java:1041)
at jdk.jlink/jdk.tools.jlink.internal.ImagePluginStack.operate(ImagePluginStack.java:194)
at jdk.jlink/jdk.tools.jlink.internal.JlinkTask.createImage(JlinkTask.java:501)
at jdk.jlink/jdk.tools.jlink.internal.JlinkTask.run(JlinkTask.java:294)
at jdk.jlink/jdk.tools.jlink.internal.Main.run(Main.java:56)
at jdk.jlink/jdk.tools.jlink.internal.Main$JlinkToolProvider.run(Main.java:73)
at tests.JImageGenerator$JLinkTask.call(JImageGenerator.java:713)
at AbstractLinkableRuntimeTest.createRuntimeLinkImage(AbstractLinkableRuntimeTest.java:273)
at AbstractLinkableRuntimeTest.createJavaImageRuntimeLink(AbstractLinkableRuntimeTest.java:126)
at AbstractLinkableRuntimeTest.createJavaImageRuntimeLink(AbstractLinkableRuntimeTest.java:121)
at AddOptionsTest.runTest(AddOptionsTest.java:64)
at AbstractLinkableRuntimeTest.run(AbstractLinkableRuntimeTest.java:62)
at AddOptionsTest.main(AddOptionsTest.java:49)
at java.base/java.lang.invoke.LambdaForm$DMH/0x0a0001009070c3e0.invokeStatic(LambdaForm$DMH)
at java.base/java.lang.invoke.LambdaForm$MH/0x0a00010090710c10.invoke(LambdaForm$MH)
at java.base/java.lang.invoke.LambdaForm$MH/0x0a00010090711408.invokeExact_MT(LambdaForm$MH)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:155)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
at java.base/java.lang.Thread.runWith(Thread.java:1589)
at java.base/java.lang.Thread.run(Thread.java:1576)

java.lang.AssertionError: Unexpected failure: 4
at tests.Result.assertSuccess(Result.java:80)

The tests need more memory and one also needs a higher timeout value to succeed .

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8344036: Tests tools/jlink/runtimeImage fail on AIX after JDK-8311302 (Bug - P4)

Reviewers

• Severin Gehwolf (@jerboaa - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/22140/head:pull/22140
$ git checkout pull/22140

Update a local copy of the PR:
$ git checkout pull/22140
$ git pull https://git.openjdk.org/jdk.git pull/22140/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 22140

View PR using the GUI difftool:
$ git pr show -t 22140

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/22140.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back mbaesken! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@MBaesken This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8344036: Tests tools/jlink/runtimeImage fail on AIX after JDK-8311302

Reviewed-by: sgehwolf

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 49 new commits pushed to the master branch:

• dfddbca: 8341916: Remove ProtectionDomain related hotspot code and tests
• 5eb0733: 8344383: Include ZipArchive and JarArchive directly
• b8b70c8: 8344379: [s390x] build failure due to missing change from JDK-8339466
• 5fc4322: 8288298: Resolve multiline message parsing ambiguities in UL
• ea8f289: 8344271: Comparison build fails due to difference in doc summary
• b9c6ce9: 8344122: IGV: Extend c2 IdealGraphPrinter to send subgraphs to IGV
• 00ff6a3: 8344105: Remove SecurityManager and related calls from jdk.attach and jdk.hotspot.agent
• 475feb0: 8344056: Use markdown format for man pages
• 6c2ae44: 8344204: IGV: Button to enable/disable cutting of long edges
• 4a7ce1d: 8344205: [PPC]: failing assertion: sharedRuntime_ppc.cpp:1652: cookie not found
• ... and 39 more: https://git.openjdk.org/jdk/compare/bfee766f035fb1b122cd3f3703b9e2a2d85abfe6...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[openjdk]

@MBaesken The following label will be automatically applied to this pull request:

• core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 01: Full - Incremental (1c911f63)
• 00: Full (b6a5aa85)

[jerboaa]

It seems a bit excessive to bump to twice the size while we won't need it for any platform other than AIX. Would -Xmx1500m work as well? Aside, those tests now potentially run on machines with 2gb of max memory with a 2gb heap setting.

Where you able to reproduce outside of testing?

at jdk.jlink/jdk.tools.jlink.internal.runtimelink.ResourcePoolReader.getResourceBytes(ResourcePoolReader.java:54)
at jdk.jlink/jdk.tools.jlink.internal.runtimelink.JimageDiffGenerator.generateDiff(JimageDiffGenerator.java:89)

It would be interesting to know whether or not the resource that is being compared is the same when the error happens for each run. Have you tried investigating?

[jerboaa]

Where you able to reproduce outside of testing?

Something like this should give you an idea what the RSS size ends up being (not sure about the AIX equivalent to get that info, though; this is Linux)

$ rm -rf ./build/linkable-runtime/ && \time --format "Max RSS: %M KB" ./build/linux-x86_64-server-fastdebug/images/jdk/bin/jlink -J-Xmx1g --add-modules java.base,jdk.jlink --generate-linkable-runtime --keep-packaged-modules=./build/linkable-runtime/jmods --output ./build/linkable-runtime/
Max RSS: 569332 KB

$ rm -rf ./build/linkable-runtime/ && \time --format "Max RSS: %M KB" ./build/linux-x86_64-server-fastdebug/images/jdk/bin/jlink -J-Xmx1g --add-modules ALL-MODULE-PATH --generate-linkable-runtime --keep-packaged-modules=./build/linkable-runtime/jmods --output ./build/linkable-runtime/
WARNING: Using incubator modules: jdk.incubator.vector
Max RSS: 593816 KB

[MBaesken]

Where you able to reproduce outside of testing?

No, it failed only in our central test environment, not for single runs of one of these tests with jtreg in my local AIX dev env .
But the central tests use fastdebug and I use locally mostly opt builds so this might be a reason too.
Exception is the test with the timeout error / timeout increase, I saw this too locally.

Regarding Xmx1500m I can try this , or I can also copy the whole test header block and use 1g for all platforms and 2g or 1500m for AIX (would be a bit code duplication of course).

[jerboaa]

Regarding Xmx1500m I can try this , or I can also copy the whole test header block and use 1g for all platforms and 2g or 1500m for AIX (would be a bit code duplication of course).

If -Xmx1500m works for you that seems ok. -1 on the bump only for AIX and requires. An alternative might be to use test/jdk/tools/jlink/runtimeImage/TEST.properties with this content:

exclusiveAccess.dirs=.

I could see why the JavaSE comparison test with packaged modules and run-time image linkage would need 2 GB, but not the others.

Both options seem preferred over the current version. It might be that a combination of -Xmx1500m, exclusiveAccess.dirs=. works. The less invasive the better.

[MBaesken]

I reduced to Xmx1400m seems that is sufficient for the memory requirements on AIX.

Unfortunately I still get now in the PackagedModulesVsRuntimeImageLinkTest
Timeout information:
Running jstack on process 16253352
16253352: Unable to open socket file /tmp/.java_pid16253352: target process 16253352 doesn't respond within 10500ms or HotSpot VM not loaded
(when running centrally in the concurrent tests)

[jerboaa]

Seems fine to me.

[MBaesken]

/integrate

[openjdk]

Going to push as commit cc8bd63.
Since your change was applied there have been 78 commits pushed to the master branch:

• 2359736: 8344534: Remove leftover import of java.security.AccessControlContext in JavaLangAccess
• 0d66689: 8344393: RISC-V: Remove option UseRVVForBigIntegerShiftIntrinsics
• 7540fa2: 8343884: [s390x] Disallow OptoScheduling
• 9d60300: 8344199: Incorrect excluded field value set by getEventWriter intrinsic
• 8bd080b: 8341901: Using 'var' keyword switch pattern matching causes compiler error
• 76a55c3: 8341334: CDS: Parallel relocation
• 499186b: 8343902: javax/swing/plaf/nimbus/8041642/bug8041642.java fails in ubuntu22.04
• b12c5b4: 8344218: Remove calls to SecurityManager and and doPrivileged in java.net.NetworkInterface after JEP 486 integration
• d85dd77: 8344365: SecurityManager cleanups in java.sql and java.sql.rowset modules
• 9e92a9e: 8344059: Remove doPrivileged calls from windows platform sources in the java.desktop module
• ... and 68 more: https://git.openjdk.org/jdk/compare/bfee766f035fb1b122cd3f3703b9e2a2d85abfe6...master

Your commit was automatically rebased without conflicts.

[openjdk]

@MBaesken Pushed as commit cc8bd63.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.