Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDK-8290845: Consider an alternative item separator for multi-item option values #11178

Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Expand Up @@ -531,14 +531,16 @@ doclet.usage.link-platform-properties.description=\
Link to platform documentation URLs declared in properties file at <url>

doclet.usage.excludedocfilessubdir.parameters=\
<name>:...
<name>,<name>,...
doclet.usage.excludedocfilessubdir.description=\
Exclude any doc-files subdirectories with given name
Exclude any doc-files subdirectories with given name.\n\
':' can also be used anywhere in the argument as a separator.

doclet.usage.group.parameters=\
<name> <g1>:<g2>...
<name> <g1>,<g2>...
doclet.usage.group.description=\
Group specified elements together in overview page
Group specified elements together in overview page.\n\
':' can also be used anywhere in the argument as a separator.

doclet.usage.legal-notices.parameters=\
'default' | 'none' | <directory>
Expand All @@ -552,9 +554,10 @@ doclet.usage.nodeprecated.description=\
Do not include @deprecated information

doclet.usage.noqualifier.parameters=\
<name1>:<name2>:...
<name1>,<name2>,...
doclet.usage.noqualifier.description=\
Exclude the list of qualifiers from the output
Exclude the list of qualifiers from the output.\n\
':' can also be used anywhere in the argument as a separator.

doclet.usage.nosince.description=\
Do not include @since information
Expand Down
Expand Up @@ -471,9 +471,13 @@ private void initTagletManager(Set<List<String>> customTagStrs) {
tagletManager.addCustomTag(args.get(1), fileManager);
continue;
}
List<String> tokens = tokenize(args.get(1), TagletManager.SIMPLE_TAGLET_OPT_SEPARATOR, 3);
/* Since there are few constraints on the characters in a tag name,
* and real world examples with ':' in the tag name, we cannot simply use
* String.split(regex); instead, we tokenize the string, allowing
* special characters to be escaped with '\'. */
List<String> tokens = tokenize(args.get(1), 3);
switch (tokens.size()) {
case 1:
case 1 -> {
String tagName = args.get(1);
if (tagletManager.isKnownCustomTag(tagName)) {
//reorder a standard tag
Expand All @@ -484,18 +488,16 @@ private void initTagletManager(Set<List<String>> customTagStrs) {
heading.setCharAt(0, Character.toUpperCase(tagName.charAt(0)));
tagletManager.addNewSimpleCustomTag(tagName, heading.toString(), "a");
}
break;
}

case 2:
case 2 ->
//Add simple taglet without heading, probably to excluding it in the output.
tagletManager.addNewSimpleCustomTag(tokens.get(0), tokens.get(1), "");
break;

case 3:
case 3 ->
tagletManager.addNewSimpleCustomTag(tokens.get(0), tokens.get(2), tokens.get(1));
break;

default:
default ->
messages.error("doclet.Error_invalid_custom_tag_argument", args.get(1));
}
}
Expand All @@ -505,18 +507,17 @@ private void initTagletManager(Set<List<String>> customTagStrs) {
}

/**
* Given a string, return an array of tokens. The separator can be escaped
* with the '\' character. The '\' character may also be escaped by the
* '\' character.
* Given a string, return an array of tokens, separated by ':'.
* The separator character can be escaped with the '\' character.
* The '\' character may also be escaped with the '\' character.
*
* @param s the string to tokenize.
* @param separator the separator char.
* @param s the string to tokenize
* @param maxTokens the maximum number of tokens returned. If the
* max is reached, the remaining part of s is appended
* to the end of the last token.
* @return an array of tokens.
* @return an array of tokens
*/
private List<String> tokenize(String s, char separator, int maxTokens) {
private List<String> tokenize(String s, int maxTokens) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not saying that we should do anything about it, but I would be surprised if there were no such tokenization functionality in the JDK API.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of the JDK simple tokenizers supports handling of escape characters.
But a Pattern/Matcher could be used.

List<String> tokens = new ArrayList<>();
StringBuilder token = new StringBuilder();
boolean prevIsEscapeChar = false;
Expand All @@ -526,7 +527,7 @@ private List<String> tokenize(String s, char separator, int maxTokens) {
// Case 1: escaped character
token.appendCodePoint(currentChar);
prevIsEscapeChar = false;
} else if (currentChar == separator && tokens.size() < maxTokens - 1) {
} else if (currentChar == ':' && tokens.size() < maxTokens - 1) {
// Case 2: separator
tokens.add(token.toString());
token = new StringBuilder();
Expand Down
Expand Up @@ -419,7 +419,7 @@ public boolean process(String opt, List<String> args) {
new Option(resources, "-excludedocfilessubdir", 1) {
@Override
public boolean process(String opt, List<String> args) {
addToSet(excludedDocFileDirs, args.get(0));
excludedDocFileDirs.addAll(List.of(args.get(0).split("[,:]")));
return true;
}
},
Expand Down Expand Up @@ -536,7 +536,7 @@ public boolean process(String opt, List<String> args) {
new Option(resources, "-noqualifier", 1) {
@Override
public boolean process(String opt, List<String> args) {
addToSet(excludedQualifiers, args.get(0));
excludedQualifiers.addAll(List.of(args.get(0).split("[,:]")));
return true;
}
},
Expand Down Expand Up @@ -764,15 +764,6 @@ private boolean checkOutputFileEncoding(String docencoding) {
return true;
}

private void addToSet(Set<String> s, String str) {
StringTokenizer st = new StringTokenizer(str, ":");
String current;
while (st.hasMoreTokens()) {
current = st.nextToken();
s.add(current);
}
}

/**
* Add a trailing file separator, if not found. Remove superfluous
* file separators if any. Preserve the front double file separator for
Expand Down
Expand Up @@ -85,11 +85,6 @@
*/
public class TagletManager {

/**
* The default separator for the simple tag option.
*/
public static final char SIMPLE_TAGLET_OPT_SEPARATOR = ':';

/**
* All taglets, keyed either by their {@link Taglet#getName() name},
* or by an alias.
Expand Down
Expand Up @@ -115,7 +115,7 @@ public Group(BaseConfiguration configuration) {
* @param moduleNameFormList List of the module name formats.
*/
public boolean checkModuleGroups(String groupname, String moduleNameFormList) {
String[] mdlPatterns = moduleNameFormList.split(":");
String[] mdlPatterns = moduleNameFormList.split("[,:]");
if (groupList.contains(groupname)) {
initMessages();
messages.warning("doclet.Groupname_already_used", groupname);
Expand Down Expand Up @@ -161,7 +161,7 @@ public boolean checkModuleGroups(String groupname, String moduleNameFormList) {
* @param pkgNameFormList List of the package name formats.
*/
public boolean checkPackageGroups(String groupname, String pkgNameFormList) {
String[] pkgPatterns = pkgNameFormList.split(":");
String[] pkgPatterns = pkgNameFormList.split("[,:]");
if (groupList.contains(groupname)) {
initMessages();
messages.warning("doclet.Groupname_already_used", groupname);
Expand Down
65 changes: 52 additions & 13 deletions src/jdk.javadoc/share/man/javadoc.1
@@ -1,4 +1,4 @@
.\" Copyright (c) 1994, 2021, Oracle and/or its affiliates. All rights reserved.
.\" Copyright (c) 1994, 2022, Oracle and/or its affiliates. All rights reserved.
.\" DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
.\"
.\" This code is free software; you can redistribute it and/or modify it
Expand All @@ -21,7 +21,7 @@
.\"
.\" Automatically generated by Pandoc 2.3.1
.\"
.TH "JAVADOC" "1" "2023" "JDK 20\-ea" "JDK Commands"
.TH "JAVADOC" "1" "2022" "JDK 20\-internal" "JDK Commands"
.hy
.SH NAME
.PP
Expand Down Expand Up @@ -609,6 +609,13 @@ javadoc\ \-docencoding\ "iso\-8859\-1"\ mypackage
.TP
.B \f[CB]\-docfilessubdirs\f[R]
Recursively copies doc\-file subdirectories.
Enables deep copying of doc\-files directories.
Subdirectories and all contents are recursively copied to the
destination.
For example, the directory \f[CB]doc\-files/example/images\f[R] and all of
its contents are copied.
The \f[B]\f[BC]\-excludedocfilessubdir\f[B]\f[R] option can be used to
exclude specific subdirectories.
.RS
.RE
.TP
Expand All @@ -625,14 +632,12 @@ For example,
.RS
.RE
.TP
.B \f[CB]\-excludedocfilessubdir\f[R] \f[I]name\f[R]
Excludes any doc files subdirectories with the given name.
Enables deep copying of doc\-files directories.
Subdirectories and all contents are recursively copied to the
destination.
For example, the directory \f[CB]doc\-files/example/images\f[R] and all of
its contents are copied.
There is also an option to exclude subdirectories.
.B \f[CB]\-excludedocfilessubdir\f[R] \f[I]name1\f[R]\f[CB],\f[R]\f[I]name2...\f[R]
Excludes any subdirectories with the given names when recursively
copying doc\-file subdirectories.
See \f[B]\f[BC]\-docfilessubdirs\f[B]\f[R].
For historical reasons, \f[CB]:\f[R] can be used anywhere in the argument
as a separator instead of \f[CB],\f[R].
.RS
.RE
.TP
Expand All @@ -648,8 +653,10 @@ Use escape characters for any internal quotation marks within a footer.
.RS
.RE
.TP
.B \f[CB]\-group\f[R] \f[I]namep1\f[R]\f[CB]:\f[R]\f[I]p2\f[R]
.B \f[CB]\-group\f[R] \f[I]name\f[R] \f[I]p1\f[R]\f[CB],\f[R]\f[I]p2...\f[R]
Group the specified packages together in the Overview page.
For historical reasons, \f[CB]:\f[R] can be used as a separator anywhere
in the argument instead of \f[CB],\f[R].
.RS
.RE
.TP
Expand Down Expand Up @@ -1004,10 +1011,12 @@ These links are generated by default.
.RS
.RE
.TP
.B \f[CB]\-noqualifier\f[R] \f[I]name1\f[R]\f[CB]:\f[R]\f[I]name2\f[R]...
.B \f[CB]\-noqualifier\f[R] \f[I]name1\f[R]\f[CB],\f[R]\f[I]name2...\f[R]
Excludes the list of qualifiers from the output.
The package name is removed from places where class or interface names
appear.
For historical reasons, \f[CB]:\f[R] can be used anywhere in the argument
as a separator instead of \f[CB],\f[R].
.RS
.PP
The following example omits all package qualifiers:
Expand Down Expand Up @@ -1134,7 +1143,7 @@ snippet, and then searches all the directories in the given list.
.RS
.RE
.TP
.B \f[CB]\-sourcetab\f[R] \f[I]tablength\f[R]
.B \f[CB]\-sourcetab\f[R] \f[I]tab\-length\f[R]
Specifies the number of spaces each tab uses in the source.
.RS
.RE
Expand Down Expand Up @@ -1162,6 +1171,8 @@ to include a \f[CB]\-tag\f[R] option for every custom tag that is present
in the source code, disabling (with \f[CB]X\f[R]) those that aren\[aq]t
being output in the current run.
The colon (\f[CB]:\f[R]) is always the separator.
To include a colon in the tag name, escape it with a backward slash
(\f[CB]\\\f[R]).
The \f[CB]\-tag\f[R] option outputs the tag heading, \f[I]header\f[R], in
bold, followed on the next line by the text from its single argument.
Similar to any block tag, the argument text can contain inline tags,
Expand All @@ -1170,7 +1181,35 @@ The output is similar to standard one\-argument tags, such as the
\f[CB]\@return\f[R] and \f[CB]\@author\f[R] tags.
Omitting a \f[I]header\f[R] value causes the \f[I]name\f[R] to be the
heading.
\f[I]locations\f[R] is a list of characters specifying the kinds of
declarations in which the tag may be used.
The following characters may be used, in either uppercase or lowercase:
.RS
.IP \[bu] 2
\f[CB]A\f[R]: all declarations
.IP \[bu] 2
\f[CB]C\f[R]: constructors
.IP \[bu] 2
\f[CB]F\f[R]: fields
.IP \[bu] 2
\f[CB]M\f[R]: methods
.IP \[bu] 2
\f[CB]O\f[R]: the overview page and other documentation files in
\f[CB]doc\-files\f[R] subdirectories
.IP \[bu] 2
\f[CB]P\f[R]: packages
.IP \[bu] 2
\f[CB]S\f[R]: modules
.IP \[bu] 2
\f[CB]T\f[R]: types (classes and interfaces)
.IP \[bu] 2
\f[CB]X\f[R]: nowhere: the tag is disabled, and will be ignored
.PP
The order in which tags are given on the command line will be used as
the order in which the tags appear in the generated output.
You can include standard tags in the order given on the command line by
using the \f[CB]\-tag\f[R] option with no \f[I]locations\f[R] or
\f[I]header\f[R].
.RE
.TP
.B \f[CB]\-taglet\f[R] \f[I]class\f[R]
Expand Down