You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "thiru-mg (via GitHub)" <gi...@apache.org> on 2023/05/21 13:57:41 UTC
[GitHub] [solr] thiru-mg opened a new pull request, #1654: SOLR-16810: Under certain situations Solr produces managed schema XML that cannot be loaded
thiru-mg opened a new pull request, #1654:
URL: https://github.com/apache/solr/pull/1654
https://issues.apache.org/jira/browse/SOLR-16810
<!--
_(If you are a project committer then you may remove some/all of the following template.)_
Before creating a pull request, please file an issue in the ASF Jira system for Solr:
* https://issues.apache.org/jira/projects/SOLR
For something minor (i.e. that wouldn't be worth putting in release notes), you can skip JIRA.
To create a Jira issue, you will need to create an account there first.
The title of the PR should reference the Jira issue number in the form:
* SOLR-####: <short description of problem or changes>
SOLR must be fully capitalized. A short description helps people scanning pull requests for items they can work on.
Properly referencing the issue in the title ensures that Jira is correctly updated with code review comments and commits. -->
# Description
While persisting the ManagedIndexSchema as XML, non-printable characters in field names get escaped as #nn;, where nn is the decimal representation of the non-printable character. For example, if the field name has the byte 0x14, it gets escaped as #20;. This in indistinguishable from the literal #20; in the field name. If we have two fields - one with the non-printable character and the other with the literal string, two fields get generated with the same name. Loading the resulting XML, naturally, causes an exception. To fix this, any occurrence of literal # in the field name should be escaped, with say ##.
A second problem is that while escaping happens when generating XML, the corresponding unescaping does not happen on loading it.
# Solution
Both the suggested fixes are here. There are tests to expose the bugs and the fixes that pass the tests.
# Tests
Added two sets of tests - one for IndexSchema.java and the other for XML.java
# Checklist
Please review the following and check all that apply:
- [x] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability.
- [x] I have created a Jira issue and added the issue ID to my pull request title.
- [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended)
- [x] I have developed this patch against the `main` branch.
- [x] I have run `./gradlew check`.
- [x] I have added tests for my changes.
- [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16810: Under certain situations Solr produces managed schema XML with duplicate fields [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #1654:
URL: https://github.com/apache/solr/pull/1654#issuecomment-1946086595
Thank you StaleBot..... I just checked the JIRA and I was last to chime in, so I'll take this and try and get it over the finish line.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16810: Under certain situations Solr produces managed schema XML with duplicate fields [solr]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1654:
URL: https://github.com/apache/solr/pull/1654#issuecomment-2087740588
This PR had no visible activity in the past 60 days, labeling it as stale. Any new activity will remove the stale label. To attract more reviewers, please tag someone or notify the dev@solr.apache.org mailing list. Thank you for your contribution!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] sonatype-lift[bot] commented on a diff in pull request #1654: SOLR-16810: Under certain situations Solr produces managed schema XML that cannot be loaded
Posted by "sonatype-lift[bot] (via GitHub)" <gi...@apache.org>.
sonatype-lift[bot] commented on code in PR #1654:
URL: https://github.com/apache/solr/pull/1654#discussion_r1199943926
##########
solr/solrj/src/java/org/apache/solr/common/util/XML.java:
##########
@@ -164,7 +212,44 @@ private static void escape(char[] chars, int offset, int length, Writer out, Str
}
}
+ private static final Pattern toEscPattern =
+ Pattern.compile("([^#]|^)(#*)(?:(([0-9]{1,2});)|([\0-\37]))");
+
+ /**
+ * Replace any character control c with value less than 32 #nn; where nn is the decimal
+ * representation of c. Since this encoding will be ambiguous with literal '#nn;', any occurrence
+ * of '#' is repeated. Now this will become ambiguous with '#'s preceding control characters. They
+ * are also repeated. Thus, when the number of consecutive '#' is even then they are literal '#'s
+ * and when it is odd, the last '#' should be considered encoding of control characters.
+ *
+ * <p>Also replace characters like '#' into "&qout;" etc.
+ *
+ * @param str Input string
+ * @param out Output writer to write into
+ * @param escapes Escape character map
+ * @throws IOException If write operation fails
+ */
private static void escape(String str, Writer out, String[] escapes) throws IOException {
+ Matcher m = toEscPattern.matcher(str);
+ if (m.find()) {
+ StringBuilder sb = new StringBuilder();
+ do {
+ m.appendReplacement(sb, m.group(1));
+ sb.append(m.group(2));
+ String g3 = m.group(3);
+ if (g3 != null) {
+ if (Integer.parseInt(m.group(4)) < 32) {
+ sb.append(m.group(2));
+ }
+ sb.append(g3);
+ } else {
+ sb.append(m.group(2));
+ sb.append(escapes[m.group(5).charAt(0)]);
Review Comment:
<picture><img alt="15% of developers fix this issue" src="https://lift.sonatype.com/api/commentimage/fixrate/15/display.svg"></picture>
<b>*NULL_DEREFERENCE:</b>* object returned by `m.group(5)` could be null and is dereferenced at line 247.
---
<details><summary>ℹ️ Expand to see all <b>@sonatype-lift</b> commands</summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16810: Under certain situations Solr produces managed schema XML with duplicate fields [solr]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1654:
URL: https://github.com/apache/solr/pull/1654#issuecomment-1945989527
This PR had no visible activity in the past 60 days, labeling it as stale. Any new activity will remove the stale label. To attract more reviewers, please tag someone or notify the dev@solr.apache.org mailing list. Thank you for your contribution!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org