You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "pongandnoon (via GitHub)" <gi...@apache.org> on 2023/09/14 04:59:24 UTC
[GitHub] [incubator-paimon] pongandnoon opened a new pull request, #2010: [core] fix ArrayIndexOutOfBoundsException when sequenceGroup repeat d…
pongandnoon opened a new pull request, #2010:
URL: https://github.com/apache/incubator-paimon/pull/2010
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
<!-- Linking this pull request to the issue -->
<!-- What is the purpose of the change -->
when sequenceGroup is repeat defined with no field in table，it cause
`java.lang.ArrayIndexOutOfBoundsException: -1
at java.util.ArrayList.elementData(ArrayList.java:424)
at java.util.ArrayList.get(ArrayList.java:437)
at org.apache.paimon.mergetree.compact.PartialUpdateMergeFunction$Factory.lambda$new$0(PartialUpdateMergeFunction.java:215)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.apache.paimon.mergetree.compact.PartialUpdateMergeFunction$Factory.<init>(PartialUpdateMergeFunction.java:209)
at org.apache.paimon.mergetree.compact.PartialUpdateMergeFunction$Factory.<init>(PartialUpdateMergeFunction.java:183)
at org.apache.paimon.mergetree.compact.PartialUpdateMergeFunction.factory(PartialUpdateMergeFunction.java:180)`
This message is not friendly.
### Tests
PartialUpdateMergeFunctionTest.testSequenceGroupRepeatDefineNoField
<!-- List UT and IT cases to verify this change -->
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi merged pull request #2010: [core] fix ArrayIndexOutOfBoundsException when sequenceGroup repeat d…
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #2010:
URL: https://github.com/apache/incubator-paimon/pull/2010
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] pongandnoon commented on a diff in pull request #2010: [core] fix ArrayIndexOutOfBoundsException when sequenceGroup repeat d…
Posted by "pongandnoon (via GitHub)" <gi...@apache.org>.
pongandnoon commented on code in PR #2010:
URL: https://github.com/apache/incubator-paimon/pull/2010#discussion_r1325632297
##########
paimon-core/src/main/java/org/apache/paimon/mergetree/compact/PartialUpdateMergeFunction.java:
##########
@@ -214,7 +214,10 @@ private Factory(Options options, RowType rowType) {
"Field %s is defined repeatedly by multiple groups: %s",
fieldNames.get(field), k));
}
- fieldSequences.put(field, sequenceGen);
+
+ if (field >= 0) {
Review Comment:
If we just define a sequenceGroup with fields which not in table schema, It don’t report any Exception, just ignore.Because it don't affect write .So I think sequenceGroup is repeat defined with fields which not in table schema should't report Exception,it can alse write.
##########
paimon-core/src/main/java/org/apache/paimon/mergetree/compact/PartialUpdateMergeFunction.java:
##########
@@ -214,7 +214,10 @@ private Factory(Options options, RowType rowType) {
"Field %s is defined repeatedly by multiple groups: %s",
fieldNames.get(field), k));
}
- fieldSequences.put(field, sequenceGen);
+
+ if (field >= 0) {
Review Comment:
If we just define a sequenceGroup with fields which not in table schema, It don’t report any Exception, just ignore.Because it don't affect write .So I think sequenceGroup is repeat defined with fields which not in table schema should't report Exception,it can alse write.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #2010: [core] fix ArrayIndexOutOfBoundsException when sequenceGroup repeat d…
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #2010:
URL: https://github.com/apache/incubator-paimon/pull/2010#discussion_r1325614681
##########
paimon-core/src/main/java/org/apache/paimon/mergetree/compact/PartialUpdateMergeFunction.java:
##########
@@ -214,7 +214,10 @@ private Factory(Options options, RowType rowType) {
"Field %s is defined repeatedly by multiple groups: %s",
fieldNames.get(field), k));
}
- fieldSequences.put(field, sequenceGen);
+
+ if (field >= 0) {
Review Comment:
I think better way is throw an exception with meaningful message?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org