You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu> on 2021/10/03 16:31:11 UTC

Change in asterixdb[master]: [ASTERIXDB-2895][RT] Vsize buffers in PyUDF IPC

From Wael Alkowaileet <wa...@gmail.com>:

Wael Alkowaileet has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643 )

Change subject: [ASTERIXDB-2895][RT] Vsize buffers in PyUDF IPC
......................................................................


Patch Set 20:

(11 comments)

https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/builders/StdToModUTF8DataOutputFactory.java 
File asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/builders/StdToModUTF8DataOutputFactory.java:

https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/builders/StdToModUTF8DataOutputFactory.java@28 
PS20, Line 28: StdToModUTF8DataOutputFactory
Minor: Maybe use the full "StandardToModifiedUTF8DataOutputFactory"?


https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/ipc/PythonMessageBuilder.java 
File asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/ipc/PythonMessageBuilder.java:

https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/ipc/PythonMessageBuilder.java@47 
PS20, Line 47: position
clear() will make position=0 and limit=capacity


https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarPythonFunctionEvaluator.java 
File asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarPythonFunctionEvaluator.java:

https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarPythonFunctionEvaluator.java@138 
PS20, Line 138: Returned result missing outer wrapper
Maybe move the message to ErrorCode instead of hard coded (for localization)


https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/msgpack/MessagePackerFromADM.java 
File asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/msgpack/MessagePackerFromADM.java:

https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/msgpack/MessagePackerFromADM.java@77 
PS20, Line 77:  private final CharsetEncoder encoder;
             :     private final CharBuffer cbuf;
Are these still used?


https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/msgpack/MessagePackerFromADM.java@91 
PS20, Line 91: TaggedValuePointable pointy = TaggedValuePointable.FACTORY.createPointable();
Is this needed? You can get the type tag from ptr.getByteArray()[ptr.getStartOffset()]


https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/msgpack/MessagePackerFromADM.java@95 
PS20, Line 95: pack
Does it expect the value to be tagged or untagged? The value here (from ptr) contains the type tag. But below, it does not.


https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/msgpack/MessagePackerFromADM.java@97 
PS20, Line 97: pack
Same. Is the value tagged or untagged?


https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/msgpack/MessagePackerFromADM.java@97 
PS20, Line 97: true
The value is not tagged?


https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/msgpack/MessagePackerFromADM.java@152 
PS20, Line 152: PARSER_ADM_DATA_PARSER_CAST_ERROR
Maybe ErrorCode#TYPE_UNSUPPORTED instead?


https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/msgpack/MessagePackerFromADM.java@264 
PS20, Line 264:         cbuf.clear();
              :         cbuf.position(0);
Still needed?


https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643/20/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/msgpack/MessagePackerFromADM.java@306 
PS20, Line 306: getClosedFieldOffset
Need to inspect the null bitmap to check whether the value is null/missing



-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12643
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ic95e592b42139b4750af8bb20291f926b3c973e2
Gerrit-Change-Number: 12643
Gerrit-PatchSet: 20
Gerrit-Owner: Ian Maxon <im...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Ian Maxon <im...@uci.edu>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Wael Alkowaileet <wa...@gmail.com>
Gerrit-Comment-Date: Sun, 03 Oct 2021 16:31:11 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Gerrit-MessageType: comment