You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/11/11 20:31:17 UTC
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6235: Data and query generator bytes-column support
Jackie-Jiang commented on a change in pull request #6235:
URL: https://github.com/apache/incubator-pinot/pull/6235#discussion_r521620768
##########
File path: pinot-tools/src/main/java/org/apache/pinot/tools/data/generator/DataGeneratorSpec.java
##########
@@ -41,6 +41,7 @@
private final Map<String, DataType> dataTypesMap;
private final Map<String, FieldType> fieldTypesMap;
+ private final Map<String, Boolean> singleValueFlagsMap;
Review comment:
Are we capable of generating MV values?
##########
File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/QueryGenerator.java
##########
@@ -182,7 +184,12 @@ public QueryGenerator(List<File> avroFiles, String pinotTableName, String h2Tabl
* @param avroValue Avro value.
*/
private static void storeAvroValueIntoValueSet(Set<String> valueSet, Object avroValue) {
- if (avroValue instanceof Number) {
+ if (avroValue instanceof ByteBuffer) {
+ // for raw bytes
+ String hexRaw = StringUtils.stripStart(Hex.encodeHexString(((ByteBuffer) avroValue).array()), "0");
+ String hexAligned = (hexRaw.length() & 0x1) == 0 ? hexRaw : "0" + hexRaw;
Review comment:
Why not directly using `Hex.encodeHexString(((ByteBuffer) avroValue).array())`?
##########
File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/QueryGenerator.java
##########
@@ -182,7 +184,12 @@ public QueryGenerator(List<File> avroFiles, String pinotTableName, String h2Tabl
* @param avroValue Avro value.
*/
private static void storeAvroValueIntoValueSet(Set<String> valueSet, Object avroValue) {
- if (avroValue instanceof Number) {
+ if (avroValue instanceof ByteBuffer) {
+ // for raw bytes
+ String hexRaw = StringUtils.stripStart(Hex.encodeHexString(((ByteBuffer) avroValue).array()), "0");
+ String hexAligned = (hexRaw.length() & 0x1) == 0 ? hexRaw : "0" + hexRaw;
Review comment:
Also, does Avro always have `ByteBuffer` backed by an array?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org