You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/03/11 04:01:38 UTC
[GitHub] [pinot] yupeng9 commented on a change in pull request #8273: Add prefixesToRename config for renaming fields upon ingestion
yupeng9 commented on a change in pull request #8273:
URL: https://github.com/apache/pinot/pull/8273#discussion_r824367859
##########
File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ComplexTypeTransformer.java
##########
@@ -279,6 +295,28 @@ protected void flattenMap(GenericRow record, List<String> columns) {
}
}
+ /**
+ * Loops through all columns and renames the column's prefix with the corresponding replacement if the prefix matches.
+ */
+ @VisibleForTesting
+ protected void renamePrefixes(GenericRow record) {
+ List<String> fields = new ArrayList<>(record.getFieldToValueMap().keySet());
Review comment:
nit: short-circuit this if `_prefixesToRename` is empty
##########
File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ComplexTypeTransformer.java
##########
@@ -279,6 +295,28 @@ protected void flattenMap(GenericRow record, List<String> columns) {
}
}
+ /**
+ * Loops through all columns and renames the column's prefix with the corresponding replacement if the prefix matches.
+ */
+ @VisibleForTesting
+ protected void renamePrefixes(GenericRow record) {
+ List<String> fields = new ArrayList<>(record.getFieldToValueMap().keySet());
+ for (String field : fields) {
+ for (String prefix : _prefixesToRename.keySet()) {
+ if (field.startsWith(prefix)) {
+ Object value = record.removeValue(field);
+ String remainingColumnName = field.substring(prefix.length());
+ String newName = _prefixesToRename.get(prefix) + remainingColumnName;
Review comment:
is it possible the new name is empty?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org