You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "snleee (via GitHub)" <gi...@apache.org> on 2023/05/05 21:09:34 UTC

[GitHub] [pinot] snleee opened a new issue, #10727: RecordReader config is not correctly wired in `SegmentGenerationAndTaskExecutor`

snleee opened a new issue, #10727:
URL: https://github.com/apache/pinot/issues/10727

   `SegmentGenerationAndTaskExecutor` doesn't correctly wire `recordReader.prop.<keys>`. Due to this, record reader config (e.g. delimiter for csv file) cannot be configured in `SegmentGenerationAndTaskExecutor`.
   
   ```
   SegmentGenerationAndPushTaskExecutor: L286
         RecordReaderSpec recordReaderSpec = new RecordReaderSpec();
         recordReaderSpec.setDataFormat(taskConfigs.get(BatchConfigProperties.INPUT_FORMAT));
         recordReaderSpec.setClassName(taskConfigs.get(BatchConfigProperties.RECORD_READER_CLASS));
         recordReaderSpec.setConfigClassName(taskConfigs.get(BatchConfigProperties.RECORD_READER_CONFIG_CLASS));
         ...
         
         --> We need to add the following line:
         recordReaderSpec.setConfigs(IngestionConfigUtils.getRecordReaderProps(taskConfigs));
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] snleee commented on issue #10727: RecordReader config is not correctly wired in `SegmentGenerationAndTaskExecutor`

Posted by "snleee (via GitHub)" <gi...@apache.org>.
snleee commented on issue #10727:
URL: https://github.com/apache/pinot/issues/10727#issuecomment-1536913062

   @abhioncbr  FYI, Johan from OSS reported this and he showed the interest to pick this ticket. I mentioned you in the discussion thread from Apache Pinot Slack Channel.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] jvenant commented on issue #10727: RecordReader config is not correctly wired in `SegmentGenerationAndTaskExecutor`

Posted by "jvenant (via GitHub)" <gi...@apache.org>.
jvenant commented on issue #10727:
URL: https://github.com/apache/pinot/issues/10727#issuecomment-1537134857

   Here it is


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] abhioncbr commented on issue #10727: RecordReader config is not correctly wired in `SegmentGenerationAndTaskExecutor`

Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on issue #10727:
URL: https://github.com/apache/pinot/issues/10727#issuecomment-1536887564

   Please assign the ticket to me. I will work on it. Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] Jackie-Jiang closed issue #10727: RecordReader config is not correctly wired in `SegmentGenerationAndTaskExecutor`

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed issue #10727: RecordReader config is not correctly wired in `SegmentGenerationAndTaskExecutor`
URL: https://github.com/apache/pinot/issues/10727


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org