You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pinot.apache.org by Pinot Slack Email Digest <sn...@apache.org> on 2020/07/27 02:00:08 UTC
Apache Pinot Daily Email Digest (2020-07-26)
<h3><u>#general</u></h3><br><strong>@moghey.apoorv: </strong>A simple question:
Does it make sense to keep segment retention 4 years in case of realtime tables?
Shall we go for a hybrid table?
What is the recommendation?
or is this decision depend upon other factors?
In our case, we will always be ingesting data via Kafka.<br><strong>@techlabassociates: </strong>@techlabassociates has joined the channel<br><strong>@moghey.apoorv: </strong>Another simple question
I was following this doc. <https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMdTeAXadp8BL3QinSdRtJdoH1omTMY3qwbAUC5IutlmIfnExuLFuaw8aCfHk5tae35jst1BsOURaWXx6d7RyTlLc9Gi1Za-2FQiA9u8Mu498Q-2FJkcu_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTxSWTRnp2Vki1mAYWQmdza6Um-2BRMBNJdQZU8QiNyG97mtuqBiEansPcd4yMoK3DffBATZn08PtKlNbV2BraIhqffA5CPgvdTq0pfqw7-2FVGFNsWKl-2FWhTuMip1cYUkBUdZcnQtLc0j6z293Qro3yON9BUX-2Bln8kVLWYjdjeRTAYOhixX9lNVuAdMwrTgCKNJ4K8-3D>
it mentioned `*pinot.server.instance.realtime.alloc.offheap: true*.`
But I am not able to get the format of Pinot server configuration, is it YAML or JSON?
There is a parameter called -configFileName. But it's description says `Broker Starter Config file`. I am little confused here. Can anybody help?<br><strong>@g.kishore: </strong>It’s a key value pairs<br><strong>@g.kishore: </strong>@npawar will be presenting Pinot at Kafka Summit. <https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMeEYm6Ej-2B2POza216FzajVPgfkpopWUSPaSEfvzgkDlKPO4V4e6rNfwBuB1NrM0cIMEJNZvAy-2BbodQnFcSy-2B27phraP7yzF-2F-2BuF2BTt-2FyR-2BkFK9tTFeP5ahonxZC7nFtyEx8zblGqNYxbIjgmxvvoh7-2FYI5-2F3myXQTjHDmFvi879Hr3s_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTxSWTRnp2Vki1mAYWQmdza6pL4rtV4A81YwWAkSUuNQ92gWNQLUz0TdrhDwCCeCoHYxMvEl3USdKBu-2F06vzduXwGI3irLY3MdYg61Gw6ofpWHJyfygHRsHqcHscnD-2FfkEyZ8OYFcPYV6TBDyvBSHjf7isnAek3NYv8FXjDpCygtxE1g4N9Ero7iyR1pZum1dj4-3D> aug 24 1pm pst. This years Kafka Summit virtual and free. <br><h3><u>#random</u></h3><br><strong>@techlabassociates: </strong>@techlabassociates has joined the channel<br><h3><u>#troubleshooting</u></h3><br><strong>@pradeepgv42: </strong>Sorry I am facing another issue while querying the table, after fixing the above issue with this code-change (<https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMSfW2QiSG4bkQpnpkSL7FiK3MHb8libOHmhAW89nP5XKm-2FonoynKraNJYoo1fPujzw-3D-3DscXI_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTxSWTRnp2Vki1mAYWQmdza6hRJw3ExaJCXNiskj-2B78wIYG1oyCuzBLkn4KhpBgCylZoZ-2Bh1ZwGY7ycZc9EazEYjpSgM2fldhtgmh7e9VZrEWYCueTKmmJLCmuCex8wxAhQbUinXfff76makDyHPeSLwRYW5ObxH-2BbC9zj4cZcHDHJnTjup13b51hBtpf6gyUjw-3D>). KafkaConsumer seems to be working fine now based on the logs on server node.
```Caught exception while processing instance request
java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
at org.apache.pinot.core.common.datatable.DataTableBuilder.setColumn(DataTableBuilder.java:157) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-39a78f3df43ac613663975844e598f07b18bf623]
at org.apache.pinot.core.query.selection.SelectionOperatorUtils.getDataTableFromRows(SelectionOperatorUtils.java:261) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-39a78f3df43ac613663975844e598f07b18bf623]
at org.apache.pinot.core.operator.blocks.IntermediateResultsBlock.getSelectionResultDataTable(IntermediateResultsBlock.java:348) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-39a78f3df43ac613663975844e598f07b18bf623]
at org.apache.pinot.core.operator.blocks.IntermediateResultsBlock.getDataTable(IntermediateResultsBlock.java:262) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-39a78f3df43ac613663975844e598f07b18bf623]
at org.apache.pinot.core.operator.blocks.InstanceResponseBlock.<init>(InstanceResponseBlock.java:43) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-39a78f3df43ac613663975844e598f07b18bf623]
at org.apache.pinot.core.operator.InstanceResponseOperator.getNextBlock(InstanceResponseOperator.java:37) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-39a78f3df43ac613663975844e598f07b18bf623]
at org.apache.pinot.core.operator.InstanceResponseOperator.getNextBlock(InstanceResponseOperator.java:26) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-39a78f3df43ac613663975844e598f07b18bf623]
at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-39a78f3df43ac613663975844e598f07b18bf623]
at org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:48) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-39a78f3df43ac613663975844e598f07b18bf623] ```
<br>