You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pinot.apache.org by Pinot Slack Email Digest <sn...@apache.org> on 2020/07/14 02:00:06 UTC
Apache Pinot Daily Email Digest (2020-07-13)
<h3><u>#general</u></h3><br><strong>@avns.ajit: </strong>@avns.ajit has joined the channel<br><strong>@anurag.rp.sharma: </strong>@anurag.rp.sharma has joined the channel<br><strong>@tmmwangioo3: </strong>Hi all<br><strong>@tmmwangioo3: </strong>I’ve filed a bunch of issues I’ve encountered recently<br><strong>@mayanks: </strong>Hello<br><strong>@tmmwangioo3: </strong><https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMSfW2QiSG4bkQpnpkSL7FiK3MHb8libOHmhAW89nP5XKkHh-2FzSdSPLz63LVYyJFuQA-3D-3Dc2D7_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTzuzBAQtVHWor8p4rIuiN3gFGWdCuMFREp4-2FedByiMZW1v-2B8Z2D1y0MG-2Byv3jZtU8IZ21ZusBk4ltBg1v4BVQMye8F395bfv4PDUev7xPompH7myk6NJKARorTIY-2B0-2BsWq0SygBMJvWIudThrHNEjtlYqzwp4eR1U7j5YazwrNtQn7MGcHvofQBClZFI-2FYTS9Y-3D><br><strong>@tmmwangioo3: </strong><https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMSfW2QiSG4bkQpnpkSL7FiK3MHb8libOHmhAW89nP5XKWrDNtdGICBhwz6iZYDHhsg-3D-3DDh_1_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTzuzBAQtVHWor8p4rIuiN3gb-2FwIOTUU5KMxmRFZ7eajjBAvJ3QP3cQKrtejoaQJvHtWO-2BQhPeW54Fyrh-2BU7fSVhNbFzaDcyehAxMXBlNmycJpP5C6F8lyamV0uqf4d5R1Cpnx2jBS2FctfooeFVj6bqTA6wxCIgxYaox1RAUhyX-2BCEQkJb2Ut8mQ4Cml-2F-2B1QFw-3D><br><strong>@tmmwangioo3: </strong><https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMSfW2QiSG4bkQpnpkSL7FiK3MHb8libOHmhAW89nP5XKPwYTxKumYiXgdNVutVjQqg-3D-3D7uWH_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTzuzBAQtVHWor8p4rIuiN3gubUGSEjxJRZY8iAS-2Bcb78Y3u1Y92KJ87aszI55WqnufafiY3Ahhkv1XoyI3LCvInep4Bta-2FcuwooJJ3KdAfpJKpexNfdar6VYkMXiCCHlzWwlSOOZ-2FvDZVmnyILPSADFoMTpkHa-2BrjeAFjWZ9Hh2JyOsiVhQTsz3VW5ELQvk9uQ-3D><br><strong>@tmmwangioo3: </strong>I don’t know if the latter 2 have been solved by this <https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMSfW2QiSG4bkQpnpkSL7FiK3MHb8libOHmhAW89nP5XKA6n1WzLjxiRXzbvLsPOKcg-3D-3Dnd42_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTzuzBAQtVHWor8p4rIuiN3gR2A-2BMuTT1A7msdw-2FSJGIS7VP7wmnbvFKJYgqdxgMwUeRFBusfhuRkGdFYIaJc-2BRmFrjmjY0JyQqNcv9sYa2EWJqadcWBTDHUmCpQ-2BvfHh-2FJ8HFhnoJQAOlCsjmedC0WRaZ0a8eKen2uJ-2FKKjNc-2BSQmfLmnXlg2ApEzxnzpykVlU-3D><br><strong>@mayanks: </strong>Can you elaborate a bit on your usage of BYTES types columns?<br><strong>@mhomaid: </strong>is there thirdeye channel ?<br><strong>@g.kishore: </strong>@mhomaid <https://u17000708.ct.sendgrid.net/ls/click?upn=iSrCRfgZvz-2BV64a3Rv7HYUsp-2BvNiWH3ChUHHwuSoHQR3Nb86TCSGfz-2FYL4LaYL-2BQR-2BTqrj95gl3AauwPUMwvFbTcpOyw5m6wi91gRPxvHpA-3Dfp8c_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTzuzBAQtVHWor8p4rIuiN3gxuu8hnNqMSU6X6VQvRjx1q8zpynHoNrK3StcOd-2F1f5buADgNiA5iX30jWwYln4BVq5Hq5Wk4Tlyn8-2BK1ntzJuyPFzdaf91E5iAQl5GvITmpX-2FvhfcNQxJbkQozvXLjOqZpK7Zy1AvJ6MNRkKN8PZLZ5bRJj-2BNhkiO15PQyF3V3E-3D><br><h3><u>#random</u></h3><br><strong>@avns.ajit: </strong>@avns.ajit has joined the channel<br><strong>@anurag.rp.sharma: </strong>@anurag.rp.sharma has joined the channel<br><h3><u>#pql-2-calcite</u></h3><br><strong>@mans4singh: </strong>@mans4singh has joined the channel<br><h3><u>#troubleshooting</u></h3><br><strong>@rgarcia: </strong>Hello, I'm newbie with Pinot and I need some help.
I have deployed Pinot into a swarm cluster with docker. There is a controller, a broker and a server. I defined an schema and a realtime table. Zookeeper and Kafka were running before than Pinot, and are shared with other applications.
I can see the table into the Data Explorer, but when I run a query ( select * from dmaTestData limit 10 ), only get an timeout error.
Can someone help me?
What's wrong in my configuration?
--------------
docker-compose
--------------
version: '3.7'
services:
pinot-controller:
image: apachepinot/pinot:latest
ports:
- "9091:9000"
command: ["StartController", "-zkAddress", "zookeeper:2181"]
volumes:
- ./data/controller_data:/opt/pinot/data
- ./data/controller_configs:/opt/pinot/configs
networks:
- public
pinot-broker:
image: apachepinot/pinot:latest
command: ["StartBroker", "-zkAddress", "zookeeper:2181"]
volumes:
- ./data/broker_data:/opt/pinot/data
- ./data/broker_configs:/opt/pinot/configs
networks:
- public
pinot-server:
image: apachepinot/pinot:latest
command: ["StartServer", "-zkAddress", "zookeeper:2181"]
volumes:
- ./data/server_data:/opt/pinot/data
- ./data/server_configs:/opt/pinot/configs
networks:
- public
networks:
public:
external: true
------
Schema
------
{
"schemaName": "dmaTestSchema",
"dimensionFieldSpecs": [
{
"name": "dma_name",
"dataType": "STRING"
},
{
"name": "dma_id",
"dataType": "INT"
}
],
"metricFieldSpecs": [
{
"name": "value",
"dataType": "FLOAT"
},
{
"name": "value_old",
"dataType": "FLOAT"
}
],
"dateTimeFieldSpecs": [
{
"name": "time_stamp",
"dataType": "STRING",
"format" : "1:DAYS:SIMPLE_DATE_FORMAT:yyyy-MM-dd",
"granularity": "1:DAYS"
}
]
}
-----
Table
-----
{
"tableName": "dmaTestData",
"tableType": "REALTIME",
"segmentsConfig": {
"timeColumnName": "time_stamp",
"timeType": "DAYS",
"schemaName": "dmaTestSchema",
"replication": "1",
"replicasPerPartition": "1"
},
"tenants": {
"broker": "pinot-broker",
"server": "pinot-server"
},
"tableIndexConfig": {
"loadMode": "MMAP",
"streamConfigs": {
"streamType": "kafka",
"stream.kafka.consumer.type": "simple",
"stream.kafka.topic.name": "transcript-topic",
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
"stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
"stream.kafka.broker.list": "kafka:9092",
"stream.kafka.hlc.zk.connect.string": "zookeeper:2181/kafka",
"stream.kafka.zk.broker.url": "zookeeper:2181/kafka",
"realtime.segment.flush.threshold.time": "12h",
"realtime.segment.flush.threshold.size": "100000",
"stream.kafka.consumer.prop.auto.offset.reset": "smallest"
}
},
"metadata": {
"customConfigs": {}
}
}<br><strong>@mans4singh: </strong>@mans4singh has joined the channel<br><h3><u>#pinot-dev</u></h3><br><strong>@tingchen: </strong>Is there any ongoing project to add "EXPLAIN" functionality to Pinot query?<br><strong>@tingchen: </strong>with more index types added, EXPLAIN can show how the query would be executed. We got a few requests for our internal customers too.<br><strong>@g.kishore: </strong>There is none, will be a good feature to have but don’t see it as top priority for this quarter <br><strong>@g.kishore: </strong><!here> we have too many PR's waiting for reviews. will be great to get some help<br><h3><u>#presto-pinot-streaming</u></h3><br><strong>@elon.azoulay: </strong>Thanks for the review! Updated <https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMSfW2QiSG4bkQpnpkSL7FiK3MHb8libOHmhAW89nP5XKlOgp56r8UA0vv6Zm3Qwxeg-3D-3DKhCK_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTzuzBAQtVHWor8p4rIuiN3gf22FKi-2F1Uknl5TSzgHzdrnvTufEetaEHnmrVxbvwFsANP-2BHcLJ1Qifrg8eJybbJln-2BVyDMyeFFBAETDDx4GtrvB8nLN11DS1W4Khee5cte9uk0qH3TeikNJTQqG0rWub0sTTMdlIvQgBZbMQ4hoJ74PQs1F5D6D-2Bp6Qv6Mjd30E-3D> and updated the doc. Let me know when you have time to talk about the next steps. I have commits I did not yet publish for handling the request, creating a plan, etc. but wanted to sync up first.<br><strong>@jackie.jxt: </strong>Merged the first PR<br><strong>@jackie.jxt: </strong>@elon.azoulay Any time this afternoon is fine<br><strong>@mayanks: </strong>@elon.azoulay Does this endpoint support ACLs?<br>