You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "eaugene (via GitHub)" <gi...@apache.org> on 2023/07/03 06:00:37 UTC
[GitHub] [pinot] eaugene opened a new issue, #11024: Web Socket Endpoints for querying in Pinot Broker
eaugene opened a new issue, #11024:
URL: https://github.com/apache/pinot/issues/11024
Currently, we have REST API endpoints in pinot broker for querying. However, adding a WebSocket endpoint helps with having low latency & high throughput queries once the connection is established between the pinot broker and the client service.
**Use Case:**
Using WebSocket to connect to a Pinot table that is being populated from a Kafka topic / some streaming source. By not directly connecting to the Kafka topic, we can take advantage of data manipulation and efficient filtering, and aggregation features in Pinot. This approach provides more control and flexibility over the data flow and analysis process.
label: `feature`
cc : @atris @kishoreg @Aravind-Suresh
Related Slack Discussion : https://apache-pinot.slack.com/archives/CDRCA57FC/p1687701635253309
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] kishoreg commented on issue #11024: Web Socket Endpoints for querying in Pinot Broker
Posted by "kishoreg (via GitHub)" <gi...@apache.org>.
kishoreg commented on issue #11024:
URL: https://github.com/apache/pinot/issues/11024#issuecomment-1618433509
Like the phased approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] eaugene commented on issue #11024: Web Socket Endpoints for querying in Pinot Broker
Posted by "eaugene (via GitHub)" <gi...@apache.org>.
eaugene commented on issue #11024:
URL: https://github.com/apache/pinot/issues/11024#issuecomment-1618142759
@kishoreg Thanks for the questions.
1. The initial version of WebSocket Endpoint won't be for the standing queries. Web Socket would serve as one additional interface ( than existing REST API way) for an external system to connect and query in Pinot, here we save the connection establishment time for each HTTP request . The client is expected to give any queries as we currently have with REST API.
2. The additional use cases of making it support standing queries to period response/return after a change in data ( with a periodic check at pinot ) can be added once the underlying web socket interface is there. This would be a nice feature to support at Pinot as it removes the work at the client system ( particularly data viz. applications ) to periodically call Pinot.
3. For incremental results bounded to standing queries, we can this support once point 2 is made. Here we can utilise the time field of real-time data flowing in & a WebSocket connection at the pinot broker end can maintain the state of the last timestamp it served.
@atris Please add if I am missing any points.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] kishoreg commented on issue #11024: Web Socket Endpoints for querying in Pinot Broker
Posted by "kishoreg (via GitHub)" <gi...@apache.org>.
kishoreg commented on issue #11024:
URL: https://github.com/apache/pinot/issues/11024#issuecomment-1617417281
thanks for creating this proposal. Can we add some additional details..
- will this be a standing query..
- will the connection return something only when there is a change in response?
- or is it periodic
- is it going to incremental? That return only the changes from previous response? For e.g it's a filter query and the client is only interested in new rows that match the filter criteria.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org