You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/04/29 22:57:40 UTC
[GitHub] [incubator-pinot] Jackie-Jiang opened a new issue #6865: Add controller rest API to list recently created/modified tables
Jackie-Jiang opened a new issue #6865:
URL: https://github.com/apache/incubator-pinot/issues/6865
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6865: Add controller rest API to list recently created/modified tables
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6865:
URL: https://github.com/apache/incubator-pinot/issues/6865#issuecomment-875903834
@mcvsubbu I agree having a pinot cluster change log table could be very useful, but that involves much bigger change. IMO checking the ZNode metadata should be good enough to solve this issue for now, and we can change the underlying implementation when the log table is available in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6865: Add controller rest API to list recently created/modified tables
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6865:
URL: https://github.com/apache/incubator-pinot/issues/6865#issuecomment-876664247
@yashrsharma44 `TableConfig` is stored as a `ZNRecord` in the Zookeeper. We can use the `ZNRecord` metadata for the create/last modify time (`ZNRecord.getCreationTime()` and `ZNRecord.getModifiedTime()`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] mcvsubbu commented on issue #6865: Add controller rest API to list recently created/modified tables
Posted by GitBox <gi...@apache.org>.
mcvsubbu commented on issue #6865:
URL: https://github.com/apache/incubator-pinot/issues/6865#issuecomment-829755695
I recently did that for the corp cluster at Linkedin, so I can see that this may be useful. In our case, the zookeeper servers had also been replaced but the new servers had retained the metadata too (surprise for me). Are there cases where this can change? Also, do we want to depend on zk metadata for our info? It will be nice if we had a pinot internal table that kept certain things.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6865: Add controller rest API to list recently created/modified tables
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6865:
URL: https://github.com/apache/incubator-pinot/issues/6865#issuecomment-829741340
@mcvsubbu I believe publishing events requires extra dependency on a streaming service, unless we can find a way to short-circuit it and directly ingest the events into a table.
What I'm thinking here is to just grab all table config ZNRecords, and sort them based on creation time or last modification time. This can help track the recent changes in a large cluster.
@yupeng9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] yashrsharma44 commented on issue #6865: Add controller rest API to list recently created/modified tables
Posted by GitBox <gi...@apache.org>.
yashrsharma44 commented on issue #6865:
URL: https://github.com/apache/incubator-pinot/issues/6865#issuecomment-876656732
I checked out the `TableConfig` - https://github.com/apache/incubator-pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/common/utils/config/TableConfigUtils.java class, but didn't find any **last related**/**last modified** parameter. Any pointers where I can get the time parameter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] yashrsharma44 commented on issue #6865: Add controller rest API to list recently created/modified tables
Posted by GitBox <gi...@apache.org>.
yashrsharma44 commented on issue #6865:
URL: https://github.com/apache/incubator-pinot/issues/6865#issuecomment-875819968
Hi @Jackie-Jiang,
I wanted to pick up this issue, if no one is working on it.
Could you provide more details about the requirements and what needs to be done?
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang closed issue #6865: Add controller rest API to list recently created/modified tables
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang closed issue #6865:
URL: https://github.com/apache/incubator-pinot/issues/6865
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang closed issue #6865: Add controller rest API to list recently created/modified tables
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang closed issue #6865:
URL: https://github.com/apache/incubator-pinot/issues/6865
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6865: Add controller rest API to list recently created/modified tables
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6865:
URL: https://github.com/apache/incubator-pinot/issues/6865#issuecomment-875908401
@yashrsharma44 Great! Thanks for volunteering. Basically we want to add a new rest API endpoint on controller to list all tables sorted by their creation/modification time. One way to do that is to enhance the `PinotTableRestletResource.listTableConfigs()` (we should rename this method because it only returns the table names instead of configs) to take an extra query parameter for the ordering. By default all the table names are ordered alphabetically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang closed issue #6865: Add controller rest API to list recently created/modified tables
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang closed issue #6865:
URL: https://github.com/apache/incubator-pinot/issues/6865
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] mcvsubbu commented on issue #6865: Add controller rest API to list recently created/modified tables
Posted by GitBox <gi...@apache.org>.
mcvsubbu commented on issue #6865:
URL: https://github.com/apache/incubator-pinot/issues/6865#issuecomment-829653376
What is the use case here?
Why not publish an event into a (lightweight pinot) table for each table creation, and query that table? We can then add other columns as well to this table, and use it to provide statistics about pinot tables? (
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org