You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "dachaostripe (via GitHub)" <gi...@apache.org> on 2023/03/29 16:45:34 UTC
[GitHub] [pinot] dachaostripe opened a new issue, #10504: Optimize controller UI for large cluster/tables
dachaostripe opened a new issue, #10504:
URL: https://github.com/apache/pinot/issues/10504
We have large tables that have over 220k and 370k offline segments, that we almost always timeout trying to open the offline table page, as it seems the page was trying to load all segment data on the page. It would be great if the UI can be redesigned or tweaked to not trying to load all those data on page load, to make UI more responsive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #10504: Optimize controller UI for large cluster/tables
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10504:
URL: https://github.com/apache/pinot/issues/10504#issuecomment-1491056065
@jayeshchoudhary Can you help list down all the data that the UI is collecting before the page is rendered? Then we can discuss how to reduce the data to be fetched for large tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jadami10 commented on issue #10504: Optimize controller UI for large cluster/tables
Posted by "jadami10 (via GitHub)" <gi...@apache.org>.
jadami10 commented on issue #10504:
URL: https://github.com/apache/pinot/issues/10504#issuecomment-1489415980
yup, i think we can take the same strategy as the tables page and render everything separately
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #10504: Optimize controller UI for large cluster/tables
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10504:
URL: https://github.com/apache/pinot/issues/10504#issuecomment-1489191513
cc @joshigaurava @jayeshchoudhary @jadami10
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] joshigaurava commented on issue #10504: Optimize controller UI for large cluster/tables
Posted by "joshigaurava (via GitHub)" <gi...@apache.org>.
joshigaurava commented on issue #10504:
URL: https://github.com/apache/pinot/issues/10504#issuecomment-1489451534
@Jackie-Jiang, noted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jayeshchoudhary commented on issue #10504: Optimize controller UI for large cluster/tables
Posted by "jayeshchoudhary (via GitHub)" <gi...@apache.org>.
jayeshchoudhary commented on issue #10504:
URL: https://github.com/apache/pinot/issues/10504#issuecomment-1490928042
I think this is more related to the backend. Currently, UI calls idealState and externalView APIs to show segments.
If there are 100 segments the API response will be faster, if there are 100k segments the API response will be slow. so UI can't do anything here to compensate for the slowness.
IMO we should have some pagination/search in the API, which UI can leverage to only show a subset of the segments.
But, I don't know the userflow here. maybe we always want to look at all the segments? or if we show only a subset of segments how do we consume them?
We can think about the common workflows here and decide based on user behavior. After this, there will be backend + UI changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Optimize controller UI for large cluster/tables [pinot]
Posted by "jadami10 (via GitHub)" <gi...@apache.org>.
jadami10 commented on issue #10504:
URL: https://github.com/apache/pinot/issues/10504#issuecomment-1852457305
I'm hoping to take a pass at this during the EOY slowdown. My approach is going to be:
- decouple rendering from data loading
- use loading skeletons while loading data
I'm going to tackle the biggest known issues first:
- /size: this reaches out to servers for table size. and depending on current load, it can take ~forever. It also only populates numbers in the UI, so there's no reason to stop rendering the whole page
- idealstate/externalview: this is known to be slow for large tables. It also causes memory issues in the browser tab, but i don't have an easy solution for that yet until things like the segment list do server side pagination. But this endpoint is mainly used to show the "status" of the table or the segments. Again, no need to block the whole UI from loading.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org