Apache Pinot Daily Email Digest (2020-09-13)

[Pinot Slack Email Digest <sn...@apache.org>]

<h3><u>#general</u></h3><br><strong>@park.aaron.y: </strong>@park.aaron.y has joined the channel<br><h3><u>#random</u></h3><br><strong>@park.aaron.y: </strong>@park.aaron.y has joined the channel<br><h3><u>#lp-pinot-poc</u></h3><br><strong>@fx19880617: </strong>Hi, we tried out pinot server deployment with a different GC option: use `g1gc` for pinot server will reduce p999 5 times from our observations.<br><strong>@fx19880617: </strong><br><strong>@fx19880617: </strong>this is a sample comparison ,the second part is just by changing the gc settings<br><strong>@fx19880617: </strong>we are adding below settings to the jvmOpts, so we can also see the gc logs
```-Xms16G -Xmx16G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:/opt/pinot/gc-pinot-server.log```<br><strong>@fx19880617: </strong>apart from this, you the place to put `pinot.server.instance.realtime.alloc.offheap=true` is at pinot serve config file, not inside the table config. You can follow this PR: <https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMSfW2QiSG4bkQpnpkSL7FiK3MHb8libOHmhAW89nP5XKhW-2FBG3g8flDyjLykkSst4g-3D-3DBpbR_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTxEbHe-2BpfKb-2Fd05AHfncZBL7HgAJbVaeDcfsd9vH1y5c82r-2FzeXux57vSPKxVkvrF0y05oitQPz9Tf4Z6tntaHH3OVpE53VLpyOM49o6hmFDGdq9kFQpLzBJtcQBvTbXIphBkojBfcPd53JU9bfwaxUOJY8Fy-2F9Ci5fQHgHWhC5-2BevsJ6TvPPpYWlv9okdTUJI-3D> to make the changes<br><strong>@fx19880617: </strong>This is the graph, once adding the offheap into server config:<br><strong>@fx19880617: </strong><br><strong>@fx19880617: </strong>p999 i s 200ms<br><strong>@fx19880617: </strong>i’m using 4 pinot-servers fo the benchmark just fyi<br><strong>@fx19880617: </strong>offheap could give about  10x latency improvement<br><strong>@fx19880617: </strong><br><strong>@fx19880617: </strong>this is another test by updating userId from `sortedColumn` to `invertedIndexColumns`<br><strong>@fx19880617: </strong>```...
        "sortedColumn": [
        ],
        "invertedIndexColumns": [
          "userId"
        ]
...```
<br><strong>@fx19880617: </strong><br>