You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/04/12 16:30:25 UTC
[GitHub] [incubator-pinot] Omkar-Halikar opened a new pull request #6774: Support REALTIME tables in reload status (segment metadata API) #6767
Omkar-Halikar opened a new pull request #6774:
URL: https://github.com/apache/incubator-pinot/pull/6774
## Description
<!-- Add a description of your PR here.
A good description should include pointers to an issue or design document, etc.
-->
This PR is to support REALTIME tables in reload status (segment metadata API) #6767 It includes pulling status from Immutable
Segments as well as Consuming Segments of REALTIME tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] npawar commented on pull request #6774: Support REALTIME tables in reload status (segment metadata API) #6767
Posted by GitBox <gi...@apache.org>.
npawar commented on pull request #6774:
URL: https://github.com/apache/incubator-pinot/pull/6774#issuecomment-822784048
Is the sample response in the description for a CONSUMING segment or an ONLINE segment of realtime?
Also, does this work in the UI now, or will some work be needed on that end?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Omkar-Halikar commented on pull request #6774: Support REALTIME tables in reload status (segment metadata API) #6767
Posted by GitBox <gi...@apache.org>.
Omkar-Halikar commented on pull request #6774:
URL: https://github.com/apache/incubator-pinot/pull/6774#issuecomment-821774894
Hi @mcvsubbu @npawar, have pulled in the latest changes from master. Could you please help with the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] npawar commented on pull request #6774: Support REALTIME tables in reload status (segment metadata API) #6767
Posted by GitBox <gi...@apache.org>.
npawar commented on pull request #6774:
URL: https://github.com/apache/incubator-pinot/pull/6774#issuecomment-822783453
> ## Description
> This PR is to support REALTIME tables in reload status (segment metadata API) #6767 It includes pulling status from Immutable
> Segments as well as Consuming Segments of REALTIME tables.
>
> ## Sample Request :
> http://localhost:9000/segments/airlineStats_REALTIME/metadata?type=REALTIME
>
> ## Sample Response
> ```
> {
> "airlineStats__0__5__20210415T2223Z": {
> "segmentName": "airlineStats__0__5__20210415T2223Z",
> "schemaName": null,
> "crc": 3618839852,
> "creationTimeMillis": 1618525449428,
> "creationTimeReadable": "2021-04-15T22:24:09:428 UTC",
> "timeGranularitySec": 86400,
> "startTimeMillis": 1388534400000,
> "startTimeReadable": "2014-01-01T00:00:00.000Z",
> "endTimeMillis": 1388620800000,
> "endTimeReadable": "2014-01-02T00:00:00.000Z",
> "pushTimeMillis": -9223372036854776000,
> "pushTimeReadable": null,
> "refreshTimeMillis": -9223372036854776000,
> "refreshTimeReadable": null,
> "segmentVersion": "v3",
> "creatorName": null,
> "paddingCharacter": "\u0000",
> "custom": {},
> "columns": [],
> "indexes": {
> "FlightNum": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Origin": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Quarter": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "LateAircraftDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivActualElapsedTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivWheelsOns": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivWheelsOffs": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "AirTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ArrDel15": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivTotalGTimes": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DepTimeBlk": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestCityMarketID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivAirportSeqIDs": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DaysSinceEpoch": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DepTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Month": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "$segmentName": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "CRSElapsedTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestStateName": {
> "bloom-filter": "YES",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Carrier": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestAirportID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Distance": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ArrTimeBlk": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivArrDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "SecurityDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "LongestAddGTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginWac": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "WheelsOff": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestAirportSeqID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "UniqueCarrier": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivReachedDest": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Diverted": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ActualElapsedTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "AirlineID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginStateName": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DepartureDelayGroups": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivAirportLandings": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "FlightDate": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginCityName": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginStateFips": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginState": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DistanceGroup": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "WeatherDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestWac": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "WheelsOn": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginAirportID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginCityMarketID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "NASDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ArrTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestState": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "$hostName": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ArrivalDelayGroups": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Flights": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DayofMonth": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "RandomAirports": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "TotalAddGTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "CRSDepTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DayOfWeek": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "CancellationCode": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Dest": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "FirstDepTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivTailNums": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DepDelayMinutes": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DepDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "TaxiIn": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginAirportSeqID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestStateFips": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ArrDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Cancelled": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivAirportIDs": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "$docId": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "TaxiOut": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "CarrierDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DepDel15": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivLongestGTimes": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivAirports": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivDistance": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Year": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ArrDelayMinutes": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "CRSArrTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "TailNum": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestCityName": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> }
> }
> }
> }
> ```
Is this sample response for a CONSUMING segment or an ONLINE segment of realtime?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Omkar-Halikar commented on a change in pull request #6774: Support REALTIME tables in reload status (segment metadata API) #6767
Posted by GitBox <gi...@apache.org>.
Omkar-Halikar commented on a change in pull request #6774:
URL: https://github.com/apache/incubator-pinot/pull/6774#discussion_r611919224
##########
File path: pinot-server/src/main/java/org/apache/pinot/server/api/resources/SegmentMetadataFetcher.java
##########
@@ -77,8 +80,13 @@ public static String getSegmentMetadata(SegmentDataManager segmentDataManager, L
if (immutableSegment instanceof ImmutableSegmentImpl) {
ImmutableSegmentImpl immutableSegmentImpl = (ImmutableSegmentImpl) immutableSegment;
Map<String, ColumnIndexContainer> columnIndexContainerMap = immutableSegmentImpl.getIndexContainerMap();
- columnIndexMap = getImmutableSegmentColumnIndexes(columnIndexContainerMap);
+ columnIndexMap = getSegmentColumnIndexes(columnIndexContainerMap);
Review comment:
Agreed, lot of type casting going on here. How about have it in the IndexSegment interface instead? Since managing the IndexContainer is really the job of the segment than the SegmentDataManager.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] npawar commented on a change in pull request #6774: Support REALTIME tables in reload status (segment metadata API) #6767
Posted by GitBox <gi...@apache.org>.
npawar commented on a change in pull request #6774:
URL: https://github.com/apache/incubator-pinot/pull/6774#discussion_r616150648
##########
File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/metadata/SegmentMetadataImpl.java
##########
@@ -554,7 +554,9 @@ public JsonNode toJson(@Nullable Set<String> columnFilter) {
String refreshTimeStr = _refreshTime != Long.MIN_VALUE ? dateFormat.format(new Date(_refreshTime)) : null;
segmentMetadata.put("refreshTimeReadable", refreshTimeStr);
- segmentMetadata.put("segmentVersion", _segmentVersion.toString());
+ if(_segmentVersion!=null){
Review comment:
nit: formatting looks off here
##########
File path: pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/IndexSegment.java
##########
@@ -86,6 +88,12 @@
*/
GenericRow getRecord(int docId, GenericRow reuse);
+ /**
+ * Returns a map of Column and its Index Container or null if there is no such map
+ * @return Column Index Container Map
+ */
+ Map<String, ? extends ColumnIndexContainer> getIndexContainerMap();
Review comment:
if this is going to return null, mark this as `@Nullable`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #6774: Support REALTIME tables in reload status (segment metadata API) #6767
Posted by GitBox <gi...@apache.org>.
mcvsubbu commented on a change in pull request #6774:
URL: https://github.com/apache/incubator-pinot/pull/6774#discussion_r611806017
##########
File path: pinot-server/src/main/java/org/apache/pinot/server/api/resources/SegmentMetadataFetcher.java
##########
@@ -77,8 +80,13 @@ public static String getSegmentMetadata(SegmentDataManager segmentDataManager, L
if (immutableSegment instanceof ImmutableSegmentImpl) {
ImmutableSegmentImpl immutableSegmentImpl = (ImmutableSegmentImpl) immutableSegment;
Map<String, ColumnIndexContainer> columnIndexContainerMap = immutableSegmentImpl.getIndexContainerMap();
- columnIndexMap = getImmutableSegmentColumnIndexes(columnIndexContainerMap);
+ columnIndexMap = getSegmentColumnIndexes(columnIndexContainerMap);
Review comment:
Instead of all this type-casting, why not extend the SegmentDataManager interface to include a method to get column indices?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] npawar removed a comment on pull request #6774: Support REALTIME tables in reload status (segment metadata API) #6767
Posted by GitBox <gi...@apache.org>.
npawar removed a comment on pull request #6774:
URL: https://github.com/apache/incubator-pinot/pull/6774#issuecomment-822783453
> ## Description
> This PR is to support REALTIME tables in reload status (segment metadata API) #6767 It includes pulling status from Immutable
> Segments as well as Consuming Segments of REALTIME tables.
>
> ## Sample Request :
> http://localhost:9000/segments/airlineStats_REALTIME/metadata?type=REALTIME
>
> ## Sample Response
> ```
> {
> "airlineStats__0__5__20210415T2223Z": {
> "segmentName": "airlineStats__0__5__20210415T2223Z",
> "schemaName": null,
> "crc": 3618839852,
> "creationTimeMillis": 1618525449428,
> "creationTimeReadable": "2021-04-15T22:24:09:428 UTC",
> "timeGranularitySec": 86400,
> "startTimeMillis": 1388534400000,
> "startTimeReadable": "2014-01-01T00:00:00.000Z",
> "endTimeMillis": 1388620800000,
> "endTimeReadable": "2014-01-02T00:00:00.000Z",
> "pushTimeMillis": -9223372036854776000,
> "pushTimeReadable": null,
> "refreshTimeMillis": -9223372036854776000,
> "refreshTimeReadable": null,
> "segmentVersion": "v3",
> "creatorName": null,
> "paddingCharacter": "\u0000",
> "custom": {},
> "columns": [],
> "indexes": {
> "FlightNum": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Origin": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Quarter": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "LateAircraftDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivActualElapsedTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivWheelsOns": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivWheelsOffs": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "AirTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ArrDel15": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivTotalGTimes": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DepTimeBlk": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestCityMarketID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivAirportSeqIDs": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DaysSinceEpoch": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DepTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Month": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "$segmentName": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "CRSElapsedTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestStateName": {
> "bloom-filter": "YES",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Carrier": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestAirportID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Distance": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ArrTimeBlk": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivArrDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "SecurityDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "LongestAddGTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginWac": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "WheelsOff": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestAirportSeqID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "UniqueCarrier": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivReachedDest": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Diverted": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ActualElapsedTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "AirlineID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginStateName": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DepartureDelayGroups": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivAirportLandings": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "FlightDate": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginCityName": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginStateFips": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginState": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DistanceGroup": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "WeatherDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestWac": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "WheelsOn": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginAirportID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginCityMarketID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "NASDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ArrTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestState": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "$hostName": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ArrivalDelayGroups": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Flights": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DayofMonth": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "RandomAirports": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "TotalAddGTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "CRSDepTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DayOfWeek": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "CancellationCode": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Dest": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "FirstDepTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivTailNums": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DepDelayMinutes": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DepDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "TaxiIn": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "OriginAirportSeqID": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestStateFips": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ArrDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Cancelled": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivAirportIDs": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "$docId": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "TaxiOut": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "CarrierDelay": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DepDel15": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivLongestGTimes": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivAirports": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DivDistance": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "Year": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "YES",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "ArrDelayMinutes": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "CRSArrTime": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "TailNum": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> },
> "DestCityName": {
> "bloom-filter": "NO",
> "dictionary": "YES",
> "forward-index": "YES",
> "inverted-index": "NO",
> "null-value-vector-reader": "NO",
> "range-index": "NO"
> }
> }
> }
> }
> ```
Is this sample response for a CONSUMING segment or an ONLINE segment of realtime?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6774: Support REALTIME tables in reload status (segment metadata API) #6767
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #6774:
URL: https://github.com/apache/incubator-pinot/pull/6774#discussion_r616133001
##########
File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/EmptyIndexSegment.java
##########
@@ -99,6 +101,11 @@ public GenericRow getRecord(int docId, GenericRow reuse) {
throw new UnsupportedOperationException();
}
+ @Override
+ public final Map<String, ColumnIndexContainer> getIndexContainerMap() {
+ return null;
Review comment:
Return `Collections.emptyMap()` here? Or it could cause NPE while processing the map
##########
File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/mutable/IndexContainer.java
##########
@@ -0,0 +1,211 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.segment.local.indexsegment.mutable;
+
+import java.util.Set;
+import javax.annotation.Nullable;
+import org.apache.pinot.segment.local.realtime.impl.geospatial.MutableH3Index;
+import org.apache.pinot.segment.local.realtime.impl.invertedindex.RealtimeInvertedIndexReader;
+import org.apache.pinot.segment.local.realtime.impl.invertedindex.RealtimeLuceneTextIndexReader;
+import org.apache.pinot.segment.local.realtime.impl.json.MutableJsonIndex;
+import org.apache.pinot.segment.local.realtime.impl.nullvalue.MutableNullValueVector;
+import org.apache.pinot.segment.local.segment.index.column.PhysicalColumnIndexContainer;
+import org.apache.pinot.segment.local.segment.index.datasource.MutableDataSource;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.segment.spi.index.column.ColumnIndexContainer;
+import org.apache.pinot.segment.spi.index.reader.BloomFilterReader;
+import org.apache.pinot.segment.spi.index.reader.Dictionary;
+import org.apache.pinot.segment.spi.index.reader.ForwardIndexReader;
+import org.apache.pinot.segment.spi.index.reader.H3IndexReader;
+import org.apache.pinot.segment.spi.index.reader.InvertedIndexReader;
+import org.apache.pinot.segment.spi.index.reader.JsonIndexReader;
+import org.apache.pinot.segment.spi.index.reader.MutableDictionary;
+import org.apache.pinot.segment.spi.index.reader.MutableForwardIndex;
+import org.apache.pinot.segment.spi.index.reader.NullValueVectorReader;
+import org.apache.pinot.segment.spi.index.reader.TextIndexReader;
+import org.apache.pinot.segment.spi.partition.PartitionFunction;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+public class IndexContainer implements ColumnIndexContainer {
Review comment:
I would recommend keeping this as a sub-class within the `MutableSegmentImpl`. Separating it out doesn't provide extra value and is not as readable because the member fields are directly accessed by `MutableSegmentImpl`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #6774: Support REALTIME tables in reload status (segment metadata API) #6767
Posted by GitBox <gi...@apache.org>.
mcvsubbu commented on a change in pull request #6774:
URL: https://github.com/apache/incubator-pinot/pull/6774#discussion_r612020196
##########
File path: pinot-server/src/main/java/org/apache/pinot/server/api/resources/SegmentMetadataFetcher.java
##########
@@ -77,8 +80,13 @@ public static String getSegmentMetadata(SegmentDataManager segmentDataManager, L
if (immutableSegment instanceof ImmutableSegmentImpl) {
ImmutableSegmentImpl immutableSegmentImpl = (ImmutableSegmentImpl) immutableSegment;
Map<String, ColumnIndexContainer> columnIndexContainerMap = immutableSegmentImpl.getIndexContainerMap();
- columnIndexMap = getImmutableSegmentColumnIndexes(columnIndexContainerMap);
+ columnIndexMap = getSegmentColumnIndexes(columnIndexContainerMap);
Review comment:
I am fine with that too. @mayankshriv is changing the way all this is organized, so we might as well get to a final state that he has in mind.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org