[GitHub] [incubator-pinot] sunithabeeram edited a comment on issue #3979: Track "freshness" timestamp across consuming segments

[GitBox <gi...@apache.org>]

sunithabeeram edited a comment on issue #3979: Track "freshness" timestamp across consuming segments
URL: https://github.com/apache/incubator-pinot/pull/3979#issuecomment-481530284
 
 
   @Jackie-Jiang - thanks for raising the concerns. I have captured some responses here. We can discuss further in person if needed.
   > We need to have a design doc for all the interfaces and do a team discussion on this
   
   We did have an internal discussion exactly on this today and its unfortunate that while you were present during that discussion the concerns weren't brought up there. We have an internal doc discussing the details - I will create an external version of the doc soon.
   
   > StreamMessageMetadata is not general enough, and is a concrete class which is introduced to the general interface of MutableSegment
   
   I am not sure its a good idea to make everything as an interface simply because its getting exposed through an interface. I can see that the name StreamMessageMetadata does tie it to streams a bit more than we want associated with MutableSegment - I can change this to RowMetadata.
   
   > 2. The ingestion time and index time do not make sense for offline (immutable) segment but got added to the SegmentMetadata interface.
   
   I definitely went down the path of splitting SegmentMetadata interface into mutable and immutable. Currently, SegmentMetadata exposes methods such as refreshTime(), pushTime() which don't quite apply to mutable data. I finally backed out of that path because of the way MutableSegmentImpl currently manages the segment-metadata (using inline variables instead of explicit setters/getters) - and given that the metadata gets update for every row ingested, I decided to keep with the current approach than introduce additional overhead by making explicit method calls everytime. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org