[GitHub] koeninger commented on issue #23747: [SPARK-26848][SQL] Introduce new option to Kafka source: offset by timestamp (starting/ending)

[GitBox <gi...@apache.org>]

koeninger commented on issue #23747: [SPARK-26848][SQL] Introduce new option to Kafka source: offset by timestamp (starting/ending)
URL: https://github.com/apache/spark/pull/23747#issuecomment-463247595
 
 
   Why not just link to e.g.
   https://kafka.apache.org/21/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#offsetsForTimes-java.util.Map-
   
   The main kafka documentation at
   https://kafka.apache.org/documentation/ has plenty of discussion of
   timestamp imho
   
   On Wed, Feb 13, 2019 at 4:01 AM Gabor Somogyi <no...@github.com> wrote:
   >
   > @gaborgsomogyi commented on this pull request.
   >
   > ________________________________
   >
   > In docs/structured-streaming-kafka-integration.md:
   >
   > > @@ -310,6 +310,23 @@ The following configurations are optional:
   >
   >  <table class="table">
   >  <tr><th>Option</th><th>value</th><th>default</th><th>query type</th><th>meaning</th></tr>
   > +<tr>
   >
   > I agree too detailed explanation is bad/redundant and that's the reason why I've suggested compacted limitations.
   >
   > Stating The start point of timestamp when a query is started is just not true because producer can overwrite this field under some circumstances (not Spark is the only producer). I would refer to Kafka configuration and mention that this field is depending on how Kafka is configured / data produced without mentioning all the details. I think based on the field users can look for KIP in need.
   >
   > Yeah, offsetsForTimes is not over-documented but that's a Kafka question.
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub, or mute the thread.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org