You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jungtaek Lim (JIRA)" <ji...@apache.org> on 2019/02/01 01:01:10 UTC
[jira] [Comment Edited] (SPARK-26783) Kafka parameter documentation
doesn't match with the reality (upper/lowercase)
[ https://issues.apache.org/jira/browse/SPARK-26783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757842#comment-16757842 ]
Jungtaek Lim edited comment on SPARK-26783 at 2/1/19 1:01 AM:
--------------------------------------------------------------
I'm not sure about what you're referring to here from SPARK-23685:
{quote}
Originally this pr was created as "failOnDataLoss" doesn't have any impact when set in structured streaming. But found out that ,the variable that needs to be used is "failondataloss" (all in lower case).
{quote}
I just played with test "failOnDataLoss=false should not return duplicated records: v1" in KafkaDontFailOnDataLossSuite, and looks like it works as intended (as case-insensitive manner).
"failOnDataLoss" -> "false" // passed
"failondataloss" -> "false" // passed
"FAILONDATALOSS" -> "false" // passed
<default value> // failed
"failOnDataLoss" -> "true" // failed
was (Author: kabhwan):
I'm not sure about what [~sindiri] left a comment on the PR:
{quote}
Originally this pr was created as "failOnDataLoss" doesn't have any impact when set in structured streaming. But found out that ,the variable that needs to be used is "failondataloss" (all in lower case).
{quote}
I just played with test "failOnDataLoss=false should not return duplicated records: v1" in KafkaDontFailOnDataLossSuite, and looks like it works as intended (as case-insensitive manner).
"failOnDataLoss" -> "false" // passed
"failondataloss" -> "false" // passed
"FAILONDATALOSS" -> "false" // passed
<default value> // failed
"failOnDataLoss" -> "true" // failed
> Kafka parameter documentation doesn't match with the reality (upper/lowercase)
> ------------------------------------------------------------------------------
>
> Key: SPARK-26783
> URL: https://issues.apache.org/jira/browse/SPARK-26783
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 3.0.0
> Reporter: Gabor Somogyi
> Priority: Minor
>
> A good example for this is "failOnDataLoss" which is reported in SPARK-23685. I've just checked and there are several other parameters which suffer from the same issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org