You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jack Foy <jf...@whitepages.com> on 2014/10/29 20:40:46 UTC
Compile failure going from kafka 0.8.1.1 to 0.8.2
My Scala project built against kafka 0.8.1.1 commits consumer offsets as follows:
connector.commitOffsets
This compiles without warnings. When I bumped the library dependency to 0.8.2-beta, the compiler started emitting this error:
[error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing arguments for method commitOffsets in trait ConsumerConnector;
[error] follow this method with `_' if you want to treat it as a partially applied function
[error] connector.commitOffsets
[error] ^
[error] one error found
[error] (compile:compile) Compilation failed
The following change resolved the error:
- connector.commitOffsets
+ connector.commitOffsets()
Should we expect compilation-breaking changes moving from 0.8.1.1 to 0.8.2-beta?
--
Jack Foy <jf...@whitepages.com>
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Posted by Neha Narkhede <ne...@gmail.com>.
Scala 2.10.4, kafka 0.8.1.1, my old code - compiled
Scala 2.10.4, kafka 0.8.2, my old code - failed
If that's the case, then we probably need to fix the issue. Can you please
file a JIRA?
On Wed, Oct 29, 2014 at 3:15 PM, Jack Foy <jf...@whitepages.com> wrote:
> On Oct 29, 2014, at 3:03 PM, Jack Foy <jf...@whitepages.com> wrote:
> > On Oct 29, 2014, at 1:46 PM, Neha Narkhede <ne...@gmail.com>
> wrote:
> >> Which version of Scala did you use for Kafka 0.8.2-beta?
> >
> > The problem reproduces under both Scala 2.10.4 and Scala 2.11.2.
>
> Sorry, to be clearer:
>
> I am using SBT with the following dependency line, to pull in the specific
> package for the version of Scala I’m running.
>
> libraryDependencies ++= Seq(
> "org.apache.kafka" %% "kafka" % "0.8.2-beta"
> exclude("javax.jms", "jms")
> exclude("com.sun.jdmk", "jmxtools")
> exclude("com.sun.jmx", "jmxri")
> exclude("log4j", "log4j")
> )
>
> I tried the following combinations:
>
> Scala 2.10.4, kafka 0.8.1.1, my old code - compiled
> Scala 2.10.4, kafka 0.8.2, my old code - failed
> Scala 2.11.2, kafka 0.8.2, my old code - failed
> Scala 2.10.4, kafka 0.8.2, my new code - compiled
> Scala 2.11.2, kafka 0.8.2, my new code - compiled
>
> --
> Jack Foy <jf...@whitepages.com>
>
>
>
>
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Posted by Jack Foy <jf...@whitepages.com>.
On Oct 29, 2014, at 3:03 PM, Jack Foy <jf...@whitepages.com> wrote:
> On Oct 29, 2014, at 1:46 PM, Neha Narkhede <ne...@gmail.com> wrote:
>> Which version of Scala did you use for Kafka 0.8.2-beta?
>
> The problem reproduces under both Scala 2.10.4 and Scala 2.11.2.
Sorry, to be clearer:
I am using SBT with the following dependency line, to pull in the specific package for the version of Scala I’m running.
libraryDependencies ++= Seq(
"org.apache.kafka" %% "kafka" % "0.8.2-beta"
exclude("javax.jms", "jms")
exclude("com.sun.jdmk", "jmxtools")
exclude("com.sun.jmx", "jmxri")
exclude("log4j", "log4j")
)
I tried the following combinations:
Scala 2.10.4, kafka 0.8.1.1, my old code - compiled
Scala 2.10.4, kafka 0.8.2, my old code - failed
Scala 2.11.2, kafka 0.8.2, my old code - failed
Scala 2.10.4, kafka 0.8.2, my new code - compiled
Scala 2.11.2, kafka 0.8.2, my new code - compiled
--
Jack Foy <jf...@whitepages.com>
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Posted by Jack Foy <jf...@whitepages.com>.
On Oct 29, 2014, at 1:46 PM, Neha Narkhede <ne...@gmail.com> wrote:
> Which version of Scala did you use for Kafka 0.8.2-beta?
The problem reproduces under both Scala 2.10.4 and Scala 2.11.2.
--
Jack Foy <jf...@whitepages.com>
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Posted by Neha Narkhede <ne...@gmail.com>.
Which version of Scala did you use for Kafka 0.8.2-beta?
On Wed, Oct 29, 2014 at 12:40 PM, Jack Foy <jf...@whitepages.com> wrote:
> My Scala project built against kafka 0.8.1.1 commits consumer offsets as
> follows:
>
> connector.commitOffsets
>
> This compiles without warnings. When I bumped the library dependency to
> 0.8.2-beta, the compiler started emitting this error:
>
> [error]
> src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing
> arguments for method commitOffsets in trait ConsumerConnector;
> [error] follow this method with `_' if you want to treat it as a
> partially applied function
> [error] connector.commitOffsets
> [error] ^
> [error] one error found
> [error] (compile:compile) Compilation failed
>
> The following change resolved the error:
>
> - connector.commitOffsets
> + connector.commitOffsets()
>
> Should we expect compilation-breaking changes moving from 0.8.1.1 to
> 0.8.2-beta?
>
> --
> Jack Foy <jf...@whitepages.com>
>
>
>
>
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Posted by Jun Rao <ju...@gmail.com>.
Yes, this is actually indeed an incompatible change.
In 0.8.1.x, ConsumerConnector has the following api:
def commitOffsets
This is changed to the following in 0.8.2 and breaks compatibility
def commitOffsets(retryOnFailure: Boolean = true)
Filed KAFKA-1743 as an 0.8.2 blocker.
Thanks,
Jun
On Fri, Oct 31, 2014 at 3:07 PM, Jack Foy <jf...@whitepages.com> wrote:
> On Thu, Oct 30, 2014 at 9:20 AM, Jay Kreps <jay.kreps@gmail.com<mailto:
> jay.kreps@gmail.com>> wrote:
> I think we should treat this like a bug for 0.8.2 final, we should be able
> to add two commitOffsets methods with and without the param which should
> fix the problem, right?
>
> On Oct 30, 2014, at 9:51 AM, Jun Rao <junrao@gmail.com<mailto:
> junrao@gmail.com>> wrote:
> Yes, we can change this to two methods in 0.8.2 final.
>
> Thanks. After some experimentation, I think there isn’t actually a form of
> this code that can compile under both kafka 0.8.1.1 and 0.8.2-beta.
>
> This form fails to compile against kafka-0.8.1.1: connector.commitOffsets()
>
> [error]
> src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: Unit does
> not take parameters
> [error] connector.commitOffsets()
> [error] ^
>
> This form fails to compile against kafka-0.8.2-beta:
> connector.commitOffsets
>
> [error]
> src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing
> arguments for method commitOffsets in trait ConsumerConnector;
> [error] follow this method with `_' if you want to treat it as a
> partially applied function
> [error] connector.commitOffsets
> [error] ^
>
> --
> Jack Foy <jf...@whitepages.com>>
>
>
>
>
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Posted by Jack Foy <jf...@whitepages.com>.
On Thu, Oct 30, 2014 at 9:20 AM, Jay Kreps <ja...@gmail.com>> wrote:
I think we should treat this like a bug for 0.8.2 final, we should be able
to add two commitOffsets methods with and without the param which should
fix the problem, right?
On Oct 30, 2014, at 9:51 AM, Jun Rao <ju...@gmail.com>> wrote:
Yes, we can change this to two methods in 0.8.2 final.
Thanks. After some experimentation, I think there isn’t actually a form of this code that can compile under both kafka 0.8.1.1 and 0.8.2-beta.
This form fails to compile against kafka-0.8.1.1: connector.commitOffsets()
[error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: Unit does not take parameters
[error] connector.commitOffsets()
[error] ^
This form fails to compile against kafka-0.8.2-beta: connector.commitOffsets
[error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing arguments for method commitOffsets in trait ConsumerConnector;
[error] follow this method with `_' if you want to treat it as a partially applied function
[error] connector.commitOffsets
[error] ^
--
Jack Foy <jf...@whitepages.com>>
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Posted by Jun Rao <ju...@gmail.com>.
Yes, we can change this to two methods in 0.8.2 final.
Thanks,
Jun
On Thu, Oct 30, 2014 at 9:20 AM, Jay Kreps <ja...@gmail.com> wrote:
> But Jun,
>
> That change breaks peoples code who weren't calling in that way. I don't
> think we should be making breaking changes in a point release like this.
>
> I think we should treat this like a bug for 0.8.2 final, we should be able
> to add two commitOffsets methods with and without the param which should
> fix the problem, right?
>
> -Jay
>
> On Thu, Oct 30, 2014 at 8:59 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > Jack,
> >
> > The commit offset api is changed slightly from
> >
> > def commitOffsets() in 0.8.1.x
> >
> > to
> >
> > def commitOffsets(retryOnFailure: Boolean = true) in 0.8.2.x.
> >
> > If you have been calling the method with parentheses like
> commitOffsets(),
> > then the code will compile in both 0.8.1.x and 0.8.2.x. In general, the
> > scala rule (http://docs.scala-lang.org/style/method-invocation.html) for
> > omitting parentheses when calling arity-0 methods is that the methods in
> > question have no side effects. In this case, commitOffsets() clearly has
> > side effect and should have been called with parentheses.
> >
> > Thanks,
> >
> > Jun
> >
> >
> >
> >
> > On Wed, Oct 29, 2014 at 12:40 PM, Jack Foy <jf...@whitepages.com> wrote:
> >
> > > My Scala project built against kafka 0.8.1.1 commits consumer offsets
> as
> > > follows:
> > >
> > > connector.commitOffsets
> > >
> > > This compiles without warnings. When I bumped the library dependency to
> > > 0.8.2-beta, the compiler started emitting this error:
> > >
> > > [error]
> > > src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21:
> missing
> > > arguments for method commitOffsets in trait ConsumerConnector;
> > > [error] follow this method with `_' if you want to treat it as a
> > > partially applied function
> > > [error] connector.commitOffsets
> > > [error] ^
> > > [error] one error found
> > > [error] (compile:compile) Compilation failed
> > >
> > > The following change resolved the error:
> > >
> > > - connector.commitOffsets
> > > + connector.commitOffsets()
> > >
> > > Should we expect compilation-breaking changes moving from 0.8.1.1 to
> > > 0.8.2-beta?
> > >
> > > --
> > > Jack Foy <jf...@whitepages.com>
> > >
> > >
> > >
> > >
> >
>
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Posted by Colin <co...@clark.ws>.
+1 on two methods
--
Colin Clark
+1-320-221-9531
> On Oct 30, 2014, at 11:24 AM, Jonathan Weeks <jo...@gmail.com> wrote:
>
> +1 on the two methods suggestion
>
> -JW
>
>> On Oct 30, 2014, at 9:20 AM, Jay Kreps <ja...@gmail.com> wrote:
>>
>> But Jun,
>>
>> That change breaks peoples code who weren't calling in that way. I don't
>> think we should be making breaking changes in a point release like this.
>>
>> I think we should treat this like a bug for 0.8.2 final, we should be able
>> to add two commitOffsets methods with and without the param which should
>> fix the problem, right?
>>
>> -Jay
>>
>>> On Thu, Oct 30, 2014 at 8:59 AM, Jun Rao <ju...@gmail.com> wrote:
>>>
>>> Jack,
>>>
>>> The commit offset api is changed slightly from
>>>
>>> def commitOffsets() in 0.8.1.x
>>>
>>> to
>>>
>>> def commitOffsets(retryOnFailure: Boolean = true) in 0.8.2.x.
>>>
>>> If you have been calling the method with parentheses like commitOffsets(),
>>> then the code will compile in both 0.8.1.x and 0.8.2.x. In general, the
>>> scala rule (http://docs.scala-lang.org/style/method-invocation.html) for
>>> omitting parentheses when calling arity-0 methods is that the methods in
>>> question have no side effects. In this case, commitOffsets() clearly has
>>> side effect and should have been called with parentheses.
>>>
>>> Thanks,
>>>
>>> Jun
>>>
>>>
>>>
>>>
>>>> On Wed, Oct 29, 2014 at 12:40 PM, Jack Foy <jf...@whitepages.com> wrote:
>>>>
>>>> My Scala project built against kafka 0.8.1.1 commits consumer offsets as
>>>> follows:
>>>>
>>>> connector.commitOffsets
>>>>
>>>> This compiles without warnings. When I bumped the library dependency to
>>>> 0.8.2-beta, the compiler started emitting this error:
>>>>
>>>> [error]
>>>> src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing
>>>> arguments for method commitOffsets in trait ConsumerConnector;
>>>> [error] follow this method with `_' if you want to treat it as a
>>>> partially applied function
>>>> [error] connector.commitOffsets
>>>> [error] ^
>>>> [error] one error found
>>>> [error] (compile:compile) Compilation failed
>>>>
>>>> The following change resolved the error:
>>>>
>>>> - connector.commitOffsets
>>>> + connector.commitOffsets()
>>>>
>>>> Should we expect compilation-breaking changes moving from 0.8.1.1 to
>>>> 0.8.2-beta?
>>>>
>>>> --
>>>> Jack Foy <jf...@whitepages.com>
>>>>
>>>>
>>>>
>>>>
>>>
>
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Posted by Jonathan Weeks <jo...@gmail.com>.
+1 on the two methods suggestion
-JW
On Oct 30, 2014, at 9:20 AM, Jay Kreps <ja...@gmail.com> wrote:
> But Jun,
>
> That change breaks peoples code who weren't calling in that way. I don't
> think we should be making breaking changes in a point release like this.
>
> I think we should treat this like a bug for 0.8.2 final, we should be able
> to add two commitOffsets methods with and without the param which should
> fix the problem, right?
>
> -Jay
>
> On Thu, Oct 30, 2014 at 8:59 AM, Jun Rao <ju...@gmail.com> wrote:
>
>> Jack,
>>
>> The commit offset api is changed slightly from
>>
>> def commitOffsets() in 0.8.1.x
>>
>> to
>>
>> def commitOffsets(retryOnFailure: Boolean = true) in 0.8.2.x.
>>
>> If you have been calling the method with parentheses like commitOffsets(),
>> then the code will compile in both 0.8.1.x and 0.8.2.x. In general, the
>> scala rule (http://docs.scala-lang.org/style/method-invocation.html) for
>> omitting parentheses when calling arity-0 methods is that the methods in
>> question have no side effects. In this case, commitOffsets() clearly has
>> side effect and should have been called with parentheses.
>>
>> Thanks,
>>
>> Jun
>>
>>
>>
>>
>> On Wed, Oct 29, 2014 at 12:40 PM, Jack Foy <jf...@whitepages.com> wrote:
>>
>>> My Scala project built against kafka 0.8.1.1 commits consumer offsets as
>>> follows:
>>>
>>> connector.commitOffsets
>>>
>>> This compiles without warnings. When I bumped the library dependency to
>>> 0.8.2-beta, the compiler started emitting this error:
>>>
>>> [error]
>>> src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing
>>> arguments for method commitOffsets in trait ConsumerConnector;
>>> [error] follow this method with `_' if you want to treat it as a
>>> partially applied function
>>> [error] connector.commitOffsets
>>> [error] ^
>>> [error] one error found
>>> [error] (compile:compile) Compilation failed
>>>
>>> The following change resolved the error:
>>>
>>> - connector.commitOffsets
>>> + connector.commitOffsets()
>>>
>>> Should we expect compilation-breaking changes moving from 0.8.1.1 to
>>> 0.8.2-beta?
>>>
>>> --
>>> Jack Foy <jf...@whitepages.com>
>>>
>>>
>>>
>>>
>>
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Posted by Jay Kreps <ja...@gmail.com>.
But Jun,
That change breaks peoples code who weren't calling in that way. I don't
think we should be making breaking changes in a point release like this.
I think we should treat this like a bug for 0.8.2 final, we should be able
to add two commitOffsets methods with and without the param which should
fix the problem, right?
-Jay
On Thu, Oct 30, 2014 at 8:59 AM, Jun Rao <ju...@gmail.com> wrote:
> Jack,
>
> The commit offset api is changed slightly from
>
> def commitOffsets() in 0.8.1.x
>
> to
>
> def commitOffsets(retryOnFailure: Boolean = true) in 0.8.2.x.
>
> If you have been calling the method with parentheses like commitOffsets(),
> then the code will compile in both 0.8.1.x and 0.8.2.x. In general, the
> scala rule (http://docs.scala-lang.org/style/method-invocation.html) for
> omitting parentheses when calling arity-0 methods is that the methods in
> question have no side effects. In this case, commitOffsets() clearly has
> side effect and should have been called with parentheses.
>
> Thanks,
>
> Jun
>
>
>
>
> On Wed, Oct 29, 2014 at 12:40 PM, Jack Foy <jf...@whitepages.com> wrote:
>
> > My Scala project built against kafka 0.8.1.1 commits consumer offsets as
> > follows:
> >
> > connector.commitOffsets
> >
> > This compiles without warnings. When I bumped the library dependency to
> > 0.8.2-beta, the compiler started emitting this error:
> >
> > [error]
> > src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing
> > arguments for method commitOffsets in trait ConsumerConnector;
> > [error] follow this method with `_' if you want to treat it as a
> > partially applied function
> > [error] connector.commitOffsets
> > [error] ^
> > [error] one error found
> > [error] (compile:compile) Compilation failed
> >
> > The following change resolved the error:
> >
> > - connector.commitOffsets
> > + connector.commitOffsets()
> >
> > Should we expect compilation-breaking changes moving from 0.8.1.1 to
> > 0.8.2-beta?
> >
> > --
> > Jack Foy <jf...@whitepages.com>
> >
> >
> >
> >
>
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Posted by Jun Rao <ju...@gmail.com>.
Jack,
The commit offset api is changed slightly from
def commitOffsets() in 0.8.1.x
to
def commitOffsets(retryOnFailure: Boolean = true) in 0.8.2.x.
If you have been calling the method with parentheses like commitOffsets(),
then the code will compile in both 0.8.1.x and 0.8.2.x. In general, the
scala rule (http://docs.scala-lang.org/style/method-invocation.html) for
omitting parentheses when calling arity-0 methods is that the methods in
question have no side effects. In this case, commitOffsets() clearly has
side effect and should have been called with parentheses.
Thanks,
Jun
On Wed, Oct 29, 2014 at 12:40 PM, Jack Foy <jf...@whitepages.com> wrote:
> My Scala project built against kafka 0.8.1.1 commits consumer offsets as
> follows:
>
> connector.commitOffsets
>
> This compiles without warnings. When I bumped the library dependency to
> 0.8.2-beta, the compiler started emitting this error:
>
> [error]
> src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing
> arguments for method commitOffsets in trait ConsumerConnector;
> [error] follow this method with `_' if you want to treat it as a
> partially applied function
> [error] connector.commitOffsets
> [error] ^
> [error] one error found
> [error] (compile:compile) Compilation failed
>
> The following change resolved the error:
>
> - connector.commitOffsets
> + connector.commitOffsets()
>
> Should we expect compilation-breaking changes moving from 0.8.1.1 to
> 0.8.2-beta?
>
> --
> Jack Foy <jf...@whitepages.com>
>
>
>
>