You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Sasaki Toru <sa...@oss.nttdata.com> on 2018/04/04 11:37:32 UTC
[DISCUSS] KIP-278: Add version option to Kafka's commands
Hello everyone,
I would like to start a discussion for KIP 278. Cloud you please give comments and advice ?
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-278+-+Add+version+option+to+Kafka%27s+commands>
JIRA ticket and Pull Request are bellow:
<https://issues.apache.org/jira/browse/KAFKA-2061>
<https://github.com/apache/kafka/pull/639>
Many thanks,
Sasaki
--
Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Sasaki Toru <sa...@oss.nttdata.com>.
I started voting thread for this KIP. Thanks, Jason and Colin.
> From: Colin McCabe <cm...@apache.org>
> Date: 2018-05-10 2:53 GMT+09:00
> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
> To: dev@kafka.apache.org
>
>
> +1. Thanks, Sasaki.
>
> Colin
>
> On Wed, May 9, 2018, at 09:15, Jason Gustafson wrote:
>> Hi Sasaki,
>>
>> Thanks for the update. The KIP looks good to me. I'd suggest moving to a
>> vote.
>>
>> Thanks,
>> Jason
>>
>> On Mon, May 7, 2018 at 7:08 AM, Sasaki Toru <sa...@oss.nttdata.com>
>> wrote:
>>
>>> Hi Manikumar, Colin,
>>>
>>> Thank you for your comment.
>>>
>>> As Colin said, I proposed adding an option to show version information
> of
>>> "local" tool,
>>> because many software have this option and I think Kafka also needs this
>>> one.
>>>
>>> As you said, the function to show remote Kafka version is useful,
>>> but I think it is better to create new KIP because this function has
> some
>>> points which should be considered.
>>>
>>> If you have any better ideas, could you please tell us?
>>>
>>>
>>> Many thanks,
>>> Sasaki
>>>
>>> From: Manikumar <ma...@gmail.com>
>>>> Date: 2018-05-03 4:11 GMT+09:00
>>>>
>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
>>>> To: dev <de...@kafka.apache.org>
>>>>
>>>>
>>>> Hi Colin,
>>>>
>>>> Thanks for explanation. It's definitely useful to have --version flag.
>>>>
>>>> kafka-broker-api-versions.sh gives the API versions, not Kafka release
>>>> version.
>>>> Is not easy to figure out release version from API versions. Currently
>>>> release version is available via metric/JMX.
>>>> If required, we can implement this in future.
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> On Wed, May 2, 2018 at 10:58 PM, Colin McCabe <cm...@apache.org>
> wrote:
>>>> Hi Manikumar,
>>>>> We already have a tool for getting the Kafka broker API versions,
>>>>> "./bin/kafka-broker-api-versions.sh". It was added as part of KIP-97.
>>>>>
>>>>> What Saski is proposing here is having a way of getting the version of
>>>>> locally installed Kafka software, which may be different from the
> server
>>>>> version. Many pieces of software offer a --version flag, and it's
> always
>>>>> understood to refer to the local version of the software, not a
> version
>>>>> running somewhere else. The user has no way to get this information
> now,
>>>>> other than perhaps trying to look at he names of jar files.
>>>>>
>>>>> cheers,
>>>>> Colin
>>>>>
>>>>> On Tue, May 1, 2018, at 08:20, Manikumar wrote:
>>>>>
>>>>>> I assume the intent of the KIP to find out the Kafka broker
> version. In
>>>>>> this case, maybe we should expose
>>>>>> version using a Kafka request. This will help the remote
> scripts/tools
>>>>> to
>>>>> query the Kafka version.
>>>>>> scripts (kafka-topics.sh, kafka-configs.sh, etc..) may run from
> remote
>>>>>> machines and may use
>>>>>> older Kafka versions. In this case, current proposal prints on the
> older
>>>>>> version.
>>>>>>
>>>>>> On Tue, May 1, 2018 at 7:47 PM, Colin McCabe <cm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>> Thanks, Sasaki.
>>>>>>> Colin
>>>>>>>
>>>>>>> On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote:
>>>>>>>
>>>>>>>> Hi Colin, Jason,
>>>>>>>>
>>>>>>>> Thank you for your beneficial comment.
>>>>>>>> I have updated my Pull Request to show git commit hash in version
>>>>>>>> information.> In my current Pull Request, we cat get the result
> such
>>>>>>> below:
>>>>>>>
>>>>>>>> $ bin/kafka-topics.sh --version
>>>>>>>> (snip)
>>>>>>>> 2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e)
>>>>>>>>
>>>>>>>>
>>>>>>>> I have also updated to accept double-dash for this option (--
>>>>>>>> version) only.>
>>>>>>>>
>>>>>>>> Many thanks,
>>>>>>>> Sasaki
>>>>>>>>
>>>>>>>> From: Jason Gustafson <ja...@confluent.io>
>>>>>>>>> Date: 2018-04-25 9:42 GMT+09:00
>>>>>>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
>>>>>>>>> commands> > To: dev <de...@kafka.apache.org>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> +1 on adding the git commit id to the output. We often encounter
>>>>>>>>> environments which are testing off of trunk or have modifications
>>>>>>>>>
>>>>>>>> on
>>>>>> top of> > an existing release.
>>>>>>>>> -Jason
>>>>>>>>>
>>>>>>>>> On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cmccabe@apache.org
>>>>>>>>> wrote:> >
>>>>>>>>>
>>>>>>>>>> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Jason, Colin,
>>>>>>>>>>>
>>>>>>>>>>> Thank you for your comment, and I'm sorry for late reply.
>>>>>>>>>>>
>>>>>>>>>>> > we refactored all of the tools so that we could use a
> common
>>>>>>>>>>> > set of> >>> base options,
>>>>>>>>>>> > it might be a little annoying to have to continue
> supporting
>>>>>>>>>>> > both> >>> variations.
>>>>>>>>>>>
>>>>>>>>>>> I feel KIP-14 is good idea (I'm sorry, I didn't know about
>>>>>>>>>>> it), and> >>> I think it's no longer necessary both single-dash
>>>>>>>>>>>
>>>>>>>>>> and
>>>>>> double-
>>>>>>>> dash are> >>> supported when this KIP will be accepted, as you
>>>>>>>>>> said.
>>>>>> I revise my Pull Request to support single-dash option only.
>>>>>>>>>>> Some my code in kafka-run-class.sh will become unnecessary when
>>>>>>>>>>> KIP-14> >>> is accepted.
>>>>>>>>>>> But I will keep this as temporary, because I seem that it's
>>>>>>>>>>>
>>>>>>>>>> useful>
>>>>>> before KIP-14 accepted.
>>>>>>>>>>> > Can you give a little more detail about what would be
>>>>>>>>>>> > displayed when> >>> the version command was used?
>>>>>>>>>>>
>>>>>>>>>>> As Ismael said, the version string is got from
>>>>>>>>>>> AppInfoParser#getVersion.> >>>
>>>>>>>>>>> In my Pull Request, we can get the result such as below::
>>>>>>>>>>>
>>>>>>>>>>> $ bin/kafka-topics.sh --version
>>>>>>>>>>> (snip)
>>>>>>>>>>> Kafka 1.2.0-SNAPSHOT
>>>>>>>>>>>
>>>>>>>>>> Hi Sasaki,
>>>>>>>>>>
>>>>>>>>>> Thanks for the info. Can you add this to the KIP?
>>>>>>>>>>
>>>>>>>>>> Also, I really think we should include the git hash.
>>>>>>>>>>
>>>>>>>>>> best,
>>>>>>>>>> Colin
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Many thanks,
>>>>>>>>>>> Sasaki
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> From: Ismael Juma <is...@juma.me.uk>
>>>>>>>>>>>> Date: 2018-04-24 3:44 GMT+09:00
>>>>>>>>>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
>>>>>>>>>>>> commands> >>>> To: dev <de...@kafka.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> FYI, the injection via the build process that is mentioned here
>>>>>>>>>>>>
>>>>>>>>>>> already
>>>>>>>>>> happens. See AppInfoParser.
>>>>>>>>>>>> Ismael
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe
>>>>>>>>>>>> <cm...@apache.org>> >> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Sasaki,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the KIP. I think a version flag is a good idea.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Can you give a little more detail about what would be
>>>>>>>>>>>>>
>>>>>>>>>>>> displayed
>>>>> when> >> the
>>>>>>>>>>>>> version command was used?
>>>>>>>>>>>>>
>>>>>>>>>>>>> We clearly want the version number, but we probably also want
>>>>>>>>>>>>>
>>>>>>>>>>>> to
>>>>>> know> >> if
>>>>>>>>>>>>> this is an official release, or a random SNAPSHOT from a
>>>>>>>>>>>>>
>>>>>>>>>>>> branch.
>>>>>> If> >> this
>>>>>>>>>>>>> is a release candidate, we probably want the RC number as
>>>>>>>>>>>>>
>>>>>>>>>>>> well,
>>>>> like> >>>>> "1.1-rc3" We also want a git hash. This can be
>>>>>>>>>>>> injected by the> > build
>>>>>>>> process. In the case of an official release, where the source
>>>>>>>>>>>>> code> >> is not
>>>>>>>>>>>>> under git, we can pull it from a file.
>>>>>>>>>>>>>
>>>>>>>>>>>>> For example, hadoop's version output looks like this:
>>>>>>>>>>>>>
>>>>>>>>>>>>> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop
>>>>>>>>>>>>> > version> >>>>> > Hadoop 2.8.3
>>>>>>>>>>>>> > Subversion
>>>>>>>>>>>>> > https://git-wip-us.apache.org/repos/asf/hadoop.git -r>
>>>>>>>>>>>>>
>>>>>>>>>>>> b3fe56402d908019d99af1f1f4fc65cb1d1436a2
>>>>>>>> > Compiled by jdu on 2017-12-05T03:43Z
>>>>>>>>>>>>> > Compiled with protoc 2.5.0
>>>>>>>>>>>>> > From source with checksum 9ff4856d824e983fa510d3f843e3f1
>>>>>>>>>>>>>
>>>>>>>>>>>> 9d>
>>>>>> > This command was run using /home/cmccabe/Downloads/
>>>>>>>>>>>>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
>>>>>>>>>>>>>
>>>>>>>>>>>>> (The "subversion" line here is a little weird -- it now refers
>>>>>>>>>>>>> to> >> git, not
>>>>>>>>>>>>> svn)
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hey Sasaki,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yeah, I don't feel too strongly about only supporting --
>>>>>>>>>>>>>> version. I> >> agree
>>>>>>>>>>>>>>
>>>>>>>>>>>>> it
>>>>>>>>>>>>>
>>>>>>>>>>>>>> may help discoverability given the current approach. On the
>>>>>>>>>>>>>> other> >> hand,
>>>>>>>>>>>>>>
>>>>>>>>>>>>> if
>>>>>>>>>>>>>
>>>>>>>>>>>>>> we refactored all of the tools so that we could use a common
>>>>>>>>>>>>>> set of> >> base
>>>>>>>>>>>>>> options, it might be a little annoying to have to continue
>>>>>>>>>>>>>>
>>>>>>>>>>>>> supporting
>>>>>>>>>> both
>>>>>>>>>>>>>> variations. For example, tool standardization was proposed in
>>>>>>>>>>>>>> KIP-14> >> and
>>>>>>>>>>>>>> I'm still holding out hope that someone will have time to
>>>>>>>>>>>>>>
>>>>>>>>>>>>> pick
>>>>> this> >> work
>>>>>>>>>>>>>> back up. It's always easier to add an option than remove one,
>>>>>>>>>>>>>> so I'm> >>>>>> slightly inclined to have only --version for
>>>>>>>>>>>>>>
>>>>>>>>>>>>> now.
>>>>>> What do you
>>>>>>>> think?> >>>>> The double dash version is more consistent with
>>>>>>>>>>>>> how
>>>>>> our other
>>>>>>>> flags> >> work.
>>>>>>>>>>>>> In general, I feel that if --version is supported, --help
>>>>>>>>>>>>>
>>>>>>>>>>>> should
>>>>>> say> >> so.
>>>>>>>>>>>>> best,
>>>>>>>>>>>>> Colin
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Jason
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <
>>>>>>>>>>>>>>
>>>>>>>>>>>>> sasakitoa@oss.nttdata.com
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> Hi Jason
>>>>>>>>>>>>>>> Thank you for helpful comments. I updated wiki based on your
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> advice.
>>>>>>>>>> I thought this option was relatively common and making
>>>>>>>>>>>>>>> maintenance> >>>> easy
>>>>>>>>>>>>>>> was also important.
>>>>>>>>>>>>>>> However, as you said, it is not good that version option
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> won't
>>>>>> be> >>>>> shown up
>>>>>>>>>>>>>>> in help description.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I thought accepting both single-dash and double-dash will
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> help
>>>>>> to> >> find
>>>>>>>>>>>>>>> this option.
>>>>>>>>>>>>>>> In my approach this option won't be showed, but most of
>>>>>>>>>>>>>>> software> >> which
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> has
>>>>>>>>>>>>>> this option accepts either single-dash or double-dash.
>>>>>>>>>>>>>>> I guess it doesn't need to support both if we take another
>>>>>>>>>>>>>>> way.> >>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> @Ted Yeah, you're right. Sorry about the confusion.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Since we're here, I think this KIP is a nice improvement.
>>>>>>>>>>>>>>>> It's> >>>>> definitely
>>>>>>>>>>>>>>>> nice to have an easy way to check the version. That said,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> do
>>>>> we> >>>> really
>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>> to support both `-version` and `--version`? The latter is
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> consistent
>>>>>>>>>>> with
>>>>>>>>>>>>>> our current tools.
>>>>>>>>>>>>>>>> Also, I think the approach we've taken is basically to
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> build
>>>>> the> >>>>> --version
>>>>>>>>>>>>>>>> functionality into the bash script. This is nice because it
>>>>>>>>>>>>>>>> saves> > a
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lot of
>>>>>>>>>>>>>> work to update the commands individually and we don't need
>>>>>>>>>>>>>>> to
>>>>>> do> >>>>> anything
>>>>>>>>>>>>>>>> when we add new tools. The downside is that `--version`
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> won't
>>>>>> show> >> up
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>> option in any of the --help output. Not sure if that is too
>>>>>>>>>>>>>>>> big of> >> a
>>>>>>>>>>>>>>>> problem, but maybe worth mentioning this in the rejected
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> alternatives
>>>>>>>>>>> section.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -Jason
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yuzhihong@gmail.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>> Jason:
>>>>>>>>>>>>>>>>> Maybe your reply was intended for another KIP ?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> KIP-278 is about adding version option, not timeout.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> jason@confluent.io>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> Hi Sasaki,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks for the KIP. I think the timeout controls the
>>>>>>>>>>>>>>>>>> maximum> >>>> allowed
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>> that the consumer will block for the next record. Maybe
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> the>
>>>>>> meaning
>>>>>>>>>> would
>>>>>>>>>>>>>>>>> be clearer with the more concise name `--timeout`? That
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> also
>>>>>> fits> >>>>> with
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> old consumer which overrides the `consumer.timeout.ms`
>>>>>>>>>>>>>>>>>> property.> >>>>>>>>>>
>>>>>>>>>>>>>>>>>> By the way, it seems like the default value was
>>>>>>>>>>>>>>>>>> intentionally> > set
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> low
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> both the old and new consumers, but I'm not sure of the
>>>>>>>>>>>>>>>>> reason.> > We
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>> leave the default as it is if we want to be safe, but
>>>>>>>>>>>>>>>>>> increasing> >> it
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>> ok to me. Perhaps we could start a little lower, though,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> say
>>>>>> 10> >>>>> seconds?
>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> any case, we should make it clear to the user that the
>>>>>>>>>>>>>>>>>> timeout> >> was
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> reached.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It's surprising to see only the incomplete reported
>>>>>>>>>>>>>>>>> results>
>>>>>> following a
>>>>>>>>>>>>> timeout.
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Jason
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> sasakitoa@oss.nttdata.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> Hello everyone,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I would like to start a discussion for KIP 278. Cloud
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> you> >
>>>>>> please
>>>>>>>> give
>>>>>>>>>>>>>> comments and advice ?
>>>>>>>>>>>>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>>>>>>>>>>>>>>>>>> 278+-> >>>>>>>>>>> +Add+version+option+to+Kafka%2
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 7s+commands>
>>>>>>>> JIRA ticket and Pull Request are bellow:
>>>>>>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
>>>>>>>>>>>>>>>>>>> <https://github.com/apache/kafka/pull/639>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Many thanks,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Sasaki
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA
>>>>>>>>>>>>>>>>>>> CORPORATION> >>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>>>>
>>>>>>>>
>>> --
>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>
>>>
--
Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Colin McCabe <cm...@apache.org>.
+1. Thanks, Sasaki.
Colin
On Wed, May 9, 2018, at 09:15, Jason Gustafson wrote:
> Hi Sasaki,
>
> Thanks for the update. The KIP looks good to me. I'd suggest moving to a
> vote.
>
> Thanks,
> Jason
>
> On Mon, May 7, 2018 at 7:08 AM, Sasaki Toru <sa...@oss.nttdata.com>
> wrote:
>
> > Hi Manikumar, Colin,
> >
> > Thank you for your comment.
> >
> > As Colin said, I proposed adding an option to show version information of
> > "local" tool,
> > because many software have this option and I think Kafka also needs this
> > one.
> >
> > As you said, the function to show remote Kafka version is useful,
> > but I think it is better to create new KIP because this function has some
> > points which should be considered.
> >
> > If you have any better ideas, could you please tell us?
> >
> >
> > Many thanks,
> > Sasaki
> >
> > From: Manikumar <ma...@gmail.com>
> >> Date: 2018-05-03 4:11 GMT+09:00
> >>
> >> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
> >> To: dev <de...@kafka.apache.org>
> >>
> >>
> >> Hi Colin,
> >>
> >> Thanks for explanation. It's definitely useful to have --version flag.
> >>
> >> kafka-broker-api-versions.sh gives the API versions, not Kafka release
> >> version.
> >> Is not easy to figure out release version from API versions. Currently
> >> release version is available via metric/JMX.
> >> If required, we can implement this in future.
> >>
> >>
> >> Thanks,
> >>
> >> On Wed, May 2, 2018 at 10:58 PM, Colin McCabe <cm...@apache.org> wrote:
> >>
> >> Hi Manikumar,
> >>>
> >>> We already have a tool for getting the Kafka broker API versions,
> >>> "./bin/kafka-broker-api-versions.sh". It was added as part of KIP-97.
> >>>
> >>> What Saski is proposing here is having a way of getting the version of
> >>> locally installed Kafka software, which may be different from the server
> >>> version. Many pieces of software offer a --version flag, and it's always
> >>> understood to refer to the local version of the software, not a version
> >>> running somewhere else. The user has no way to get this information now,
> >>> other than perhaps trying to look at he names of jar files.
> >>>
> >>> cheers,
> >>> Colin
> >>>
> >>> On Tue, May 1, 2018, at 08:20, Manikumar wrote:
> >>>
> >>>> I assume the intent of the KIP to find out the Kafka broker version. In
> >>>> this case, maybe we should expose
> >>>> version using a Kafka request. This will help the remote scripts/tools
> >>>>
> >>> to
> >>
> >>> query the Kafka version.
> >>>> scripts (kafka-topics.sh, kafka-configs.sh, etc..) may run from remote
> >>>> machines and may use
> >>>> older Kafka versions. In this case, current proposal prints on the older
> >>>> version.
> >>>>
> >>>> On Tue, May 1, 2018 at 7:47 PM, Colin McCabe <cm...@apache.org>
> >>>> wrote:
> >>>>
> >>>> Thanks, Sasaki.
> >>>>>
> >>>>> Colin
> >>>>>
> >>>>> On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote:
> >>>>>
> >>>>>> Hi Colin, Jason,
> >>>>>>
> >>>>>> Thank you for your beneficial comment.
> >>>>>> I have updated my Pull Request to show git commit hash in version
> >>>>>> information.> In my current Pull Request, we cat get the result such
> >>>>>>
> >>>>> below:
> >>>>>
> >>>>>> $ bin/kafka-topics.sh --version
> >>>>>> (snip)
> >>>>>> 2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e)
> >>>>>>
> >>>>>>
> >>>>>> I have also updated to accept double-dash for this option (--
> >>>>>> version) only.>
> >>>>>>
> >>>>>> Many thanks,
> >>>>>> Sasaki
> >>>>>>
> >>>>>> From: Jason Gustafson <ja...@confluent.io>
> >>>>>>> Date: 2018-04-25 9:42 GMT+09:00
> >>>>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> >>>>>>> commands> > To: dev <de...@kafka.apache.org>
> >>>>>>>
> >>>>>>>
> >>>>>>> +1 on adding the git commit id to the output. We often encounter
> >>>>>>> environments which are testing off of trunk or have modifications
> >>>>>>>
> >>>>>> on
> >>>
> >>>> top of> > an existing release.
> >>>>>>>
> >>>>>>> -Jason
> >>>>>>>
> >>>>>>> On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cmccabe@apache.org
> >>>>>>> wrote:> >
> >>>>>>>
> >>>>>>>> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
> >>>>>>>>
> >>>>>>>>> Hi Jason, Colin,
> >>>>>>>>>
> >>>>>>>>> Thank you for your comment, and I'm sorry for late reply.
> >>>>>>>>>
> >>>>>>>>> > we refactored all of the tools so that we could use a common
> >>>>>>>>> > set of> >>> base options,
> >>>>>>>>> > it might be a little annoying to have to continue supporting
> >>>>>>>>> > both> >>> variations.
> >>>>>>>>>
> >>>>>>>>> I feel KIP-14 is good idea (I'm sorry, I didn't know about
> >>>>>>>>> it), and> >>> I think it's no longer necessary both single-dash
> >>>>>>>>>
> >>>>>>>> and
> >>>
> >>>> double-
> >>>>>
> >>>>>> dash are> >>> supported when this KIP will be accepted, as you
> >>>>>>>>>
> >>>>>>>> said.
> >>>
> >>>> I revise my Pull Request to support single-dash option only.
> >>>>>>>>>
> >>>>>>>>> Some my code in kafka-run-class.sh will become unnecessary when
> >>>>>>>>> KIP-14> >>> is accepted.
> >>>>>>>>> But I will keep this as temporary, because I seem that it's
> >>>>>>>>>
> >>>>>>>> useful>
> >>>
> >>>> before KIP-14 accepted.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> > Can you give a little more detail about what would be
> >>>>>>>>> > displayed when> >>> the version command was used?
> >>>>>>>>>
> >>>>>>>>> As Ismael said, the version string is got from
> >>>>>>>>> AppInfoParser#getVersion.> >>>
> >>>>>>>>> In my Pull Request, we can get the result such as below::
> >>>>>>>>>
> >>>>>>>>> $ bin/kafka-topics.sh --version
> >>>>>>>>> (snip)
> >>>>>>>>> Kafka 1.2.0-SNAPSHOT
> >>>>>>>>>
> >>>>>>>> Hi Sasaki,
> >>>>>>>>
> >>>>>>>> Thanks for the info. Can you add this to the KIP?
> >>>>>>>>
> >>>>>>>> Also, I really think we should include the git hash.
> >>>>>>>>
> >>>>>>>> best,
> >>>>>>>> Colin
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Many thanks,
> >>>>>>>>> Sasaki
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> From: Ismael Juma <is...@juma.me.uk>
> >>>>>>>>>> Date: 2018-04-24 3:44 GMT+09:00
> >>>>>>>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> >>>>>>>>>> commands> >>>> To: dev <de...@kafka.apache.org>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> FYI, the injection via the build process that is mentioned here
> >>>>>>>>>>
> >>>>>>>>> already
> >>>>>>>
> >>>>>>>> happens. See AppInfoParser.
> >>>>>>>>>>
> >>>>>>>>>> Ismael
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe
> >>>>>>>>>> <cm...@apache.org>> >> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Sasaki,
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks for the KIP. I think a version flag is a good idea.
> >>>>>>>>>>>
> >>>>>>>>>>> Can you give a little more detail about what would be
> >>>>>>>>>>>
> >>>>>>>>>> displayed
> >>
> >>> when> >> the
> >>>>>>>>>>> version command was used?
> >>>>>>>>>>>
> >>>>>>>>>>> We clearly want the version number, but we probably also want
> >>>>>>>>>>>
> >>>>>>>>>> to
> >>>
> >>>> know> >> if
> >>>>>>>>>>> this is an official release, or a random SNAPSHOT from a
> >>>>>>>>>>>
> >>>>>>>>>> branch.
> >>>
> >>>> If> >> this
> >>>>>>>>>>> is a release candidate, we probably want the RC number as
> >>>>>>>>>>>
> >>>>>>>>>> well,
> >>
> >>> like> >>>>> "1.1-rc3" We also want a git hash. This can be
> >>>>>>>>>>>
> >>>>>>>>>> injected by the> > build
> >>>>>
> >>>>>> process. In the case of an official release, where the source
> >>>>>>>>>>> code> >> is not
> >>>>>>>>>>> under git, we can pull it from a file.
> >>>>>>>>>>>
> >>>>>>>>>>> For example, hadoop's version output looks like this:
> >>>>>>>>>>>
> >>>>>>>>>>> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop
> >>>>>>>>>>> > version> >>>>> > Hadoop 2.8.3
> >>>>>>>>>>> > Subversion
> >>>>>>>>>>> > https://git-wip-us.apache.org/repos/asf/hadoop.git -r>
> >>>>>>>>>>>
> >>>>>>>>>> b3fe56402d908019d99af1f1f4fc65cb1d1436a2
> >>>>>
> >>>>>> > Compiled by jdu on 2017-12-05T03:43Z
> >>>>>>>>>>> > Compiled with protoc 2.5.0
> >>>>>>>>>>> > From source with checksum 9ff4856d824e983fa510d3f843e3f1
> >>>>>>>>>>>
> >>>>>>>>>> 9d>
> >>>
> >>>> > This command was run using /home/cmccabe/Downloads/
> >>>>>>>>>>
> >>>>>>>>>>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
> >>>>>>>>>>>
> >>>>>>>>>>> (The "subversion" line here is a little weird -- it now refers
> >>>>>>>>>>> to> >> git, not
> >>>>>>>>>>> svn)
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hey Sasaki,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Yeah, I don't feel too strongly about only supporting --
> >>>>>>>>>>>> version. I> >> agree
> >>>>>>>>>>>>
> >>>>>>>>>>> it
> >>>>>>>>>>>
> >>>>>>>>>>>> may help discoverability given the current approach. On the
> >>>>>>>>>>>> other> >> hand,
> >>>>>>>>>>>>
> >>>>>>>>>>> if
> >>>>>>>>>>>
> >>>>>>>>>>>> we refactored all of the tools so that we could use a common
> >>>>>>>>>>>> set of> >> base
> >>>>>>>>>>>> options, it might be a little annoying to have to continue
> >>>>>>>>>>>>
> >>>>>>>>>>> supporting
> >>>>>>>
> >>>>>>>> both
> >>>>>>>>>>>
> >>>>>>>>>>>> variations. For example, tool standardization was proposed in
> >>>>>>>>>>>> KIP-14> >> and
> >>>>>>>>>>>> I'm still holding out hope that someone will have time to
> >>>>>>>>>>>>
> >>>>>>>>>>> pick
> >>
> >>> this> >> work
> >>>>>>>>>>>> back up. It's always easier to add an option than remove one,
> >>>>>>>>>>>> so I'm> >>>>>> slightly inclined to have only --version for
> >>>>>>>>>>>>
> >>>>>>>>>>> now.
> >>>
> >>>> What do you
> >>>>>
> >>>>>> think?> >>>>> The double dash version is more consistent with
> >>>>>>>>>>>>
> >>>>>>>>>>> how
> >>>
> >>>> our other
> >>>>>
> >>>>>> flags> >> work.
> >>>>>>>>>>> In general, I feel that if --version is supported, --help
> >>>>>>>>>>>
> >>>>>>>>>> should
> >>>
> >>>> say> >> so.
> >>>>>>>>>>> best,
> >>>>>>>>>>> Colin
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Jason
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <
> >>>>>>>>>>>>
> >>>>>>>>>>> sasakitoa@oss.nttdata.com
> >>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi Jason
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thank you for helpful comments. I updated wiki based on your
> >>>>>>>>>>>>>
> >>>>>>>>>>>> advice.
> >>>>>>>
> >>>>>>>> I thought this option was relatively common and making
> >>>>>>>>>>>>> maintenance> >>>> easy
> >>>>>>>>>>>>> was also important.
> >>>>>>>>>>>>> However, as you said, it is not good that version option
> >>>>>>>>>>>>>
> >>>>>>>>>>>> won't
> >>>
> >>>> be> >>>>> shown up
> >>>>>>>>>>>>> in help description.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I thought accepting both single-dash and double-dash will
> >>>>>>>>>>>>>
> >>>>>>>>>>>> help
> >>>
> >>>> to> >> find
> >>>>>>>>>>>>> this option.
> >>>>>>>>>>>>> In my approach this option won't be showed, but most of
> >>>>>>>>>>>>> software> >> which
> >>>>>>>>>>>>>
> >>>>>>>>>>>> has
> >>>>>>>>>>>
> >>>>>>>>>>>> this option accepts either single-dash or double-dash.
> >>>>>>>>>>>>> I guess it doesn't need to support both if we take another
> >>>>>>>>>>>>> way.> >>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> @Ted Yeah, you're right. Sorry about the confusion.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Since we're here, I think this KIP is a nice improvement.
> >>>>>>>>>>>>>> It's> >>>>> definitely
> >>>>>>>>>>>>>> nice to have an easy way to check the version. That said,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> do
> >>
> >>> we> >>>> really
> >>>>>>>>>>>>>> need
> >>>>>>>>>>>>>> to support both `-version` and `--version`? The latter is
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> consistent
> >>>>>>>>
> >>>>>>>>> with
> >>>>>>>>>>>
> >>>>>>>>>>>> our current tools.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Also, I think the approach we've taken is basically to
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> build
> >>
> >>> the> >>>>> --version
> >>>>>>>>>>>>>> functionality into the bash script. This is nice because it
> >>>>>>>>>>>>>> saves> > a
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> lot of
> >>>>>>>>>>>
> >>>>>>>>>>>> work to update the commands individually and we don't need
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> to
> >>>
> >>>> do> >>>>> anything
> >>>>>>>>>>>>>> when we add new tools. The downside is that `--version`
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> won't
> >>>
> >>>> show> >> up
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> as
> >>>>>>>>>>>
> >>>>>>>>>>>> an
> >>>>>>>>>>>>>> option in any of the --help output. Not sure if that is too
> >>>>>>>>>>>>>> big of> >> a
> >>>>>>>>>>>>>> problem, but maybe worth mentioning this in the rejected
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> alternatives
> >>>>>>>>
> >>>>>>>>> section.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -Jason
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yuzhihong@gmail.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Jason:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Maybe your reply was intended for another KIP ?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> KIP-278 is about adding version option, not timeout.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Cheers
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> jason@confluent.io>
> >>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Sasaki,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks for the KIP. I think the timeout controls the
> >>>>>>>>>>>>>>>> maximum> >>>> allowed
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>> that the consumer will block for the next record. Maybe
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> the>
> >>>
> >>>> meaning
> >>>>>>>
> >>>>>>>> would
> >>>>>>>>>>>>>>> be clearer with the more concise name `--timeout`? That
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> also
> >>>
> >>>> fits> >>>>> with
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> old consumer which overrides the `consumer.timeout.ms`
> >>>>>>>>>>>>>>>> property.> >>>>>>>>>>
> >>>>>>>>>>>>>>>> By the way, it seems like the default value was
> >>>>>>>>>>>>>>>> intentionally> > set
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> low
> >>>>>>>>>>>
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>> both the old and new consumers, but I'm not sure of the
> >>>>>>>>>>>>>>> reason.> > We
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> could
> >>>>>>>>>>>
> >>>>>>>>>>>> leave the default as it is if we want to be safe, but
> >>>>>>>>>>>>>>>> increasing> >> it
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>> ok to me. Perhaps we could start a little lower, though,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> say
> >>>
> >>>> 10> >>>>> seconds?
> >>>>>>>>>>>>>>> In
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> any case, we should make it clear to the user that the
> >>>>>>>>>>>>>>>> timeout> >> was
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> reached.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It's surprising to see only the incomplete reported
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> results>
> >>>
> >>>> following a
> >>>>>>>>>>
> >>>>>>>>>>> timeout.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> Jason
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> sasakitoa@oss.nttdata.com>
> >>>>>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hello everyone,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I would like to start a discussion for KIP 278. Cloud
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> you> >
> >>>
> >>>> please
> >>>>>
> >>>>>> give
> >>>>>>>>>>>
> >>>>>>>>>>>> comments and advice ?
> >>>>>>>>>>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>>>>>>>>>>>>>>>> 278+-> >>>>>>>>>>> +Add+version+option+to+Kafka%2
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 7s+commands>
> >>>>>
> >>>>>> JIRA ticket and Pull Request are bellow:
> >>>>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
> >>>>>>>>>>>>>>>>> <https://github.com/apache/kafka/pull/639>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Many thanks,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Sasaki
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA
> >>>>>>>>>>>>>>>>> CORPORATION> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >>>>>>
> >>>>>>
> >>>>>
> > --
> > Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >
> >
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Jason Gustafson <ja...@confluent.io>.
Hi Sasaki,
Thanks for the update. The KIP looks good to me. I'd suggest moving to a
vote.
Thanks,
Jason
On Mon, May 7, 2018 at 7:08 AM, Sasaki Toru <sa...@oss.nttdata.com>
wrote:
> Hi Manikumar, Colin,
>
> Thank you for your comment.
>
> As Colin said, I proposed adding an option to show version information of
> "local" tool,
> because many software have this option and I think Kafka also needs this
> one.
>
> As you said, the function to show remote Kafka version is useful,
> but I think it is better to create new KIP because this function has some
> points which should be considered.
>
> If you have any better ideas, could you please tell us?
>
>
> Many thanks,
> Sasaki
>
> From: Manikumar <ma...@gmail.com>
>> Date: 2018-05-03 4:11 GMT+09:00
>>
>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
>> To: dev <de...@kafka.apache.org>
>>
>>
>> Hi Colin,
>>
>> Thanks for explanation. It's definitely useful to have --version flag.
>>
>> kafka-broker-api-versions.sh gives the API versions, not Kafka release
>> version.
>> Is not easy to figure out release version from API versions. Currently
>> release version is available via metric/JMX.
>> If required, we can implement this in future.
>>
>>
>> Thanks,
>>
>> On Wed, May 2, 2018 at 10:58 PM, Colin McCabe <cm...@apache.org> wrote:
>>
>> Hi Manikumar,
>>>
>>> We already have a tool for getting the Kafka broker API versions,
>>> "./bin/kafka-broker-api-versions.sh". It was added as part of KIP-97.
>>>
>>> What Saski is proposing here is having a way of getting the version of
>>> locally installed Kafka software, which may be different from the server
>>> version. Many pieces of software offer a --version flag, and it's always
>>> understood to refer to the local version of the software, not a version
>>> running somewhere else. The user has no way to get this information now,
>>> other than perhaps trying to look at he names of jar files.
>>>
>>> cheers,
>>> Colin
>>>
>>> On Tue, May 1, 2018, at 08:20, Manikumar wrote:
>>>
>>>> I assume the intent of the KIP to find out the Kafka broker version. In
>>>> this case, maybe we should expose
>>>> version using a Kafka request. This will help the remote scripts/tools
>>>>
>>> to
>>
>>> query the Kafka version.
>>>> scripts (kafka-topics.sh, kafka-configs.sh, etc..) may run from remote
>>>> machines and may use
>>>> older Kafka versions. In this case, current proposal prints on the older
>>>> version.
>>>>
>>>> On Tue, May 1, 2018 at 7:47 PM, Colin McCabe <cm...@apache.org>
>>>> wrote:
>>>>
>>>> Thanks, Sasaki.
>>>>>
>>>>> Colin
>>>>>
>>>>> On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote:
>>>>>
>>>>>> Hi Colin, Jason,
>>>>>>
>>>>>> Thank you for your beneficial comment.
>>>>>> I have updated my Pull Request to show git commit hash in version
>>>>>> information.> In my current Pull Request, we cat get the result such
>>>>>>
>>>>> below:
>>>>>
>>>>>> $ bin/kafka-topics.sh --version
>>>>>> (snip)
>>>>>> 2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e)
>>>>>>
>>>>>>
>>>>>> I have also updated to accept double-dash for this option (--
>>>>>> version) only.>
>>>>>>
>>>>>> Many thanks,
>>>>>> Sasaki
>>>>>>
>>>>>> From: Jason Gustafson <ja...@confluent.io>
>>>>>>> Date: 2018-04-25 9:42 GMT+09:00
>>>>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
>>>>>>> commands> > To: dev <de...@kafka.apache.org>
>>>>>>>
>>>>>>>
>>>>>>> +1 on adding the git commit id to the output. We often encounter
>>>>>>> environments which are testing off of trunk or have modifications
>>>>>>>
>>>>>> on
>>>
>>>> top of> > an existing release.
>>>>>>>
>>>>>>> -Jason
>>>>>>>
>>>>>>> On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cmccabe@apache.org
>>>>>>> wrote:> >
>>>>>>>
>>>>>>>> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
>>>>>>>>
>>>>>>>>> Hi Jason, Colin,
>>>>>>>>>
>>>>>>>>> Thank you for your comment, and I'm sorry for late reply.
>>>>>>>>>
>>>>>>>>> > we refactored all of the tools so that we could use a common
>>>>>>>>> > set of> >>> base options,
>>>>>>>>> > it might be a little annoying to have to continue supporting
>>>>>>>>> > both> >>> variations.
>>>>>>>>>
>>>>>>>>> I feel KIP-14 is good idea (I'm sorry, I didn't know about
>>>>>>>>> it), and> >>> I think it's no longer necessary both single-dash
>>>>>>>>>
>>>>>>>> and
>>>
>>>> double-
>>>>>
>>>>>> dash are> >>> supported when this KIP will be accepted, as you
>>>>>>>>>
>>>>>>>> said.
>>>
>>>> I revise my Pull Request to support single-dash option only.
>>>>>>>>>
>>>>>>>>> Some my code in kafka-run-class.sh will become unnecessary when
>>>>>>>>> KIP-14> >>> is accepted.
>>>>>>>>> But I will keep this as temporary, because I seem that it's
>>>>>>>>>
>>>>>>>> useful>
>>>
>>>> before KIP-14 accepted.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> > Can you give a little more detail about what would be
>>>>>>>>> > displayed when> >>> the version command was used?
>>>>>>>>>
>>>>>>>>> As Ismael said, the version string is got from
>>>>>>>>> AppInfoParser#getVersion.> >>>
>>>>>>>>> In my Pull Request, we can get the result such as below::
>>>>>>>>>
>>>>>>>>> $ bin/kafka-topics.sh --version
>>>>>>>>> (snip)
>>>>>>>>> Kafka 1.2.0-SNAPSHOT
>>>>>>>>>
>>>>>>>> Hi Sasaki,
>>>>>>>>
>>>>>>>> Thanks for the info. Can you add this to the KIP?
>>>>>>>>
>>>>>>>> Also, I really think we should include the git hash.
>>>>>>>>
>>>>>>>> best,
>>>>>>>> Colin
>>>>>>>>
>>>>>>>>
>>>>>>>> Many thanks,
>>>>>>>>> Sasaki
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> From: Ismael Juma <is...@juma.me.uk>
>>>>>>>>>> Date: 2018-04-24 3:44 GMT+09:00
>>>>>>>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
>>>>>>>>>> commands> >>>> To: dev <de...@kafka.apache.org>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> FYI, the injection via the build process that is mentioned here
>>>>>>>>>>
>>>>>>>>> already
>>>>>>>
>>>>>>>> happens. See AppInfoParser.
>>>>>>>>>>
>>>>>>>>>> Ismael
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe
>>>>>>>>>> <cm...@apache.org>> >> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Sasaki,
>>>>>>>>>>>
>>>>>>>>>>> Thanks for the KIP. I think a version flag is a good idea.
>>>>>>>>>>>
>>>>>>>>>>> Can you give a little more detail about what would be
>>>>>>>>>>>
>>>>>>>>>> displayed
>>
>>> when> >> the
>>>>>>>>>>> version command was used?
>>>>>>>>>>>
>>>>>>>>>>> We clearly want the version number, but we probably also want
>>>>>>>>>>>
>>>>>>>>>> to
>>>
>>>> know> >> if
>>>>>>>>>>> this is an official release, or a random SNAPSHOT from a
>>>>>>>>>>>
>>>>>>>>>> branch.
>>>
>>>> If> >> this
>>>>>>>>>>> is a release candidate, we probably want the RC number as
>>>>>>>>>>>
>>>>>>>>>> well,
>>
>>> like> >>>>> "1.1-rc3" We also want a git hash. This can be
>>>>>>>>>>>
>>>>>>>>>> injected by the> > build
>>>>>
>>>>>> process. In the case of an official release, where the source
>>>>>>>>>>> code> >> is not
>>>>>>>>>>> under git, we can pull it from a file.
>>>>>>>>>>>
>>>>>>>>>>> For example, hadoop's version output looks like this:
>>>>>>>>>>>
>>>>>>>>>>> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop
>>>>>>>>>>> > version> >>>>> > Hadoop 2.8.3
>>>>>>>>>>> > Subversion
>>>>>>>>>>> > https://git-wip-us.apache.org/repos/asf/hadoop.git -r>
>>>>>>>>>>>
>>>>>>>>>> b3fe56402d908019d99af1f1f4fc65cb1d1436a2
>>>>>
>>>>>> > Compiled by jdu on 2017-12-05T03:43Z
>>>>>>>>>>> > Compiled with protoc 2.5.0
>>>>>>>>>>> > From source with checksum 9ff4856d824e983fa510d3f843e3f1
>>>>>>>>>>>
>>>>>>>>>> 9d>
>>>
>>>> > This command was run using /home/cmccabe/Downloads/
>>>>>>>>>>
>>>>>>>>>>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
>>>>>>>>>>>
>>>>>>>>>>> (The "subversion" line here is a little weird -- it now refers
>>>>>>>>>>> to> >> git, not
>>>>>>>>>>> svn)
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hey Sasaki,
>>>>>>>>>>>>
>>>>>>>>>>>> Yeah, I don't feel too strongly about only supporting --
>>>>>>>>>>>> version. I> >> agree
>>>>>>>>>>>>
>>>>>>>>>>> it
>>>>>>>>>>>
>>>>>>>>>>>> may help discoverability given the current approach. On the
>>>>>>>>>>>> other> >> hand,
>>>>>>>>>>>>
>>>>>>>>>>> if
>>>>>>>>>>>
>>>>>>>>>>>> we refactored all of the tools so that we could use a common
>>>>>>>>>>>> set of> >> base
>>>>>>>>>>>> options, it might be a little annoying to have to continue
>>>>>>>>>>>>
>>>>>>>>>>> supporting
>>>>>>>
>>>>>>>> both
>>>>>>>>>>>
>>>>>>>>>>>> variations. For example, tool standardization was proposed in
>>>>>>>>>>>> KIP-14> >> and
>>>>>>>>>>>> I'm still holding out hope that someone will have time to
>>>>>>>>>>>>
>>>>>>>>>>> pick
>>
>>> this> >> work
>>>>>>>>>>>> back up. It's always easier to add an option than remove one,
>>>>>>>>>>>> so I'm> >>>>>> slightly inclined to have only --version for
>>>>>>>>>>>>
>>>>>>>>>>> now.
>>>
>>>> What do you
>>>>>
>>>>>> think?> >>>>> The double dash version is more consistent with
>>>>>>>>>>>>
>>>>>>>>>>> how
>>>
>>>> our other
>>>>>
>>>>>> flags> >> work.
>>>>>>>>>>> In general, I feel that if --version is supported, --help
>>>>>>>>>>>
>>>>>>>>>> should
>>>
>>>> say> >> so.
>>>>>>>>>>> best,
>>>>>>>>>>> Colin
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Jason
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <
>>>>>>>>>>>>
>>>>>>>>>>> sasakitoa@oss.nttdata.com
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Jason
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank you for helpful comments. I updated wiki based on your
>>>>>>>>>>>>>
>>>>>>>>>>>> advice.
>>>>>>>
>>>>>>>> I thought this option was relatively common and making
>>>>>>>>>>>>> maintenance> >>>> easy
>>>>>>>>>>>>> was also important.
>>>>>>>>>>>>> However, as you said, it is not good that version option
>>>>>>>>>>>>>
>>>>>>>>>>>> won't
>>>
>>>> be> >>>>> shown up
>>>>>>>>>>>>> in help description.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I thought accepting both single-dash and double-dash will
>>>>>>>>>>>>>
>>>>>>>>>>>> help
>>>
>>>> to> >> find
>>>>>>>>>>>>> this option.
>>>>>>>>>>>>> In my approach this option won't be showed, but most of
>>>>>>>>>>>>> software> >> which
>>>>>>>>>>>>>
>>>>>>>>>>>> has
>>>>>>>>>>>
>>>>>>>>>>>> this option accepts either single-dash or double-dash.
>>>>>>>>>>>>> I guess it doesn't need to support both if we take another
>>>>>>>>>>>>> way.> >>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>
>>>>>>>>>>>>> @Ted Yeah, you're right. Sorry about the confusion.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Since we're here, I think this KIP is a nice improvement.
>>>>>>>>>>>>>> It's> >>>>> definitely
>>>>>>>>>>>>>> nice to have an easy way to check the version. That said,
>>>>>>>>>>>>>>
>>>>>>>>>>>>> do
>>
>>> we> >>>> really
>>>>>>>>>>>>>> need
>>>>>>>>>>>>>> to support both `-version` and `--version`? The latter is
>>>>>>>>>>>>>>
>>>>>>>>>>>>> consistent
>>>>>>>>
>>>>>>>>> with
>>>>>>>>>>>
>>>>>>>>>>>> our current tools.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Also, I think the approach we've taken is basically to
>>>>>>>>>>>>>>
>>>>>>>>>>>>> build
>>
>>> the> >>>>> --version
>>>>>>>>>>>>>> functionality into the bash script. This is nice because it
>>>>>>>>>>>>>> saves> > a
>>>>>>>>>>>>>>
>>>>>>>>>>>>> lot of
>>>>>>>>>>>
>>>>>>>>>>>> work to update the commands individually and we don't need
>>>>>>>>>>>>>>
>>>>>>>>>>>>> to
>>>
>>>> do> >>>>> anything
>>>>>>>>>>>>>> when we add new tools. The downside is that `--version`
>>>>>>>>>>>>>>
>>>>>>>>>>>>> won't
>>>
>>>> show> >> up
>>>>>>>>>>>>>>
>>>>>>>>>>>>> as
>>>>>>>>>>>
>>>>>>>>>>>> an
>>>>>>>>>>>>>> option in any of the --help output. Not sure if that is too
>>>>>>>>>>>>>> big of> >> a
>>>>>>>>>>>>>> problem, but maybe worth mentioning this in the rejected
>>>>>>>>>>>>>>
>>>>>>>>>>>>> alternatives
>>>>>>>>
>>>>>>>>> section.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -Jason
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yuzhihong@gmail.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>
>>>>>>>> Jason:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Maybe your reply was intended for another KIP ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> KIP-278 is about adding version option, not timeout.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> jason@confluent.io>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Sasaki,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks for the KIP. I think the timeout controls the
>>>>>>>>>>>>>>>> maximum> >>>> allowed
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>> that the consumer will block for the next record. Maybe
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> the>
>>>
>>>> meaning
>>>>>>>
>>>>>>>> would
>>>>>>>>>>>>>>> be clearer with the more concise name `--timeout`? That
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> also
>>>
>>>> fits> >>>>> with
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> old consumer which overrides the `consumer.timeout.ms`
>>>>>>>>>>>>>>>> property.> >>>>>>>>>>
>>>>>>>>>>>>>>>> By the way, it seems like the default value was
>>>>>>>>>>>>>>>> intentionally> > set
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> low
>>>>>>>>>>>
>>>>>>>>>>>> for
>>>>>>>>>>>>>>> both the old and new consumers, but I'm not sure of the
>>>>>>>>>>>>>>> reason.> > We
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> could
>>>>>>>>>>>
>>>>>>>>>>>> leave the default as it is if we want to be safe, but
>>>>>>>>>>>>>>>> increasing> >> it
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>> ok to me. Perhaps we could start a little lower, though,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> say
>>>
>>>> 10> >>>>> seconds?
>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> any case, we should make it clear to the user that the
>>>>>>>>>>>>>>>> timeout> >> was
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> reached.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It's surprising to see only the incomplete reported
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> results>
>>>
>>>> following a
>>>>>>>>>>
>>>>>>>>>>> timeout.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Jason
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> sasakitoa@oss.nttdata.com>
>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hello everyone,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I would like to start a discussion for KIP 278. Cloud
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> you> >
>>>
>>>> please
>>>>>
>>>>>> give
>>>>>>>>>>>
>>>>>>>>>>>> comments and advice ?
>>>>>>>>>>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>>>>>>>>>>>>>>>> 278+-> >>>>>>>>>>> +Add+version+option+to+Kafka%2
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 7s+commands>
>>>>>
>>>>>> JIRA ticket and Pull Request are bellow:
>>>>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
>>>>>>>>>>>>>>>>> <https://github.com/apache/kafka/pull/639>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Many thanks,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Sasaki
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA
>>>>>>>>>>>>>>>>> CORPORATION> >>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>>>>>
>>>>>>>>> --
>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>>
>>>>>>
>>>>>
> --
> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>
>
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Sasaki Toru <sa...@oss.nttdata.com>.
Hi Manikumar, Colin,
Thank you for your comment.
As Colin said, I proposed adding an option to show version information of "local" tool,
because many software have this option and I think Kafka also needs this one.
As you said, the function to show remote Kafka version is useful,
but I think it is better to create new KIP because this function has some points which should be considered.
If you have any better ideas, could you please tell us?
Many thanks,
Sasaki
> From: Manikumar <ma...@gmail.com>
> Date: 2018-05-03 4:11 GMT+09:00
> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
> To: dev <de...@kafka.apache.org>
>
>
> Hi Colin,
>
> Thanks for explanation. It's definitely useful to have --version flag.
>
> kafka-broker-api-versions.sh gives the API versions, not Kafka release
> version.
> Is not easy to figure out release version from API versions. Currently
> release version is available via metric/JMX.
> If required, we can implement this in future.
>
>
> Thanks,
>
> On Wed, May 2, 2018 at 10:58 PM, Colin McCabe <cm...@apache.org> wrote:
>
>> Hi Manikumar,
>>
>> We already have a tool for getting the Kafka broker API versions,
>> "./bin/kafka-broker-api-versions.sh". It was added as part of KIP-97.
>>
>> What Saski is proposing here is having a way of getting the version of
>> locally installed Kafka software, which may be different from the server
>> version. Many pieces of software offer a --version flag, and it's always
>> understood to refer to the local version of the software, not a version
>> running somewhere else. The user has no way to get this information now,
>> other than perhaps trying to look at he names of jar files.
>>
>> cheers,
>> Colin
>>
>> On Tue, May 1, 2018, at 08:20, Manikumar wrote:
>>> I assume the intent of the KIP to find out the Kafka broker version. In
>>> this case, maybe we should expose
>>> version using a Kafka request. This will help the remote scripts/tools
> to
>>> query the Kafka version.
>>> scripts (kafka-topics.sh, kafka-configs.sh, etc..) may run from remote
>>> machines and may use
>>> older Kafka versions. In this case, current proposal prints on the older
>>> version.
>>>
>>> On Tue, May 1, 2018 at 7:47 PM, Colin McCabe <cm...@apache.org> wrote:
>>>
>>>> Thanks, Sasaki.
>>>>
>>>> Colin
>>>>
>>>> On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote:
>>>>> Hi Colin, Jason,
>>>>>
>>>>> Thank you for your beneficial comment.
>>>>> I have updated my Pull Request to show git commit hash in version
>>>>> information.> In my current Pull Request, we cat get the result such
>>>> below:
>>>>> $ bin/kafka-topics.sh --version
>>>>> (snip)
>>>>> 2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e)
>>>>>
>>>>>
>>>>> I have also updated to accept double-dash for this option (--
>>>>> version) only.>
>>>>>
>>>>> Many thanks,
>>>>> Sasaki
>>>>>
>>>>>> From: Jason Gustafson <ja...@confluent.io>
>>>>>> Date: 2018-04-25 9:42 GMT+09:00
>>>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
>>>>>> commands> > To: dev <de...@kafka.apache.org>
>>>>>>
>>>>>>
>>>>>> +1 on adding the git commit id to the output. We often encounter
>>>>>> environments which are testing off of trunk or have modifications
>> on
>>>>>> top of> > an existing release.
>>>>>>
>>>>>> -Jason
>>>>>>
>>>>>> On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cmccabe@apache.org
>>>>>> wrote:> >
>>>>>>> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
>>>>>>>> Hi Jason, Colin,
>>>>>>>>
>>>>>>>> Thank you for your comment, and I'm sorry for late reply.
>>>>>>>>
>>>>>>>> > we refactored all of the tools so that we could use a common
>>>>>>>> > set of> >>> base options,
>>>>>>>> > it might be a little annoying to have to continue supporting
>>>>>>>> > both> >>> variations.
>>>>>>>>
>>>>>>>> I feel KIP-14 is good idea (I'm sorry, I didn't know about
>>>>>>>> it), and> >>> I think it's no longer necessary both single-dash
>> and
>>>> double-
>>>>>>>> dash are> >>> supported when this KIP will be accepted, as you
>> said.
>>>>>>>> I revise my Pull Request to support single-dash option only.
>>>>>>>>
>>>>>>>> Some my code in kafka-run-class.sh will become unnecessary when
>>>>>>>> KIP-14> >>> is accepted.
>>>>>>>> But I will keep this as temporary, because I seem that it's
>> useful>
>>>>>>> before KIP-14 accepted.
>>>>>>>>
>>>>>>>> > Can you give a little more detail about what would be
>>>>>>>> > displayed when> >>> the version command was used?
>>>>>>>>
>>>>>>>> As Ismael said, the version string is got from
>>>>>>>> AppInfoParser#getVersion.> >>>
>>>>>>>> In my Pull Request, we can get the result such as below::
>>>>>>>>
>>>>>>>> $ bin/kafka-topics.sh --version
>>>>>>>> (snip)
>>>>>>>> Kafka 1.2.0-SNAPSHOT
>>>>>>> Hi Sasaki,
>>>>>>>
>>>>>>> Thanks for the info. Can you add this to the KIP?
>>>>>>>
>>>>>>> Also, I really think we should include the git hash.
>>>>>>>
>>>>>>> best,
>>>>>>> Colin
>>>>>>>
>>>>>>>
>>>>>>>> Many thanks,
>>>>>>>> Sasaki
>>>>>>>>
>>>>>>>>
>>>>>>>>> From: Ismael Juma <is...@juma.me.uk>
>>>>>>>>> Date: 2018-04-24 3:44 GMT+09:00
>>>>>>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
>>>>>>>>> commands> >>>> To: dev <de...@kafka.apache.org>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> FYI, the injection via the build process that is mentioned here
>>>>>> already
>>>>>>>>> happens. See AppInfoParser.
>>>>>>>>>
>>>>>>>>> Ismael
>>>>>>>>>
>>>>>>>>> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe
>>>>>>>>> <cm...@apache.org>> >> wrote:
>>>>>>>>>> Hi Sasaki,
>>>>>>>>>>
>>>>>>>>>> Thanks for the KIP. I think a version flag is a good idea.
>>>>>>>>>>
>>>>>>>>>> Can you give a little more detail about what would be
> displayed
>>>>>>>>>> when> >> the
>>>>>>>>>> version command was used?
>>>>>>>>>>
>>>>>>>>>> We clearly want the version number, but we probably also want
>> to
>>>>>>>>>> know> >> if
>>>>>>>>>> this is an official release, or a random SNAPSHOT from a
>> branch.
>>>>>>>>>> If> >> this
>>>>>>>>>> is a release candidate, we probably want the RC number as
> well,
>>>>>>>>>> like> >>>>> "1.1-rc3" We also want a git hash. This can be
>>>> injected by the> > build
>>>>>>>>>> process. In the case of an official release, where the source
>>>>>>>>>> code> >> is not
>>>>>>>>>> under git, we can pull it from a file.
>>>>>>>>>>
>>>>>>>>>> For example, hadoop's version output looks like this:
>>>>>>>>>>
>>>>>>>>>> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop
>>>>>>>>>> > version> >>>>> > Hadoop 2.8.3
>>>>>>>>>> > Subversion
>>>>>>>>>> > https://git-wip-us.apache.org/repos/asf/hadoop.git -r>
>>>> b3fe56402d908019d99af1f1f4fc65cb1d1436a2
>>>>>>>>>> > Compiled by jdu on 2017-12-05T03:43Z
>>>>>>>>>> > Compiled with protoc 2.5.0
>>>>>>>>>> > From source with checksum 9ff4856d824e983fa510d3f843e3f1
>> 9d>
>>>>>>>>> > This command was run using /home/cmccabe/Downloads/
>>>>>>>>>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
>>>>>>>>>>
>>>>>>>>>> (The "subversion" line here is a little weird -- it now refers
>>>>>>>>>> to> >> git, not
>>>>>>>>>> svn)
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
>>>>>>>>>>> Hey Sasaki,
>>>>>>>>>>>
>>>>>>>>>>> Yeah, I don't feel too strongly about only supporting --
>>>>>>>>>>> version. I> >> agree
>>>>>>>>>> it
>>>>>>>>>>> may help discoverability given the current approach. On the
>>>>>>>>>>> other> >> hand,
>>>>>>>>>> if
>>>>>>>>>>> we refactored all of the tools so that we could use a common
>>>>>>>>>>> set of> >> base
>>>>>>>>>>> options, it might be a little annoying to have to continue
>>>>>> supporting
>>>>>>>>>> both
>>>>>>>>>>> variations. For example, tool standardization was proposed in
>>>>>>>>>>> KIP-14> >> and
>>>>>>>>>>> I'm still holding out hope that someone will have time to
> pick
>>>>>>>>>>> this> >> work
>>>>>>>>>>> back up. It's always easier to add an option than remove one,
>>>>>>>>>>> so I'm> >>>>>> slightly inclined to have only --version for
>> now.
>>>> What do you
>>>>>>>>>>> think?> >>>>> The double dash version is more consistent with
>> how
>>>> our other
>>>>>>>>>> flags> >> work.
>>>>>>>>>> In general, I feel that if --version is supported, --help
>> should
>>>>>>>>>> say> >> so.
>>>>>>>>>> best,
>>>>>>>>>> Colin
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Jason
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <
>>>>>>> sasakitoa@oss.nttdata.com
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Jason
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you for helpful comments. I updated wiki based on your
>>>>>> advice.
>>>>>>>>>>>> I thought this option was relatively common and making
>>>>>>>>>>>> maintenance> >>>> easy
>>>>>>>>>>>> was also important.
>>>>>>>>>>>> However, as you said, it is not good that version option
>> won't
>>>>>>>>>>>> be> >>>>> shown up
>>>>>>>>>>>> in help description.
>>>>>>>>>>>>
>>>>>>>>>>>> I thought accepting both single-dash and double-dash will
>> help
>>>>>>>>>>>> to> >> find
>>>>>>>>>>>> this option.
>>>>>>>>>>>> In my approach this option won't be showed, but most of
>>>>>>>>>>>> software> >> which
>>>>>>>>>> has
>>>>>>>>>>>> this option accepts either single-dash or double-dash.
>>>>>>>>>>>> I guess it doesn't need to support both if we take another
>>>>>>>>>>>> way.> >>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>>
>>>>>>>>>>>> @Ted Yeah, you're right. Sorry about the confusion.
>>>>>>>>>>>>> Since we're here, I think this KIP is a nice improvement.
>>>>>>>>>>>>> It's> >>>>> definitely
>>>>>>>>>>>>> nice to have an easy way to check the version. That said,
> do
>>>>>>>>>>>>> we> >>>> really
>>>>>>>>>>>>> need
>>>>>>>>>>>>> to support both `-version` and `--version`? The latter is
>>>>>>> consistent
>>>>>>>>>> with
>>>>>>>>>>>>> our current tools.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, I think the approach we've taken is basically to
> build
>>>>>>>>>>>>> the> >>>>> --version
>>>>>>>>>>>>> functionality into the bash script. This is nice because it
>>>>>>>>>>>>> saves> > a
>>>>>>>>>> lot of
>>>>>>>>>>>>> work to update the commands individually and we don't need
>> to
>>>>>>>>>>>>> do> >>>>> anything
>>>>>>>>>>>>> when we add new tools. The downside is that `--version`
>> won't
>>>>>>>>>>>>> show> >> up
>>>>>>>>>> as
>>>>>>>>>>>>> an
>>>>>>>>>>>>> option in any of the --help output. Not sure if that is too
>>>>>>>>>>>>> big of> >> a
>>>>>>>>>>>>> problem, but maybe worth mentioning this in the rejected
>>>>>>> alternatives
>>>>>>>>>>>>> section.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Jason
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yuzhihong@gmail.com
>>>>>> wrote:
>>>>>>>>>>>>> Jason:
>>>>>>>>>>>>>> Maybe your reply was intended for another KIP ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> KIP-278 is about adding version option, not timeout.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <
>>>>>>> jason@confluent.io>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Sasaki,
>>>>>>>>>>>>>>> Thanks for the KIP. I think the timeout controls the
>>>>>>>>>>>>>>> maximum> >>>> allowed
>>>>>>>>>>>>>> time
>>>>>>>>>>>>>> that the consumer will block for the next record. Maybe
>> the>
>>>>>> meaning
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>> be clearer with the more concise name `--timeout`? That
>> also
>>>>>>>>>>>>>> fits> >>>>> with
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> old consumer which overrides the `consumer.timeout.ms`
>>>>>>>>>>>>>>> property.> >>>>>>>>>>
>>>>>>>>>>>>>>> By the way, it seems like the default value was
>>>>>>>>>>>>>>> intentionally> > set
>>>>>>>>>> low
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>> both the old and new consumers, but I'm not sure of the
>>>>>>>>>>>>>> reason.> > We
>>>>>>>>>> could
>>>>>>>>>>>>>>> leave the default as it is if we want to be safe, but
>>>>>>>>>>>>>>> increasing> >> it
>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>> ok to me. Perhaps we could start a little lower, though,
>> say
>>>>>>>>>>>>>> 10> >>>>> seconds?
>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> any case, we should make it clear to the user that the
>>>>>>>>>>>>>>> timeout> >> was
>>>>>>>>>>>>>> reached.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It's surprising to see only the incomplete reported
>> results>
>>>>>>>>> following a
>>>>>>>>>>>>>>> timeout.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Jason
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
>>>>>>>>>> sasakitoa@oss.nttdata.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello everyone,
>>>>>>>>>>>>>>>> I would like to start a discussion for KIP 278. Cloud
>> you> >
>>>> please
>>>>>>>>>> give
>>>>>>>>>>>>>>>> comments and advice ?
>>>>>>>>>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>>>>>>>>>>>>>>> 278+-> >>>>>>>>>>> +Add+version+option+to+Kafka%2
>>>> 7s+commands>
>>>>>>>>>>>>>>>> JIRA ticket and Pull Request are bellow:
>>>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
>>>>>>>>>>>>>>>> <https://github.com/apache/kafka/pull/639>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Many thanks,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sasaki
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA
>>>>>>>>>>>>>>>> CORPORATION> >>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>> --
>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>>>>
>>>>> --
>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>
>>>>
--
Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Manikumar <ma...@gmail.com>.
Hi Colin,
Thanks for explanation. It's definitely useful to have --version flag.
kafka-broker-api-versions.sh gives the API versions, not Kafka release
version.
Is not easy to figure out release version from API versions. Currently
release version is available via metric/JMX.
If required, we can implement this in future.
Thanks,
On Wed, May 2, 2018 at 10:58 PM, Colin McCabe <cm...@apache.org> wrote:
> Hi Manikumar,
>
> We already have a tool for getting the Kafka broker API versions,
> "./bin/kafka-broker-api-versions.sh". It was added as part of KIP-97.
>
> What Saski is proposing here is having a way of getting the version of
> locally installed Kafka software, which may be different from the server
> version. Many pieces of software offer a --version flag, and it's always
> understood to refer to the local version of the software, not a version
> running somewhere else. The user has no way to get this information now,
> other than perhaps trying to look at he names of jar files.
>
> cheers,
> Colin
>
> On Tue, May 1, 2018, at 08:20, Manikumar wrote:
> > I assume the intent of the KIP to find out the Kafka broker version. In
> > this case, maybe we should expose
> > version using a Kafka request. This will help the remote scripts/tools to
> > query the Kafka version.
> > scripts (kafka-topics.sh, kafka-configs.sh, etc..) may run from remote
> > machines and may use
> > older Kafka versions. In this case, current proposal prints on the older
> > version.
> >
> > On Tue, May 1, 2018 at 7:47 PM, Colin McCabe <cm...@apache.org> wrote:
> >
> > > Thanks, Sasaki.
> > >
> > > Colin
> > >
> > > On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote:
> > > > Hi Colin, Jason,
> > > >
> > > > Thank you for your beneficial comment.
> > > > I have updated my Pull Request to show git commit hash in version
> > > > information.> In my current Pull Request, we cat get the result such
> > > below:
> > > >
> > > > $ bin/kafka-topics.sh --version
> > > > (snip)
> > > > 2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e)
> > > >
> > > >
> > > > I have also updated to accept double-dash for this option (--
> > > > version) only.>
> > > >
> > > > Many thanks,
> > > > Sasaki
> > > >
> > > > > From: Jason Gustafson <ja...@confluent.io>
> > > > > Date: 2018-04-25 9:42 GMT+09:00
> > > > > Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> > > > > commands> > To: dev <de...@kafka.apache.org>
> > > > >
> > > > >
> > > > > +1 on adding the git commit id to the output. We often encounter
> > > > > environments which are testing off of trunk or have modifications
> on
> > > > > top of> > an existing release.
> > > > >
> > > > > -Jason
> > > > >
> > > > > On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cmccabe@apache.org
> >
> > > > > wrote:> >
> > > > >> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
> > > > >>> Hi Jason, Colin,
> > > > >>>
> > > > >>> Thank you for your comment, and I'm sorry for late reply.
> > > > >>>
> > > > >>> > we refactored all of the tools so that we could use a common
> > > > >>> > set of> >>> base options,
> > > > >>> > it might be a little annoying to have to continue supporting
> > > > >>> > both> >>> variations.
> > > > >>>
> > > > >>> I feel KIP-14 is good idea (I'm sorry, I didn't know about
> > > > >>> it), and> >>> I think it's no longer necessary both single-dash
> and
> > > double-
> > > > >>> dash are> >>> supported when this KIP will be accepted, as you
> said.
> > > > >>> I revise my Pull Request to support single-dash option only.
> > > > >>>
> > > > >>> Some my code in kafka-run-class.sh will become unnecessary when
> > > > >>> KIP-14> >>> is accepted.
> > > > >>> But I will keep this as temporary, because I seem that it's
> useful>
> > > >>> before KIP-14 accepted.
> > > > >>>
> > > > >>>
> > > > >>> > Can you give a little more detail about what would be
> > > > >>> > displayed when> >>> the version command was used?
> > > > >>>
> > > > >>> As Ismael said, the version string is got from
> > > > >>> AppInfoParser#getVersion.> >>>
> > > > >>> In my Pull Request, we can get the result such as below::
> > > > >>>
> > > > >>> $ bin/kafka-topics.sh --version
> > > > >>> (snip)
> > > > >>> Kafka 1.2.0-SNAPSHOT
> > > > >> Hi Sasaki,
> > > > >>
> > > > >> Thanks for the info. Can you add this to the KIP?
> > > > >>
> > > > >> Also, I really think we should include the git hash.
> > > > >>
> > > > >> best,
> > > > >> Colin
> > > > >>
> > > > >>
> > > > >>>
> > > > >>> Many thanks,
> > > > >>> Sasaki
> > > > >>>
> > > > >>>
> > > > >>>> From: Ismael Juma <is...@juma.me.uk>
> > > > >>>> Date: 2018-04-24 3:44 GMT+09:00
> > > > >>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> > > > >>>> commands> >>>> To: dev <de...@kafka.apache.org>
> > > > >>>>
> > > > >>>>
> > > > >>>> FYI, the injection via the build process that is mentioned here
> > > > > already
> > > > >>>> happens. See AppInfoParser.
> > > > >>>>
> > > > >>>> Ismael
> > > > >>>>
> > > > >>>> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe
> > > > >>>> <cm...@apache.org>> >> wrote:
> > > > >>>>> Hi Sasaki,
> > > > >>>>>
> > > > >>>>> Thanks for the KIP. I think a version flag is a good idea.
> > > > >>>>>
> > > > >>>>> Can you give a little more detail about what would be displayed
> > > > >>>>> when> >> the
> > > > >>>>> version command was used?
> > > > >>>>>
> > > > >>>>> We clearly want the version number, but we probably also want
> to
> > > > >>>>> know> >> if
> > > > >>>>> this is an official release, or a random SNAPSHOT from a
> branch.
> > > > >>>>> If> >> this
> > > > >>>>> is a release candidate, we probably want the RC number as well,
> > > > >>>>> like> >>>>> "1.1-rc3" We also want a git hash. This can be
> > > injected by the> > build
> > > > >>>>> process. In the case of an official release, where the source
> > > > >>>>> code> >> is not
> > > > >>>>> under git, we can pull it from a file.
> > > > >>>>>
> > > > >>>>> For example, hadoop's version output looks like this:
> > > > >>>>>
> > > > >>>>> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop
> > > > >>>>> > version> >>>>> > Hadoop 2.8.3
> > > > >>>>> > Subversion
> > > > >>>>> > https://git-wip-us.apache.org/repos/asf/hadoop.git -r>
> >>>>>
> > > b3fe56402d908019d99af1f1f4fc65cb1d1436a2
> > > > >>>>> > Compiled by jdu on 2017-12-05T03:43Z
> > > > >>>>> > Compiled with protoc 2.5.0
> > > > >>>>> > From source with checksum 9ff4856d824e983fa510d3f843e3f1
> 9d>
> > > >>>>> > This command was run using /home/cmccabe/Downloads/
> > > > >>>>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
> > > > >>>>>
> > > > >>>>> (The "subversion" line here is a little weird -- it now refers
> > > > >>>>> to> >> git, not
> > > > >>>>> svn)
> > > > >>>>>
> > > > >>>>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
> > > > >>>>>> Hey Sasaki,
> > > > >>>>>>
> > > > >>>>>> Yeah, I don't feel too strongly about only supporting --
> > > > >>>>>> version. I> >> agree
> > > > >>>>> it
> > > > >>>>>> may help discoverability given the current approach. On the
> > > > >>>>>> other> >> hand,
> > > > >>>>> if
> > > > >>>>>> we refactored all of the tools so that we could use a common
> > > > >>>>>> set of> >> base
> > > > >>>>>> options, it might be a little annoying to have to continue
> > > > > supporting
> > > > >>>>> both
> > > > >>>>>> variations. For example, tool standardization was proposed in
> > > > >>>>>> KIP-14> >> and
> > > > >>>>>> I'm still holding out hope that someone will have time to pick
> > > > >>>>>> this> >> work
> > > > >>>>>> back up. It's always easier to add an option than remove one,
> > > > >>>>>> so I'm> >>>>>> slightly inclined to have only --version for
> now.
> > > What do you
> > > > >>>>>> think?> >>>>> The double dash version is more consistent with
> how
> > > our other
> > > > >>>>> flags> >> work.
> > > > >>>>> In general, I feel that if --version is supported, --help
> should
> > > > >>>>> say> >> so.
> > > > >>>>> best,
> > > > >>>>> Colin
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>> Thanks,
> > > > >>>>>> Jason
> > > > >>>>>>
> > > > >>>>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <
> > > > >> sasakitoa@oss.nttdata.com
> > > > >>>>>> wrote:
> > > > >>>>>>
> > > > >>>>>>> Hi Jason
> > > > >>>>>>>
> > > > >>>>>>> Thank you for helpful comments. I updated wiki based on your
> > > > > advice.
> > > > >>>>>>> I thought this option was relatively common and making
> > > > >>>>>>> maintenance> >>>> easy
> > > > >>>>>>> was also important.
> > > > >>>>>>> However, as you said, it is not good that version option
> won't
> > > > >>>>>>> be> >>>>> shown up
> > > > >>>>>>> in help description.
> > > > >>>>>>>
> > > > >>>>>>> I thought accepting both single-dash and double-dash will
> help
> > > > >>>>>>> to> >> find
> > > > >>>>>>> this option.
> > > > >>>>>>> In my approach this option won't be showed, but most of
> > > > >>>>>>> software> >> which
> > > > >>>>> has
> > > > >>>>>>> this option accepts either single-dash or double-dash.
> > > > >>>>>>> I guess it doesn't need to support both if we take another
> > > > >>>>>>> way.> >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> Thanks
> > > > >>>>>>>
> > > > >>>>>>> @Ted Yeah, you're right. Sorry about the confusion.
> > > > >>>>>>>> Since we're here, I think this KIP is a nice improvement.
> > > > >>>>>>>> It's> >>>>> definitely
> > > > >>>>>>>> nice to have an easy way to check the version. That said, do
> > > > >>>>>>>> we> >>>> really
> > > > >>>>>>>> need
> > > > >>>>>>>> to support both `-version` and `--version`? The latter is
> > > > >> consistent
> > > > >>>>> with
> > > > >>>>>>>> our current tools.
> > > > >>>>>>>>
> > > > >>>>>>>> Also, I think the approach we've taken is basically to build
> > > > >>>>>>>> the> >>>>> --version
> > > > >>>>>>>> functionality into the bash script. This is nice because it
> > > > >>>>>>>> saves> > a
> > > > >>>>> lot of
> > > > >>>>>>>> work to update the commands individually and we don't need
> to
> > > > >>>>>>>> do> >>>>> anything
> > > > >>>>>>>> when we add new tools. The downside is that `--version`
> won't
> > > > >>>>>>>> show> >> up
> > > > >>>>> as
> > > > >>>>>>>> an
> > > > >>>>>>>> option in any of the --help output. Not sure if that is too
> > > > >>>>>>>> big of> >> a
> > > > >>>>>>>> problem, but maybe worth mentioning this in the rejected
> > > > >> alternatives
> > > > >>>>>>>> section.
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> -Jason
> > > > >>>>>>>>
> > > > >>>>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yuzhihong@gmail.com
> >>
> > > >> wrote:
> > > > >>>>>>>> Jason:
> > > > >>>>>>>>> Maybe your reply was intended for another KIP ?
> > > > >>>>>>>>>
> > > > >>>>>>>>> KIP-278 is about adding version option, not timeout.
> > > > >>>>>>>>>
> > > > >>>>>>>>> Cheers
> > > > >>>>>>>>>
> > > > >>>>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <
> > > > >> jason@confluent.io>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>> Hi Sasaki,
> > > > >>>>>>>>>> Thanks for the KIP. I think the timeout controls the
> > > > >>>>>>>>>> maximum> >>>> allowed
> > > > >>>>>>>>> time
> > > > >>>>>>>>> that the consumer will block for the next record. Maybe
> the>
> > > >> meaning
> > > > >>>>>>>>> would
> > > > >>>>>>>>> be clearer with the more concise name `--timeout`? That
> also
> > > > >>>>>>>>> fits> >>>>> with
> > > > >>>>>>>>> the
> > > > >>>>>>>>>
> > > > >>>>>>>>>> old consumer which overrides the `consumer.timeout.ms`
> > > > >>>>>>>>>> property.> >>>>>>>>>>
> > > > >>>>>>>>>> By the way, it seems like the default value was
> > > > >>>>>>>>>> intentionally> > set
> > > > >>>>> low
> > > > >>>>>>>>> for
> > > > >>>>>>>>> both the old and new consumers, but I'm not sure of the
> > > > >>>>>>>>> reason.> > We
> > > > >>>>> could
> > > > >>>>>>>>>> leave the default as it is if we want to be safe, but
> > > > >>>>>>>>>> increasing> >> it
> > > > >>>>>>>>> seems
> > > > >>>>>>>>> ok to me. Perhaps we could start a little lower, though,
> say
> > > > >>>>>>>>> 10> >>>>> seconds?
> > > > >>>>>>>>> In
> > > > >>>>>>>>>
> > > > >>>>>>>>>> any case, we should make it clear to the user that the
> > > > >>>>>>>>>> timeout> >> was
> > > > >>>>>>>>> reached.
> > > > >>>>>>>>>
> > > > >>>>>>>>>> It's surprising to see only the incomplete reported
> results>
> > > >>>>> following a
> > > > >>>>>>>>>> timeout.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Thanks,
> > > > >>>>>>>>>> Jason
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
> > > > >>>>> sasakitoa@oss.nttdata.com>
> > > > >>>>>>>>>> wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Hello everyone,
> > > > >>>>>>>>>>> I would like to start a discussion for KIP 278. Cloud
> you> >
> > > please
> > > > >>>>> give
> > > > >>>>>>>>>>> comments and advice ?
> > > > >>>>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > >>>>>>>>>>> 278+-> >>>>>>>>>>> +Add+version+option+to+Kafka%2
> > > 7s+commands>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> JIRA ticket and Pull Request are bellow:
> > > > >>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
> > > > >>>>>>>>>>> <https://github.com/apache/kafka/pull/639>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Many thanks,
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Sasaki
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> --
> > > > >>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA
> > > > >>>>>>>>>>> CORPORATION> >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>> --
> > > > >>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>> --
> > > > >>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > > > >>>
> > > >
> > > > --
> > > > Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > > >
> > >
> > >
>
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Colin McCabe <cm...@apache.org>.
Hi Manikumar,
We already have a tool for getting the Kafka broker API versions, "./bin/kafka-broker-api-versions.sh". It was added as part of KIP-97.
What Saski is proposing here is having a way of getting the version of locally installed Kafka software, which may be different from the server version. Many pieces of software offer a --version flag, and it's always understood to refer to the local version of the software, not a version running somewhere else. The user has no way to get this information now, other than perhaps trying to look at he names of jar files.
cheers,
Colin
On Tue, May 1, 2018, at 08:20, Manikumar wrote:
> I assume the intent of the KIP to find out the Kafka broker version. In
> this case, maybe we should expose
> version using a Kafka request. This will help the remote scripts/tools to
> query the Kafka version.
> scripts (kafka-topics.sh, kafka-configs.sh, etc..) may run from remote
> machines and may use
> older Kafka versions. In this case, current proposal prints on the older
> version.
>
> On Tue, May 1, 2018 at 7:47 PM, Colin McCabe <cm...@apache.org> wrote:
>
> > Thanks, Sasaki.
> >
> > Colin
> >
> > On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote:
> > > Hi Colin, Jason,
> > >
> > > Thank you for your beneficial comment.
> > > I have updated my Pull Request to show git commit hash in version
> > > information.> In my current Pull Request, we cat get the result such
> > below:
> > >
> > > $ bin/kafka-topics.sh --version
> > > (snip)
> > > 2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e)
> > >
> > >
> > > I have also updated to accept double-dash for this option (--
> > > version) only.>
> > >
> > > Many thanks,
> > > Sasaki
> > >
> > > > From: Jason Gustafson <ja...@confluent.io>
> > > > Date: 2018-04-25 9:42 GMT+09:00
> > > > Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> > > > commands> > To: dev <de...@kafka.apache.org>
> > > >
> > > >
> > > > +1 on adding the git commit id to the output. We often encounter
> > > > environments which are testing off of trunk or have modifications on
> > > > top of> > an existing release.
> > > >
> > > > -Jason
> > > >
> > > > On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cm...@apache.org>
> > > > wrote:> >
> > > >> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
> > > >>> Hi Jason, Colin,
> > > >>>
> > > >>> Thank you for your comment, and I'm sorry for late reply.
> > > >>>
> > > >>> > we refactored all of the tools so that we could use a common
> > > >>> > set of> >>> base options,
> > > >>> > it might be a little annoying to have to continue supporting
> > > >>> > both> >>> variations.
> > > >>>
> > > >>> I feel KIP-14 is good idea (I'm sorry, I didn't know about
> > > >>> it), and> >>> I think it's no longer necessary both single-dash and
> > double-
> > > >>> dash are> >>> supported when this KIP will be accepted, as you said.
> > > >>> I revise my Pull Request to support single-dash option only.
> > > >>>
> > > >>> Some my code in kafka-run-class.sh will become unnecessary when
> > > >>> KIP-14> >>> is accepted.
> > > >>> But I will keep this as temporary, because I seem that it's useful>
> > >>> before KIP-14 accepted.
> > > >>>
> > > >>>
> > > >>> > Can you give a little more detail about what would be
> > > >>> > displayed when> >>> the version command was used?
> > > >>>
> > > >>> As Ismael said, the version string is got from
> > > >>> AppInfoParser#getVersion.> >>>
> > > >>> In my Pull Request, we can get the result such as below::
> > > >>>
> > > >>> $ bin/kafka-topics.sh --version
> > > >>> (snip)
> > > >>> Kafka 1.2.0-SNAPSHOT
> > > >> Hi Sasaki,
> > > >>
> > > >> Thanks for the info. Can you add this to the KIP?
> > > >>
> > > >> Also, I really think we should include the git hash.
> > > >>
> > > >> best,
> > > >> Colin
> > > >>
> > > >>
> > > >>>
> > > >>> Many thanks,
> > > >>> Sasaki
> > > >>>
> > > >>>
> > > >>>> From: Ismael Juma <is...@juma.me.uk>
> > > >>>> Date: 2018-04-24 3:44 GMT+09:00
> > > >>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> > > >>>> commands> >>>> To: dev <de...@kafka.apache.org>
> > > >>>>
> > > >>>>
> > > >>>> FYI, the injection via the build process that is mentioned here
> > > > already
> > > >>>> happens. See AppInfoParser.
> > > >>>>
> > > >>>> Ismael
> > > >>>>
> > > >>>> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe
> > > >>>> <cm...@apache.org>> >> wrote:
> > > >>>>> Hi Sasaki,
> > > >>>>>
> > > >>>>> Thanks for the KIP. I think a version flag is a good idea.
> > > >>>>>
> > > >>>>> Can you give a little more detail about what would be displayed
> > > >>>>> when> >> the
> > > >>>>> version command was used?
> > > >>>>>
> > > >>>>> We clearly want the version number, but we probably also want to
> > > >>>>> know> >> if
> > > >>>>> this is an official release, or a random SNAPSHOT from a branch.
> > > >>>>> If> >> this
> > > >>>>> is a release candidate, we probably want the RC number as well,
> > > >>>>> like> >>>>> "1.1-rc3" We also want a git hash. This can be
> > injected by the> > build
> > > >>>>> process. In the case of an official release, where the source
> > > >>>>> code> >> is not
> > > >>>>> under git, we can pull it from a file.
> > > >>>>>
> > > >>>>> For example, hadoop's version output looks like this:
> > > >>>>>
> > > >>>>> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop
> > > >>>>> > version> >>>>> > Hadoop 2.8.3
> > > >>>>> > Subversion
> > > >>>>> > https://git-wip-us.apache.org/repos/asf/hadoop.git -r> >>>>>
> > b3fe56402d908019d99af1f1f4fc65cb1d1436a2
> > > >>>>> > Compiled by jdu on 2017-12-05T03:43Z
> > > >>>>> > Compiled with protoc 2.5.0
> > > >>>>> > From source with checksum 9ff4856d824e983fa510d3f843e3f19d>
> > >>>>> > This command was run using /home/cmccabe/Downloads/
> > > >>>>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
> > > >>>>>
> > > >>>>> (The "subversion" line here is a little weird -- it now refers
> > > >>>>> to> >> git, not
> > > >>>>> svn)
> > > >>>>>
> > > >>>>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
> > > >>>>>> Hey Sasaki,
> > > >>>>>>
> > > >>>>>> Yeah, I don't feel too strongly about only supporting --
> > > >>>>>> version. I> >> agree
> > > >>>>> it
> > > >>>>>> may help discoverability given the current approach. On the
> > > >>>>>> other> >> hand,
> > > >>>>> if
> > > >>>>>> we refactored all of the tools so that we could use a common
> > > >>>>>> set of> >> base
> > > >>>>>> options, it might be a little annoying to have to continue
> > > > supporting
> > > >>>>> both
> > > >>>>>> variations. For example, tool standardization was proposed in
> > > >>>>>> KIP-14> >> and
> > > >>>>>> I'm still holding out hope that someone will have time to pick
> > > >>>>>> this> >> work
> > > >>>>>> back up. It's always easier to add an option than remove one,
> > > >>>>>> so I'm> >>>>>> slightly inclined to have only --version for now.
> > What do you
> > > >>>>>> think?> >>>>> The double dash version is more consistent with how
> > our other
> > > >>>>> flags> >> work.
> > > >>>>> In general, I feel that if --version is supported, --help should
> > > >>>>> say> >> so.
> > > >>>>> best,
> > > >>>>> Colin
> > > >>>>>
> > > >>>>>
> > > >>>>>> Thanks,
> > > >>>>>> Jason
> > > >>>>>>
> > > >>>>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <
> > > >> sasakitoa@oss.nttdata.com
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> Hi Jason
> > > >>>>>>>
> > > >>>>>>> Thank you for helpful comments. I updated wiki based on your
> > > > advice.
> > > >>>>>>> I thought this option was relatively common and making
> > > >>>>>>> maintenance> >>>> easy
> > > >>>>>>> was also important.
> > > >>>>>>> However, as you said, it is not good that version option won't
> > > >>>>>>> be> >>>>> shown up
> > > >>>>>>> in help description.
> > > >>>>>>>
> > > >>>>>>> I thought accepting both single-dash and double-dash will help
> > > >>>>>>> to> >> find
> > > >>>>>>> this option.
> > > >>>>>>> In my approach this option won't be showed, but most of
> > > >>>>>>> software> >> which
> > > >>>>> has
> > > >>>>>>> this option accepts either single-dash or double-dash.
> > > >>>>>>> I guess it doesn't need to support both if we take another
> > > >>>>>>> way.> >>>>>>>
> > > >>>>>>>
> > > >>>>>>> Thanks
> > > >>>>>>>
> > > >>>>>>> @Ted Yeah, you're right. Sorry about the confusion.
> > > >>>>>>>> Since we're here, I think this KIP is a nice improvement.
> > > >>>>>>>> It's> >>>>> definitely
> > > >>>>>>>> nice to have an easy way to check the version. That said, do
> > > >>>>>>>> we> >>>> really
> > > >>>>>>>> need
> > > >>>>>>>> to support both `-version` and `--version`? The latter is
> > > >> consistent
> > > >>>>> with
> > > >>>>>>>> our current tools.
> > > >>>>>>>>
> > > >>>>>>>> Also, I think the approach we've taken is basically to build
> > > >>>>>>>> the> >>>>> --version
> > > >>>>>>>> functionality into the bash script. This is nice because it
> > > >>>>>>>> saves> > a
> > > >>>>> lot of
> > > >>>>>>>> work to update the commands individually and we don't need to
> > > >>>>>>>> do> >>>>> anything
> > > >>>>>>>> when we add new tools. The downside is that `--version` won't
> > > >>>>>>>> show> >> up
> > > >>>>> as
> > > >>>>>>>> an
> > > >>>>>>>> option in any of the --help output. Not sure if that is too
> > > >>>>>>>> big of> >> a
> > > >>>>>>>> problem, but maybe worth mentioning this in the rejected
> > > >> alternatives
> > > >>>>>>>> section.
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> -Jason
> > > >>>>>>>>
> > > >>>>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yu...@gmail.com>>
> > >> wrote:
> > > >>>>>>>> Jason:
> > > >>>>>>>>> Maybe your reply was intended for another KIP ?
> > > >>>>>>>>>
> > > >>>>>>>>> KIP-278 is about adding version option, not timeout.
> > > >>>>>>>>>
> > > >>>>>>>>> Cheers
> > > >>>>>>>>>
> > > >>>>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <
> > > >> jason@confluent.io>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>> Hi Sasaki,
> > > >>>>>>>>>> Thanks for the KIP. I think the timeout controls the
> > > >>>>>>>>>> maximum> >>>> allowed
> > > >>>>>>>>> time
> > > >>>>>>>>> that the consumer will block for the next record. Maybe the>
> > >> meaning
> > > >>>>>>>>> would
> > > >>>>>>>>> be clearer with the more concise name `--timeout`? That also
> > > >>>>>>>>> fits> >>>>> with
> > > >>>>>>>>> the
> > > >>>>>>>>>
> > > >>>>>>>>>> old consumer which overrides the `consumer.timeout.ms`
> > > >>>>>>>>>> property.> >>>>>>>>>>
> > > >>>>>>>>>> By the way, it seems like the default value was
> > > >>>>>>>>>> intentionally> > set
> > > >>>>> low
> > > >>>>>>>>> for
> > > >>>>>>>>> both the old and new consumers, but I'm not sure of the
> > > >>>>>>>>> reason.> > We
> > > >>>>> could
> > > >>>>>>>>>> leave the default as it is if we want to be safe, but
> > > >>>>>>>>>> increasing> >> it
> > > >>>>>>>>> seems
> > > >>>>>>>>> ok to me. Perhaps we could start a little lower, though, say
> > > >>>>>>>>> 10> >>>>> seconds?
> > > >>>>>>>>> In
> > > >>>>>>>>>
> > > >>>>>>>>>> any case, we should make it clear to the user that the
> > > >>>>>>>>>> timeout> >> was
> > > >>>>>>>>> reached.
> > > >>>>>>>>>
> > > >>>>>>>>>> It's surprising to see only the incomplete reported results>
> > >>>>> following a
> > > >>>>>>>>>> timeout.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Thanks,
> > > >>>>>>>>>> Jason
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
> > > >>>>> sasakitoa@oss.nttdata.com>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>> Hello everyone,
> > > >>>>>>>>>>> I would like to start a discussion for KIP 278. Cloud you> >
> > please
> > > >>>>> give
> > > >>>>>>>>>>> comments and advice ?
> > > >>>>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > >>>>>>>>>>> 278+-> >>>>>>>>>>> +Add+version+option+to+Kafka%2
> > 7s+commands>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> JIRA ticket and Pull Request are bellow:
> > > >>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
> > > >>>>>>>>>>> <https://github.com/apache/kafka/pull/639>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Many thanks,
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Sasaki
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> --
> > > >>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA
> > > >>>>>>>>>>> CORPORATION> >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>> --
> > > >>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > > >>>>>>>
> > > >>>>>>>
> > > >>> --
> > > >>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > > >>>
> > >
> > > --
> > > Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > >
> >
> >
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Manikumar <ma...@gmail.com>.
I assume the intent of the KIP to find out the Kafka broker version. In
this case, maybe we should expose
version using a Kafka request. This will help the remote scripts/tools to
query the Kafka version.
scripts (kafka-topics.sh, kafka-configs.sh, etc..) may run from remote
machines and may use
older Kafka versions. In this case, current proposal prints on the older
version.
On Tue, May 1, 2018 at 7:47 PM, Colin McCabe <cm...@apache.org> wrote:
> Thanks, Sasaki.
>
> Colin
>
> On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote:
> > Hi Colin, Jason,
> >
> > Thank you for your beneficial comment.
> > I have updated my Pull Request to show git commit hash in version
> > information.> In my current Pull Request, we cat get the result such
> below:
> >
> > $ bin/kafka-topics.sh --version
> > (snip)
> > 2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e)
> >
> >
> > I have also updated to accept double-dash for this option (--
> > version) only.>
> >
> > Many thanks,
> > Sasaki
> >
> > > From: Jason Gustafson <ja...@confluent.io>
> > > Date: 2018-04-25 9:42 GMT+09:00
> > > Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> > > commands> > To: dev <de...@kafka.apache.org>
> > >
> > >
> > > +1 on adding the git commit id to the output. We often encounter
> > > environments which are testing off of trunk or have modifications on
> > > top of> > an existing release.
> > >
> > > -Jason
> > >
> > > On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cm...@apache.org>
> > > wrote:> >
> > >> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
> > >>> Hi Jason, Colin,
> > >>>
> > >>> Thank you for your comment, and I'm sorry for late reply.
> > >>>
> > >>> > we refactored all of the tools so that we could use a common
> > >>> > set of> >>> base options,
> > >>> > it might be a little annoying to have to continue supporting
> > >>> > both> >>> variations.
> > >>>
> > >>> I feel KIP-14 is good idea (I'm sorry, I didn't know about
> > >>> it), and> >>> I think it's no longer necessary both single-dash and
> double-
> > >>> dash are> >>> supported when this KIP will be accepted, as you said.
> > >>> I revise my Pull Request to support single-dash option only.
> > >>>
> > >>> Some my code in kafka-run-class.sh will become unnecessary when
> > >>> KIP-14> >>> is accepted.
> > >>> But I will keep this as temporary, because I seem that it's useful>
> >>> before KIP-14 accepted.
> > >>>
> > >>>
> > >>> > Can you give a little more detail about what would be
> > >>> > displayed when> >>> the version command was used?
> > >>>
> > >>> As Ismael said, the version string is got from
> > >>> AppInfoParser#getVersion.> >>>
> > >>> In my Pull Request, we can get the result such as below::
> > >>>
> > >>> $ bin/kafka-topics.sh --version
> > >>> (snip)
> > >>> Kafka 1.2.0-SNAPSHOT
> > >> Hi Sasaki,
> > >>
> > >> Thanks for the info. Can you add this to the KIP?
> > >>
> > >> Also, I really think we should include the git hash.
> > >>
> > >> best,
> > >> Colin
> > >>
> > >>
> > >>>
> > >>> Many thanks,
> > >>> Sasaki
> > >>>
> > >>>
> > >>>> From: Ismael Juma <is...@juma.me.uk>
> > >>>> Date: 2018-04-24 3:44 GMT+09:00
> > >>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> > >>>> commands> >>>> To: dev <de...@kafka.apache.org>
> > >>>>
> > >>>>
> > >>>> FYI, the injection via the build process that is mentioned here
> > > already
> > >>>> happens. See AppInfoParser.
> > >>>>
> > >>>> Ismael
> > >>>>
> > >>>> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe
> > >>>> <cm...@apache.org>> >> wrote:
> > >>>>> Hi Sasaki,
> > >>>>>
> > >>>>> Thanks for the KIP. I think a version flag is a good idea.
> > >>>>>
> > >>>>> Can you give a little more detail about what would be displayed
> > >>>>> when> >> the
> > >>>>> version command was used?
> > >>>>>
> > >>>>> We clearly want the version number, but we probably also want to
> > >>>>> know> >> if
> > >>>>> this is an official release, or a random SNAPSHOT from a branch.
> > >>>>> If> >> this
> > >>>>> is a release candidate, we probably want the RC number as well,
> > >>>>> like> >>>>> "1.1-rc3" We also want a git hash. This can be
> injected by the> > build
> > >>>>> process. In the case of an official release, where the source
> > >>>>> code> >> is not
> > >>>>> under git, we can pull it from a file.
> > >>>>>
> > >>>>> For example, hadoop's version output looks like this:
> > >>>>>
> > >>>>> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop
> > >>>>> > version> >>>>> > Hadoop 2.8.3
> > >>>>> > Subversion
> > >>>>> > https://git-wip-us.apache.org/repos/asf/hadoop.git -r> >>>>>
> b3fe56402d908019d99af1f1f4fc65cb1d1436a2
> > >>>>> > Compiled by jdu on 2017-12-05T03:43Z
> > >>>>> > Compiled with protoc 2.5.0
> > >>>>> > From source with checksum 9ff4856d824e983fa510d3f843e3f19d>
> >>>>> > This command was run using /home/cmccabe/Downloads/
> > >>>>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
> > >>>>>
> > >>>>> (The "subversion" line here is a little weird -- it now refers
> > >>>>> to> >> git, not
> > >>>>> svn)
> > >>>>>
> > >>>>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
> > >>>>>> Hey Sasaki,
> > >>>>>>
> > >>>>>> Yeah, I don't feel too strongly about only supporting --
> > >>>>>> version. I> >> agree
> > >>>>> it
> > >>>>>> may help discoverability given the current approach. On the
> > >>>>>> other> >> hand,
> > >>>>> if
> > >>>>>> we refactored all of the tools so that we could use a common
> > >>>>>> set of> >> base
> > >>>>>> options, it might be a little annoying to have to continue
> > > supporting
> > >>>>> both
> > >>>>>> variations. For example, tool standardization was proposed in
> > >>>>>> KIP-14> >> and
> > >>>>>> I'm still holding out hope that someone will have time to pick
> > >>>>>> this> >> work
> > >>>>>> back up. It's always easier to add an option than remove one,
> > >>>>>> so I'm> >>>>>> slightly inclined to have only --version for now.
> What do you
> > >>>>>> think?> >>>>> The double dash version is more consistent with how
> our other
> > >>>>> flags> >> work.
> > >>>>> In general, I feel that if --version is supported, --help should
> > >>>>> say> >> so.
> > >>>>> best,
> > >>>>> Colin
> > >>>>>
> > >>>>>
> > >>>>>> Thanks,
> > >>>>>> Jason
> > >>>>>>
> > >>>>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <
> > >> sasakitoa@oss.nttdata.com
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Hi Jason
> > >>>>>>>
> > >>>>>>> Thank you for helpful comments. I updated wiki based on your
> > > advice.
> > >>>>>>> I thought this option was relatively common and making
> > >>>>>>> maintenance> >>>> easy
> > >>>>>>> was also important.
> > >>>>>>> However, as you said, it is not good that version option won't
> > >>>>>>> be> >>>>> shown up
> > >>>>>>> in help description.
> > >>>>>>>
> > >>>>>>> I thought accepting both single-dash and double-dash will help
> > >>>>>>> to> >> find
> > >>>>>>> this option.
> > >>>>>>> In my approach this option won't be showed, but most of
> > >>>>>>> software> >> which
> > >>>>> has
> > >>>>>>> this option accepts either single-dash or double-dash.
> > >>>>>>> I guess it doesn't need to support both if we take another
> > >>>>>>> way.> >>>>>>>
> > >>>>>>>
> > >>>>>>> Thanks
> > >>>>>>>
> > >>>>>>> @Ted Yeah, you're right. Sorry about the confusion.
> > >>>>>>>> Since we're here, I think this KIP is a nice improvement.
> > >>>>>>>> It's> >>>>> definitely
> > >>>>>>>> nice to have an easy way to check the version. That said, do
> > >>>>>>>> we> >>>> really
> > >>>>>>>> need
> > >>>>>>>> to support both `-version` and `--version`? The latter is
> > >> consistent
> > >>>>> with
> > >>>>>>>> our current tools.
> > >>>>>>>>
> > >>>>>>>> Also, I think the approach we've taken is basically to build
> > >>>>>>>> the> >>>>> --version
> > >>>>>>>> functionality into the bash script. This is nice because it
> > >>>>>>>> saves> > a
> > >>>>> lot of
> > >>>>>>>> work to update the commands individually and we don't need to
> > >>>>>>>> do> >>>>> anything
> > >>>>>>>> when we add new tools. The downside is that `--version` won't
> > >>>>>>>> show> >> up
> > >>>>> as
> > >>>>>>>> an
> > >>>>>>>> option in any of the --help output. Not sure if that is too
> > >>>>>>>> big of> >> a
> > >>>>>>>> problem, but maybe worth mentioning this in the rejected
> > >> alternatives
> > >>>>>>>> section.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> -Jason
> > >>>>>>>>
> > >>>>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yu...@gmail.com>>
> >> wrote:
> > >>>>>>>> Jason:
> > >>>>>>>>> Maybe your reply was intended for another KIP ?
> > >>>>>>>>>
> > >>>>>>>>> KIP-278 is about adding version option, not timeout.
> > >>>>>>>>>
> > >>>>>>>>> Cheers
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <
> > >> jason@confluent.io>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Hi Sasaki,
> > >>>>>>>>>> Thanks for the KIP. I think the timeout controls the
> > >>>>>>>>>> maximum> >>>> allowed
> > >>>>>>>>> time
> > >>>>>>>>> that the consumer will block for the next record. Maybe the>
> >> meaning
> > >>>>>>>>> would
> > >>>>>>>>> be clearer with the more concise name `--timeout`? That also
> > >>>>>>>>> fits> >>>>> with
> > >>>>>>>>> the
> > >>>>>>>>>
> > >>>>>>>>>> old consumer which overrides the `consumer.timeout.ms`
> > >>>>>>>>>> property.> >>>>>>>>>>
> > >>>>>>>>>> By the way, it seems like the default value was
> > >>>>>>>>>> intentionally> > set
> > >>>>> low
> > >>>>>>>>> for
> > >>>>>>>>> both the old and new consumers, but I'm not sure of the
> > >>>>>>>>> reason.> > We
> > >>>>> could
> > >>>>>>>>>> leave the default as it is if we want to be safe, but
> > >>>>>>>>>> increasing> >> it
> > >>>>>>>>> seems
> > >>>>>>>>> ok to me. Perhaps we could start a little lower, though, say
> > >>>>>>>>> 10> >>>>> seconds?
> > >>>>>>>>> In
> > >>>>>>>>>
> > >>>>>>>>>> any case, we should make it clear to the user that the
> > >>>>>>>>>> timeout> >> was
> > >>>>>>>>> reached.
> > >>>>>>>>>
> > >>>>>>>>>> It's surprising to see only the incomplete reported results>
> >>>>> following a
> > >>>>>>>>>> timeout.
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks,
> > >>>>>>>>>> Jason
> > >>>>>>>>>>
> > >>>>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
> > >>>>> sasakitoa@oss.nttdata.com>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Hello everyone,
> > >>>>>>>>>>> I would like to start a discussion for KIP 278. Cloud you> >
> please
> > >>>>> give
> > >>>>>>>>>>> comments and advice ?
> > >>>>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >>>>>>>>>>> 278+-> >>>>>>>>>>> +Add+version+option+to+Kafka%2
> 7s+commands>
> > >>>>>>>>>>>
> > >>>>>>>>>>> JIRA ticket and Pull Request are bellow:
> > >>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
> > >>>>>>>>>>> <https://github.com/apache/kafka/pull/639>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> Many thanks,
> > >>>>>>>>>>>
> > >>>>>>>>>>> Sasaki
> > >>>>>>>>>>>
> > >>>>>>>>>>> --
> > >>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA
> > >>>>>>>>>>> CORPORATION> >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>> --
> > >>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > >>>>>>>
> > >>>>>>>
> > >>> --
> > >>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > >>>
> >
> > --
> > Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >
>
>
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Colin McCabe <cm...@apache.org>.
Thanks, Sasaki.
Colin
On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote:
> Hi Colin, Jason,
>
> Thank you for your beneficial comment.
> I have updated my Pull Request to show git commit hash in version
> information.> In my current Pull Request, we cat get the result such below:
>
> $ bin/kafka-topics.sh --version
> (snip)
> 2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e)
>
>
> I have also updated to accept double-dash for this option (--
> version) only.>
>
> Many thanks,
> Sasaki
>
> > From: Jason Gustafson <ja...@confluent.io>
> > Date: 2018-04-25 9:42 GMT+09:00
> > Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> > commands> > To: dev <de...@kafka.apache.org>
> >
> >
> > +1 on adding the git commit id to the output. We often encounter
> > environments which are testing off of trunk or have modifications on
> > top of> > an existing release.
> >
> > -Jason
> >
> > On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cm...@apache.org>
> > wrote:> >
> >> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
> >>> Hi Jason, Colin,
> >>>
> >>> Thank you for your comment, and I'm sorry for late reply.
> >>>
> >>> > we refactored all of the tools so that we could use a common
> >>> > set of> >>> base options,
> >>> > it might be a little annoying to have to continue supporting
> >>> > both> >>> variations.
> >>>
> >>> I feel KIP-14 is good idea (I'm sorry, I didn't know about
> >>> it), and> >>> I think it's no longer necessary both single-dash and double-
> >>> dash are> >>> supported when this KIP will be accepted, as you said.
> >>> I revise my Pull Request to support single-dash option only.
> >>>
> >>> Some my code in kafka-run-class.sh will become unnecessary when
> >>> KIP-14> >>> is accepted.
> >>> But I will keep this as temporary, because I seem that it's useful> >>> before KIP-14 accepted.
> >>>
> >>>
> >>> > Can you give a little more detail about what would be
> >>> > displayed when> >>> the version command was used?
> >>>
> >>> As Ismael said, the version string is got from
> >>> AppInfoParser#getVersion.> >>>
> >>> In my Pull Request, we can get the result such as below::
> >>>
> >>> $ bin/kafka-topics.sh --version
> >>> (snip)
> >>> Kafka 1.2.0-SNAPSHOT
> >> Hi Sasaki,
> >>
> >> Thanks for the info. Can you add this to the KIP?
> >>
> >> Also, I really think we should include the git hash.
> >>
> >> best,
> >> Colin
> >>
> >>
> >>>
> >>> Many thanks,
> >>> Sasaki
> >>>
> >>>
> >>>> From: Ismael Juma <is...@juma.me.uk>
> >>>> Date: 2018-04-24 3:44 GMT+09:00
> >>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> >>>> commands> >>>> To: dev <de...@kafka.apache.org>
> >>>>
> >>>>
> >>>> FYI, the injection via the build process that is mentioned here
> > already
> >>>> happens. See AppInfoParser.
> >>>>
> >>>> Ismael
> >>>>
> >>>> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe
> >>>> <cm...@apache.org>> >> wrote:
> >>>>> Hi Sasaki,
> >>>>>
> >>>>> Thanks for the KIP. I think a version flag is a good idea.
> >>>>>
> >>>>> Can you give a little more detail about what would be displayed
> >>>>> when> >> the
> >>>>> version command was used?
> >>>>>
> >>>>> We clearly want the version number, but we probably also want to
> >>>>> know> >> if
> >>>>> this is an official release, or a random SNAPSHOT from a branch.
> >>>>> If> >> this
> >>>>> is a release candidate, we probably want the RC number as well,
> >>>>> like> >>>>> "1.1-rc3" We also want a git hash. This can be injected by the> > build
> >>>>> process. In the case of an official release, where the source
> >>>>> code> >> is not
> >>>>> under git, we can pull it from a file.
> >>>>>
> >>>>> For example, hadoop's version output looks like this:
> >>>>>
> >>>>> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop
> >>>>> > version> >>>>> > Hadoop 2.8.3
> >>>>> > Subversion
> >>>>> > https://git-wip-us.apache.org/repos/asf/hadoop.git -r> >>>>> b3fe56402d908019d99af1f1f4fc65cb1d1436a2
> >>>>> > Compiled by jdu on 2017-12-05T03:43Z
> >>>>> > Compiled with protoc 2.5.0
> >>>>> > From source with checksum 9ff4856d824e983fa510d3f843e3f19d> >>>>> > This command was run using /home/cmccabe/Downloads/
> >>>>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
> >>>>>
> >>>>> (The "subversion" line here is a little weird -- it now refers
> >>>>> to> >> git, not
> >>>>> svn)
> >>>>>
> >>>>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
> >>>>>> Hey Sasaki,
> >>>>>>
> >>>>>> Yeah, I don't feel too strongly about only supporting --
> >>>>>> version. I> >> agree
> >>>>> it
> >>>>>> may help discoverability given the current approach. On the
> >>>>>> other> >> hand,
> >>>>> if
> >>>>>> we refactored all of the tools so that we could use a common
> >>>>>> set of> >> base
> >>>>>> options, it might be a little annoying to have to continue
> > supporting
> >>>>> both
> >>>>>> variations. For example, tool standardization was proposed in
> >>>>>> KIP-14> >> and
> >>>>>> I'm still holding out hope that someone will have time to pick
> >>>>>> this> >> work
> >>>>>> back up. It's always easier to add an option than remove one,
> >>>>>> so I'm> >>>>>> slightly inclined to have only --version for now. What do you
> >>>>>> think?> >>>>> The double dash version is more consistent with how our other
> >>>>> flags> >> work.
> >>>>> In general, I feel that if --version is supported, --help should
> >>>>> say> >> so.
> >>>>> best,
> >>>>> Colin
> >>>>>
> >>>>>
> >>>>>> Thanks,
> >>>>>> Jason
> >>>>>>
> >>>>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <
> >> sasakitoa@oss.nttdata.com
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi Jason
> >>>>>>>
> >>>>>>> Thank you for helpful comments. I updated wiki based on your
> > advice.
> >>>>>>> I thought this option was relatively common and making
> >>>>>>> maintenance> >>>> easy
> >>>>>>> was also important.
> >>>>>>> However, as you said, it is not good that version option won't
> >>>>>>> be> >>>>> shown up
> >>>>>>> in help description.
> >>>>>>>
> >>>>>>> I thought accepting both single-dash and double-dash will help
> >>>>>>> to> >> find
> >>>>>>> this option.
> >>>>>>> In my approach this option won't be showed, but most of
> >>>>>>> software> >> which
> >>>>> has
> >>>>>>> this option accepts either single-dash or double-dash.
> >>>>>>> I guess it doesn't need to support both if we take another
> >>>>>>> way.> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>>
> >>>>>>> @Ted Yeah, you're right. Sorry about the confusion.
> >>>>>>>> Since we're here, I think this KIP is a nice improvement.
> >>>>>>>> It's> >>>>> definitely
> >>>>>>>> nice to have an easy way to check the version. That said, do
> >>>>>>>> we> >>>> really
> >>>>>>>> need
> >>>>>>>> to support both `-version` and `--version`? The latter is
> >> consistent
> >>>>> with
> >>>>>>>> our current tools.
> >>>>>>>>
> >>>>>>>> Also, I think the approach we've taken is basically to build
> >>>>>>>> the> >>>>> --version
> >>>>>>>> functionality into the bash script. This is nice because it
> >>>>>>>> saves> > a
> >>>>> lot of
> >>>>>>>> work to update the commands individually and we don't need to
> >>>>>>>> do> >>>>> anything
> >>>>>>>> when we add new tools. The downside is that `--version` won't
> >>>>>>>> show> >> up
> >>>>> as
> >>>>>>>> an
> >>>>>>>> option in any of the --help output. Not sure if that is too
> >>>>>>>> big of> >> a
> >>>>>>>> problem, but maybe worth mentioning this in the rejected
> >> alternatives
> >>>>>>>> section.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> -Jason
> >>>>>>>>
> >>>>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yu...@gmail.com>> >> wrote:
> >>>>>>>> Jason:
> >>>>>>>>> Maybe your reply was intended for another KIP ?
> >>>>>>>>>
> >>>>>>>>> KIP-278 is about adding version option, not timeout.
> >>>>>>>>>
> >>>>>>>>> Cheers
> >>>>>>>>>
> >>>>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <
> >> jason@confluent.io>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi Sasaki,
> >>>>>>>>>> Thanks for the KIP. I think the timeout controls the
> >>>>>>>>>> maximum> >>>> allowed
> >>>>>>>>> time
> >>>>>>>>> that the consumer will block for the next record. Maybe the> >> meaning
> >>>>>>>>> would
> >>>>>>>>> be clearer with the more concise name `--timeout`? That also
> >>>>>>>>> fits> >>>>> with
> >>>>>>>>> the
> >>>>>>>>>
> >>>>>>>>>> old consumer which overrides the `consumer.timeout.ms`
> >>>>>>>>>> property.> >>>>>>>>>>
> >>>>>>>>>> By the way, it seems like the default value was
> >>>>>>>>>> intentionally> > set
> >>>>> low
> >>>>>>>>> for
> >>>>>>>>> both the old and new consumers, but I'm not sure of the
> >>>>>>>>> reason.> > We
> >>>>> could
> >>>>>>>>>> leave the default as it is if we want to be safe, but
> >>>>>>>>>> increasing> >> it
> >>>>>>>>> seems
> >>>>>>>>> ok to me. Perhaps we could start a little lower, though, say
> >>>>>>>>> 10> >>>>> seconds?
> >>>>>>>>> In
> >>>>>>>>>
> >>>>>>>>>> any case, we should make it clear to the user that the
> >>>>>>>>>> timeout> >> was
> >>>>>>>>> reached.
> >>>>>>>>>
> >>>>>>>>>> It's surprising to see only the incomplete reported results> >>>>> following a
> >>>>>>>>>> timeout.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Jason
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
> >>>>> sasakitoa@oss.nttdata.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hello everyone,
> >>>>>>>>>>> I would like to start a discussion for KIP 278. Cloud you> > please
> >>>>> give
> >>>>>>>>>>> comments and advice ?
> >>>>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>>>>>>>>>> 278+-> >>>>>>>>>>> +Add+version+option+to+Kafka%27s+commands>
> >>>>>>>>>>>
> >>>>>>>>>>> JIRA ticket and Pull Request are bellow:
> >>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
> >>>>>>>>>>> <https://github.com/apache/kafka/pull/639>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Many thanks,
> >>>>>>>>>>>
> >>>>>>>>>>> Sasaki
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA
> >>>>>>>>>>> CORPORATION> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>> --
> >>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >>>>>>>
> >>>>>>>
> >>> --
> >>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >>>
>
> --
> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Sasaki Toru <sa...@oss.nttdata.com>.
Hi Colin, Jason,
Thank you for your beneficial comment.
I have updated my Pull Request to show git commit hash in version information.
In my current Pull Request, we cat get the result such below:
$ bin/kafka-topics.sh --version
(snip)
2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e)
I have also updated to accept double-dash for this option (--version) only.
Many thanks,
Sasaki
> From: Jason Gustafson <ja...@confluent.io>
> Date: 2018-04-25 9:42 GMT+09:00
> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
> To: dev <de...@kafka.apache.org>
>
>
> +1 on adding the git commit id to the output. We often encounter
> environments which are testing off of trunk or have modifications on top of
> an existing release.
>
> -Jason
>
> On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cm...@apache.org> wrote:
>
>> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
>>> Hi Jason, Colin,
>>>
>>> Thank you for your comment, and I'm sorry for late reply.
>>>
>>> > we refactored all of the tools so that we could use a common set of
>>> base options,
>>> > it might be a little annoying to have to continue supporting both
>>> variations.
>>>
>>> I feel KIP-14 is good idea (I'm sorry, I didn't know about it), and
>>> I think it's no longer necessary both single-dash and double-dash are
>>> supported when this KIP will be accepted, as you said.
>>> I revise my Pull Request to support single-dash option only.
>>>
>>> Some my code in kafka-run-class.sh will become unnecessary when KIP-14
>>> is accepted.
>>> But I will keep this as temporary, because I seem that it's useful
>>> before KIP-14 accepted.
>>>
>>>
>>> > Can you give a little more detail about what would be displayed when
>>> the version command was used?
>>>
>>> As Ismael said, the version string is got from AppInfoParser#getVersion.
>>>
>>> In my Pull Request, we can get the result such as below::
>>>
>>> $ bin/kafka-topics.sh --version
>>> (snip)
>>> Kafka 1.2.0-SNAPSHOT
>> Hi Sasaki,
>>
>> Thanks for the info. Can you add this to the KIP?
>>
>> Also, I really think we should include the git hash.
>>
>> best,
>> Colin
>>
>>
>>>
>>> Many thanks,
>>> Sasaki
>>>
>>>
>>>> From: Ismael Juma <is...@juma.me.uk>
>>>> Date: 2018-04-24 3:44 GMT+09:00
>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
>>>> To: dev <de...@kafka.apache.org>
>>>>
>>>>
>>>> FYI, the injection via the build process that is mentioned here
> already
>>>> happens. See AppInfoParser.
>>>>
>>>> Ismael
>>>>
>>>> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe <cm...@apache.org>
>> wrote:
>>>>> Hi Sasaki,
>>>>>
>>>>> Thanks for the KIP. I think a version flag is a good idea.
>>>>>
>>>>> Can you give a little more detail about what would be displayed when
>> the
>>>>> version command was used?
>>>>>
>>>>> We clearly want the version number, but we probably also want to know
>> if
>>>>> this is an official release, or a random SNAPSHOT from a branch. If
>> this
>>>>> is a release candidate, we probably want the RC number as well, like
>>>>> "1.1-rc3" We also want a git hash. This can be injected by the
> build
>>>>> process. In the case of an official release, where the source code
>> is not
>>>>> under git, we can pull it from a file.
>>>>>
>>>>> For example, hadoop's version output looks like this:
>>>>>
>>>>> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop version
>>>>> > Hadoop 2.8.3
>>>>> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
>>>>> b3fe56402d908019d99af1f1f4fc65cb1d1436a2
>>>>> > Compiled by jdu on 2017-12-05T03:43Z
>>>>> > Compiled with protoc 2.5.0
>>>>> > From source with checksum 9ff4856d824e983fa510d3f843e3f19d
>>>>> > This command was run using /home/cmccabe/Downloads/
>>>>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
>>>>>
>>>>> (The "subversion" line here is a little weird -- it now refers to
>> git, not
>>>>> svn)
>>>>>
>>>>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
>>>>>> Hey Sasaki,
>>>>>>
>>>>>> Yeah, I don't feel too strongly about only supporting --version. I
>> agree
>>>>> it
>>>>>> may help discoverability given the current approach. On the other
>> hand,
>>>>> if
>>>>>> we refactored all of the tools so that we could use a common set of
>> base
>>>>>> options, it might be a little annoying to have to continue
> supporting
>>>>> both
>>>>>> variations. For example, tool standardization was proposed in KIP-14
>> and
>>>>>> I'm still holding out hope that someone will have time to pick this
>> work
>>>>>> back up. It's always easier to add an option than remove one, so I'm
>>>>>> slightly inclined to have only --version for now. What do you think?
>>>>> The double dash version is more consistent with how our other flags
>> work.
>>>>> In general, I feel that if --version is supported, --help should say
>> so.
>>>>> best,
>>>>> Colin
>>>>>
>>>>>
>>>>>> Thanks,
>>>>>> Jason
>>>>>>
>>>>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <
>> sasakitoa@oss.nttdata.com
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Jason
>>>>>>>
>>>>>>> Thank you for helpful comments. I updated wiki based on your
> advice.
>>>>>>> I thought this option was relatively common and making maintenance
>>>> easy
>>>>>>> was also important.
>>>>>>> However, as you said, it is not good that version option won't be
>>>>> shown up
>>>>>>> in help description.
>>>>>>>
>>>>>>> I thought accepting both single-dash and double-dash will help to
>> find
>>>>>>> this option.
>>>>>>> In my approach this option won't be showed, but most of software
>> which
>>>>> has
>>>>>>> this option accepts either single-dash or double-dash.
>>>>>>> I guess it doesn't need to support both if we take another way.
>>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> @Ted Yeah, you're right. Sorry about the confusion.
>>>>>>>> Since we're here, I think this KIP is a nice improvement. It's
>>>>> definitely
>>>>>>>> nice to have an easy way to check the version. That said, do we
>>>> really
>>>>>>>> need
>>>>>>>> to support both `-version` and `--version`? The latter is
>> consistent
>>>>> with
>>>>>>>> our current tools.
>>>>>>>>
>>>>>>>> Also, I think the approach we've taken is basically to build the
>>>>> --version
>>>>>>>> functionality into the bash script. This is nice because it saves
> a
>>>>> lot of
>>>>>>>> work to update the commands individually and we don't need to do
>>>>> anything
>>>>>>>> when we add new tools. The downside is that `--version` won't show
>> up
>>>>> as
>>>>>>>> an
>>>>>>>> option in any of the --help output. Not sure if that is too big of
>> a
>>>>>>>> problem, but maybe worth mentioning this in the rejected
>> alternatives
>>>>>>>> section.
>>>>>>>>
>>>>>>>>
>>>>>>>> -Jason
>>>>>>>>
>>>>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yu...@gmail.com>
>> wrote:
>>>>>>>> Jason:
>>>>>>>>> Maybe your reply was intended for another KIP ?
>>>>>>>>>
>>>>>>>>> KIP-278 is about adding version option, not timeout.
>>>>>>>>>
>>>>>>>>> Cheers
>>>>>>>>>
>>>>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <
>> jason@confluent.io>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi Sasaki,
>>>>>>>>>> Thanks for the KIP. I think the timeout controls the maximum
>>>> allowed
>>>>>>>>> time
>>>>>>>>> that the consumer will block for the next record. Maybe the
>> meaning
>>>>>>>>> would
>>>>>>>>> be clearer with the more concise name `--timeout`? That also fits
>>>>> with
>>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> old consumer which overrides the `consumer.timeout.ms` property.
>>>>>>>>>>
>>>>>>>>>> By the way, it seems like the default value was intentionally
> set
>>>>> low
>>>>>>>>> for
>>>>>>>>> both the old and new consumers, but I'm not sure of the reason.
> We
>>>>> could
>>>>>>>>>> leave the default as it is if we want to be safe, but increasing
>> it
>>>>>>>>> seems
>>>>>>>>> ok to me. Perhaps we could start a little lower, though, say 10
>>>>> seconds?
>>>>>>>>> In
>>>>>>>>>
>>>>>>>>>> any case, we should make it clear to the user that the timeout
>> was
>>>>>>>>> reached.
>>>>>>>>>
>>>>>>>>>> It's surprising to see only the incomplete reported results
>>>>> following a
>>>>>>>>>> timeout.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Jason
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
>>>>> sasakitoa@oss.nttdata.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello everyone,
>>>>>>>>>>> I would like to start a discussion for KIP 278. Cloud you
> please
>>>>> give
>>>>>>>>>>> comments and advice ?
>>>>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-278+-
>>>>>>>>>>> +Add+version+option+to+Kafka%27s+commands>
>>>>>>>>>>>
>>>>>>>>>>> JIRA ticket and Pull Request are bellow:
>>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
>>>>>>>>>>> <https://github.com/apache/kafka/pull/639>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Many thanks,
>>>>>>>>>>>
>>>>>>>>>>> Sasaki
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>> --
>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>>>
>>>>>>>
>>> --
>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>
--
Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Jason Gustafson <ja...@confluent.io>.
+1 on adding the git commit id to the output. We often encounter
environments which are testing off of trunk or have modifications on top of
an existing release.
-Jason
On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cm...@apache.org> wrote:
> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
> > Hi Jason, Colin,
> >
> > Thank you for your comment, and I'm sorry for late reply.
> >
> > > we refactored all of the tools so that we could use a common set of
> > base options,
> > > it might be a little annoying to have to continue supporting both
> > variations.
> >
> > I feel KIP-14 is good idea (I'm sorry, I didn't know about it), and
> > I think it's no longer necessary both single-dash and double-dash are
> > supported when this KIP will be accepted, as you said.
> > I revise my Pull Request to support single-dash option only.
> >
> > Some my code in kafka-run-class.sh will become unnecessary when KIP-14
> > is accepted.
> > But I will keep this as temporary, because I seem that it's useful
> > before KIP-14 accepted.
> >
> >
> > > Can you give a little more detail about what would be displayed when
> > the version command was used?
> >
> > As Ismael said, the version string is got from AppInfoParser#getVersion.
> >
> > In my Pull Request, we can get the result such as below::
> >
> > $ bin/kafka-topics.sh --version
> > (snip)
> > Kafka 1.2.0-SNAPSHOT
>
> Hi Sasaki,
>
> Thanks for the info. Can you add this to the KIP?
>
> Also, I really think we should include the git hash.
>
> best,
> Colin
>
>
> >
> >
> > Many thanks,
> > Sasaki
> >
> >
> > > From: Ismael Juma <is...@juma.me.uk>
> > > Date: 2018-04-24 3:44 GMT+09:00
> > > Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
> > > To: dev <de...@kafka.apache.org>
> > >
> > >
> > > FYI, the injection via the build process that is mentioned here already
> > > happens. See AppInfoParser.
> > >
> > > Ismael
> > >
> > > On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe <cm...@apache.org>
> wrote:
> > >
> > >> Hi Sasaki,
> > >>
> > >> Thanks for the KIP. I think a version flag is a good idea.
> > >>
> > >> Can you give a little more detail about what would be displayed when
> the
> > >> version command was used?
> > >>
> > >> We clearly want the version number, but we probably also want to know
> if
> > >> this is an official release, or a random SNAPSHOT from a branch. If
> this
> > >> is a release candidate, we probably want the RC number as well, like
> > >> "1.1-rc3" We also want a git hash. This can be injected by the build
> > >> process. In the case of an official release, where the source code
> is not
> > >> under git, we can pull it from a file.
> > >>
> > >> For example, hadoop's version output looks like this:
> > >>
> > >> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop version
> > >> > Hadoop 2.8.3
> > >> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
> > >> b3fe56402d908019d99af1f1f4fc65cb1d1436a2
> > >> > Compiled by jdu on 2017-12-05T03:43Z
> > >> > Compiled with protoc 2.5.0
> > >> > From source with checksum 9ff4856d824e983fa510d3f843e3f19d
> > >> > This command was run using /home/cmccabe/Downloads/
> > >> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
> > >>
> > >> (The "subversion" line here is a little weird -- it now refers to
> git, not
> > >> svn)
> > >>
> > >> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
> > >>> Hey Sasaki,
> > >>>
> > >>> Yeah, I don't feel too strongly about only supporting --version. I
> agree
> > >> it
> > >>> may help discoverability given the current approach. On the other
> hand,
> > >> if
> > >>> we refactored all of the tools so that we could use a common set of
> base
> > >>> options, it might be a little annoying to have to continue supporting
> > >> both
> > >>> variations. For example, tool standardization was proposed in KIP-14
> and
> > >>> I'm still holding out hope that someone will have time to pick this
> work
> > >>> back up. It's always easier to add an option than remove one, so I'm
> > >>> slightly inclined to have only --version for now. What do you think?
> > >> The double dash version is more consistent with how our other flags
> work.
> > >>
> > >> In general, I feel that if --version is supported, --help should say
> so.
> > >>
> > >> best,
> > >> Colin
> > >>
> > >>
> > >>> Thanks,
> > >>> Jason
> > >>>
> > >>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <
> sasakitoa@oss.nttdata.com
> > >>>
> > >>> wrote:
> > >>>
> > >>>> Hi Jason
> > >>>>
> > >>>> Thank you for helpful comments. I updated wiki based on your advice.
> > >>>>
> > >>>> I thought this option was relatively common and making maintenance
> > > easy
> > >>>> was also important.
> > >>>> However, as you said, it is not good that version option won't be
> > >> shown up
> > >>>> in help description.
> > >>>>
> > >>>> I thought accepting both single-dash and double-dash will help to
> find
> > >>>> this option.
> > >>>> In my approach this option won't be showed, but most of software
> which
> > >> has
> > >>>> this option accepts either single-dash or double-dash.
> > >>>> I guess it doesn't need to support both if we take another way.
> > >>>>
> > >>>>
> > >>>> Thanks
> > >>>>
> > >>>> @Ted Yeah, you're right. Sorry about the confusion.
> > >>>>> Since we're here, I think this KIP is a nice improvement. It's
> > >> definitely
> > >>>>> nice to have an easy way to check the version. That said, do we
> > > really
> > >>>>> need
> > >>>>> to support both `-version` and `--version`? The latter is
> consistent
> > >> with
> > >>>>> our current tools.
> > >>>>>
> > >>>>> Also, I think the approach we've taken is basically to build the
> > >> --version
> > >>>>> functionality into the bash script. This is nice because it saves a
> > >> lot of
> > >>>>> work to update the commands individually and we don't need to do
> > >> anything
> > >>>>> when we add new tools. The downside is that `--version` won't show
> up
> > >> as
> > >>>>> an
> > >>>>> option in any of the --help output. Not sure if that is too big of
> a
> > >>>>> problem, but maybe worth mentioning this in the rejected
> alternatives
> > >>>>> section.
> > >>>>>
> > >>>>>
> > >>>>> -Jason
> > >>>>>
> > >>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yu...@gmail.com>
> wrote:
> > >>>>>
> > >>>>> Jason:
> > >>>>>> Maybe your reply was intended for another KIP ?
> > >>>>>>
> > >>>>>> KIP-278 is about adding version option, not timeout.
> > >>>>>>
> > >>>>>> Cheers
> > >>>>>>
> > >>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <
> jason@confluent.io>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>> Hi Sasaki,
> > >>>>>>> Thanks for the KIP. I think the timeout controls the maximum
> > > allowed
> > >>>>>> time
> > >>>>>> that the consumer will block for the next record. Maybe the
> meaning
> > >>>>>> would
> > >>>>>> be clearer with the more concise name `--timeout`? That also fits
> > >> with
> > >>>>>> the
> > >>>>>>
> > >>>>>>> old consumer which overrides the `consumer.timeout.ms` property.
> > >>>>>>>
> > >>>>>>> By the way, it seems like the default value was intentionally set
> > >> low
> > >>>>>> for
> > >>>>>> both the old and new consumers, but I'm not sure of the reason. We
> > >> could
> > >>>>>>> leave the default as it is if we want to be safe, but increasing
> it
> > >>>>>>>
> > >>>>>> seems
> > >>>>>> ok to me. Perhaps we could start a little lower, though, say 10
> > >> seconds?
> > >>>>>> In
> > >>>>>>
> > >>>>>>> any case, we should make it clear to the user that the timeout
> was
> > >>>>>>>
> > >>>>>> reached.
> > >>>>>>
> > >>>>>>> It's surprising to see only the incomplete reported results
> > >> following a
> > >>>>>>> timeout.
> > >>>>>>>
> > >>>>>>> Thanks,
> > >>>>>>> Jason
> > >>>>>>>
> > >>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
> > >> sasakitoa@oss.nttdata.com>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>> Hello everyone,
> > >>>>>>>> I would like to start a discussion for KIP 278. Cloud you please
> > >> give
> > >>>>>>>> comments and advice ?
> > >>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-278+-
> > >>>>>>>> +Add+version+option+to+Kafka%27s+commands>
> > >>>>>>>>
> > >>>>>>>> JIRA ticket and Pull Request are bellow:
> > >>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
> > >>>>>>>> <https://github.com/apache/kafka/pull/639>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Many thanks,
> > >>>>>>>>
> > >>>>>>>> Sasaki
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>> --
> > >>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > >>>>
> > >>>>
> >
> > --
> > Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >
>
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Colin McCabe <cm...@apache.org>.
On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
> Hi Jason, Colin,
>
> Thank you for your comment, and I'm sorry for late reply.
>
> > we refactored all of the tools so that we could use a common set of
> base options,
> > it might be a little annoying to have to continue supporting both
> variations.
>
> I feel KIP-14 is good idea (I'm sorry, I didn't know about it), and
> I think it's no longer necessary both single-dash and double-dash are
> supported when this KIP will be accepted, as you said.
> I revise my Pull Request to support single-dash option only.
>
> Some my code in kafka-run-class.sh will become unnecessary when KIP-14
> is accepted.
> But I will keep this as temporary, because I seem that it's useful
> before KIP-14 accepted.
>
>
> > Can you give a little more detail about what would be displayed when
> the version command was used?
>
> As Ismael said, the version string is got from AppInfoParser#getVersion.
>
> In my Pull Request, we can get the result such as below::
>
> $ bin/kafka-topics.sh --version
> (snip)
> Kafka 1.2.0-SNAPSHOT
Hi Sasaki,
Thanks for the info. Can you add this to the KIP?
Also, I really think we should include the git hash.
best,
Colin
>
>
> Many thanks,
> Sasaki
>
>
> > From: Ismael Juma <is...@juma.me.uk>
> > Date: 2018-04-24 3:44 GMT+09:00
> > Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
> > To: dev <de...@kafka.apache.org>
> >
> >
> > FYI, the injection via the build process that is mentioned here already
> > happens. See AppInfoParser.
> >
> > Ismael
> >
> > On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe <cm...@apache.org> wrote:
> >
> >> Hi Sasaki,
> >>
> >> Thanks for the KIP. I think a version flag is a good idea.
> >>
> >> Can you give a little more detail about what would be displayed when the
> >> version command was used?
> >>
> >> We clearly want the version number, but we probably also want to know if
> >> this is an official release, or a random SNAPSHOT from a branch. If this
> >> is a release candidate, we probably want the RC number as well, like
> >> "1.1-rc3" We also want a git hash. This can be injected by the build
> >> process. In the case of an official release, where the source code is not
> >> under git, we can pull it from a file.
> >>
> >> For example, hadoop's version output looks like this:
> >>
> >> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop version
> >> > Hadoop 2.8.3
> >> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
> >> b3fe56402d908019d99af1f1f4fc65cb1d1436a2
> >> > Compiled by jdu on 2017-12-05T03:43Z
> >> > Compiled with protoc 2.5.0
> >> > From source with checksum 9ff4856d824e983fa510d3f843e3f19d
> >> > This command was run using /home/cmccabe/Downloads/
> >> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
> >>
> >> (The "subversion" line here is a little weird -- it now refers to git, not
> >> svn)
> >>
> >> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
> >>> Hey Sasaki,
> >>>
> >>> Yeah, I don't feel too strongly about only supporting --version. I agree
> >> it
> >>> may help discoverability given the current approach. On the other hand,
> >> if
> >>> we refactored all of the tools so that we could use a common set of base
> >>> options, it might be a little annoying to have to continue supporting
> >> both
> >>> variations. For example, tool standardization was proposed in KIP-14 and
> >>> I'm still holding out hope that someone will have time to pick this work
> >>> back up. It's always easier to add an option than remove one, so I'm
> >>> slightly inclined to have only --version for now. What do you think?
> >> The double dash version is more consistent with how our other flags work.
> >>
> >> In general, I feel that if --version is supported, --help should say so.
> >>
> >> best,
> >> Colin
> >>
> >>
> >>> Thanks,
> >>> Jason
> >>>
> >>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <sasakitoa@oss.nttdata.com
> >>>
> >>> wrote:
> >>>
> >>>> Hi Jason
> >>>>
> >>>> Thank you for helpful comments. I updated wiki based on your advice.
> >>>>
> >>>> I thought this option was relatively common and making maintenance
> > easy
> >>>> was also important.
> >>>> However, as you said, it is not good that version option won't be
> >> shown up
> >>>> in help description.
> >>>>
> >>>> I thought accepting both single-dash and double-dash will help to find
> >>>> this option.
> >>>> In my approach this option won't be showed, but most of software which
> >> has
> >>>> this option accepts either single-dash or double-dash.
> >>>> I guess it doesn't need to support both if we take another way.
> >>>>
> >>>>
> >>>> Thanks
> >>>>
> >>>> @Ted Yeah, you're right. Sorry about the confusion.
> >>>>> Since we're here, I think this KIP is a nice improvement. It's
> >> definitely
> >>>>> nice to have an easy way to check the version. That said, do we
> > really
> >>>>> need
> >>>>> to support both `-version` and `--version`? The latter is consistent
> >> with
> >>>>> our current tools.
> >>>>>
> >>>>> Also, I think the approach we've taken is basically to build the
> >> --version
> >>>>> functionality into the bash script. This is nice because it saves a
> >> lot of
> >>>>> work to update the commands individually and we don't need to do
> >> anything
> >>>>> when we add new tools. The downside is that `--version` won't show up
> >> as
> >>>>> an
> >>>>> option in any of the --help output. Not sure if that is too big of a
> >>>>> problem, but maybe worth mentioning this in the rejected alternatives
> >>>>> section.
> >>>>>
> >>>>>
> >>>>> -Jason
> >>>>>
> >>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yu...@gmail.com> wrote:
> >>>>>
> >>>>> Jason:
> >>>>>> Maybe your reply was intended for another KIP ?
> >>>>>>
> >>>>>> KIP-278 is about adding version option, not timeout.
> >>>>>>
> >>>>>> Cheers
> >>>>>>
> >>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <ja...@confluent.io>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Hi Sasaki,
> >>>>>>> Thanks for the KIP. I think the timeout controls the maximum
> > allowed
> >>>>>> time
> >>>>>> that the consumer will block for the next record. Maybe the meaning
> >>>>>> would
> >>>>>> be clearer with the more concise name `--timeout`? That also fits
> >> with
> >>>>>> the
> >>>>>>
> >>>>>>> old consumer which overrides the `consumer.timeout.ms` property.
> >>>>>>>
> >>>>>>> By the way, it seems like the default value was intentionally set
> >> low
> >>>>>> for
> >>>>>> both the old and new consumers, but I'm not sure of the reason. We
> >> could
> >>>>>>> leave the default as it is if we want to be safe, but increasing it
> >>>>>>>
> >>>>>> seems
> >>>>>> ok to me. Perhaps we could start a little lower, though, say 10
> >> seconds?
> >>>>>> In
> >>>>>>
> >>>>>>> any case, we should make it clear to the user that the timeout was
> >>>>>>>
> >>>>>> reached.
> >>>>>>
> >>>>>>> It's surprising to see only the incomplete reported results
> >> following a
> >>>>>>> timeout.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Jason
> >>>>>>>
> >>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
> >> sasakitoa@oss.nttdata.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> Hello everyone,
> >>>>>>>> I would like to start a discussion for KIP 278. Cloud you please
> >> give
> >>>>>>>> comments and advice ?
> >>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-278+-
> >>>>>>>> +Add+version+option+to+Kafka%27s+commands>
> >>>>>>>>
> >>>>>>>> JIRA ticket and Pull Request are bellow:
> >>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
> >>>>>>>> <https://github.com/apache/kafka/pull/639>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Many thanks,
> >>>>>>>>
> >>>>>>>> Sasaki
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>> --
> >>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >>>>
> >>>>
>
> --
> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Sasaki Toru <sa...@oss.nttdata.com>.
Hi Jason, Colin,
Thank you for your comment, and I'm sorry for late reply.
> we refactored all of the tools so that we could use a common set of base options,
> it might be a little annoying to have to continue supporting both variations.
I feel KIP-14 is good idea (I'm sorry, I didn't know about it), and
I think it's no longer necessary both single-dash and double-dash are supported when this KIP will be accepted, as you said.
I revise my Pull Request to support single-dash option only.
Some my code in kafka-run-class.sh will become unnecessary when KIP-14 is accepted.
But I will keep this as temporary, because I seem that it's useful before KIP-14 accepted.
> Can you give a little more detail about what would be displayed when the version command was used?
As Ismael said, the version string is got from AppInfoParser#getVersion.
In my Pull Request, we can get the result such as below::
$ bin/kafka-topics.sh --version
(snip)
Kafka 1.2.0-SNAPSHOT
Many thanks,
Sasaki
> From: Ismael Juma <is...@juma.me.uk>
> Date: 2018-04-24 3:44 GMT+09:00
> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
> To: dev <de...@kafka.apache.org>
>
>
> FYI, the injection via the build process that is mentioned here already
> happens. See AppInfoParser.
>
> Ismael
>
> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe <cm...@apache.org> wrote:
>
>> Hi Sasaki,
>>
>> Thanks for the KIP. I think a version flag is a good idea.
>>
>> Can you give a little more detail about what would be displayed when the
>> version command was used?
>>
>> We clearly want the version number, but we probably also want to know if
>> this is an official release, or a random SNAPSHOT from a branch. If this
>> is a release candidate, we probably want the RC number as well, like
>> "1.1-rc3" We also want a git hash. This can be injected by the build
>> process. In the case of an official release, where the source code is not
>> under git, we can pull it from a file.
>>
>> For example, hadoop's version output looks like this:
>>
>> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop version
>> > Hadoop 2.8.3
>> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
>> b3fe56402d908019d99af1f1f4fc65cb1d1436a2
>> > Compiled by jdu on 2017-12-05T03:43Z
>> > Compiled with protoc 2.5.0
>> > From source with checksum 9ff4856d824e983fa510d3f843e3f19d
>> > This command was run using /home/cmccabe/Downloads/
>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
>>
>> (The "subversion" line here is a little weird -- it now refers to git, not
>> svn)
>>
>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
>>> Hey Sasaki,
>>>
>>> Yeah, I don't feel too strongly about only supporting --version. I agree
>> it
>>> may help discoverability given the current approach. On the other hand,
>> if
>>> we refactored all of the tools so that we could use a common set of base
>>> options, it might be a little annoying to have to continue supporting
>> both
>>> variations. For example, tool standardization was proposed in KIP-14 and
>>> I'm still holding out hope that someone will have time to pick this work
>>> back up. It's always easier to add an option than remove one, so I'm
>>> slightly inclined to have only --version for now. What do you think?
>> The double dash version is more consistent with how our other flags work.
>>
>> In general, I feel that if --version is supported, --help should say so.
>>
>> best,
>> Colin
>>
>>
>>> Thanks,
>>> Jason
>>>
>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <sasakitoa@oss.nttdata.com
>>>
>>> wrote:
>>>
>>>> Hi Jason
>>>>
>>>> Thank you for helpful comments. I updated wiki based on your advice.
>>>>
>>>> I thought this option was relatively common and making maintenance
> easy
>>>> was also important.
>>>> However, as you said, it is not good that version option won't be
>> shown up
>>>> in help description.
>>>>
>>>> I thought accepting both single-dash and double-dash will help to find
>>>> this option.
>>>> In my approach this option won't be showed, but most of software which
>> has
>>>> this option accepts either single-dash or double-dash.
>>>> I guess it doesn't need to support both if we take another way.
>>>>
>>>>
>>>> Thanks
>>>>
>>>> @Ted Yeah, you're right. Sorry about the confusion.
>>>>> Since we're here, I think this KIP is a nice improvement. It's
>> definitely
>>>>> nice to have an easy way to check the version. That said, do we
> really
>>>>> need
>>>>> to support both `-version` and `--version`? The latter is consistent
>> with
>>>>> our current tools.
>>>>>
>>>>> Also, I think the approach we've taken is basically to build the
>> --version
>>>>> functionality into the bash script. This is nice because it saves a
>> lot of
>>>>> work to update the commands individually and we don't need to do
>> anything
>>>>> when we add new tools. The downside is that `--version` won't show up
>> as
>>>>> an
>>>>> option in any of the --help output. Not sure if that is too big of a
>>>>> problem, but maybe worth mentioning this in the rejected alternatives
>>>>> section.
>>>>>
>>>>>
>>>>> -Jason
>>>>>
>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yu...@gmail.com> wrote:
>>>>>
>>>>> Jason:
>>>>>> Maybe your reply was intended for another KIP ?
>>>>>>
>>>>>> KIP-278 is about adding version option, not timeout.
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <ja...@confluent.io>
>>>>>> wrote:
>>>>>>
>>>>>> Hi Sasaki,
>>>>>>> Thanks for the KIP. I think the timeout controls the maximum
> allowed
>>>>>> time
>>>>>> that the consumer will block for the next record. Maybe the meaning
>>>>>> would
>>>>>> be clearer with the more concise name `--timeout`? That also fits
>> with
>>>>>> the
>>>>>>
>>>>>>> old consumer which overrides the `consumer.timeout.ms` property.
>>>>>>>
>>>>>>> By the way, it seems like the default value was intentionally set
>> low
>>>>>> for
>>>>>> both the old and new consumers, but I'm not sure of the reason. We
>> could
>>>>>>> leave the default as it is if we want to be safe, but increasing it
>>>>>>>
>>>>>> seems
>>>>>> ok to me. Perhaps we could start a little lower, though, say 10
>> seconds?
>>>>>> In
>>>>>>
>>>>>>> any case, we should make it clear to the user that the timeout was
>>>>>>>
>>>>>> reached.
>>>>>>
>>>>>>> It's surprising to see only the incomplete reported results
>> following a
>>>>>>> timeout.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Jason
>>>>>>>
>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
>> sasakitoa@oss.nttdata.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hello everyone,
>>>>>>>> I would like to start a discussion for KIP 278. Cloud you please
>> give
>>>>>>>> comments and advice ?
>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-278+-
>>>>>>>> +Add+version+option+to+Kafka%27s+commands>
>>>>>>>>
>>>>>>>> JIRA ticket and Pull Request are bellow:
>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
>>>>>>>> <https://github.com/apache/kafka/pull/639>
>>>>>>>>
>>>>>>>>
>>>>>>>> Many thanks,
>>>>>>>>
>>>>>>>> Sasaki
>>>>>>>>
>>>>>>>> --
>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>> --
>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>
>>>>
--
Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Ismael Juma <is...@juma.me.uk>.
FYI, the injection via the build process that is mentioned here already
happens. See AppInfoParser.
Ismael
On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe <cm...@apache.org> wrote:
> Hi Sasaki,
>
> Thanks for the KIP. I think a version flag is a good idea.
>
> Can you give a little more detail about what would be displayed when the
> version command was used?
>
> We clearly want the version number, but we probably also want to know if
> this is an official release, or a random SNAPSHOT from a branch. If this
> is a release candidate, we probably want the RC number as well, like
> "1.1-rc3" We also want a git hash. This can be injected by the build
> process. In the case of an official release, where the source code is not
> under git, we can pull it from a file.
>
> For example, hadoop's version output looks like this:
>
> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop version
> > Hadoop 2.8.3
> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
> b3fe56402d908019d99af1f1f4fc65cb1d1436a2
> > Compiled by jdu on 2017-12-05T03:43Z
> > Compiled with protoc 2.5.0
> > From source with checksum 9ff4856d824e983fa510d3f843e3f19d
> > This command was run using /home/cmccabe/Downloads/
> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
>
> (The "subversion" line here is a little weird -- it now refers to git, not
> svn)
>
> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
> > Hey Sasaki,
> >
> > Yeah, I don't feel too strongly about only supporting --version. I agree
> it
> > may help discoverability given the current approach. On the other hand,
> if
> > we refactored all of the tools so that we could use a common set of base
> > options, it might be a little annoying to have to continue supporting
> both
> > variations. For example, tool standardization was proposed in KIP-14 and
> > I'm still holding out hope that someone will have time to pick this work
> > back up. It's always easier to add an option than remove one, so I'm
> > slightly inclined to have only --version for now. What do you think?
>
> The double dash version is more consistent with how our other flags work.
>
> In general, I feel that if --version is supported, --help should say so.
>
> best,
> Colin
>
>
> >
> > Thanks,
> > Jason
> >
> > On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <sasakitoa@oss.nttdata.com
> >
> > wrote:
> >
> > > Hi Jason
> > >
> > > Thank you for helpful comments. I updated wiki based on your advice.
> > >
> > > I thought this option was relatively common and making maintenance easy
> > > was also important.
> > > However, as you said, it is not good that version option won't be
> shown up
> > > in help description.
> > >
> > > I thought accepting both single-dash and double-dash will help to find
> > > this option.
> > > In my approach this option won't be showed, but most of software which
> has
> > > this option accepts either single-dash or double-dash.
> > > I guess it doesn't need to support both if we take another way.
> > >
> > >
> > > Thanks
> > >
> > > @Ted Yeah, you're right. Sorry about the confusion.
> > >>
> > >> Since we're here, I think this KIP is a nice improvement. It's
> definitely
> > >> nice to have an easy way to check the version. That said, do we really
> > >> need
> > >> to support both `-version` and `--version`? The latter is consistent
> with
> > >> our current tools.
> > >>
> > >> Also, I think the approach we've taken is basically to build the
> --version
> > >> functionality into the bash script. This is nice because it saves a
> lot of
> > >> work to update the commands individually and we don't need to do
> anything
> > >> when we add new tools. The downside is that `--version` won't show up
> as
> > >> an
> > >> option in any of the --help output. Not sure if that is too big of a
> > >> problem, but maybe worth mentioning this in the rejected alternatives
> > >> section.
> > >>
> > >>
> > >> -Jason
> > >>
> > >> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yu...@gmail.com> wrote:
> > >>
> > >> Jason:
> > >>> Maybe your reply was intended for another KIP ?
> > >>>
> > >>> KIP-278 is about adding version option, not timeout.
> > >>>
> > >>> Cheers
> > >>>
> > >>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <ja...@confluent.io>
> > >>> wrote:
> > >>>
> > >>> Hi Sasaki,
> > >>>>
> > >>>> Thanks for the KIP. I think the timeout controls the maximum allowed
> > >>>>
> > >>> time
> > >>
> > >>> that the consumer will block for the next record. Maybe the meaning
> > >>>>
> > >>> would
> > >>
> > >>> be clearer with the more concise name `--timeout`? That also fits
> with
> > >>>>
> > >>> the
> > >>>
> > >>>> old consumer which overrides the `consumer.timeout.ms` property.
> > >>>>
> > >>>> By the way, it seems like the default value was intentionally set
> low
> > >>>>
> > >>> for
> > >>
> > >>> both the old and new consumers, but I'm not sure of the reason. We
> could
> > >>>> leave the default as it is if we want to be safe, but increasing it
> > >>>>
> > >>> seems
> > >>
> > >>> ok to me. Perhaps we could start a little lower, though, say 10
> seconds?
> > >>>>
> > >>> In
> > >>>
> > >>>> any case, we should make it clear to the user that the timeout was
> > >>>>
> > >>> reached.
> > >>>
> > >>>> It's surprising to see only the incomplete reported results
> following a
> > >>>> timeout.
> > >>>>
> > >>>> Thanks,
> > >>>> Jason
> > >>>>
> > >>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <
> sasakitoa@oss.nttdata.com>
> > >>>> wrote:
> > >>>>
> > >>>> Hello everyone,
> > >>>>>
> > >>>>> I would like to start a discussion for KIP 278. Cloud you please
> give
> > >>>>> comments and advice ?
> > >>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-278+-
> > >>>>> +Add+version+option+to+Kafka%27s+commands>
> > >>>>>
> > >>>>> JIRA ticket and Pull Request are bellow:
> > >>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
> > >>>>> <https://github.com/apache/kafka/pull/639>
> > >>>>>
> > >>>>>
> > >>>>> Many thanks,
> > >>>>>
> > >>>>> Sasaki
> > >>>>>
> > >>>>> --
> > >>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > >>>>>
> > >>>>>
> > >>>>>
> > > --
> > > Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > >
> > >
>
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Colin McCabe <cm...@apache.org>.
Hi Sasaki,
Thanks for the KIP. I think a version flag is a good idea.
Can you give a little more detail about what would be displayed when the version command was used?
We clearly want the version number, but we probably also want to know if this is an official release, or a random SNAPSHOT from a branch. If this is a release candidate, we probably want the RC number as well, like "1.1-rc3" We also want a git hash. This can be injected by the build process. In the case of an official release, where the source code is not under git, we can pull it from a file.
For example, hadoop's version output looks like this:
> cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop version
> Hadoop 2.8.3
> Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r b3fe56402d908019d99af1f1f4fc65cb1d1436a2
> Compiled by jdu on 2017-12-05T03:43Z
> Compiled with protoc 2.5.0
> From source with checksum 9ff4856d824e983fa510d3f843e3f19d
> This command was run using /home/cmccabe/Downloads/hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
(The "subversion" line here is a little weird -- it now refers to git, not svn)
On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
> Hey Sasaki,
>
> Yeah, I don't feel too strongly about only supporting --version. I agree it
> may help discoverability given the current approach. On the other hand, if
> we refactored all of the tools so that we could use a common set of base
> options, it might be a little annoying to have to continue supporting both
> variations. For example, tool standardization was proposed in KIP-14 and
> I'm still holding out hope that someone will have time to pick this work
> back up. It's always easier to add an option than remove one, so I'm
> slightly inclined to have only --version for now. What do you think?
The double dash version is more consistent with how our other flags work.
In general, I feel that if --version is supported, --help should say so.
best,
Colin
>
> Thanks,
> Jason
>
> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <sa...@oss.nttdata.com>
> wrote:
>
> > Hi Jason
> >
> > Thank you for helpful comments. I updated wiki based on your advice.
> >
> > I thought this option was relatively common and making maintenance easy
> > was also important.
> > However, as you said, it is not good that version option won't be shown up
> > in help description.
> >
> > I thought accepting both single-dash and double-dash will help to find
> > this option.
> > In my approach this option won't be showed, but most of software which has
> > this option accepts either single-dash or double-dash.
> > I guess it doesn't need to support both if we take another way.
> >
> >
> > Thanks
> >
> > @Ted Yeah, you're right. Sorry about the confusion.
> >>
> >> Since we're here, I think this KIP is a nice improvement. It's definitely
> >> nice to have an easy way to check the version. That said, do we really
> >> need
> >> to support both `-version` and `--version`? The latter is consistent with
> >> our current tools.
> >>
> >> Also, I think the approach we've taken is basically to build the --version
> >> functionality into the bash script. This is nice because it saves a lot of
> >> work to update the commands individually and we don't need to do anything
> >> when we add new tools. The downside is that `--version` won't show up as
> >> an
> >> option in any of the --help output. Not sure if that is too big of a
> >> problem, but maybe worth mentioning this in the rejected alternatives
> >> section.
> >>
> >>
> >> -Jason
> >>
> >> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yu...@gmail.com> wrote:
> >>
> >> Jason:
> >>> Maybe your reply was intended for another KIP ?
> >>>
> >>> KIP-278 is about adding version option, not timeout.
> >>>
> >>> Cheers
> >>>
> >>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <ja...@confluent.io>
> >>> wrote:
> >>>
> >>> Hi Sasaki,
> >>>>
> >>>> Thanks for the KIP. I think the timeout controls the maximum allowed
> >>>>
> >>> time
> >>
> >>> that the consumer will block for the next record. Maybe the meaning
> >>>>
> >>> would
> >>
> >>> be clearer with the more concise name `--timeout`? That also fits with
> >>>>
> >>> the
> >>>
> >>>> old consumer which overrides the `consumer.timeout.ms` property.
> >>>>
> >>>> By the way, it seems like the default value was intentionally set low
> >>>>
> >>> for
> >>
> >>> both the old and new consumers, but I'm not sure of the reason. We could
> >>>> leave the default as it is if we want to be safe, but increasing it
> >>>>
> >>> seems
> >>
> >>> ok to me. Perhaps we could start a little lower, though, say 10 seconds?
> >>>>
> >>> In
> >>>
> >>>> any case, we should make it clear to the user that the timeout was
> >>>>
> >>> reached.
> >>>
> >>>> It's surprising to see only the incomplete reported results following a
> >>>> timeout.
> >>>>
> >>>> Thanks,
> >>>> Jason
> >>>>
> >>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <sa...@oss.nttdata.com>
> >>>> wrote:
> >>>>
> >>>> Hello everyone,
> >>>>>
> >>>>> I would like to start a discussion for KIP 278. Cloud you please give
> >>>>> comments and advice ?
> >>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-278+-
> >>>>> +Add+version+option+to+Kafka%27s+commands>
> >>>>>
> >>>>> JIRA ticket and Pull Request are bellow:
> >>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
> >>>>> <https://github.com/apache/kafka/pull/639>
> >>>>>
> >>>>>
> >>>>> Many thanks,
> >>>>>
> >>>>> Sasaki
> >>>>>
> >>>>> --
> >>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >>>>>
> >>>>>
> >>>>>
> > --
> > Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >
> >
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Jason Gustafson <ja...@confluent.io>.
Hey Sasaki,
Yeah, I don't feel too strongly about only supporting --version. I agree it
may help discoverability given the current approach. On the other hand, if
we refactored all of the tools so that we could use a common set of base
options, it might be a little annoying to have to continue supporting both
variations. For example, tool standardization was proposed in KIP-14 and
I'm still holding out hope that someone will have time to pick this work
back up. It's always easier to add an option than remove one, so I'm
slightly inclined to have only --version for now. What do you think?
Thanks,
Jason
On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru <sa...@oss.nttdata.com>
wrote:
> Hi Jason
>
> Thank you for helpful comments. I updated wiki based on your advice.
>
> I thought this option was relatively common and making maintenance easy
> was also important.
> However, as you said, it is not good that version option won't be shown up
> in help description.
>
> I thought accepting both single-dash and double-dash will help to find
> this option.
> In my approach this option won't be showed, but most of software which has
> this option accepts either single-dash or double-dash.
> I guess it doesn't need to support both if we take another way.
>
>
> Thanks
>
> @Ted Yeah, you're right. Sorry about the confusion.
>>
>> Since we're here, I think this KIP is a nice improvement. It's definitely
>> nice to have an easy way to check the version. That said, do we really
>> need
>> to support both `-version` and `--version`? The latter is consistent with
>> our current tools.
>>
>> Also, I think the approach we've taken is basically to build the --version
>> functionality into the bash script. This is nice because it saves a lot of
>> work to update the commands individually and we don't need to do anything
>> when we add new tools. The downside is that `--version` won't show up as
>> an
>> option in any of the --help output. Not sure if that is too big of a
>> problem, but maybe worth mentioning this in the rejected alternatives
>> section.
>>
>>
>> -Jason
>>
>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yu...@gmail.com> wrote:
>>
>> Jason:
>>> Maybe your reply was intended for another KIP ?
>>>
>>> KIP-278 is about adding version option, not timeout.
>>>
>>> Cheers
>>>
>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <ja...@confluent.io>
>>> wrote:
>>>
>>> Hi Sasaki,
>>>>
>>>> Thanks for the KIP. I think the timeout controls the maximum allowed
>>>>
>>> time
>>
>>> that the consumer will block for the next record. Maybe the meaning
>>>>
>>> would
>>
>>> be clearer with the more concise name `--timeout`? That also fits with
>>>>
>>> the
>>>
>>>> old consumer which overrides the `consumer.timeout.ms` property.
>>>>
>>>> By the way, it seems like the default value was intentionally set low
>>>>
>>> for
>>
>>> both the old and new consumers, but I'm not sure of the reason. We could
>>>> leave the default as it is if we want to be safe, but increasing it
>>>>
>>> seems
>>
>>> ok to me. Perhaps we could start a little lower, though, say 10 seconds?
>>>>
>>> In
>>>
>>>> any case, we should make it clear to the user that the timeout was
>>>>
>>> reached.
>>>
>>>> It's surprising to see only the incomplete reported results following a
>>>> timeout.
>>>>
>>>> Thanks,
>>>> Jason
>>>>
>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <sa...@oss.nttdata.com>
>>>> wrote:
>>>>
>>>> Hello everyone,
>>>>>
>>>>> I would like to start a discussion for KIP 278. Cloud you please give
>>>>> comments and advice ?
>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-278+-
>>>>> +Add+version+option+to+Kafka%27s+commands>
>>>>>
>>>>> JIRA ticket and Pull Request are bellow:
>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
>>>>> <https://github.com/apache/kafka/pull/639>
>>>>>
>>>>>
>>>>> Many thanks,
>>>>>
>>>>> Sasaki
>>>>>
>>>>> --
>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>>
>>>>>
>>>>>
> --
> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>
>
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Sasaki Toru <sa...@oss.nttdata.com>.
Hi Jason
Thank you for helpful comments. I updated wiki based on your advice.
I thought this option was relatively common and making maintenance easy was also important.
However, as you said, it is not good that version option won't be shown up in help description.
I thought accepting both single-dash and double-dash will help to find this option.
In my approach this option won't be showed, but most of software which has this option accepts either single-dash or double-dash.
I guess it doesn't need to support both if we take another way.
Thanks
> @Ted Yeah, you're right. Sorry about the confusion.
>
> Since we're here, I think this KIP is a nice improvement. It's definitely
> nice to have an easy way to check the version. That said, do we really need
> to support both `-version` and `--version`? The latter is consistent with
> our current tools.
>
> Also, I think the approach we've taken is basically to build the --version
> functionality into the bash script. This is nice because it saves a lot of
> work to update the commands individually and we don't need to do anything
> when we add new tools. The downside is that `--version` won't show up as an
> option in any of the --help output. Not sure if that is too big of a
> problem, but maybe worth mentioning this in the rejected alternatives
> section.
>
>
> -Jason
>
> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Jason:
>> Maybe your reply was intended for another KIP ?
>>
>> KIP-278 is about adding version option, not timeout.
>>
>> Cheers
>>
>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <ja...@confluent.io>
>> wrote:
>>
>>> Hi Sasaki,
>>>
>>> Thanks for the KIP. I think the timeout controls the maximum allowed
> time
>>> that the consumer will block for the next record. Maybe the meaning
> would
>>> be clearer with the more concise name `--timeout`? That also fits with
>> the
>>> old consumer which overrides the `consumer.timeout.ms` property.
>>>
>>> By the way, it seems like the default value was intentionally set low
> for
>>> both the old and new consumers, but I'm not sure of the reason. We could
>>> leave the default as it is if we want to be safe, but increasing it
> seems
>>> ok to me. Perhaps we could start a little lower, though, say 10 seconds?
>> In
>>> any case, we should make it clear to the user that the timeout was
>> reached.
>>> It's surprising to see only the incomplete reported results following a
>>> timeout.
>>>
>>> Thanks,
>>> Jason
>>>
>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <sa...@oss.nttdata.com>
>>> wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> I would like to start a discussion for KIP 278. Cloud you please give
>>>> comments and advice ?
>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-278+-
>>>> +Add+version+option+to+Kafka%27s+commands>
>>>>
>>>> JIRA ticket and Pull Request are bellow:
>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
>>>> <https://github.com/apache/kafka/pull/639>
>>>>
>>>>
>>>> Many thanks,
>>>>
>>>> Sasaki
>>>>
>>>> --
>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>>>>
>>>>
--
Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Jason Gustafson <ja...@confluent.io>.
@Ted Yeah, you're right. Sorry about the confusion.
Since we're here, I think this KIP is a nice improvement. It's definitely
nice to have an easy way to check the version. That said, do we really need
to support both `-version` and `--version`? The latter is consistent with
our current tools.
Also, I think the approach we've taken is basically to build the --version
functionality into the bash script. This is nice because it saves a lot of
work to update the commands individually and we don't need to do anything
when we add new tools. The downside is that `--version` won't show up as an
option in any of the --help output. Not sure if that is too big of a
problem, but maybe worth mentioning this in the rejected alternatives
section.
-Jason
On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yu...@gmail.com> wrote:
> Jason:
> Maybe your reply was intended for another KIP ?
>
> KIP-278 is about adding version option, not timeout.
>
> Cheers
>
> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > Hi Sasaki,
> >
> > Thanks for the KIP. I think the timeout controls the maximum allowed time
> > that the consumer will block for the next record. Maybe the meaning would
> > be clearer with the more concise name `--timeout`? That also fits with
> the
> > old consumer which overrides the `consumer.timeout.ms` property.
> >
> > By the way, it seems like the default value was intentionally set low for
> > both the old and new consumers, but I'm not sure of the reason. We could
> > leave the default as it is if we want to be safe, but increasing it seems
> > ok to me. Perhaps we could start a little lower, though, say 10 seconds?
> In
> > any case, we should make it clear to the user that the timeout was
> reached.
> > It's surprising to see only the incomplete reported results following a
> > timeout.
> >
> > Thanks,
> > Jason
> >
> > On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <sa...@oss.nttdata.com>
> > wrote:
> >
> > > Hello everyone,
> > >
> > > I would like to start a discussion for KIP 278. Cloud you please give
> > > comments and advice ?
> > > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-278+-
> > > +Add+version+option+to+Kafka%27s+commands>
> > >
> > > JIRA ticket and Pull Request are bellow:
> > > <https://issues.apache.org/jira/browse/KAFKA-2061>
> > > <https://github.com/apache/kafka/pull/639>
> > >
> > >
> > > Many thanks,
> > >
> > > Sasaki
> > >
> > > --
> > > Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > >
> > >
> >
>
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Ted Yu <yu...@gmail.com>.
Jason:
Maybe your reply was intended for another KIP ?
KIP-278 is about adding version option, not timeout.
Cheers
On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson <ja...@confluent.io> wrote:
> Hi Sasaki,
>
> Thanks for the KIP. I think the timeout controls the maximum allowed time
> that the consumer will block for the next record. Maybe the meaning would
> be clearer with the more concise name `--timeout`? That also fits with the
> old consumer which overrides the `consumer.timeout.ms` property.
>
> By the way, it seems like the default value was intentionally set low for
> both the old and new consumers, but I'm not sure of the reason. We could
> leave the default as it is if we want to be safe, but increasing it seems
> ok to me. Perhaps we could start a little lower, though, say 10 seconds? In
> any case, we should make it clear to the user that the timeout was reached.
> It's surprising to see only the incomplete reported results following a
> timeout.
>
> Thanks,
> Jason
>
> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <sa...@oss.nttdata.com>
> wrote:
>
> > Hello everyone,
> >
> > I would like to start a discussion for KIP 278. Cloud you please give
> > comments and advice ?
> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-278+-
> > +Add+version+option+to+Kafka%27s+commands>
> >
> > JIRA ticket and Pull Request are bellow:
> > <https://issues.apache.org/jira/browse/KAFKA-2061>
> > <https://github.com/apache/kafka/pull/639>
> >
> >
> > Many thanks,
> >
> > Sasaki
> >
> > --
> > Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> >
> >
>
Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Posted by Jason Gustafson <ja...@confluent.io>.
Hi Sasaki,
Thanks for the KIP. I think the timeout controls the maximum allowed time
that the consumer will block for the next record. Maybe the meaning would
be clearer with the more concise name `--timeout`? That also fits with the
old consumer which overrides the `consumer.timeout.ms` property.
By the way, it seems like the default value was intentionally set low for
both the old and new consumers, but I'm not sure of the reason. We could
leave the default as it is if we want to be safe, but increasing it seems
ok to me. Perhaps we could start a little lower, though, say 10 seconds? In
any case, we should make it clear to the user that the timeout was reached.
It's surprising to see only the incomplete reported results following a
timeout.
Thanks,
Jason
On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru <sa...@oss.nttdata.com>
wrote:
> Hello everyone,
>
> I would like to start a discussion for KIP 278. Cloud you please give
> comments and advice ?
> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-278+-
> +Add+version+option+to+Kafka%27s+commands>
>
> JIRA ticket and Pull Request are bellow:
> <https://issues.apache.org/jira/browse/KAFKA-2061>
> <https://github.com/apache/kafka/pull/639>
>
>
> Many thanks,
>
> Sasaki
>
> --
> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
>
>