You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Ismaël Mejía <ie...@gmail.com> on 2019/01/28 14:21:52 UTC
Re: Beam Dependency Check Report (2018-06-13)
Hello,
The dependency update report has been working fine. However I found some
issues that I summarized in this issue.
https://issues.apache.org/jira/browse/BEAM-6524
Can Yifan or someone else that knows that area please take a look.
Regards,
Ismaël
On Thu, Jun 14, 2018 at 11:37 PM Yifan Zou <yi...@google.com> wrote:
> Thank you Paul for letting us know this issue. We will take care of it
> when upgrading dependencies.
>
> On Thu, Jun 14, 2018 at 7:23 AM Paul Gerver <pf...@gmail.com> wrote:
>
>> I do have one request to be added to the Java SDK version updates:
>> Beam-3831 [1]. The Google Core depends on the old org.json package which
>> ASF discourages using because of the "Use only for good, not evil" clause.
>>
>> [1] https://issues.apache.org/jira/browse/BEAM-3831
>>
>> On Thu, Jun 14, 2018 at 3:03 AM Etienne Chauchot <ec...@apache.org>
>> wrote:
>>
>>> Thanks Yifan,
>>>
>>> This is great ! It would help us maintain Beam more easily and probably
>>> help us fixing CVE as well.
>>>
>>> Etienne
>>>
>>> Le mercredi 13 juin 2018 à 07:45 -0700, Yifan Zou a écrit :
>>>
>>> Hi,
>>>
>>>
>>> I want to follow up and explain this email.
>>>
>>>
>>> This is a sample email that reports the results of Beam SDK dependency
>>> check, which was proposed here
>>> <https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/edit#heading=h.u75g8bk11ngp>.
>>> The goal is finding updates for all Beam Python & Java SDKs' dependencies
>>> and prioritize them. The job will be auto triggered in Jenkins once a week
>>> and generate a report. The report lists the high priority updates base on
>>> the following criteria:
>>>
>>>
>>> The dependency update is high priority if:
>>>
>>> 1. It has major versions update available;
>>>
>>> e.g. org.assertj:assertj-core 2.5.0 -> 3.10.0
>>>
>>> 2. or, it is over 3 minor versions behind the latest version;
>>>
>>> e.g. org.tukaani:xz 1.5 -> 1.8
>>>
>>> 3. or, the current version is behind the later version for over 180 days.
>>>
>>>
>>> e.g. com.google.auto.service:auto-service 2014-10-24 -> 2017-12-11
>>>
>>>
>>> This job helps Beam contributors to determine the dependency which is
>>> far behind the latest released version. The next step would be automating
>>> filing JIRA bugs for dep updates, group dependencies and identify owners to
>>> take care of the upgrades follow Chamikara's proposal
>>> <https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit>
>>> .
>>>
>>>
>>> For more readings:
>>>
>>> [Proposal] Beam dependency check automation
>>> <https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/edit#heading=h.u75g8bk11ngp>
>>> by Yifan Zou
>>>
>>> [Proposal] Beam dependency update policy
>>> <https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit>
>>> by *Chamikara Jayalath*
>>>
>>> Thank you.
>>>
>>> Yifan Zou
>>>
>>> On Wed, Jun 13, 2018 at 7:41 AM Apache Jenkins Server <
>>> jenkins@builds.apache.org> wrote:
>>>
>>> High Priority Dependency Updates Of Beam Python SDK:
>>> *Dependency Name* *Current Version* *Later Version* *Current Version
>>> Release Date* *Later Version Release Date*
>>> google-cloud-bigquery 0.25.0 1.3.0 2017-06-26 2018-06-08
>>> httplib2 0.9.2 0.11.3 2015-09-28 2018-03-30 High Priority Dependency
>>> Updates Of Beam Java SDK:
>>> *Dependency Name* *Current Version* *Later Version* *Current Version
>>> Release Date* *Later Version Release Date*
>>> org.assertj:assertj-core 2.5.0 3.10.0 2016-07-03 2018-05-11
>>> com.google.auto.service:auto-service 1.0-rc2 1.0-rc4 2014-10-24
>>> 2017-12-11
>>> biz.aQute:bndlib 1.43.0 2.0.0.20130123-133441 2011-04-01 2013-02-27
>>> org.apache.cassandra:cassandra-all 3.9 3.11.2 2016-09-26 2018-02-14
>>> commons-cli:commons-cli 1.2 1.4 2009-03-19 2017-03-09
>>> commons-codec:commons-codec 1.9 1.11 2013-12-20 2017-10-17
>>> org.apache.commons:commons-dbcp2 2.1.1 2.3.0 2015-08-02 2018-05-08
>>> com.typesafe:config 1.3.0 1.3.3 2015-05-08 2018-02-21
>>> de.flapdoodle.embed:de.flapdoodle.embed.mongo 1.50.1 2.0.3 2015-12-11
>>> 2018-02-14
>>> de.flapdoodle.embed:de.flapdoodle.embed.process 1.50.1 2.0.3 2015-12-11
>>> 2018-02-14
>>> org.apache.derby:derby 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03
>>> org.apache.derby:derbyclient 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03
>>> org.apache.derby:derbynet 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03
>>> org.elasticsearch:elasticsearch 5.6.3 6.2.4 2017-10-06 2018-04-12
>>> org.elasticsearch:elasticsearch-hadoop 5.0.0 6.2.4 2016-10-26 2018-04-12
>>> org.elasticsearch.client:elasticsearch-rest-client 5.6.3 6.2.4
>>> 2017-10-06 2018-04-12
>>> com.alibaba:fastjson 1.2.12 1.2.47 2016-05-21 2018-03-15
>>> org.elasticsearch.test:framework 5.6.3 6.2.4 2017-10-06 2018-04-12
>>> org.freemarker:freemarker 2.3.25-incubating 2.3.28 2016-06-14 2018-03-30
>>> org.codehaus.groovy:groovy-all 2.4.13 3.0.0-alpha-2 2017-11-22
>>> 2018-04-16
>>> org.apache.hbase:hbase-common 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>> 2018-05-31
>>> org.apache.hbase:hbase-hadoop-compat 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>> 2018-05-31
>>> org.apache.hbase:hbase-hadoop2-compat 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>> 2018-05-31
>>> org.apache.hbase:hbase-server 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>> 2018-05-31
>>> org.apache.hbase:hbase-shaded-client 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>> 2018-05-31
>>> org.apache.hbase:hbase-shaded-server 1.2.6 2.0.0-alpha2 2017-05-29
>>> 2018-05-31
>>> org.apache.hive:hive-cli 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21
>>> org.apache.hive:hive-common 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21
>>> org.apache.hive:hive-exec 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21
>>> org.apache.hive.hcatalog:hive-hcatalog-core 2.1.0 3.0.0.3.0.0.3-2
>>> 2016-06-16 2018-05-21
>>> org.apache.httpcomponents:httpasyncclient 4.1.2 4.1.3 2016-06-18
>>> 2017-02-05
>>> org.apache.httpcomponents:httpclient 4.5.2 4.5.5 2016-02-21 2018-01-18
>>> org.apache.httpcomponents:httpcore 4.4.5 4.4.9 2016-06-08 2018-01-11
>>> net.java.dev.javacc:javacc 4.0 7.0.3 2018-06-08 2017-11-06
>>> jline:jline 2.14.6 3.0.0.M1 2018-03-26 2018-06-08
>>> net.java.dev.jna:jna 4.1.0 4.5.1 2014-03-06 2017-12-27
>>> com.esotericsoftware.kryo:kryo 2.21 2.24.0 2013-02-27 2014-05-04
>>> io.dropwizard.metrics:metrics-core 3.1.2 4.1.0-rc2 2015-04-25 2018-05-03
>>> org.mongodb:mongo-java-driver 3.2.2 3.8.0-beta3 2016-02-15 2018-05-29
>>> io.netty:netty-all 4.1.17.Final 5.0.0.Alpha2 2017-11-08 2018-06-06
>>> io.grpc:protoc-gen-grpc-java 1.2.0 1.12.0 2017-03-15 2018-05-07
>>> org.apache.qpid:proton-j 0.13.1 0.27.1 2016-07-01 2018-04-25
>>> com.carrotsearch.randomizedtesting:randomizedtesting-runner 2.5.0 2.6.3
>>> 2017-01-23 2018-06-11
>>> org.scala-lang:scala-library 2.11.8 2.13.0-M4 2017-03-08 2018-05-14
>>> org.slf4j:slf4j-api 1.7.25 1.8.0-beta2 2017-03-16 2018-03-21
>>> org.slf4j:slf4j-jdk14 1.7.25 1.8.0-beta2 2017-03-16 2018-03-21
>>> org.apache.solr:solr-core 5.5.4 7.3.1 2017-10-20 2018-05-17
>>> org.apache.solr:solr-solrj 5.5.4 7.3.1 2017-10-20 2018-05-17
>>> org.apache.solr:solr-test-framework 5.5.4 7.3.1 2017-10-20 2018-05-17
>>> org.springframework:spring-expression 4.3.5.RELEASE 5.0.7.RELEASE
>>> 2017-01-25 2018-06-12
>>> sqlline:sqlline 1.3.0 1.4.0 2017-05-30 2018-05-30
>>> com.clearspring.analytics:stream 2.9.5 2.9.6 2016-08-10 2018-01-10
>>> org.elasticsearch.client:transport 5.0.0 6.2.4 2016-10-25 2018-04-12
>>> org.elasticsearch.plugin:transport-netty4-client 5.6.3 6.2.4 2017-11-06
>>> 2018-04-12
>>> org.tukaani:xz 1.5 1.8 2014-03-08 2018-01-04
>>>
>>>
>>
>> --
>> *Paul Gerver*
>>
>
Re: Beam Dependency Check Report (2018-06-13)
Posted by Ahmet Altay <al...@google.com>.
Looking at the latest report for January 28, there are lots of stale
dependencies. Their associated JIRAs are open for more than 3 months in
some cases. We have a decent policy and tooling to support that, but we are
not following up on those with actions. How could we keep the identified
stale dependencies up to date? Could we mark the issue with a fix version
and triage during releases? That will allow us to at least sort out major
dependency updates from the less urgent ones.
On Mon, Jan 28, 2019 at 9:43 AM Yifan Zou <yi...@google.com> wrote:
> Hi,
>
> You're looking at the old versions dependency bugs which were created
> before Oct, 2018 (e.g BEAM-4904
> <https://issues.apache.org/jira/browse/BEAM-4904>). Based on the
> discussion [1]
> <https://lists.apache.org/thread.html/28d3c349a5021c3598379b6f6b9210b4ef150a6235e55c0499250034@%3Cdev.beam.apache.org%3E>,
> we modified the tool with the new Beam Dependency Policy
> <https://beam.apache.org/contribute/dependencies/>, and closed the old
> bugs (most of them were marked as won't fix, and they will never get
> updated).
>
> The current dependency JIRA looks like this: BEAM-5549
> <https://issues.apache.org/jira/browse/BEAM-5549>. The major changes
> including [2] <https://issues.apache.org/jira/browse/BEAM-5339>:
>
> 1. A JIRA will be created if a dependency has more then 1 major version or
> 3 minor versions behind the latest version. Or, there is new version
> available for more then a year that the dep didn't update in Beam.
> 2. A JIRA could be closed if the new version is not appropriate to be used
> in Beam. In this case, the tool will stop checking updates on this dep
> until the next major version available or after 3 months.
> 3. Stop specifying the target version number in the issue's title. This
> ensures that only one JIRA would be opened for a dep that people can easily
> track the update history.
> 4. Stop directly assigning bugs to a person. Instead, cc owners in the
> descriptions.
>
> Please use the new dependency JIRAs to track the updates. Thanks for
> taking care of Beam dependencies and let me know if you have any questions
> and concerns.
>
> Regards.
> Yifan
>
> [1]:
> https://lists.apache.org/thread.html/28d3c349a5021c3598379b6f6b9210b4ef150a6235e55c0499250034@%3Cdev.beam.apache.org%3E
> [2]: https://issues.apache.org/jira/browse/BEAM-5339
>
> On Mon, Jan 28, 2019 at 6:22 AM Ismaël Mejía <ie...@gmail.com> wrote:
>
>> Hello,
>>
>> The dependency update report has been working fine. However I found some
>> issues that I summarized in this issue.
>> https://issues.apache.org/jira/browse/BEAM-6524
>> Can Yifan or someone else that knows that area please take a look.
>>
>> Regards,
>> Ismaël
>>
>>
>> On Thu, Jun 14, 2018 at 11:37 PM Yifan Zou <yi...@google.com> wrote:
>>
>>> Thank you Paul for letting us know this issue. We will take care of it
>>> when upgrading dependencies.
>>>
>>> On Thu, Jun 14, 2018 at 7:23 AM Paul Gerver <pf...@gmail.com> wrote:
>>>
>>>> I do have one request to be added to the Java SDK version updates:
>>>> Beam-3831 [1]. The Google Core depends on the old org.json package which
>>>> ASF discourages using because of the "Use only for good, not evil" clause.
>>>>
>>>> [1] https://issues.apache.org/jira/browse/BEAM-3831
>>>>
>>>> On Thu, Jun 14, 2018 at 3:03 AM Etienne Chauchot <ec...@apache.org>
>>>> wrote:
>>>>
>>>>> Thanks Yifan,
>>>>>
>>>>> This is great ! It would help us maintain Beam more easily and
>>>>> probably help us fixing CVE as well.
>>>>>
>>>>> Etienne
>>>>>
>>>>> Le mercredi 13 juin 2018 à 07:45 -0700, Yifan Zou a écrit :
>>>>>
>>>>> Hi,
>>>>>
>>>>>
>>>>> I want to follow up and explain this email.
>>>>>
>>>>>
>>>>> This is a sample email that reports the results of Beam SDK dependency
>>>>> check, which was proposed here
>>>>> <https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/edit#heading=h.u75g8bk11ngp>.
>>>>> The goal is finding updates for all Beam Python & Java SDKs' dependencies
>>>>> and prioritize them. The job will be auto triggered in Jenkins once a week
>>>>> and generate a report. The report lists the high priority updates base on
>>>>> the following criteria:
>>>>>
>>>>>
>>>>> The dependency update is high priority if:
>>>>>
>>>>> 1. It has major versions update available;
>>>>>
>>>>> e.g. org.assertj:assertj-core 2.5.0 -> 3.10.0
>>>>>
>>>>> 2. or, it is over 3 minor versions behind the latest version;
>>>>>
>>>>> e.g. org.tukaani:xz 1.5 -> 1.8
>>>>>
>>>>> 3. or, the current version is behind the later version for over 180
>>>>> days.
>>>>>
>>>>> e.g. com.google.auto.service:auto-service 2014-10-24 ->
>>>>> 2017-12-11
>>>>>
>>>>>
>>>>> This job helps Beam contributors to determine the dependency which is
>>>>> far behind the latest released version. The next step would be automating
>>>>> filing JIRA bugs for dep updates, group dependencies and identify owners to
>>>>> take care of the upgrades follow Chamikara's proposal
>>>>> <https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit>
>>>>> .
>>>>>
>>>>>
>>>>> For more readings:
>>>>>
>>>>> [Proposal] Beam dependency check automation
>>>>> <https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/edit#heading=h.u75g8bk11ngp>
>>>>> by Yifan Zou
>>>>>
>>>>> [Proposal] Beam dependency update policy
>>>>> <https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit>
>>>>> by *Chamikara Jayalath*
>>>>>
>>>>> Thank you.
>>>>>
>>>>> Yifan Zou
>>>>>
>>>>> On Wed, Jun 13, 2018 at 7:41 AM Apache Jenkins Server <
>>>>> jenkins@builds.apache.org> wrote:
>>>>>
>>>>> High Priority Dependency Updates Of Beam Python SDK:
>>>>> *Dependency Name* *Current Version* *Later Version* *Current Version
>>>>> Release Date* *Later Version Release Date*
>>>>> google-cloud-bigquery 0.25.0 1.3.0 2017-06-26 2018-06-08
>>>>> httplib2 0.9.2 0.11.3 2015-09-28 2018-03-30 High Priority Dependency
>>>>> Updates Of Beam Java SDK:
>>>>> *Dependency Name* *Current Version* *Later Version* *Current Version
>>>>> Release Date* *Later Version Release Date*
>>>>> org.assertj:assertj-core 2.5.0 3.10.0 2016-07-03 2018-05-11
>>>>> com.google.auto.service:auto-service 1.0-rc2 1.0-rc4 2014-10-24
>>>>> 2017-12-11
>>>>> biz.aQute:bndlib 1.43.0 2.0.0.20130123-133441 2011-04-01 2013-02-27
>>>>> org.apache.cassandra:cassandra-all 3.9 3.11.2 2016-09-26 2018-02-14
>>>>> commons-cli:commons-cli 1.2 1.4 2009-03-19 2017-03-09
>>>>> commons-codec:commons-codec 1.9 1.11 2013-12-20 2017-10-17
>>>>> org.apache.commons:commons-dbcp2 2.1.1 2.3.0 2015-08-02 2018-05-08
>>>>> com.typesafe:config 1.3.0 1.3.3 2015-05-08 2018-02-21
>>>>> de.flapdoodle.embed:de.flapdoodle.embed.mongo 1.50.1 2.0.3 2015-12-11
>>>>> 2018-02-14
>>>>> de.flapdoodle.embed:de.flapdoodle.embed.process 1.50.1 2.0.3
>>>>> 2015-12-11 2018-02-14
>>>>> org.apache.derby:derby 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03
>>>>> org.apache.derby:derbyclient 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03
>>>>> org.apache.derby:derbynet 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03
>>>>> org.elasticsearch:elasticsearch 5.6.3 6.2.4 2017-10-06 2018-04-12
>>>>> org.elasticsearch:elasticsearch-hadoop 5.0.0 6.2.4 2016-10-26
>>>>> 2018-04-12
>>>>> org.elasticsearch.client:elasticsearch-rest-client 5.6.3 6.2.4
>>>>> 2017-10-06 2018-04-12
>>>>> com.alibaba:fastjson 1.2.12 1.2.47 2016-05-21 2018-03-15
>>>>> org.elasticsearch.test:framework 5.6.3 6.2.4 2017-10-06 2018-04-12
>>>>> org.freemarker:freemarker 2.3.25-incubating 2.3.28 2016-06-14
>>>>> 2018-03-30
>>>>> org.codehaus.groovy:groovy-all 2.4.13 3.0.0-alpha-2 2017-11-22
>>>>> 2018-04-16
>>>>> org.apache.hbase:hbase-common 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>>>> 2018-05-31
>>>>> org.apache.hbase:hbase-hadoop-compat 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>>>> 2018-05-31
>>>>> org.apache.hbase:hbase-hadoop2-compat 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>>>> 2018-05-31
>>>>> org.apache.hbase:hbase-server 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>>>> 2018-05-31
>>>>> org.apache.hbase:hbase-shaded-client 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>>>> 2018-05-31
>>>>> org.apache.hbase:hbase-shaded-server 1.2.6 2.0.0-alpha2 2017-05-29
>>>>> 2018-05-31
>>>>> org.apache.hive:hive-cli 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21
>>>>> org.apache.hive:hive-common 2.1.0 3.0.0.3.0.0.3-2 2016-06-16
>>>>> 2018-05-21
>>>>> org.apache.hive:hive-exec 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21
>>>>> org.apache.hive.hcatalog:hive-hcatalog-core 2.1.0 3.0.0.3.0.0.3-2
>>>>> 2016-06-16 2018-05-21
>>>>> org.apache.httpcomponents:httpasyncclient 4.1.2 4.1.3 2016-06-18
>>>>> 2017-02-05
>>>>> org.apache.httpcomponents:httpclient 4.5.2 4.5.5 2016-02-21 2018-01-18
>>>>> org.apache.httpcomponents:httpcore 4.4.5 4.4.9 2016-06-08 2018-01-11
>>>>> net.java.dev.javacc:javacc 4.0 7.0.3 2018-06-08 2017-11-06
>>>>> jline:jline 2.14.6 3.0.0.M1 2018-03-26 2018-06-08
>>>>> net.java.dev.jna:jna 4.1.0 4.5.1 2014-03-06 2017-12-27
>>>>> com.esotericsoftware.kryo:kryo 2.21 2.24.0 2013-02-27 2014-05-04
>>>>> io.dropwizard.metrics:metrics-core 3.1.2 4.1.0-rc2 2015-04-25
>>>>> 2018-05-03
>>>>> org.mongodb:mongo-java-driver 3.2.2 3.8.0-beta3 2016-02-15 2018-05-29
>>>>> io.netty:netty-all 4.1.17.Final 5.0.0.Alpha2 2017-11-08 2018-06-06
>>>>> io.grpc:protoc-gen-grpc-java 1.2.0 1.12.0 2017-03-15 2018-05-07
>>>>> org.apache.qpid:proton-j 0.13.1 0.27.1 2016-07-01 2018-04-25
>>>>> com.carrotsearch.randomizedtesting:randomizedtesting-runner 2.5.0
>>>>> 2.6.3 2017-01-23 2018-06-11
>>>>> org.scala-lang:scala-library 2.11.8 2.13.0-M4 2017-03-08 2018-05-14
>>>>> org.slf4j:slf4j-api 1.7.25 1.8.0-beta2 2017-03-16 2018-03-21
>>>>> org.slf4j:slf4j-jdk14 1.7.25 1.8.0-beta2 2017-03-16 2018-03-21
>>>>> org.apache.solr:solr-core 5.5.4 7.3.1 2017-10-20 2018-05-17
>>>>> org.apache.solr:solr-solrj 5.5.4 7.3.1 2017-10-20 2018-05-17
>>>>> org.apache.solr:solr-test-framework 5.5.4 7.3.1 2017-10-20 2018-05-17
>>>>> org.springframework:spring-expression 4.3.5.RELEASE 5.0.7.RELEASE
>>>>> 2017-01-25 2018-06-12
>>>>> sqlline:sqlline 1.3.0 1.4.0 2017-05-30 2018-05-30
>>>>> com.clearspring.analytics:stream 2.9.5 2.9.6 2016-08-10 2018-01-10
>>>>> org.elasticsearch.client:transport 5.0.0 6.2.4 2016-10-25 2018-04-12
>>>>> org.elasticsearch.plugin:transport-netty4-client 5.6.3 6.2.4
>>>>> 2017-11-06 2018-04-12
>>>>> org.tukaani:xz 1.5 1.8 2014-03-08 2018-01-04
>>>>>
>>>>>
>>>>
>>>> --
>>>> *Paul Gerver*
>>>>
>>>
Re: Beam Dependency Check Report (2018-06-13)
Posted by Yifan Zou <yi...@google.com>.
Hi,
You're looking at the old versions dependency bugs which were created
before Oct, 2018 (e.g BEAM-4904
<https://issues.apache.org/jira/browse/BEAM-4904>). Based on the discussion
[1]
<https://lists.apache.org/thread.html/28d3c349a5021c3598379b6f6b9210b4ef150a6235e55c0499250034@%3Cdev.beam.apache.org%3E>,
we modified the tool with the new Beam Dependency Policy
<https://beam.apache.org/contribute/dependencies/>, and closed the old bugs
(most of them were marked as won't fix, and they will never get updated).
The current dependency JIRA looks like this: BEAM-5549
<https://issues.apache.org/jira/browse/BEAM-5549>. The major changes
including [2] <https://issues.apache.org/jira/browse/BEAM-5339>:
1. A JIRA will be created if a dependency has more then 1 major version or
3 minor versions behind the latest version. Or, there is new version
available for more then a year that the dep didn't update in Beam.
2. A JIRA could be closed if the new version is not appropriate to be used
in Beam. In this case, the tool will stop checking updates on this dep
until the next major version available or after 3 months.
3. Stop specifying the target version number in the issue's title. This
ensures that only one JIRA would be opened for a dep that people can easily
track the update history.
4. Stop directly assigning bugs to a person. Instead, cc owners in the
descriptions.
Please use the new dependency JIRAs to track the updates. Thanks for taking
care of Beam dependencies and let me know if you have any questions and
concerns.
Regards.
Yifan
[1]:
https://lists.apache.org/thread.html/28d3c349a5021c3598379b6f6b9210b4ef150a6235e55c0499250034@%3Cdev.beam.apache.org%3E
[2]: https://issues.apache.org/jira/browse/BEAM-5339
On Mon, Jan 28, 2019 at 6:22 AM Ismaël Mejía <ie...@gmail.com> wrote:
> Hello,
>
> The dependency update report has been working fine. However I found some
> issues that I summarized in this issue.
> https://issues.apache.org/jira/browse/BEAM-6524
> Can Yifan or someone else that knows that area please take a look.
>
> Regards,
> Ismaël
>
>
> On Thu, Jun 14, 2018 at 11:37 PM Yifan Zou <yi...@google.com> wrote:
>
>> Thank you Paul for letting us know this issue. We will take care of it
>> when upgrading dependencies.
>>
>> On Thu, Jun 14, 2018 at 7:23 AM Paul Gerver <pf...@gmail.com> wrote:
>>
>>> I do have one request to be added to the Java SDK version updates:
>>> Beam-3831 [1]. The Google Core depends on the old org.json package which
>>> ASF discourages using because of the "Use only for good, not evil" clause.
>>>
>>> [1] https://issues.apache.org/jira/browse/BEAM-3831
>>>
>>> On Thu, Jun 14, 2018 at 3:03 AM Etienne Chauchot <ec...@apache.org>
>>> wrote:
>>>
>>>> Thanks Yifan,
>>>>
>>>> This is great ! It would help us maintain Beam more easily and probably
>>>> help us fixing CVE as well.
>>>>
>>>> Etienne
>>>>
>>>> Le mercredi 13 juin 2018 à 07:45 -0700, Yifan Zou a écrit :
>>>>
>>>> Hi,
>>>>
>>>>
>>>> I want to follow up and explain this email.
>>>>
>>>>
>>>> This is a sample email that reports the results of Beam SDK dependency
>>>> check, which was proposed here
>>>> <https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/edit#heading=h.u75g8bk11ngp>.
>>>> The goal is finding updates for all Beam Python & Java SDKs' dependencies
>>>> and prioritize them. The job will be auto triggered in Jenkins once a week
>>>> and generate a report. The report lists the high priority updates base on
>>>> the following criteria:
>>>>
>>>>
>>>> The dependency update is high priority if:
>>>>
>>>> 1. It has major versions update available;
>>>>
>>>> e.g. org.assertj:assertj-core 2.5.0 -> 3.10.0
>>>>
>>>> 2. or, it is over 3 minor versions behind the latest version;
>>>>
>>>> e.g. org.tukaani:xz 1.5 -> 1.8
>>>>
>>>> 3. or, the current version is behind the later version for over 180
>>>> days.
>>>>
>>>> e.g. com.google.auto.service:auto-service 2014-10-24 ->
>>>> 2017-12-11
>>>>
>>>>
>>>> This job helps Beam contributors to determine the dependency which is
>>>> far behind the latest released version. The next step would be automating
>>>> filing JIRA bugs for dep updates, group dependencies and identify owners to
>>>> take care of the upgrades follow Chamikara's proposal
>>>> <https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit>
>>>> .
>>>>
>>>>
>>>> For more readings:
>>>>
>>>> [Proposal] Beam dependency check automation
>>>> <https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/edit#heading=h.u75g8bk11ngp>
>>>> by Yifan Zou
>>>>
>>>> [Proposal] Beam dependency update policy
>>>> <https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit>
>>>> by *Chamikara Jayalath*
>>>>
>>>> Thank you.
>>>>
>>>> Yifan Zou
>>>>
>>>> On Wed, Jun 13, 2018 at 7:41 AM Apache Jenkins Server <
>>>> jenkins@builds.apache.org> wrote:
>>>>
>>>> High Priority Dependency Updates Of Beam Python SDK:
>>>> *Dependency Name* *Current Version* *Later Version* *Current Version
>>>> Release Date* *Later Version Release Date*
>>>> google-cloud-bigquery 0.25.0 1.3.0 2017-06-26 2018-06-08
>>>> httplib2 0.9.2 0.11.3 2015-09-28 2018-03-30 High Priority Dependency
>>>> Updates Of Beam Java SDK:
>>>> *Dependency Name* *Current Version* *Later Version* *Current Version
>>>> Release Date* *Later Version Release Date*
>>>> org.assertj:assertj-core 2.5.0 3.10.0 2016-07-03 2018-05-11
>>>> com.google.auto.service:auto-service 1.0-rc2 1.0-rc4 2014-10-24
>>>> 2017-12-11
>>>> biz.aQute:bndlib 1.43.0 2.0.0.20130123-133441 2011-04-01 2013-02-27
>>>> org.apache.cassandra:cassandra-all 3.9 3.11.2 2016-09-26 2018-02-14
>>>> commons-cli:commons-cli 1.2 1.4 2009-03-19 2017-03-09
>>>> commons-codec:commons-codec 1.9 1.11 2013-12-20 2017-10-17
>>>> org.apache.commons:commons-dbcp2 2.1.1 2.3.0 2015-08-02 2018-05-08
>>>> com.typesafe:config 1.3.0 1.3.3 2015-05-08 2018-02-21
>>>> de.flapdoodle.embed:de.flapdoodle.embed.mongo 1.50.1 2.0.3 2015-12-11
>>>> 2018-02-14
>>>> de.flapdoodle.embed:de.flapdoodle.embed.process 1.50.1 2.0.3 2015-12-11
>>>> 2018-02-14
>>>> org.apache.derby:derby 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03
>>>> org.apache.derby:derbyclient 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03
>>>> org.apache.derby:derbynet 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03
>>>> org.elasticsearch:elasticsearch 5.6.3 6.2.4 2017-10-06 2018-04-12
>>>> org.elasticsearch:elasticsearch-hadoop 5.0.0 6.2.4 2016-10-26
>>>> 2018-04-12
>>>> org.elasticsearch.client:elasticsearch-rest-client 5.6.3 6.2.4
>>>> 2017-10-06 2018-04-12
>>>> com.alibaba:fastjson 1.2.12 1.2.47 2016-05-21 2018-03-15
>>>> org.elasticsearch.test:framework 5.6.3 6.2.4 2017-10-06 2018-04-12
>>>> org.freemarker:freemarker 2.3.25-incubating 2.3.28 2016-06-14
>>>> 2018-03-30
>>>> org.codehaus.groovy:groovy-all 2.4.13 3.0.0-alpha-2 2017-11-22
>>>> 2018-04-16
>>>> org.apache.hbase:hbase-common 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>>> 2018-05-31
>>>> org.apache.hbase:hbase-hadoop-compat 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>>> 2018-05-31
>>>> org.apache.hbase:hbase-hadoop2-compat 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>>> 2018-05-31
>>>> org.apache.hbase:hbase-server 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>>> 2018-05-31
>>>> org.apache.hbase:hbase-shaded-client 1.2.6 2.0.0.3.0.0.3-2 2017-05-29
>>>> 2018-05-31
>>>> org.apache.hbase:hbase-shaded-server 1.2.6 2.0.0-alpha2 2017-05-29
>>>> 2018-05-31
>>>> org.apache.hive:hive-cli 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21
>>>> org.apache.hive:hive-common 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21
>>>> org.apache.hive:hive-exec 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21
>>>> org.apache.hive.hcatalog:hive-hcatalog-core 2.1.0 3.0.0.3.0.0.3-2
>>>> 2016-06-16 2018-05-21
>>>> org.apache.httpcomponents:httpasyncclient 4.1.2 4.1.3 2016-06-18
>>>> 2017-02-05
>>>> org.apache.httpcomponents:httpclient 4.5.2 4.5.5 2016-02-21 2018-01-18
>>>> org.apache.httpcomponents:httpcore 4.4.5 4.4.9 2016-06-08 2018-01-11
>>>> net.java.dev.javacc:javacc 4.0 7.0.3 2018-06-08 2017-11-06
>>>> jline:jline 2.14.6 3.0.0.M1 2018-03-26 2018-06-08
>>>> net.java.dev.jna:jna 4.1.0 4.5.1 2014-03-06 2017-12-27
>>>> com.esotericsoftware.kryo:kryo 2.21 2.24.0 2013-02-27 2014-05-04
>>>> io.dropwizard.metrics:metrics-core 3.1.2 4.1.0-rc2 2015-04-25
>>>> 2018-05-03
>>>> org.mongodb:mongo-java-driver 3.2.2 3.8.0-beta3 2016-02-15 2018-05-29
>>>> io.netty:netty-all 4.1.17.Final 5.0.0.Alpha2 2017-11-08 2018-06-06
>>>> io.grpc:protoc-gen-grpc-java 1.2.0 1.12.0 2017-03-15 2018-05-07
>>>> org.apache.qpid:proton-j 0.13.1 0.27.1 2016-07-01 2018-04-25
>>>> com.carrotsearch.randomizedtesting:randomizedtesting-runner 2.5.0 2.6.3
>>>> 2017-01-23 2018-06-11
>>>> org.scala-lang:scala-library 2.11.8 2.13.0-M4 2017-03-08 2018-05-14
>>>> org.slf4j:slf4j-api 1.7.25 1.8.0-beta2 2017-03-16 2018-03-21
>>>> org.slf4j:slf4j-jdk14 1.7.25 1.8.0-beta2 2017-03-16 2018-03-21
>>>> org.apache.solr:solr-core 5.5.4 7.3.1 2017-10-20 2018-05-17
>>>> org.apache.solr:solr-solrj 5.5.4 7.3.1 2017-10-20 2018-05-17
>>>> org.apache.solr:solr-test-framework 5.5.4 7.3.1 2017-10-20 2018-05-17
>>>> org.springframework:spring-expression 4.3.5.RELEASE 5.0.7.RELEASE
>>>> 2017-01-25 2018-06-12
>>>> sqlline:sqlline 1.3.0 1.4.0 2017-05-30 2018-05-30
>>>> com.clearspring.analytics:stream 2.9.5 2.9.6 2016-08-10 2018-01-10
>>>> org.elasticsearch.client:transport 5.0.0 6.2.4 2016-10-25 2018-04-12
>>>> org.elasticsearch.plugin:transport-netty4-client 5.6.3 6.2.4 2017-11-06
>>>> 2018-04-12
>>>> org.tukaani:xz 1.5 1.8 2014-03-08 2018-01-04
>>>>
>>>>
>>>
>>> --
>>> *Paul Gerver*
>>>
>>