You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Parth Chandra <pa...@apache.org> on 2018/03/01 07:16:19 UTC
Re: [DISCUSS] 1.13.0 release
Moved Ted's PR's down in the list. Let's see where we are at the end of the
week.
Arina, Volodymyr, ank ETA on JDK 8 work? It's the gating factor for the
release.
Meanwhile, people, feel free to commit your work as usual.
Updated list:
DRILL-6185: Error is displaying while accessing query profiles via the
Web-UI -- Ready to commit
DRILL-6174: Parquet pushdown planning improvements -- Ready to commit
DRILL-6188: Fix C++ client build on Centos 7 and OS X -- Ready to commit
DRILL-1491: Support for JDK 8 --* In progress.*
DRILL-6191: Need more information on TCP flags -- *In progress*
DRILL-6190: Packets can be bigger than strictly legal -- *In progress*
DRILL-1170: YARN support for Drill -- Needs Committer +1 and Travis fix.
DRILL-6027: Implement spill to disk for the Hash Join --- No PR and is a
major feature that should be reviewed (properly!).
DRILL-6173: Support transitive closure during filter push down and
partition pruning. -- No PR and depends on 3 Apache Calcite issues that
are open.
DRILL-6023: Graceful shutdown improvements -- No PR. Consists of 6 sub
JIra's none of which have PRs.
On Wed, Feb 28, 2018 at 5:45 PM, Ted Dunning <te...@gmail.com> wrote:
> 6190 and/or 6191 cause test failures that I have been unable to spend time
> on yet. I don't think that they are ready to commit.
>
> At least one of these is likely to be something very simple like a test
> that didn't clean up after itself. The other should be as simple, but I
> can't understand it yet. It may be a memory pressure thing rather than a
> real problem with the test.
>
>
> On Wed, Feb 28, 2018 at 3:18 AM, Parth Chandra <pa...@apache.org> wrote:
>
> > OK. So let's try to get as many of the following as we can without
> breaking
> > anything. As far as I can see none of the open items below are show
> > stoppers for a release, but I'm happy to give in to popular demand for
> JDK
> > 8 :).
> >
> > Note that the last three appear to be big ticket items that have no PR
> yet.
> > Usually, it is a mistake to rush these into a release (one advantage of
> > frequent, predictable releases is that they won't have to wait too long
> for
> > the next release).
> >
> > Here's what I'm tracking :
> >
> > DRILL-6185: Error is displaying while accessing query profiles via the
> > Web-UI -- Ready to commit
> > DRILL-6174: Parquet pushdown planning improvements -- Ready to commit
> > DRILL-6191: Need more information on TCP flags -- Ready to commit
> > DRILL-6190: Packets can be bigger than strictly legal -- Ready to commit
> >
> > DRILL-6188: Fix C++ client build on Centos 7 and OS X -- Needs
> committer
> > +1
> >
> > DRILL-1491: Support for JDK 8 --* In progress.*
> >
> > DRILL-1170: YARN support for Drill -- Needs Committer +1 and Travis fix.
> >
> > DRILL-6027: Implement spill to disk for the Hash Join --- No PR and is
> a
> > major feature that should be reviewed (properly!).
> >
> > DRILL-6173: Support transitive closure during filter push down and
> > partition pruning. -- No PR and depends on 3 Apache Calcite issues that
> > are open.
> >
> > DRILL-6023: Graceful shutdown improvements -- No PR. Consists of 6 sub
> > JIra's none of which have PRs.
> >
> >
> >
> >
> >
> >
> >
> >
> > On Wed, Feb 28, 2018 at 12:32 AM, Ted Dunning <te...@gmail.com>
> > wrote:
> >
> > > I have two very small improvements to PCAP support with DRILL-6190 and
> > > DRILL-6191 that I would like to get in.
> > >
> > > I think that PCAP-NG support is too far from ready.
> > >
> > >
> > >
> > > On Tue, Feb 27, 2018 at 10:52 AM, Pritesh Maker <pm...@mapr.com>
> wrote:
> > >
> > > > I see a few more issues that are in review and worth including for
> the
> > > > 1.13 release (maybe give another week to resolve this before the 1st
> RC
> > > is
> > > > created?)
> > > >
> > > > DRILL-6027 Implement spill to disk for the Hash Join -- Boaz and Tim
> > > > DRILL-6173 Support transitive closure during filter push down and
> > > > partition pruning - Vitalii
> > > > DRILL-6023 Graceful shutdown improvements -- Jyothsna
> > > >
> > > > There are several other bugs/ improvements that are marked in
> progress
> > -
> > > > https://issues.apache.org/jira/secure/Dashboard.jspa?
> > > selectPageId=12332152
> > > > - if folks are not working on them, we should remove the fixVersion
> for
> > > > 1.13.
> > > >
> > > > Pritesh
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Abhishek Girish <ag...@apache.org>
> > > > Sent: February 27, 2018 10:44 AM
> > > > To: dev@drill.apache.org
> > > > Subject: Re: [DISCUSS] 1.13.0 release
> > > >
> > > > For JDK 8, we've run through both unit tests & regression tests from
> > [1]
> > > > and have observed no issues - so i think we should get the fixes into
> > > > 1.13.0 and claim support for JDK 8.
> > > >
> > > >
> > > > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.
> > > > com_mapr_drill-2Dtest-2Dframework&d=DwIBaQ&c=
> cskdkSMqhcnjZxdQVpwTXg&r=
> > > > zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > 1fr4O7tpGKl1SddY0&s=
> > > > J6FGSG0CSPNSx1abvNe53qFNBwCr3UaO8ILqmUYfuWg&e=
> > > >
> > > > On Tue, Feb 27, 2018 at 9:46 AM, Aman Sinha <am...@apache.org>
> > > wrote:
> > > >
> > > > > Agree with Arina on JDK8 support..let's get through the last
> > remaining
> > > > > hurdles for this for 1.13 release.
> > > > > Calcite in fact is dropping support for JDK 7 in their next
> release.
> > > > > They are even looking ahead to JDK 9.
> > > > >
> > > > > For Drill-on-Yarn, I think ultimately it is at the discretion of
> the
> > > > > release manager. My personal opinion is as long as a committer (in
> > > > > this case Arina) feels comfortable with it, and given that the
> > > > > functionality has been well-tested, I am okay to include it.
> > > > >
> > > > > -Aman
> > > > >
> > > > > On Tue, Feb 27, 2018 at 8:51 AM, Arina Yelchiyeva <
> > > > > arina.yelchiyeva@gmail.com> wrote:
> > > > >
> > > > > > I want to include DRILL-6174. It has already passed code review.
> > > > > >
> > > > > > Regarding JDK8 support. Volodymyr Tkach is working on this issue.
> > > > > Currently
> > > > > > all unit tests have passed. Now he is working on enforcing Java 8
> > > > > (changes
> > > > > > in travis.yml, drill-config.sh, pom.xml etc).
> > > > > >
> > > > > > Regarding Drill on Yarn, Salim has done code review. Failing
> Travis
> > > > > > check is easy to fix. Tim has already proposed the solution (Paul
> > > > > > just needs to add the dependency).
> > > > > > I think its safe to include these changes in this release. They
> go
> > > > > > in separate module and have no impact any existing functionality.
> > > > > > Even if there are some flows, users will be able to give it a try
> > > > > > and feedback in case of issues.
> > > > > >
> > > > > > On Tue, Feb 27, 2018 at 8:53 AM, Parth Chandra <
> parthc@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > There are two issues marked as blockers for 1.13.0 - TPatch
> > > > > > > InfoKeySummaryReporterPStatusResolutionCreatedUpdatedDueFix
> > > > > > > Version/s
> > > > > > > Assignee
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_browse_DRILL-2D6185&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > 1fr4O7tpGKl1Sd
> > > > > > > dY0&s=ZFTj2m_MKC_o4_s9du-VuNWvzjx4ab8CJLGZMzhGkGg&e=>[image:
> > Bug]
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_browse_DRILL-2D6185&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > 1fr4O7tpGKl1Sd
> > > > > > > dY0&s=ZFTj2m_MKC_o4_s9du-VuNWvzjx4ab8CJLGZMzhGkGg&e=>
> DRILL-6185
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_browse_DRILL-2D6185&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > 1fr4O7tpGKl1Sd
> > > > > > > dY0&s=ZFTj2m_MKC_o4_s9du-VuNWvzjx4ab8CJLGZMzhGkGg&e=>
> > > > > > >
> > > > > > > Error is displaying while accessing query profiles via the
> Web-UI
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_browse_DRILL-2D6185&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > 1fr4O7tpGKl1Sd
> > > > > > > dY0&s=ZFTj2m_MKC_o4_s9du-VuNWvzjx4ab8CJLGZMzhGkGg&e=>
> > > > > > > Anton Gozhiy
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_secure_ViewProfile.jspa-3Fname-3Dangozhiy&d=
> > DwIBaQ&c=cs
> > > > > > > kdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=DWH_
> > joEaRjyZJAvkuL
> > > > > > > xyCK5ln-1fr4O7tpGKl1SddY0&s=RgS2wqYAu_
> > VHlOHCeLA1RDhSkXdeB4iBFDoE5k
> > > > > > > xlbAA&e=>
> > > > > > > [image:
> > > > > > > Blocker] OPEN *Unresolved* 26/Feb/18 26/Feb/18 1.13.0
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_issues_-3Fjql-3Dproject-2B-253D-2B&d=DwIBaQ&
> > c=cskdkSMqh
> > > > > > > cnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=DWH_
> > joEaRjyZJAvkuLxyCK5ln
> > > > > > > -1fr4O7tpGKl1SddY0&s=wWbDmtFEWvWuGsnOIe1NCWslFGraA6
> > kS2EYn1NKpYsA&e
> > > > > > > =
> > > > > > > DRILL+AND+fixVersion+%3D+1.13.0>
> > > > > > > Kunal
> > > > > > > Khatua
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_secure_ViewProfile.jspa-3F&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQV
> > > > > > > pwTXg&r=zySISmkmM4WNViCKijENtQ&m=DWH_
> > joEaRjyZJAvkuLxyCK5ln-1fr4O7t
> > > > > > > pGKl1SddY0&s=roeA-sW1VokgfvGn5dw37qWNIjBku2uKnpjCKJqaeg0&e=
> > > > > > > name=kkhatua>
> > > > > > > Actions
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_rest_api_1.0_issues_&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQVpwTXg&
> > > > > > > r=zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > 1fr4O7tpGKl1S
> > > > > > > ddY0&s=4YiyewNsaFHgEIQkn3y3tfUgczHbSpkn-9WCR0EJjCM&e=
> > > > > > > 13140915/ActionsAndOperations?atl_token=A5KQ-2QAV-T4JA-FDED|
> > > > > > > 582202a248949006c997dc02e0361c023d2c491d|lin>
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_browse_DRILL-2D1491&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > 1fr4O7tpGKl1Sd
> > > > > > > dY0&s=WnrFj6sDbesjt-UWX1QEMR0fnZxNDREzvU6YtHaavvs&e=>[image:
> > Task]
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_browse_DRILL-2D1491&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > 1fr4O7tpGKl1Sd
> > > > > > > dY0&s=WnrFj6sDbesjt-UWX1QEMR0fnZxNDREzvU6YtHaavvs&e=>
> DRILL-1491
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_browse_DRILL-2D1491&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > 1fr4O7tpGKl1Sd
> > > > > > > dY0&s=WnrFj6sDbesjt-UWX1QEMR0fnZxNDREzvU6YtHaavvs&e=>
> > > > > > >
> > > > > > > Support for JDK 8
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_browse_DRILL-2D1491&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > 1fr4O7tpGKl1Sd
> > > > > > > dY0&s=WnrFj6sDbesjt-UWX1QEMR0fnZxNDREzvU6YtHaavvs&e=>
> > > > > > > Aditya Kishore
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_secure_ViewProfile.jspa-3F&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQV
> > > > > > > pwTXg&r=zySISmkmM4WNViCKijENtQ&m=DWH_
> > joEaRjyZJAvkuLxyCK5ln-1fr4O7t
> > > > > > > pGKl1SddY0&s=roeA-sW1VokgfvGn5dw37qWNIjBku2uKnpjCKJqaeg0&e=
> > > > > > name=adityakishore
> > > > > > > >
> > > > > > > [image:
> > > > > > > Blocker] OPEN *Unresolved* 03/Oct/14 22/Feb/18 1.13.0
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_issues_-3Fjql-3Dproject-2B-253D-2B&d=DwIBaQ&
> > c=cskdkSMqh
> > > > > > > cnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=DWH_
> > joEaRjyZJAvkuLxyCK5ln
> > > > > > > -1fr4O7tpGKl1SddY0&s=wWbDmtFEWvWuGsnOIe1NCWslFGraA6
> > kS2EYn1NKpYsA&e
> > > > > > > =
> > > > > > > DRILL+AND+fixVersion+%3D+1.13.0>
> > > > > > > Volodymyr
> > > > > > > Tkach
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_secure_ViewProfile.jspa-3F&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQV
> > > > > > > pwTXg&r=zySISmkmM4WNViCKijENtQ&m=DWH_
> > joEaRjyZJAvkuLxyCK5ln-1fr4O7t
> > > > > > > pGKl1SddY0&s=roeA-sW1VokgfvGn5dw37qWNIjBku2uKnpjCKJqaeg0&e=
> > > > > > > name=volodymyr.tkach>
> > > > > > > Actions
> > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__issues.apach
> > > > > > > e.org_jira_rest_api_1.0_issues_&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQVpwTXg&
> > > > > > > r=zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > 1fr4O7tpGKl1S
> > > > > > > ddY0&s=4YiyewNsaFHgEIQkn3y3tfUgczHbSpkn-9WCR0EJjCM&e=
> > > > > > > 12745818/ActionsAndOperations?atl_token=A5KQ-2QAV-T4JA-FDED|
> > > > > > > 582202a248949006c997dc02e0361c023d2c491d|lin>
> > > > > > >
> > > > > > > DRILL-6185 is definitely a blocker. We cannot do a release
> with a
> > > > > > > UI regression.
> > > > > > >
> > > > > > > From a quick perusal of DRILL-1491, JDK-8 support has issues
> that
> > > > > > > no
> > > > > one
> > > > > > is
> > > > > > > looking at. Is someone addressing these and JIRA has not been
> > > > updated?
> > > > > > >
> > > > > > > Also, - Drill 1170: Drill-on-YARN #1011, is failing Travis
> > checks.
> > > > > Call
> > > > > > me
> > > > > > > paranoid but a 21K line PR with one review comment doesn't
> sound
> > > > > > > like
> > > > > the
> > > > > > > review is completed. Any committer looking at it?
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Feb 27, 2018 at 9:28 AM, Ted Dunning
> > > > > > > <te...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > There is a pcap improvement about to be ready. And a bug
> (with
> > a
> > > > > > > > fix)
> > > > > > > that
> > > > > > > > just turned up. I will push for a PR
> > > > > > > >
> > > > > > > > On Feb 23, 2018 00:14, "Parth Chandra" <pa...@apache.org>
> > > wrote:
> > > > > > > >
> > > > > > > > > Bit of a tepid response from dev; but Aman's approval is
> all
> > > > > > > > > the encouragement I need to roll out a release :)
> > > > > > > > >
> > > > > > > > > Thoughts on pending PRs?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Thu, Feb 22, 2018 at 9:54 PM, Aman Sinha
> > > > > > > > > <am...@apache.org>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Agreed...it would be good to get the ball rolling on the
> > > > > > > > > > 1.13.0
> > > > > > > > release.
> > > > > > > > > > Among other things, this release has the long pending
> > > > > > > > > > Calcite rebase changes and the sooner we get
> > > > > > it
> > > > > > > it
> > > > > > > > > out
> > > > > > > > > > for users, the better.
> > > > > > > > > >
> > > > > > > > > > Thanks for volunteering !
> > > > > > > > > >
> > > > > > > > > > -Aman
> > > > > > > > > >
> > > > > > > > > > On Wed, Feb 21, 2018 at 9:03 PM, Parth Chandra <
> > > > > parthc@apache.org>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hello Drillers,
> > > > > > > > > > >
> > > > > > > > > > > I feel we might benefit from a early release for
> > 1.13.0.
> > > > > > > > > > > We
> > > > > > took
> > > > > > > > > longer
> > > > > > > > > > > to do the previous release so it would be nice to bring
> > > > > > > > > > > the
> > > > > > release
> > > > > > > > > train
> > > > > > > > > > > back on track.
> > > > > > > > > > >
> > > > > > > > > > > I'll volunteer (!) to manage the release :)
> > > > > > > > > > >
> > > > > > > > > > > What do you guys think?
> > > > > > > > > > >
> > > > > > > > > > > If we are in agreement on starting the release cycle
> > and
> > > > > there
> > > > > > > are
> > > > > > > > > any
> > > > > > > > > > > issues on which work is in progress, that you feel we
> > > > > > > > > > > *must*
> > > > > > > include
> > > > > > > > in
> > > > > > > > > > the
> > > > > > > > > > > release, please post in reply to this thread. Let's at
> > > > > > > > > > > least
> > > > > get
> > > > > > a
> > > > > > > > head
> > > > > > > > > > > start on closing pending PRs since these are usually
> what
> > > > > delays
> > > > > > > > > > releases.
> > > > > > > > > > >
> > > > > > > > > > > Thanks
> > > > > > > > > > >
> > > > > > > > > > > Parth
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
So it is. My mistake :). Travis is good. Will start the release process.
On Mon, Mar 12, 2018 at 6:00 PM, Arina Yelchiyeva <
arina.yelchiyeva@gmail.com> wrote:
> Parth,
>
> looks like build on Travis is still running [1].
>
> [1]
> https://travis-ci.org/apache/drill/builds/352294695?utm_
> source=github_status&utm_medium=notification
>
>
> Kind regards
> Arina
>
> On Mon, Mar 12, 2018 at 2:26 PM, Parth Chandra <pa...@apache.org> wrote:
>
> > Travis build is broken after the merge. I'm almost sure this was already
> > addressed. @Volodymyr I thought this was part of the PR for DRILL-1491?
> >
> > On Sat, Mar 10, 2018 at 11:08 AM, Parth Chandra <pa...@apache.org>
> wrote:
> >
> > > Update on DRILL-1491 -
> > > The illegal state transition (cancellation_requested -> enqueued) is
> no
> > > longer happening for me after I updated my JDK to the latest version.
> > I'll
> > > try on and off over the weekend and if it cannot be reproduced, I'll
> let
> > it
> > > go.
> > >
> > > Release plan -
> > > I'll merge the PRs for DRILL-1491 and DRILL-4547 in a bit.
> > > I'm planning on rolling out release candidate RC0 on Monday morning
> > > (PDT). I'll create a branch for the release.
> > > Apache master is still open for commits. (Try not to break anything).
> > >
> > >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Arina Yelchiyeva <ar...@gmail.com>.
Parth,
looks like build on Travis is still running [1].
[1]
https://travis-ci.org/apache/drill/builds/352294695?utm_source=github_status&utm_medium=notification
Kind regards
Arina
On Mon, Mar 12, 2018 at 2:26 PM, Parth Chandra <pa...@apache.org> wrote:
> Travis build is broken after the merge. I'm almost sure this was already
> addressed. @Volodymyr I thought this was part of the PR for DRILL-1491?
>
> On Sat, Mar 10, 2018 at 11:08 AM, Parth Chandra <pa...@apache.org> wrote:
>
> > Update on DRILL-1491 -
> > The illegal state transition (cancellation_requested -> enqueued) is no
> > longer happening for me after I updated my JDK to the latest version.
> I'll
> > try on and off over the weekend and if it cannot be reproduced, I'll let
> it
> > go.
> >
> > Release plan -
> > I'll merge the PRs for DRILL-1491 and DRILL-4547 in a bit.
> > I'm planning on rolling out release candidate RC0 on Monday morning
> > (PDT). I'll create a branch for the release.
> > Apache master is still open for commits. (Try not to break anything).
> >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
Travis build is broken after the merge. I'm almost sure this was already
addressed. @Volodymyr I thought this was part of the PR for DRILL-1491?
On Sat, Mar 10, 2018 at 11:08 AM, Parth Chandra <pa...@apache.org> wrote:
> Update on DRILL-1491 -
> The illegal state transition (cancellation_requested -> enqueued) is no
> longer happening for me after I updated my JDK to the latest version. I'll
> try on and off over the weekend and if it cannot be reproduced, I'll let it
> go.
>
> Release plan -
> I'll merge the PRs for DRILL-1491 and DRILL-4547 in a bit.
> I'm planning on rolling out release candidate RC0 on Monday morning
> (PDT). I'll create a branch for the release.
> Apache master is still open for commits. (Try not to break anything).
>
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
Update on DRILL-1491 -
The illegal state transition (cancellation_requested -> enqueued) is no
longer happening for me after I updated my JDK to the latest version. I'll
try on and off over the weekend and if it cannot be reproduced, I'll let it
go.
Release plan -
I'll merge the PRs for DRILL-1491 and DRILL-4547 in a bit.
I'm planning on rolling out release candidate RC0 on Monday morning
(PDT). I'll create a branch for the release.
Apache master is still open for commits. (Try not to break anything).
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
Latest status -
DRILL-1491 - Will continue to investigate the Cancel failure over the
weekend. Others please feel free to pitch in.
DRILL-6216 - Resolved.
DRILL-4547 - Use Jyothsna's patch. The JIRA should remain open so that we
can fix the Javadoc.
Meanwhile, please report any issues found in the latest master immediately.
On Fri, Mar 9, 2018 at 1:59 PM, Parth Chandra <pa...@apache.org> wrote:
> Resolved this. Needed to upgrade my JDK. I would rather we had fixed the
> actual javadoc but this will have to do.
>
>
>
> On Fri, Mar 9, 2018 at 12:20 PM, Parth Chandra <pa...@apache.org> wrote:
>
>> Running the build with the apache-release profile still fails with a
>> different error. (The release script gives the same error).
>> Anyone have an idea what this one is? A quick web search did not help.
>>
>>
>> mvn clean install -DskipTests -Papache-release
>>
>>
>> ...
>>
>>
>> [INFO] javadoc: error - com.sun.tools.doclets.internal
>> .toolkit.util.DocletAbortException: com.sun.tools.javac.code.Symbol$CompletionFailure:
>> class file for org.junit.runners.model.Statement not found
>>
>> On Fri, Mar 9, 2018 at 7:01 AM, Aman Sinha <am...@apache.org> wrote:
>>
>>> Hmm...322 warnings seems too much to address in a day or two. Parth, if
>>> you are able to complete a build (with warnings but no errors) with the
>>> branch link that Jyotsna I think we could go ahead with making the change
>>> in the pom file and fix the warnings post 1.13.0. Any thoughts ?
>>>
>>> Also, note that this would mean that anyone who wants to build the source
>>> with JDK 7 would have to use the 1.12 release.
>>>
>>> -Aman
>>>
>>> On Thu, Mar 8, 2018 at 3:59 PM, Jyothsna Reddy <jy...@gmail.com>
>>> wrote:
>>>
>>> > When I disabled doclint and there were around 322 java doc warnings.
>>> I'm
>>> > not sure if we can fix all of them and I'm attaching a link to the log
>>> file
>>> > that contains all the warnings. Please let me know your thoughts on
>>> this.
>>> >
>>> > https://drive.google.com/file/d/1GQ6xORr0HkoEmorzvTcl0XZS4NeLX
>>> > u83/view?usp=sharing
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Thu, Mar 8, 2018 at 10:39 AM, Jyothsna Reddy <
>>> jyothsna.dvj@gmail.com>
>>> > wrote:
>>> >
>>> > > I tried to disable it using -Dadditionalparam=-Xdoclint:none.
>>> Somewhere
>>> > I
>>> > > have read that -DXdoclint:none is for older versions of maven. But
>>> I'm
>>> > not
>>> > > sure of it.
>>> > >
>>> > >
>>> > >
>>> > >
>>> > > On Thu, Mar 8, 2018 at 10:18 AM, Jyothsna Reddy <
>>> jyothsna.dvj@gmail.com>
>>> > > wrote:
>>> > >
>>> > >> Following branch has changed related to disabling doclint in
>>> pom.xml.
>>> > >> Please check it out.
>>> > >>
>>> > >> https://github.com/dvjyothsna/drill.git DRILL-4547
>>> > >>
>>> > >>
>>> > >>
>>> > >>
>>> > >>
>>> > >>
>>> > >> On Thu, Mar 8, 2018 at 4:19 AM, Parth Chandra <pa...@apache.org>
>>> > wrote:
>>> > >>
>>> > >>> The issue with the failing test in TestDrillbitResilience.
>>> > >>> cancelAfterAllResultsProduced is similar to DRILL-3967 (
>>> > >>> TestDrillbitResilience.cancelAfterEverythingIsCompleted failure).
>>> > >>>
>>> > >>> In both this test case, and DRILL-3967, the query is paused (in
>>> > different
>>> > >>> places) and a cancel is sent. The query is then resumed and the
>>> > resulting
>>> > >>> state is checked. The problem is that the tests have a race
>>> condition
>>> > >>> between the cancellation and the resuming of the query. Sometimes
>>> the
>>> > >>> resume reaches first and sometimes the cancel reaches first. The
>>> > failure
>>> > >>> described by Volodymyr is caused by this race condition. I don't
>>> know
>>> > why
>>> > >>> the test was done like this, but this is an existing problem and
>>> > >>> shouldn't
>>> > >>> hold up the release.
>>> > >>>
>>> > >>> However, I also see a failure where we encounter an illegal state
>>> > >>> transition (the query state is CANCELLATION_REQUESTED and the
>>> Foreman
>>> > >>> tries
>>> > >>> to move to an ENQUEUED state). This happens once in about
>>> twenty-five
>>> > >>> executions. The Foreman only requests ENQUEUE once, when the query
>>> is
>>> > >>> about
>>> > >>> to start, so this means the cancellation request reached before the
>>> > query
>>> > >>> start request. How this happens in the unit test I have not been
>>> able
>>> > to
>>> > >>> determine yet (because it really shouldn't be possible).
>>> > >>>
>>> > >>> To recreate the problem I simply added a repeat rule in the class
>>> and
>>> > >>> set a
>>> > >>> repeat count of 1000. The problem occurs easily if the test is run
>>> from
>>> > >>> the
>>> > >>> command line. When running in debug, I was unable to see the
>>> problem.
>>> > >>>
>>> > >>> I'll spend some more time on this, but just in case someone wants
>>> to
>>> > >>> investigate further, feel free ...
>>> > >>>
>>> > >>>
>>> > >>> On Thu, Mar 8, 2018 at 10:59 AM, Parth Chandra <pa...@apache.org>
>>> > >>> wrote:
>>> > >>>
>>> > >>> > Not sure if that would work. The release build does not allow
>>> > >>> uncommitted
>>> > >>> > files, so I have to commit pom.xml changes to at least the local
>>> > repo,
>>> > >>> > which will get pushed to my public repo when the release is
>>> done. Not
>>> > >>> > committing this to Apache master would be cheating would leave us
>>> > with
>>> > >>> a
>>> > >>> > build that does not match any source in Apache master? Javadoc
>>> > >>> generated is
>>> > >>> > never committed to any repo. It is part of the src release jars,
>>> > AFAIK.
>>> > >>> >
>>> > >>> > Also, I'm not sure where in the pom Jyothsna made the change; I
>>> added
>>> > >>> the
>>> > >>> > -Dxoclint:none to the build section of the apache-release
>>> profile and
>>> > >>> java
>>> > >>> > exec still gives over 100 javadoc errors.
>>> > >>> >
>>> > >>> > We have to fix these one of these days. Might as well do it now.
>>> > >>> Knowing
>>> > >>> > how it works, if we don't fix these now, someone will be
>>> scrambling
>>> > to
>>> > >>> fix
>>> > >>> > these just before the 1.14.0 release :(
>>> > >>> >
>>> > >>> >
>>> > >>> >
>>> > >>> > On Thu, Mar 8, 2018 at 10:06 AM, Aman Sinha <
>>> amansinha@apache.org>
>>> > >>> wrote:
>>> > >>> >
>>> > >>> >> Parth, would it work if you made the pom.xml changes locally in
>>> > your
>>> > >>> >> branch, generated the javadoc but only commit the javadoc jar
>>> files
>>> > >>> to the
>>> > >>> >> release branch, not the pom.xml changes ?
>>> > >>> >> Anyone downloading Drill source code to build should not run
>>> into
>>> > this
>>> > >>> >> since typically they won't be building javadoc.
>>> > >>> >>
>>> > >>> >> -Aman
>>> > >>> >>
>>> > >>> >> On Wed, Mar 7, 2018 at 6:37 PM, Parth Chandra <
>>> parthc@apache.org>
>>> > >>> wrote:
>>> > >>> >>
>>> > >>> >> > Unfortunately, we cannot do that since we also want to be
>>> able to
>>> > >>> build
>>> > >>> >> > with JDK 7 for at least a couple of releases to allow for a
>>> > >>> reasonable
>>> > >>> >> > transition time. doclint was introduced in JDK 8 so JDK 7
>>> fails
>>> > >>> >> because it
>>> > >>> >> > doesn't recognize the parameter.
>>> > >>> >> >
>>> > >>> >> >
>>> > >>> >> >
>>> > >>> >> > On Thu, Mar 8, 2018 at 7:03 AM, Jyothsna Reddy <
>>> > >>> jyothsna.dvj@gmail.com>
>>> > >>> >> > wrote:
>>> > >>> >> >
>>> > >>> >> > > Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491)
>>> and
>>> > >>> added
>>> > >>> >> > > following lines to pom.xml to disable doc lint. The javadoc
>>> > >>> doesn't
>>> > >>> >> throw
>>> > >>> >> > > any errors and the build is successful.
>>> > >>> >> > >
>>> > >>> >> > > <activation>
>>> > >>> >> > >
>>> > >>> >> > > <jdk>[1.8,)</jdk>
>>> > >>> >> > >
>>> > >>> >> > > </activation>
>>> > >>> >> > >
>>> > >>> >> > > <properties>
>>> > >>> >> > >
>>> > >>> >> > > <additionalparam>-Xdoclint:none</additionalparam>
>>> > >>> >> > >
>>> > >>> >> > > </properties>
>>> > >>> >> > >
>>> > >>> >> > >
>>> > >>> >> > >
>>> > >>> >> > >
>>> > >>> >> > >
>>> > >>> >> > > On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri <
>>> > >>> >> hanu.ncr@gmail.com>
>>> > >>> >> > > wrote:
>>> > >>> >> > >
>>> > >>> >> > > > On my machine I couldn't repro the issue related to
>>> > >>> >> > > TestDrillbitResilience.
>>> > >>> >> > > > cancelAfterAllResultsProduced.
>>> > >>> >> > > > I used the vladimir's branch (i.e DRILL-1491).
>>> > >>> >> > > > Used the maven test command for testing it.
>>> > >>> >> > > >
>>> > >>> >> > > > output of the test run.
>>> > >>> >> > > > ... 4 common frames omitted
>>> > >>> >> > > > Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time
>>> > elapsed:
>>> > >>> >> > 124.187
>>> > >>> >> > > > sec - in org.apache.drill.exec.server.T
>>> estDrillbitResilience
>>> > >>> >> > > >
>>> > >>> >> > > >
>>> > >>> >> > > >
>>> > >>> >> > > >
>>> > >>> >> > > > On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <
>>> > >>> parthc@apache.org>
>>> > >>> >> > > wrote:
>>> > >>> >> > > >
>>> > >>> >> > > > > Yes I agree. JDBC would be a new feature that we can
>>> defer
>>> > to
>>> > >>> >> 1.14.0.
>>> > >>> >> > > > > I'm hoping we can resolve the other three in the next
>>> few
>>> > >>> days.
>>> > >>> >> > Target
>>> > >>> >> > > > date
>>> > >>> >> > > > > for starting release process - Friday Mar 9th
>>> > >>> >> > > > >
>>> > >>> >> > > > > Once these are resolved, I will create a branch for the
>>> > >>> release so
>>> > >>> >> > that
>>> > >>> >> > > > > Apache master remains open for commits. If any issues
>>> are
>>> > >>> found in
>>> > >>> >> > the
>>> > >>> >> > > > > release branch, we will fix them in master and I will
>>> > >>> cherry-pick
>>> > >>> >> the
>>> > >>> >> > > > into
>>> > >>> >> > > > > the release branch. Once the release is finalized I will
>>> > add a
>>> > >>> >> > release
>>> > >>> >> > > > tag
>>> > >>> >> > > > > and remove the branch.
>>> > >>> >> > > > >
>>> > >>> >> > > > > Also note if QA folks want to get started on testing the
>>> > >>> release,
>>> > >>> >> the
>>> > >>> >> > > > > current head of Apache master is close to final. Javadoc
>>> > >>> >> generation
>>> > >>> >> > is
>>> > >>> >> > > > only
>>> > >>> >> > > > > a release build issue, and the other issues are
>>> localized
>>> > to
>>> > >>> >> > specific
>>> > >>> >> > > > > cases.
>>> > >>> >> > > > >
>>> > >>> >> > > > > Note: to reproduce the javadoc issues:
>>> > >>> >> > > > > # set JAVA_HOME to JDK 8
>>> > >>> >> > > > > mvn javadoc:javadoc -Papache-release
>>> > >>> >> > > > >
>>> > >>> >> > > > >
>>> > >>> >> > > > >
>>> > >>> >> > > > > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <
>>> > >>> amansinha@apache.org
>>> > >>> >> >
>>> > >>> >> > > > wrote:
>>> > >>> >> > > > >
>>> > >>> >> > > > > > It seems to me the main blockers are:
>>> > >>> >> > > > > >
>>> > >>> >> > > > > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we
>>> > >>> split up
>>> > >>> >> the
>>> > >>> >> > > > work
>>> > >>> >> > > > > > among few people to resolve these ?
>>> > >>> >> > > > > > 2. DRILL-6216 Metadata mismatch.. <--
>>> Agreement
>>> > >>> was
>>> > >>> >> to
>>> > >>> >> > > > revert
>>> > >>> >> > > > > > one small piece of code and it appears Sorabh is
>>> looking
>>> > >>> into it
>>> > >>> >> > > > > > 3. TestDrillbitResilience.cancelA
>>> fterAllResultsProduced
>>> > >>> <--
>>> > >>> >> need
>>> > >>> >> > > > > someone
>>> > >>> >> > > > > > to look into this
>>> > >>> >> > > > > >
>>> > >>> >> > > > > > Regarding the JDBC issues that Parth mentioned,
>>> looking at
>>> > >>> the
>>> > >>> >> > JIRAs,
>>> > >>> >> > > > it
>>> > >>> >> > > > > > seems they are not showstoppers...Parth do you agree ?
>>> > >>> >> > > > > >
>>> > >>> >> > > > > > Since we are close to the finish line for JDK 8, IMO
>>> we
>>> > >>> should
>>> > >>> >> try
>>> > >>> >> > > and
>>> > >>> >> > > > > see
>>> > >>> >> > > > > > if in another day or two we can get over these
>>> hurdles.
>>> > >>> >> > > > > >
>>> > >>> >> > > > > > -Aman
>>> > >>> >> > > > > >
>>> > >>> >> > > > > >
>>> > >>> >> > > > > >
>>> > >>> >> > > > > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <
>>> > >>> pmaker@mapr.com>
>>> > >>> >> > > wrote:
>>> > >>> >> > > > > >
>>> > >>> >> > > > > > > The JDK 8 issues will likely require more time to
>>> harden
>>> > >>> for
>>> > >>> >> it
>>> > >>> >> > to
>>> > >>> >> > > be
>>> > >>> >> > > > > > > included in the 1.13 release. My recommendation
>>> would be
>>> > >>> to
>>> > >>> >> move
>>> > >>> >> > > > ahead
>>> > >>> >> > > > > > with
>>> > >>> >> > > > > > > the 1.13 release now and address these issues right.
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > Pritesh
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > -----Original Message-----
>>> > >>> >> > > > > > > From: Parth Chandra <pa...@apache.org>
>>> > >>> >> > > > > > > Sent: March 7, 2018 3:34 AM
>>> > >>> >> > > > > > > To: dev <de...@drill.apache.org>
>>> > >>> >> > > > > > > Subject: Re: [DISCUSS] 1.13.0 release
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > My mistake Volodymyr.
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > Found some other JDK 8 issues in JIRA not tracked in
>>> > >>> >> DRILL-1491
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > DRILL-4547 Javadoc fails with Java8
>>> > >>> >> > > > > > > DRILL-6163 Switch Travis To Java 8
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > The following are tracked in DRILL-1491, but it
>>> doesn't
>>> > >>> look
>>> > >>> >> like
>>> > >>> >> > > > we're
>>> > >>> >> > > > > > > addressing these. Are we?
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > DRILL-4329 13 Unit tests are failing with JDK 8
>>> > >>> >> > > > > > > DRILL-4333 DRILL-4329 tests in
>>> > >>> >> > > > > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in
>>> > Java
>>> > >>> 8
>>> > >>> >> > > > > > > DRILL-5120 Upgrade JDBC Driver for new Java 8
>>> > methods
>>> > >>> >> > > > > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in
>>> > >>> Eclipse
>>> > >>> >> with
>>> > >>> >> > > > Java
>>> > >>> >> > > > > 8
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > *DRILL-4547 is a showstopper*. The release build
>>> > >>> >> > (-Papache-release)
>>> > >>> >> > > > > fails
>>> > >>> >> > > > > > > with far too many Javadoc errors even with doc lint
>>> > turned
>>> > >>> >> off.
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related
>>> > which
>>> > >>> is a
>>> > >>> >> > > > project
>>> > >>> >> > > > > by
>>> > >>> >> > > > > > > itself.
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > Note that fixing JDBC related issues and adding the
>>> > >>> command
>>> > >>> >> line
>>> > >>> >> > > > option
>>> > >>> >> > > > > > to
>>> > >>> >> > > > > > > turn doc lint off will likely break Java 7 builds.
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > Folks who voted to get JDK 8 into this release,
>>> what is
>>> > >>> the
>>> > >>> >> > > consensus
>>> > >>> >> > > > > on
>>> > >>> >> > > > > > > JDBC/Java8 ?
>>> > >>> >> > > > > > > Also, any volunteers on helping debug
>>> > >>> >> > > > > > > TestDrillbitResilience.cancelA
>>> fterAllResultsProduced
>>> > >>> >> > > > > > > ?
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <
>>> > >>> >> > > > vovatkach75@gmail.com
>>> > >>> >> > > > > >
>>> > >>> >> > > > > > > wrote:
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > > Addition to my last message:
>>> > >>> >> > > > > > > > The link with PR for DRILL-1491
>>> > >>> >> https://urldefense.proofpoint
>>> > >>> >> > .
>>> > >>> >> > > > > > > com/v2/url?u=https-3A__github.
>>> > >>> com_apache_drill_pull_1143&d=
>>> > >>> >> > > DwIBaQ&c=
>>> > >>> >> > > > > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=
>>> > >>> >> > > oTnKwfjj5hFBosMrq_
>>> > >>> >> > > > > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3
>>> > >>> uialdRhgnf3XRY22R4SWXGZIq66a
>>> > >>> >> > > > > > Pijuy-Ms0J_-4&e=
>>> > >>> >> > > > > > > > on which the we can see TestDrillbitResilience.
>>> > >>> >> > > > > > > > cancelAfterAllResultsProduced
>>> > >>> >> > > > > > > > failure.
>>> > >>> >> > > > > > > >
>>> > >>> >> > > > > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <
>>> > >>> >> > > vovatkach75@gmail.com
>>> > >>> >> > > > >:
>>> > >>> >> > > > > > > >
>>> > >>> >> > > > > > > > > *To Parth:*
>>> > >>> >> > > > > > > > > The failure can only be seen if run on
>>> DRILL-1491
>>> > >>> branch,
>>> > >>> >> > > because
>>> > >>> >> > > > > it
>>> > >>> >> > > > > > > uses
>>> > >>> >> > > > > > > > > jdk 1.8 in pom.xml
>>> > >>> >> > > > > > > > >
>>> > >>> >> > > > > > > > > <source>1.8</source>
>>> > >>> >> > > > > > > > > <target>1.8</target>
>>> > >>> >> > > > > > > > >
>>> > >>> >> > > > > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <
>>> > >>> >> > > > shamirwasia@mapr.com
>>> > >>> >> > > > > >:
>>> > >>> >> > > > > > > > >
>>> > >>> >> > > > > > > > >> Just sent an email on RCA of DRILL-6216 to
>>> discuss
>>> > >>> next
>>> > >>> >> > steps.
>>> > >>> >> > > > > > > > >>
>>> > >>> >> > > > > > > > >>
>>> > >>> >> > > > > > > > >> Thanks,
>>> > >>> >> > > > > > > > >> Sorabh
>>> > >>> >> > > > > > > > >>
>>> > >>> >> > > > > > > > >> ________________________________
>>> > >>> >> > > > > > > > >> From: Parth Chandra <pa...@apache.org>
>>> > >>> >> > > > > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
>>> > >>> >> > > > > > > > >> To: dev
>>> > >>> >> > > > > > > > >> Subject: Re: [DISCUSS] 1.13.0 release
>>> > >>> >> > > > > > > > >>
>>> > >>> >> > > > > > > > >> We have two items remaining -
>>> > >>> >> > > > > > > > >>
>>> > >>> >> > > > > > > > >> DRILL-1491 - Ideally, I would like to make sure
>>> > that
>>> > >>> >> CANCEL
>>> > >>> >> > is
>>> > >>> >> > > > > > handled
>>> > >>> >> > > > > > > > >> correctly with JDK 8. If the failure of the
>>> unit
>>> > >>> test is
>>> > >>> >> > > because
>>> > >>> >> > > > > the
>>> > >>> >> > > > > > > > >> cancel
>>> > >>> >> > > > > > > > >> is received after the query is completed, then
>>> the
>>> > >>> issue
>>> > >>> >> is
>>> > >>> >> > > less
>>> > >>> >> > > > > > > severe,
>>> > >>> >> > > > > > > > >> but I would like to be sure that this is the
>>> case.
>>> > >>> >> > > > > > > > >> Are there others who see the DrillbitResilience
>>> > tests
>>> > >>> >> > failing
>>> > >>> >> > > > for
>>> > >>> >> > > > > > > them?
>>> > >>> >> > > > > > > > >> Can
>>> > >>> >> > > > > > > > >> we try to assist Volodymyr? I don't see the
>>> > failures
>>> > >>> >> myself.
>>> > >>> >> > > > > > > > >>
>>> > >>> >> > > > > > > > >> DRILL-6216 - this is a showstopper.
>>> > >>> >> > > > > > > > >>
>>> > >>> >> > > > > > > > >>
>>> > >>> >> > > > > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
>>> > >>> >> > > > > kunalkhatua@gmail.com
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > > > > >> wrote:
>>> > >>> >> > > > > > > > >>
>>> > >>> >> > > > > > > > >> > Hi Parth
>>> > >>> >> > > > > > > > >> >
>>> > >>> >> > > > > > > > >> > DRILL-6216 is a release blocker that is being
>>> > >>> currently
>>> > >>> >> > > looked
>>> > >>> >> > > > > > into.
>>> > >>> >> > > > > > > > >> >
>>> > >>> >> > > > > > > > >> > Ref:
>>> > >>> >> > > > > > > > >> > DRILL-6216: Metadata mismatch when
>>> connecting to
>>> > a
>>> > >>> >> Drill
>>> > >>> >> > > > 1.12.0
>>> > >>> >> > > > > > > with a
>>> > >>> >> > > > > > > > >> > Drill-1.13.0-SNAPSHOT driver
>>> > >>> >> > > > > > > > >> > https://urldefense.proofpoint.
>>> > >>> >> > com/v2/url?u=https-3A__issues
>>> > >>> >> > > .
>>> > >>> >> > > > > > > > >> apache.org_jira_browse_DRILL-2
>>> D6216&d=DwIBaQ&c=
>>> > >>> >> > cskdkSMqhcnjZ
>>> > >>> >> > > > > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOt
>>> > >>> >> > hF2jlYxvhTlQg&m=xu
>>> > >>> >> > > > > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz
>>> > >>> >> > 3sxpP5-wHPs&s=txeKaKzF67flAi48
>>> > >>> >> > > > > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
>>> > >>> >> > > > > > > > >> >
>>> > >>> >> > > > > > > > >> > Please add it to the list of required
>>> commits as
>>> > >>> well.
>>> > >>> >> > > > > > > > >> >
>>> > >>> >> > > > > > > > >> > Thanks
>>> > >>> >> > > > > > > > >> > ~ Kunal
>>> > >>> >> > > > > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <
>>> > >>> >> > > > vovatkach75@gmail.com>
>>> > >>> >> > > > > > > > wrote:
>>> > >>> >> > > > > > > > >> > Right now i haven't found the reason of
>>> > >>> >> > > > > > > > >> > TestDrillbitResilience.cancelA
>>> > >>> fterAllResultsProduced
>>> > >>> >> > > failure,
>>> > >>> >> > > > > > most
>>> > >>> >> > > > > > > > >> likely
>>> > >>> >> > > > > > > > >> > the cause of the failure is that the query is
>>> > able
>>> > >>> to
>>> > >>> >> have
>>> > >>> >> > > > been
>>> > >>> >> > > > > > > > >> completed
>>> > >>> >> > > > > > > > >> > before cancellation request is processed.
>>> > >>> >> > > > > > > > >> > This test not only the case, there is one
>>> more
>>> > >>> ignored
>>> > >>> >> > test
>>> > >>> >> > > > > > > > >> > TestDrillbitResilience.cancelA
>>> > >>> >> fterEverythingIsCompleted
>>> > >>> >> > and
>>> > >>> >> > > > > jira
>>> > >>> >> > > > > > > > >> > DRILL-3967
>>> > >>> >> > > > > > > > >> > created, although the environment is AWS.
>>> > >>> >> > > > > > > > >> >
>>> > >>> >> > > > > > > > >> > Maybe it makes sense to ignore this test to
>>> > >>> unblock the
>>> > >>> >> > > > release
>>> > >>> >> > > > > > and
>>> > >>> >> > > > > > > > >> merge
>>> > >>> >> > > > > > > > >> > JDK8 changes?
>>> > >>> >> > > > > > > > >> >
>>> > >>> >> > > > > > > > >>
>>> > >>> >> > > > > > > > >
>>> > >>> >> > > > > > > > >
>>> > >>> >> > > > > > > >
>>> > >>> >> > > > > > >
>>> > >>> >> > > > > >
>>> > >>> >> > > > >
>>> > >>> >> > > >
>>> > >>> >> > >
>>> > >>> >> >
>>> > >>> >>
>>> > >>> >
>>> > >>> >
>>> > >>>
>>> > >>
>>> > >>
>>> > >
>>> >
>>>
>>
>>
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
Resolved this. Needed to upgrade my JDK. I would rather we had fixed the
actual javadoc but this will have to do.
On Fri, Mar 9, 2018 at 12:20 PM, Parth Chandra <pa...@apache.org> wrote:
> Running the build with the apache-release profile still fails with a
> different error. (The release script gives the same error).
> Anyone have an idea what this one is? A quick web search did not help.
>
>
> mvn clean install -DskipTests -Papache-release
>
>
> ...
>
>
> [INFO] javadoc: error - com.sun.tools.doclets.internal.toolkit.util.DocletAbortException:
> com.sun.tools.javac.code.Symbol$CompletionFailure: class file for
> org.junit.runners.model.Statement not found
>
> On Fri, Mar 9, 2018 at 7:01 AM, Aman Sinha <am...@apache.org> wrote:
>
>> Hmm...322 warnings seems too much to address in a day or two. Parth, if
>> you are able to complete a build (with warnings but no errors) with the
>> branch link that Jyotsna I think we could go ahead with making the change
>> in the pom file and fix the warnings post 1.13.0. Any thoughts ?
>>
>> Also, note that this would mean that anyone who wants to build the source
>> with JDK 7 would have to use the 1.12 release.
>>
>> -Aman
>>
>> On Thu, Mar 8, 2018 at 3:59 PM, Jyothsna Reddy <jy...@gmail.com>
>> wrote:
>>
>> > When I disabled doclint and there were around 322 java doc warnings. I'm
>> > not sure if we can fix all of them and I'm attaching a link to the log
>> file
>> > that contains all the warnings. Please let me know your thoughts on
>> this.
>> >
>> > https://drive.google.com/file/d/1GQ6xORr0HkoEmorzvTcl0XZS4NeLX
>> > u83/view?usp=sharing
>> >
>> >
>> >
>> >
>> >
>> > On Thu, Mar 8, 2018 at 10:39 AM, Jyothsna Reddy <jyothsna.dvj@gmail.com
>> >
>> > wrote:
>> >
>> > > I tried to disable it using -Dadditionalparam=-Xdoclint:none.
>> Somewhere
>> > I
>> > > have read that -DXdoclint:none is for older versions of maven. But I'm
>> > not
>> > > sure of it.
>> > >
>> > >
>> > >
>> > >
>> > > On Thu, Mar 8, 2018 at 10:18 AM, Jyothsna Reddy <
>> jyothsna.dvj@gmail.com>
>> > > wrote:
>> > >
>> > >> Following branch has changed related to disabling doclint in pom.xml.
>> > >> Please check it out.
>> > >>
>> > >> https://github.com/dvjyothsna/drill.git DRILL-4547
>> > >>
>> > >>
>> > >>
>> > >>
>> > >>
>> > >>
>> > >> On Thu, Mar 8, 2018 at 4:19 AM, Parth Chandra <pa...@apache.org>
>> > wrote:
>> > >>
>> > >>> The issue with the failing test in TestDrillbitResilience.
>> > >>> cancelAfterAllResultsProduced is similar to DRILL-3967 (
>> > >>> TestDrillbitResilience.cancelAfterEverythingIsCompleted failure).
>> > >>>
>> > >>> In both this test case, and DRILL-3967, the query is paused (in
>> > different
>> > >>> places) and a cancel is sent. The query is then resumed and the
>> > resulting
>> > >>> state is checked. The problem is that the tests have a race
>> condition
>> > >>> between the cancellation and the resuming of the query. Sometimes
>> the
>> > >>> resume reaches first and sometimes the cancel reaches first. The
>> > failure
>> > >>> described by Volodymyr is caused by this race condition. I don't
>> know
>> > why
>> > >>> the test was done like this, but this is an existing problem and
>> > >>> shouldn't
>> > >>> hold up the release.
>> > >>>
>> > >>> However, I also see a failure where we encounter an illegal state
>> > >>> transition (the query state is CANCELLATION_REQUESTED and the
>> Foreman
>> > >>> tries
>> > >>> to move to an ENQUEUED state). This happens once in about
>> twenty-five
>> > >>> executions. The Foreman only requests ENQUEUE once, when the query
>> is
>> > >>> about
>> > >>> to start, so this means the cancellation request reached before the
>> > query
>> > >>> start request. How this happens in the unit test I have not been
>> able
>> > to
>> > >>> determine yet (because it really shouldn't be possible).
>> > >>>
>> > >>> To recreate the problem I simply added a repeat rule in the class
>> and
>> > >>> set a
>> > >>> repeat count of 1000. The problem occurs easily if the test is run
>> from
>> > >>> the
>> > >>> command line. When running in debug, I was unable to see the
>> problem.
>> > >>>
>> > >>> I'll spend some more time on this, but just in case someone wants to
>> > >>> investigate further, feel free ...
>> > >>>
>> > >>>
>> > >>> On Thu, Mar 8, 2018 at 10:59 AM, Parth Chandra <pa...@apache.org>
>> > >>> wrote:
>> > >>>
>> > >>> > Not sure if that would work. The release build does not allow
>> > >>> uncommitted
>> > >>> > files, so I have to commit pom.xml changes to at least the local
>> > repo,
>> > >>> > which will get pushed to my public repo when the release is done.
>> Not
>> > >>> > committing this to Apache master would be cheating would leave us
>> > with
>> > >>> a
>> > >>> > build that does not match any source in Apache master? Javadoc
>> > >>> generated is
>> > >>> > never committed to any repo. It is part of the src release jars,
>> > AFAIK.
>> > >>> >
>> > >>> > Also, I'm not sure where in the pom Jyothsna made the change; I
>> added
>> > >>> the
>> > >>> > -Dxoclint:none to the build section of the apache-release profile
>> and
>> > >>> java
>> > >>> > exec still gives over 100 javadoc errors.
>> > >>> >
>> > >>> > We have to fix these one of these days. Might as well do it now.
>> > >>> Knowing
>> > >>> > how it works, if we don't fix these now, someone will be
>> scrambling
>> > to
>> > >>> fix
>> > >>> > these just before the 1.14.0 release :(
>> > >>> >
>> > >>> >
>> > >>> >
>> > >>> > On Thu, Mar 8, 2018 at 10:06 AM, Aman Sinha <amansinha@apache.org
>> >
>> > >>> wrote:
>> > >>> >
>> > >>> >> Parth, would it work if you made the pom.xml changes locally in
>> > your
>> > >>> >> branch, generated the javadoc but only commit the javadoc jar
>> files
>> > >>> to the
>> > >>> >> release branch, not the pom.xml changes ?
>> > >>> >> Anyone downloading Drill source code to build should not run into
>> > this
>> > >>> >> since typically they won't be building javadoc.
>> > >>> >>
>> > >>> >> -Aman
>> > >>> >>
>> > >>> >> On Wed, Mar 7, 2018 at 6:37 PM, Parth Chandra <parthc@apache.org
>> >
>> > >>> wrote:
>> > >>> >>
>> > >>> >> > Unfortunately, we cannot do that since we also want to be able
>> to
>> > >>> build
>> > >>> >> > with JDK 7 for at least a couple of releases to allow for a
>> > >>> reasonable
>> > >>> >> > transition time. doclint was introduced in JDK 8 so JDK 7
>> fails
>> > >>> >> because it
>> > >>> >> > doesn't recognize the parameter.
>> > >>> >> >
>> > >>> >> >
>> > >>> >> >
>> > >>> >> > On Thu, Mar 8, 2018 at 7:03 AM, Jyothsna Reddy <
>> > >>> jyothsna.dvj@gmail.com>
>> > >>> >> > wrote:
>> > >>> >> >
>> > >>> >> > > Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491)
>> and
>> > >>> added
>> > >>> >> > > following lines to pom.xml to disable doc lint. The javadoc
>> > >>> doesn't
>> > >>> >> throw
>> > >>> >> > > any errors and the build is successful.
>> > >>> >> > >
>> > >>> >> > > <activation>
>> > >>> >> > >
>> > >>> >> > > <jdk>[1.8,)</jdk>
>> > >>> >> > >
>> > >>> >> > > </activation>
>> > >>> >> > >
>> > >>> >> > > <properties>
>> > >>> >> > >
>> > >>> >> > > <additionalparam>-Xdoclint:none</additionalparam>
>> > >>> >> > >
>> > >>> >> > > </properties>
>> > >>> >> > >
>> > >>> >> > >
>> > >>> >> > >
>> > >>> >> > >
>> > >>> >> > >
>> > >>> >> > > On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri <
>> > >>> >> hanu.ncr@gmail.com>
>> > >>> >> > > wrote:
>> > >>> >> > >
>> > >>> >> > > > On my machine I couldn't repro the issue related to
>> > >>> >> > > TestDrillbitResilience.
>> > >>> >> > > > cancelAfterAllResultsProduced.
>> > >>> >> > > > I used the vladimir's branch (i.e DRILL-1491).
>> > >>> >> > > > Used the maven test command for testing it.
>> > >>> >> > > >
>> > >>> >> > > > output of the test run.
>> > >>> >> > > > ... 4 common frames omitted
>> > >>> >> > > > Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time
>> > elapsed:
>> > >>> >> > 124.187
>> > >>> >> > > > sec - in org.apache.drill.exec.server.T
>> estDrillbitResilience
>> > >>> >> > > >
>> > >>> >> > > >
>> > >>> >> > > >
>> > >>> >> > > >
>> > >>> >> > > > On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <
>> > >>> parthc@apache.org>
>> > >>> >> > > wrote:
>> > >>> >> > > >
>> > >>> >> > > > > Yes I agree. JDBC would be a new feature that we can
>> defer
>> > to
>> > >>> >> 1.14.0.
>> > >>> >> > > > > I'm hoping we can resolve the other three in the next few
>> > >>> days.
>> > >>> >> > Target
>> > >>> >> > > > date
>> > >>> >> > > > > for starting release process - Friday Mar 9th
>> > >>> >> > > > >
>> > >>> >> > > > > Once these are resolved, I will create a branch for the
>> > >>> release so
>> > >>> >> > that
>> > >>> >> > > > > Apache master remains open for commits. If any issues are
>> > >>> found in
>> > >>> >> > the
>> > >>> >> > > > > release branch, we will fix them in master and I will
>> > >>> cherry-pick
>> > >>> >> the
>> > >>> >> > > > into
>> > >>> >> > > > > the release branch. Once the release is finalized I will
>> > add a
>> > >>> >> > release
>> > >>> >> > > > tag
>> > >>> >> > > > > and remove the branch.
>> > >>> >> > > > >
>> > >>> >> > > > > Also note if QA folks want to get started on testing the
>> > >>> release,
>> > >>> >> the
>> > >>> >> > > > > current head of Apache master is close to final. Javadoc
>> > >>> >> generation
>> > >>> >> > is
>> > >>> >> > > > only
>> > >>> >> > > > > a release build issue, and the other issues are
>> localized
>> > to
>> > >>> >> > specific
>> > >>> >> > > > > cases.
>> > >>> >> > > > >
>> > >>> >> > > > > Note: to reproduce the javadoc issues:
>> > >>> >> > > > > # set JAVA_HOME to JDK 8
>> > >>> >> > > > > mvn javadoc:javadoc -Papache-release
>> > >>> >> > > > >
>> > >>> >> > > > >
>> > >>> >> > > > >
>> > >>> >> > > > > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <
>> > >>> amansinha@apache.org
>> > >>> >> >
>> > >>> >> > > > wrote:
>> > >>> >> > > > >
>> > >>> >> > > > > > It seems to me the main blockers are:
>> > >>> >> > > > > >
>> > >>> >> > > > > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we
>> > >>> split up
>> > >>> >> the
>> > >>> >> > > > work
>> > >>> >> > > > > > among few people to resolve these ?
>> > >>> >> > > > > > 2. DRILL-6216 Metadata mismatch.. <--
>> Agreement
>> > >>> was
>> > >>> >> to
>> > >>> >> > > > revert
>> > >>> >> > > > > > one small piece of code and it appears Sorabh is
>> looking
>> > >>> into it
>> > >>> >> > > > > > 3. TestDrillbitResilience.cancelA
>> fterAllResultsProduced
>> > >>> <--
>> > >>> >> need
>> > >>> >> > > > > someone
>> > >>> >> > > > > > to look into this
>> > >>> >> > > > > >
>> > >>> >> > > > > > Regarding the JDBC issues that Parth mentioned,
>> looking at
>> > >>> the
>> > >>> >> > JIRAs,
>> > >>> >> > > > it
>> > >>> >> > > > > > seems they are not showstoppers...Parth do you agree ?
>> > >>> >> > > > > >
>> > >>> >> > > > > > Since we are close to the finish line for JDK 8, IMO we
>> > >>> should
>> > >>> >> try
>> > >>> >> > > and
>> > >>> >> > > > > see
>> > >>> >> > > > > > if in another day or two we can get over these hurdles.
>> > >>> >> > > > > >
>> > >>> >> > > > > > -Aman
>> > >>> >> > > > > >
>> > >>> >> > > > > >
>> > >>> >> > > > > >
>> > >>> >> > > > > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <
>> > >>> pmaker@mapr.com>
>> > >>> >> > > wrote:
>> > >>> >> > > > > >
>> > >>> >> > > > > > > The JDK 8 issues will likely require more time to
>> harden
>> > >>> for
>> > >>> >> it
>> > >>> >> > to
>> > >>> >> > > be
>> > >>> >> > > > > > > included in the 1.13 release. My recommendation
>> would be
>> > >>> to
>> > >>> >> move
>> > >>> >> > > > ahead
>> > >>> >> > > > > > with
>> > >>> >> > > > > > > the 1.13 release now and address these issues right.
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > Pritesh
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > -----Original Message-----
>> > >>> >> > > > > > > From: Parth Chandra <pa...@apache.org>
>> > >>> >> > > > > > > Sent: March 7, 2018 3:34 AM
>> > >>> >> > > > > > > To: dev <de...@drill.apache.org>
>> > >>> >> > > > > > > Subject: Re: [DISCUSS] 1.13.0 release
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > My mistake Volodymyr.
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > Found some other JDK 8 issues in JIRA not tracked in
>> > >>> >> DRILL-1491
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > DRILL-4547 Javadoc fails with Java8
>> > >>> >> > > > > > > DRILL-6163 Switch Travis To Java 8
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > The following are tracked in DRILL-1491, but it
>> doesn't
>> > >>> look
>> > >>> >> like
>> > >>> >> > > > we're
>> > >>> >> > > > > > > addressing these. Are we?
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > DRILL-4329 13 Unit tests are failing with JDK 8
>> > >>> >> > > > > > > DRILL-4333 DRILL-4329 tests in
>> > >>> >> > > > > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in
>> > Java
>> > >>> 8
>> > >>> >> > > > > > > DRILL-5120 Upgrade JDBC Driver for new Java 8
>> > methods
>> > >>> >> > > > > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in
>> > >>> Eclipse
>> > >>> >> with
>> > >>> >> > > > Java
>> > >>> >> > > > > 8
>> > >>> >> > > > > > >
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > *DRILL-4547 is a showstopper*. The release build
>> > >>> >> > (-Papache-release)
>> > >>> >> > > > > fails
>> > >>> >> > > > > > > with far too many Javadoc errors even with doc lint
>> > turned
>> > >>> >> off.
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related
>> > which
>> > >>> is a
>> > >>> >> > > > project
>> > >>> >> > > > > by
>> > >>> >> > > > > > > itself.
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > Note that fixing JDBC related issues and adding the
>> > >>> command
>> > >>> >> line
>> > >>> >> > > > option
>> > >>> >> > > > > > to
>> > >>> >> > > > > > > turn doc lint off will likely break Java 7 builds.
>> > >>> >> > > > > > >
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > Folks who voted to get JDK 8 into this release, what
>> is
>> > >>> the
>> > >>> >> > > consensus
>> > >>> >> > > > > on
>> > >>> >> > > > > > > JDBC/Java8 ?
>> > >>> >> > > > > > > Also, any volunteers on helping debug
>> > >>> >> > > > > > > TestDrillbitResilience.cancelAfterAllResultsProduced
>> > >>> >> > > > > > > ?
>> > >>> >> > > > > > >
>> > >>> >> > > > > > >
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <
>> > >>> >> > > > vovatkach75@gmail.com
>> > >>> >> > > > > >
>> > >>> >> > > > > > > wrote:
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > > Addition to my last message:
>> > >>> >> > > > > > > > The link with PR for DRILL-1491
>> > >>> >> https://urldefense.proofpoint
>> > >>> >> > .
>> > >>> >> > > > > > > com/v2/url?u=https-3A__github.
>> > >>> com_apache_drill_pull_1143&d=
>> > >>> >> > > DwIBaQ&c=
>> > >>> >> > > > > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=
>> > >>> >> > > oTnKwfjj5hFBosMrq_
>> > >>> >> > > > > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3
>> > >>> uialdRhgnf3XRY22R4SWXGZIq66a
>> > >>> >> > > > > > Pijuy-Ms0J_-4&e=
>> > >>> >> > > > > > > > on which the we can see TestDrillbitResilience.
>> > >>> >> > > > > > > > cancelAfterAllResultsProduced
>> > >>> >> > > > > > > > failure.
>> > >>> >> > > > > > > >
>> > >>> >> > > > > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <
>> > >>> >> > > vovatkach75@gmail.com
>> > >>> >> > > > >:
>> > >>> >> > > > > > > >
>> > >>> >> > > > > > > > > *To Parth:*
>> > >>> >> > > > > > > > > The failure can only be seen if run on DRILL-1491
>> > >>> branch,
>> > >>> >> > > because
>> > >>> >> > > > > it
>> > >>> >> > > > > > > uses
>> > >>> >> > > > > > > > > jdk 1.8 in pom.xml
>> > >>> >> > > > > > > > >
>> > >>> >> > > > > > > > > <source>1.8</source>
>> > >>> >> > > > > > > > > <target>1.8</target>
>> > >>> >> > > > > > > > >
>> > >>> >> > > > > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <
>> > >>> >> > > > shamirwasia@mapr.com
>> > >>> >> > > > > >:
>> > >>> >> > > > > > > > >
>> > >>> >> > > > > > > > >> Just sent an email on RCA of DRILL-6216 to
>> discuss
>> > >>> next
>> > >>> >> > steps.
>> > >>> >> > > > > > > > >>
>> > >>> >> > > > > > > > >>
>> > >>> >> > > > > > > > >> Thanks,
>> > >>> >> > > > > > > > >> Sorabh
>> > >>> >> > > > > > > > >>
>> > >>> >> > > > > > > > >> ________________________________
>> > >>> >> > > > > > > > >> From: Parth Chandra <pa...@apache.org>
>> > >>> >> > > > > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
>> > >>> >> > > > > > > > >> To: dev
>> > >>> >> > > > > > > > >> Subject: Re: [DISCUSS] 1.13.0 release
>> > >>> >> > > > > > > > >>
>> > >>> >> > > > > > > > >> We have two items remaining -
>> > >>> >> > > > > > > > >>
>> > >>> >> > > > > > > > >> DRILL-1491 - Ideally, I would like to make sure
>> > that
>> > >>> >> CANCEL
>> > >>> >> > is
>> > >>> >> > > > > > handled
>> > >>> >> > > > > > > > >> correctly with JDK 8. If the failure of the unit
>> > >>> test is
>> > >>> >> > > because
>> > >>> >> > > > > the
>> > >>> >> > > > > > > > >> cancel
>> > >>> >> > > > > > > > >> is received after the query is completed, then
>> the
>> > >>> issue
>> > >>> >> is
>> > >>> >> > > less
>> > >>> >> > > > > > > severe,
>> > >>> >> > > > > > > > >> but I would like to be sure that this is the
>> case.
>> > >>> >> > > > > > > > >> Are there others who see the DrillbitResilience
>> > tests
>> > >>> >> > failing
>> > >>> >> > > > for
>> > >>> >> > > > > > > them?
>> > >>> >> > > > > > > > >> Can
>> > >>> >> > > > > > > > >> we try to assist Volodymyr? I don't see the
>> > failures
>> > >>> >> myself.
>> > >>> >> > > > > > > > >>
>> > >>> >> > > > > > > > >> DRILL-6216 - this is a showstopper.
>> > >>> >> > > > > > > > >>
>> > >>> >> > > > > > > > >>
>> > >>> >> > > > > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
>> > >>> >> > > > > kunalkhatua@gmail.com
>> > >>> >> > > > > > >
>> > >>> >> > > > > > > > >> wrote:
>> > >>> >> > > > > > > > >>
>> > >>> >> > > > > > > > >> > Hi Parth
>> > >>> >> > > > > > > > >> >
>> > >>> >> > > > > > > > >> > DRILL-6216 is a release blocker that is being
>> > >>> currently
>> > >>> >> > > looked
>> > >>> >> > > > > > into.
>> > >>> >> > > > > > > > >> >
>> > >>> >> > > > > > > > >> > Ref:
>> > >>> >> > > > > > > > >> > DRILL-6216: Metadata mismatch when connecting
>> to
>> > a
>> > >>> >> Drill
>> > >>> >> > > > 1.12.0
>> > >>> >> > > > > > > with a
>> > >>> >> > > > > > > > >> > Drill-1.13.0-SNAPSHOT driver
>> > >>> >> > > > > > > > >> > https://urldefense.proofpoint.
>> > >>> >> > com/v2/url?u=https-3A__issues
>> > >>> >> > > .
>> > >>> >> > > > > > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=
>> > >>> >> > cskdkSMqhcnjZ
>> > >>> >> > > > > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOt
>> > >>> >> > hF2jlYxvhTlQg&m=xu
>> > >>> >> > > > > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz
>> > >>> >> > 3sxpP5-wHPs&s=txeKaKzF67flAi48
>> > >>> >> > > > > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
>> > >>> >> > > > > > > > >> >
>> > >>> >> > > > > > > > >> > Please add it to the list of required commits
>> as
>> > >>> well.
>> > >>> >> > > > > > > > >> >
>> > >>> >> > > > > > > > >> > Thanks
>> > >>> >> > > > > > > > >> > ~ Kunal
>> > >>> >> > > > > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <
>> > >>> >> > > > vovatkach75@gmail.com>
>> > >>> >> > > > > > > > wrote:
>> > >>> >> > > > > > > > >> > Right now i haven't found the reason of
>> > >>> >> > > > > > > > >> > TestDrillbitResilience.cancelA
>> > >>> fterAllResultsProduced
>> > >>> >> > > failure,
>> > >>> >> > > > > > most
>> > >>> >> > > > > > > > >> likely
>> > >>> >> > > > > > > > >> > the cause of the failure is that the query is
>> > able
>> > >>> to
>> > >>> >> have
>> > >>> >> > > > been
>> > >>> >> > > > > > > > >> completed
>> > >>> >> > > > > > > > >> > before cancellation request is processed.
>> > >>> >> > > > > > > > >> > This test not only the case, there is one more
>> > >>> ignored
>> > >>> >> > test
>> > >>> >> > > > > > > > >> > TestDrillbitResilience.cancelA
>> > >>> >> fterEverythingIsCompleted
>> > >>> >> > and
>> > >>> >> > > > > jira
>> > >>> >> > > > > > > > >> > DRILL-3967
>> > >>> >> > > > > > > > >> > created, although the environment is AWS.
>> > >>> >> > > > > > > > >> >
>> > >>> >> > > > > > > > >> > Maybe it makes sense to ignore this test to
>> > >>> unblock the
>> > >>> >> > > > release
>> > >>> >> > > > > > and
>> > >>> >> > > > > > > > >> merge
>> > >>> >> > > > > > > > >> > JDK8 changes?
>> > >>> >> > > > > > > > >> >
>> > >>> >> > > > > > > > >>
>> > >>> >> > > > > > > > >
>> > >>> >> > > > > > > > >
>> > >>> >> > > > > > > >
>> > >>> >> > > > > > >
>> > >>> >> > > > > >
>> > >>> >> > > > >
>> > >>> >> > > >
>> > >>> >> > >
>> > >>> >> >
>> > >>> >>
>> > >>> >
>> > >>> >
>> > >>>
>> > >>
>> > >>
>> > >
>> >
>>
>
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
Running the build with the apache-release profile still fails with a
different error. (The release script gives the same error).
Anyone have an idea what this one is? A quick web search did not help.
mvn clean install -DskipTests -Papache-release
...
[INFO] javadoc: error -
com.sun.tools.doclets.internal.toolkit.util.DocletAbortException:
com.sun.tools.javac.code.Symbol$CompletionFailure: class file for
org.junit.runners.model.Statement not found
On Fri, Mar 9, 2018 at 7:01 AM, Aman Sinha <am...@apache.org> wrote:
> Hmm...322 warnings seems too much to address in a day or two. Parth, if
> you are able to complete a build (with warnings but no errors) with the
> branch link that Jyotsna I think we could go ahead with making the change
> in the pom file and fix the warnings post 1.13.0. Any thoughts ?
>
> Also, note that this would mean that anyone who wants to build the source
> with JDK 7 would have to use the 1.12 release.
>
> -Aman
>
> On Thu, Mar 8, 2018 at 3:59 PM, Jyothsna Reddy <jy...@gmail.com>
> wrote:
>
> > When I disabled doclint and there were around 322 java doc warnings. I'm
> > not sure if we can fix all of them and I'm attaching a link to the log
> file
> > that contains all the warnings. Please let me know your thoughts on this.
> >
> > https://drive.google.com/file/d/1GQ6xORr0HkoEmorzvTcl0XZS4NeLX
> > u83/view?usp=sharing
> >
> >
> >
> >
> >
> > On Thu, Mar 8, 2018 at 10:39 AM, Jyothsna Reddy <jy...@gmail.com>
> > wrote:
> >
> > > I tried to disable it using -Dadditionalparam=-Xdoclint:none.
> Somewhere
> > I
> > > have read that -DXdoclint:none is for older versions of maven. But I'm
> > not
> > > sure of it.
> > >
> > >
> > >
> > >
> > > On Thu, Mar 8, 2018 at 10:18 AM, Jyothsna Reddy <
> jyothsna.dvj@gmail.com>
> > > wrote:
> > >
> > >> Following branch has changed related to disabling doclint in pom.xml.
> > >> Please check it out.
> > >>
> > >> https://github.com/dvjyothsna/drill.git DRILL-4547
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> On Thu, Mar 8, 2018 at 4:19 AM, Parth Chandra <pa...@apache.org>
> > wrote:
> > >>
> > >>> The issue with the failing test in TestDrillbitResilience.
> > >>> cancelAfterAllResultsProduced is similar to DRILL-3967 (
> > >>> TestDrillbitResilience.cancelAfterEverythingIsCompleted failure).
> > >>>
> > >>> In both this test case, and DRILL-3967, the query is paused (in
> > different
> > >>> places) and a cancel is sent. The query is then resumed and the
> > resulting
> > >>> state is checked. The problem is that the tests have a race condition
> > >>> between the cancellation and the resuming of the query. Sometimes the
> > >>> resume reaches first and sometimes the cancel reaches first. The
> > failure
> > >>> described by Volodymyr is caused by this race condition. I don't know
> > why
> > >>> the test was done like this, but this is an existing problem and
> > >>> shouldn't
> > >>> hold up the release.
> > >>>
> > >>> However, I also see a failure where we encounter an illegal state
> > >>> transition (the query state is CANCELLATION_REQUESTED and the Foreman
> > >>> tries
> > >>> to move to an ENQUEUED state). This happens once in about twenty-five
> > >>> executions. The Foreman only requests ENQUEUE once, when the query is
> > >>> about
> > >>> to start, so this means the cancellation request reached before the
> > query
> > >>> start request. How this happens in the unit test I have not been able
> > to
> > >>> determine yet (because it really shouldn't be possible).
> > >>>
> > >>> To recreate the problem I simply added a repeat rule in the class and
> > >>> set a
> > >>> repeat count of 1000. The problem occurs easily if the test is run
> from
> > >>> the
> > >>> command line. When running in debug, I was unable to see the problem.
> > >>>
> > >>> I'll spend some more time on this, but just in case someone wants to
> > >>> investigate further, feel free ...
> > >>>
> > >>>
> > >>> On Thu, Mar 8, 2018 at 10:59 AM, Parth Chandra <pa...@apache.org>
> > >>> wrote:
> > >>>
> > >>> > Not sure if that would work. The release build does not allow
> > >>> uncommitted
> > >>> > files, so I have to commit pom.xml changes to at least the local
> > repo,
> > >>> > which will get pushed to my public repo when the release is done.
> Not
> > >>> > committing this to Apache master would be cheating would leave us
> > with
> > >>> a
> > >>> > build that does not match any source in Apache master? Javadoc
> > >>> generated is
> > >>> > never committed to any repo. It is part of the src release jars,
> > AFAIK.
> > >>> >
> > >>> > Also, I'm not sure where in the pom Jyothsna made the change; I
> added
> > >>> the
> > >>> > -Dxoclint:none to the build section of the apache-release profile
> and
> > >>> java
> > >>> > exec still gives over 100 javadoc errors.
> > >>> >
> > >>> > We have to fix these one of these days. Might as well do it now.
> > >>> Knowing
> > >>> > how it works, if we don't fix these now, someone will be scrambling
> > to
> > >>> fix
> > >>> > these just before the 1.14.0 release :(
> > >>> >
> > >>> >
> > >>> >
> > >>> > On Thu, Mar 8, 2018 at 10:06 AM, Aman Sinha <am...@apache.org>
> > >>> wrote:
> > >>> >
> > >>> >> Parth, would it work if you made the pom.xml changes locally in
> > your
> > >>> >> branch, generated the javadoc but only commit the javadoc jar
> files
> > >>> to the
> > >>> >> release branch, not the pom.xml changes ?
> > >>> >> Anyone downloading Drill source code to build should not run into
> > this
> > >>> >> since typically they won't be building javadoc.
> > >>> >>
> > >>> >> -Aman
> > >>> >>
> > >>> >> On Wed, Mar 7, 2018 at 6:37 PM, Parth Chandra <pa...@apache.org>
> > >>> wrote:
> > >>> >>
> > >>> >> > Unfortunately, we cannot do that since we also want to be able
> to
> > >>> build
> > >>> >> > with JDK 7 for at least a couple of releases to allow for a
> > >>> reasonable
> > >>> >> > transition time. doclint was introduced in JDK 8 so JDK 7 fails
> > >>> >> because it
> > >>> >> > doesn't recognize the parameter.
> > >>> >> >
> > >>> >> >
> > >>> >> >
> > >>> >> > On Thu, Mar 8, 2018 at 7:03 AM, Jyothsna Reddy <
> > >>> jyothsna.dvj@gmail.com>
> > >>> >> > wrote:
> > >>> >> >
> > >>> >> > > Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491) and
> > >>> added
> > >>> >> > > following lines to pom.xml to disable doc lint. The javadoc
> > >>> doesn't
> > >>> >> throw
> > >>> >> > > any errors and the build is successful.
> > >>> >> > >
> > >>> >> > > <activation>
> > >>> >> > >
> > >>> >> > > <jdk>[1.8,)</jdk>
> > >>> >> > >
> > >>> >> > > </activation>
> > >>> >> > >
> > >>> >> > > <properties>
> > >>> >> > >
> > >>> >> > > <additionalparam>-Xdoclint:none</additionalparam>
> > >>> >> > >
> > >>> >> > > </properties>
> > >>> >> > >
> > >>> >> > >
> > >>> >> > >
> > >>> >> > >
> > >>> >> > >
> > >>> >> > > On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri <
> > >>> >> hanu.ncr@gmail.com>
> > >>> >> > > wrote:
> > >>> >> > >
> > >>> >> > > > On my machine I couldn't repro the issue related to
> > >>> >> > > TestDrillbitResilience.
> > >>> >> > > > cancelAfterAllResultsProduced.
> > >>> >> > > > I used the vladimir's branch (i.e DRILL-1491).
> > >>> >> > > > Used the maven test command for testing it.
> > >>> >> > > >
> > >>> >> > > > output of the test run.
> > >>> >> > > > ... 4 common frames omitted
> > >>> >> > > > Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time
> > elapsed:
> > >>> >> > 124.187
> > >>> >> > > > sec - in org.apache.drill.exec.server.
> TestDrillbitResilience
> > >>> >> > > >
> > >>> >> > > >
> > >>> >> > > >
> > >>> >> > > >
> > >>> >> > > > On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <
> > >>> parthc@apache.org>
> > >>> >> > > wrote:
> > >>> >> > > >
> > >>> >> > > > > Yes I agree. JDBC would be a new feature that we can defer
> > to
> > >>> >> 1.14.0.
> > >>> >> > > > > I'm hoping we can resolve the other three in the next few
> > >>> days.
> > >>> >> > Target
> > >>> >> > > > date
> > >>> >> > > > > for starting release process - Friday Mar 9th
> > >>> >> > > > >
> > >>> >> > > > > Once these are resolved, I will create a branch for the
> > >>> release so
> > >>> >> > that
> > >>> >> > > > > Apache master remains open for commits. If any issues are
> > >>> found in
> > >>> >> > the
> > >>> >> > > > > release branch, we will fix them in master and I will
> > >>> cherry-pick
> > >>> >> the
> > >>> >> > > > into
> > >>> >> > > > > the release branch. Once the release is finalized I will
> > add a
> > >>> >> > release
> > >>> >> > > > tag
> > >>> >> > > > > and remove the branch.
> > >>> >> > > > >
> > >>> >> > > > > Also note if QA folks want to get started on testing the
> > >>> release,
> > >>> >> the
> > >>> >> > > > > current head of Apache master is close to final. Javadoc
> > >>> >> generation
> > >>> >> > is
> > >>> >> > > > only
> > >>> >> > > > > a release build issue, and the other issues are localized
> > to
> > >>> >> > specific
> > >>> >> > > > > cases.
> > >>> >> > > > >
> > >>> >> > > > > Note: to reproduce the javadoc issues:
> > >>> >> > > > > # set JAVA_HOME to JDK 8
> > >>> >> > > > > mvn javadoc:javadoc -Papache-release
> > >>> >> > > > >
> > >>> >> > > > >
> > >>> >> > > > >
> > >>> >> > > > > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <
> > >>> amansinha@apache.org
> > >>> >> >
> > >>> >> > > > wrote:
> > >>> >> > > > >
> > >>> >> > > > > > It seems to me the main blockers are:
> > >>> >> > > > > >
> > >>> >> > > > > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we
> > >>> split up
> > >>> >> the
> > >>> >> > > > work
> > >>> >> > > > > > among few people to resolve these ?
> > >>> >> > > > > > 2. DRILL-6216 Metadata mismatch.. <--
> Agreement
> > >>> was
> > >>> >> to
> > >>> >> > > > revert
> > >>> >> > > > > > one small piece of code and it appears Sorabh is looking
> > >>> into it
> > >>> >> > > > > > 3. TestDrillbitResilience.cancelAfterAllResultsProduced
> > >>> <--
> > >>> >> need
> > >>> >> > > > > someone
> > >>> >> > > > > > to look into this
> > >>> >> > > > > >
> > >>> >> > > > > > Regarding the JDBC issues that Parth mentioned, looking
> at
> > >>> the
> > >>> >> > JIRAs,
> > >>> >> > > > it
> > >>> >> > > > > > seems they are not showstoppers...Parth do you agree ?
> > >>> >> > > > > >
> > >>> >> > > > > > Since we are close to the finish line for JDK 8, IMO we
> > >>> should
> > >>> >> try
> > >>> >> > > and
> > >>> >> > > > > see
> > >>> >> > > > > > if in another day or two we can get over these hurdles.
> > >>> >> > > > > >
> > >>> >> > > > > > -Aman
> > >>> >> > > > > >
> > >>> >> > > > > >
> > >>> >> > > > > >
> > >>> >> > > > > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <
> > >>> pmaker@mapr.com>
> > >>> >> > > wrote:
> > >>> >> > > > > >
> > >>> >> > > > > > > The JDK 8 issues will likely require more time to
> harden
> > >>> for
> > >>> >> it
> > >>> >> > to
> > >>> >> > > be
> > >>> >> > > > > > > included in the 1.13 release. My recommendation would
> be
> > >>> to
> > >>> >> move
> > >>> >> > > > ahead
> > >>> >> > > > > > with
> > >>> >> > > > > > > the 1.13 release now and address these issues right.
> > >>> >> > > > > > >
> > >>> >> > > > > > > Pritesh
> > >>> >> > > > > > >
> > >>> >> > > > > > > -----Original Message-----
> > >>> >> > > > > > > From: Parth Chandra <pa...@apache.org>
> > >>> >> > > > > > > Sent: March 7, 2018 3:34 AM
> > >>> >> > > > > > > To: dev <de...@drill.apache.org>
> > >>> >> > > > > > > Subject: Re: [DISCUSS] 1.13.0 release
> > >>> >> > > > > > >
> > >>> >> > > > > > > My mistake Volodymyr.
> > >>> >> > > > > > >
> > >>> >> > > > > > > Found some other JDK 8 issues in JIRA not tracked in
> > >>> >> DRILL-1491
> > >>> >> > > > > > >
> > >>> >> > > > > > > DRILL-4547 Javadoc fails with Java8
> > >>> >> > > > > > > DRILL-6163 Switch Travis To Java 8
> > >>> >> > > > > > >
> > >>> >> > > > > > > The following are tracked in DRILL-1491, but it
> doesn't
> > >>> look
> > >>> >> like
> > >>> >> > > > we're
> > >>> >> > > > > > > addressing these. Are we?
> > >>> >> > > > > > >
> > >>> >> > > > > > > DRILL-4329 13 Unit tests are failing with JDK 8
> > >>> >> > > > > > > DRILL-4333 DRILL-4329 tests in
> > >>> >> > > > > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in
> > Java
> > >>> 8
> > >>> >> > > > > > > DRILL-5120 Upgrade JDBC Driver for new Java 8
> > methods
> > >>> >> > > > > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in
> > >>> Eclipse
> > >>> >> with
> > >>> >> > > > Java
> > >>> >> > > > > 8
> > >>> >> > > > > > >
> > >>> >> > > > > > >
> > >>> >> > > > > > > *DRILL-4547 is a showstopper*. The release build
> > >>> >> > (-Papache-release)
> > >>> >> > > > > fails
> > >>> >> > > > > > > with far too many Javadoc errors even with doc lint
> > turned
> > >>> >> off.
> > >>> >> > > > > > >
> > >>> >> > > > > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related
> > which
> > >>> is a
> > >>> >> > > > project
> > >>> >> > > > > by
> > >>> >> > > > > > > itself.
> > >>> >> > > > > > >
> > >>> >> > > > > > > Note that fixing JDBC related issues and adding the
> > >>> command
> > >>> >> line
> > >>> >> > > > option
> > >>> >> > > > > > to
> > >>> >> > > > > > > turn doc lint off will likely break Java 7 builds.
> > >>> >> > > > > > >
> > >>> >> > > > > > >
> > >>> >> > > > > > > Folks who voted to get JDK 8 into this release, what
> is
> > >>> the
> > >>> >> > > consensus
> > >>> >> > > > > on
> > >>> >> > > > > > > JDBC/Java8 ?
> > >>> >> > > > > > > Also, any volunteers on helping debug
> > >>> >> > > > > > > TestDrillbitResilience.cancelAfterAllResultsProduced
> > >>> >> > > > > > > ?
> > >>> >> > > > > > >
> > >>> >> > > > > > >
> > >>> >> > > > > > >
> > >>> >> > > > > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <
> > >>> >> > > > vovatkach75@gmail.com
> > >>> >> > > > > >
> > >>> >> > > > > > > wrote:
> > >>> >> > > > > > >
> > >>> >> > > > > > > > Addition to my last message:
> > >>> >> > > > > > > > The link with PR for DRILL-1491
> > >>> >> https://urldefense.proofpoint
> > >>> >> > .
> > >>> >> > > > > > > com/v2/url?u=https-3A__github.
> > >>> com_apache_drill_pull_1143&d=
> > >>> >> > > DwIBaQ&c=
> > >>> >> > > > > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=
> > >>> >> > > oTnKwfjj5hFBosMrq_
> > >>> >> > > > > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3
> > >>> uialdRhgnf3XRY22R4SWXGZIq66a
> > >>> >> > > > > > Pijuy-Ms0J_-4&e=
> > >>> >> > > > > > > > on which the we can see TestDrillbitResilience.
> > >>> >> > > > > > > > cancelAfterAllResultsProduced
> > >>> >> > > > > > > > failure.
> > >>> >> > > > > > > >
> > >>> >> > > > > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <
> > >>> >> > > vovatkach75@gmail.com
> > >>> >> > > > >:
> > >>> >> > > > > > > >
> > >>> >> > > > > > > > > *To Parth:*
> > >>> >> > > > > > > > > The failure can only be seen if run on DRILL-1491
> > >>> branch,
> > >>> >> > > because
> > >>> >> > > > > it
> > >>> >> > > > > > > uses
> > >>> >> > > > > > > > > jdk 1.8 in pom.xml
> > >>> >> > > > > > > > >
> > >>> >> > > > > > > > > <source>1.8</source>
> > >>> >> > > > > > > > > <target>1.8</target>
> > >>> >> > > > > > > > >
> > >>> >> > > > > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <
> > >>> >> > > > shamirwasia@mapr.com
> > >>> >> > > > > >:
> > >>> >> > > > > > > > >
> > >>> >> > > > > > > > >> Just sent an email on RCA of DRILL-6216 to
> discuss
> > >>> next
> > >>> >> > steps.
> > >>> >> > > > > > > > >>
> > >>> >> > > > > > > > >>
> > >>> >> > > > > > > > >> Thanks,
> > >>> >> > > > > > > > >> Sorabh
> > >>> >> > > > > > > > >>
> > >>> >> > > > > > > > >> ________________________________
> > >>> >> > > > > > > > >> From: Parth Chandra <pa...@apache.org>
> > >>> >> > > > > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
> > >>> >> > > > > > > > >> To: dev
> > >>> >> > > > > > > > >> Subject: Re: [DISCUSS] 1.13.0 release
> > >>> >> > > > > > > > >>
> > >>> >> > > > > > > > >> We have two items remaining -
> > >>> >> > > > > > > > >>
> > >>> >> > > > > > > > >> DRILL-1491 - Ideally, I would like to make sure
> > that
> > >>> >> CANCEL
> > >>> >> > is
> > >>> >> > > > > > handled
> > >>> >> > > > > > > > >> correctly with JDK 8. If the failure of the unit
> > >>> test is
> > >>> >> > > because
> > >>> >> > > > > the
> > >>> >> > > > > > > > >> cancel
> > >>> >> > > > > > > > >> is received after the query is completed, then
> the
> > >>> issue
> > >>> >> is
> > >>> >> > > less
> > >>> >> > > > > > > severe,
> > >>> >> > > > > > > > >> but I would like to be sure that this is the
> case.
> > >>> >> > > > > > > > >> Are there others who see the DrillbitResilience
> > tests
> > >>> >> > failing
> > >>> >> > > > for
> > >>> >> > > > > > > them?
> > >>> >> > > > > > > > >> Can
> > >>> >> > > > > > > > >> we try to assist Volodymyr? I don't see the
> > failures
> > >>> >> myself.
> > >>> >> > > > > > > > >>
> > >>> >> > > > > > > > >> DRILL-6216 - this is a showstopper.
> > >>> >> > > > > > > > >>
> > >>> >> > > > > > > > >>
> > >>> >> > > > > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
> > >>> >> > > > > kunalkhatua@gmail.com
> > >>> >> > > > > > >
> > >>> >> > > > > > > > >> wrote:
> > >>> >> > > > > > > > >>
> > >>> >> > > > > > > > >> > Hi Parth
> > >>> >> > > > > > > > >> >
> > >>> >> > > > > > > > >> > DRILL-6216 is a release blocker that is being
> > >>> currently
> > >>> >> > > looked
> > >>> >> > > > > > into.
> > >>> >> > > > > > > > >> >
> > >>> >> > > > > > > > >> > Ref:
> > >>> >> > > > > > > > >> > DRILL-6216: Metadata mismatch when connecting
> to
> > a
> > >>> >> Drill
> > >>> >> > > > 1.12.0
> > >>> >> > > > > > > with a
> > >>> >> > > > > > > > >> > Drill-1.13.0-SNAPSHOT driver
> > >>> >> > > > > > > > >> > https://urldefense.proofpoint.
> > >>> >> > com/v2/url?u=https-3A__issues
> > >>> >> > > .
> > >>> >> > > > > > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=
> > >>> >> > cskdkSMqhcnjZ
> > >>> >> > > > > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOt
> > >>> >> > hF2jlYxvhTlQg&m=xu
> > >>> >> > > > > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz
> > >>> >> > 3sxpP5-wHPs&s=txeKaKzF67flAi48
> > >>> >> > > > > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> > >>> >> > > > > > > > >> >
> > >>> >> > > > > > > > >> > Please add it to the list of required commits
> as
> > >>> well.
> > >>> >> > > > > > > > >> >
> > >>> >> > > > > > > > >> > Thanks
> > >>> >> > > > > > > > >> > ~ Kunal
> > >>> >> > > > > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <
> > >>> >> > > > vovatkach75@gmail.com>
> > >>> >> > > > > > > > wrote:
> > >>> >> > > > > > > > >> > Right now i haven't found the reason of
> > >>> >> > > > > > > > >> > TestDrillbitResilience.cancelA
> > >>> fterAllResultsProduced
> > >>> >> > > failure,
> > >>> >> > > > > > most
> > >>> >> > > > > > > > >> likely
> > >>> >> > > > > > > > >> > the cause of the failure is that the query is
> > able
> > >>> to
> > >>> >> have
> > >>> >> > > > been
> > >>> >> > > > > > > > >> completed
> > >>> >> > > > > > > > >> > before cancellation request is processed.
> > >>> >> > > > > > > > >> > This test not only the case, there is one more
> > >>> ignored
> > >>> >> > test
> > >>> >> > > > > > > > >> > TestDrillbitResilience.cancelA
> > >>> >> fterEverythingIsCompleted
> > >>> >> > and
> > >>> >> > > > > jira
> > >>> >> > > > > > > > >> > DRILL-3967
> > >>> >> > > > > > > > >> > created, although the environment is AWS.
> > >>> >> > > > > > > > >> >
> > >>> >> > > > > > > > >> > Maybe it makes sense to ignore this test to
> > >>> unblock the
> > >>> >> > > > release
> > >>> >> > > > > > and
> > >>> >> > > > > > > > >> merge
> > >>> >> > > > > > > > >> > JDK8 changes?
> > >>> >> > > > > > > > >> >
> > >>> >> > > > > > > > >>
> > >>> >> > > > > > > > >
> > >>> >> > > > > > > > >
> > >>> >> > > > > > > >
> > >>> >> > > > > > >
> > >>> >> > > > > >
> > >>> >> > > > >
> > >>> >> > > >
> > >>> >> > >
> > >>> >> >
> > >>> >>
> > >>> >
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Aman Sinha <am...@apache.org>.
Hmm...322 warnings seems too much to address in a day or two. Parth, if
you are able to complete a build (with warnings but no errors) with the
branch link that Jyotsna I think we could go ahead with making the change
in the pom file and fix the warnings post 1.13.0. Any thoughts ?
Also, note that this would mean that anyone who wants to build the source
with JDK 7 would have to use the 1.12 release.
-Aman
On Thu, Mar 8, 2018 at 3:59 PM, Jyothsna Reddy <jy...@gmail.com>
wrote:
> When I disabled doclint and there were around 322 java doc warnings. I'm
> not sure if we can fix all of them and I'm attaching a link to the log file
> that contains all the warnings. Please let me know your thoughts on this.
>
> https://drive.google.com/file/d/1GQ6xORr0HkoEmorzvTcl0XZS4NeLX
> u83/view?usp=sharing
>
>
>
>
>
> On Thu, Mar 8, 2018 at 10:39 AM, Jyothsna Reddy <jy...@gmail.com>
> wrote:
>
> > I tried to disable it using -Dadditionalparam=-Xdoclint:none. Somewhere
> I
> > have read that -DXdoclint:none is for older versions of maven. But I'm
> not
> > sure of it.
> >
> >
> >
> >
> > On Thu, Mar 8, 2018 at 10:18 AM, Jyothsna Reddy <jy...@gmail.com>
> > wrote:
> >
> >> Following branch has changed related to disabling doclint in pom.xml.
> >> Please check it out.
> >>
> >> https://github.com/dvjyothsna/drill.git DRILL-4547
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Thu, Mar 8, 2018 at 4:19 AM, Parth Chandra <pa...@apache.org>
> wrote:
> >>
> >>> The issue with the failing test in TestDrillbitResilience.
> >>> cancelAfterAllResultsProduced is similar to DRILL-3967 (
> >>> TestDrillbitResilience.cancelAfterEverythingIsCompleted failure).
> >>>
> >>> In both this test case, and DRILL-3967, the query is paused (in
> different
> >>> places) and a cancel is sent. The query is then resumed and the
> resulting
> >>> state is checked. The problem is that the tests have a race condition
> >>> between the cancellation and the resuming of the query. Sometimes the
> >>> resume reaches first and sometimes the cancel reaches first. The
> failure
> >>> described by Volodymyr is caused by this race condition. I don't know
> why
> >>> the test was done like this, but this is an existing problem and
> >>> shouldn't
> >>> hold up the release.
> >>>
> >>> However, I also see a failure where we encounter an illegal state
> >>> transition (the query state is CANCELLATION_REQUESTED and the Foreman
> >>> tries
> >>> to move to an ENQUEUED state). This happens once in about twenty-five
> >>> executions. The Foreman only requests ENQUEUE once, when the query is
> >>> about
> >>> to start, so this means the cancellation request reached before the
> query
> >>> start request. How this happens in the unit test I have not been able
> to
> >>> determine yet (because it really shouldn't be possible).
> >>>
> >>> To recreate the problem I simply added a repeat rule in the class and
> >>> set a
> >>> repeat count of 1000. The problem occurs easily if the test is run from
> >>> the
> >>> command line. When running in debug, I was unable to see the problem.
> >>>
> >>> I'll spend some more time on this, but just in case someone wants to
> >>> investigate further, feel free ...
> >>>
> >>>
> >>> On Thu, Mar 8, 2018 at 10:59 AM, Parth Chandra <pa...@apache.org>
> >>> wrote:
> >>>
> >>> > Not sure if that would work. The release build does not allow
> >>> uncommitted
> >>> > files, so I have to commit pom.xml changes to at least the local
> repo,
> >>> > which will get pushed to my public repo when the release is done. Not
> >>> > committing this to Apache master would be cheating would leave us
> with
> >>> a
> >>> > build that does not match any source in Apache master? Javadoc
> >>> generated is
> >>> > never committed to any repo. It is part of the src release jars,
> AFAIK.
> >>> >
> >>> > Also, I'm not sure where in the pom Jyothsna made the change; I added
> >>> the
> >>> > -Dxoclint:none to the build section of the apache-release profile and
> >>> java
> >>> > exec still gives over 100 javadoc errors.
> >>> >
> >>> > We have to fix these one of these days. Might as well do it now.
> >>> Knowing
> >>> > how it works, if we don't fix these now, someone will be scrambling
> to
> >>> fix
> >>> > these just before the 1.14.0 release :(
> >>> >
> >>> >
> >>> >
> >>> > On Thu, Mar 8, 2018 at 10:06 AM, Aman Sinha <am...@apache.org>
> >>> wrote:
> >>> >
> >>> >> Parth, would it work if you made the pom.xml changes locally in
> your
> >>> >> branch, generated the javadoc but only commit the javadoc jar files
> >>> to the
> >>> >> release branch, not the pom.xml changes ?
> >>> >> Anyone downloading Drill source code to build should not run into
> this
> >>> >> since typically they won't be building javadoc.
> >>> >>
> >>> >> -Aman
> >>> >>
> >>> >> On Wed, Mar 7, 2018 at 6:37 PM, Parth Chandra <pa...@apache.org>
> >>> wrote:
> >>> >>
> >>> >> > Unfortunately, we cannot do that since we also want to be able to
> >>> build
> >>> >> > with JDK 7 for at least a couple of releases to allow for a
> >>> reasonable
> >>> >> > transition time. doclint was introduced in JDK 8 so JDK 7 fails
> >>> >> because it
> >>> >> > doesn't recognize the parameter.
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> > On Thu, Mar 8, 2018 at 7:03 AM, Jyothsna Reddy <
> >>> jyothsna.dvj@gmail.com>
> >>> >> > wrote:
> >>> >> >
> >>> >> > > Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491) and
> >>> added
> >>> >> > > following lines to pom.xml to disable doc lint. The javadoc
> >>> doesn't
> >>> >> throw
> >>> >> > > any errors and the build is successful.
> >>> >> > >
> >>> >> > > <activation>
> >>> >> > >
> >>> >> > > <jdk>[1.8,)</jdk>
> >>> >> > >
> >>> >> > > </activation>
> >>> >> > >
> >>> >> > > <properties>
> >>> >> > >
> >>> >> > > <additionalparam>-Xdoclint:none</additionalparam>
> >>> >> > >
> >>> >> > > </properties>
> >>> >> > >
> >>> >> > >
> >>> >> > >
> >>> >> > >
> >>> >> > >
> >>> >> > > On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri <
> >>> >> hanu.ncr@gmail.com>
> >>> >> > > wrote:
> >>> >> > >
> >>> >> > > > On my machine I couldn't repro the issue related to
> >>> >> > > TestDrillbitResilience.
> >>> >> > > > cancelAfterAllResultsProduced.
> >>> >> > > > I used the vladimir's branch (i.e DRILL-1491).
> >>> >> > > > Used the maven test command for testing it.
> >>> >> > > >
> >>> >> > > > output of the test run.
> >>> >> > > > ... 4 common frames omitted
> >>> >> > > > Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time
> elapsed:
> >>> >> > 124.187
> >>> >> > > > sec - in org.apache.drill.exec.server.TestDrillbitResilience
> >>> >> > > >
> >>> >> > > >
> >>> >> > > >
> >>> >> > > >
> >>> >> > > > On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <
> >>> parthc@apache.org>
> >>> >> > > wrote:
> >>> >> > > >
> >>> >> > > > > Yes I agree. JDBC would be a new feature that we can defer
> to
> >>> >> 1.14.0.
> >>> >> > > > > I'm hoping we can resolve the other three in the next few
> >>> days.
> >>> >> > Target
> >>> >> > > > date
> >>> >> > > > > for starting release process - Friday Mar 9th
> >>> >> > > > >
> >>> >> > > > > Once these are resolved, I will create a branch for the
> >>> release so
> >>> >> > that
> >>> >> > > > > Apache master remains open for commits. If any issues are
> >>> found in
> >>> >> > the
> >>> >> > > > > release branch, we will fix them in master and I will
> >>> cherry-pick
> >>> >> the
> >>> >> > > > into
> >>> >> > > > > the release branch. Once the release is finalized I will
> add a
> >>> >> > release
> >>> >> > > > tag
> >>> >> > > > > and remove the branch.
> >>> >> > > > >
> >>> >> > > > > Also note if QA folks want to get started on testing the
> >>> release,
> >>> >> the
> >>> >> > > > > current head of Apache master is close to final. Javadoc
> >>> >> generation
> >>> >> > is
> >>> >> > > > only
> >>> >> > > > > a release build issue, and the other issues are localized
> to
> >>> >> > specific
> >>> >> > > > > cases.
> >>> >> > > > >
> >>> >> > > > > Note: to reproduce the javadoc issues:
> >>> >> > > > > # set JAVA_HOME to JDK 8
> >>> >> > > > > mvn javadoc:javadoc -Papache-release
> >>> >> > > > >
> >>> >> > > > >
> >>> >> > > > >
> >>> >> > > > > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <
> >>> amansinha@apache.org
> >>> >> >
> >>> >> > > > wrote:
> >>> >> > > > >
> >>> >> > > > > > It seems to me the main blockers are:
> >>> >> > > > > >
> >>> >> > > > > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we
> >>> split up
> >>> >> the
> >>> >> > > > work
> >>> >> > > > > > among few people to resolve these ?
> >>> >> > > > > > 2. DRILL-6216 Metadata mismatch.. <-- Agreement
> >>> was
> >>> >> to
> >>> >> > > > revert
> >>> >> > > > > > one small piece of code and it appears Sorabh is looking
> >>> into it
> >>> >> > > > > > 3. TestDrillbitResilience.cancelAfterAllResultsProduced
> >>> <--
> >>> >> need
> >>> >> > > > > someone
> >>> >> > > > > > to look into this
> >>> >> > > > > >
> >>> >> > > > > > Regarding the JDBC issues that Parth mentioned, looking at
> >>> the
> >>> >> > JIRAs,
> >>> >> > > > it
> >>> >> > > > > > seems they are not showstoppers...Parth do you agree ?
> >>> >> > > > > >
> >>> >> > > > > > Since we are close to the finish line for JDK 8, IMO we
> >>> should
> >>> >> try
> >>> >> > > and
> >>> >> > > > > see
> >>> >> > > > > > if in another day or two we can get over these hurdles.
> >>> >> > > > > >
> >>> >> > > > > > -Aman
> >>> >> > > > > >
> >>> >> > > > > >
> >>> >> > > > > >
> >>> >> > > > > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <
> >>> pmaker@mapr.com>
> >>> >> > > wrote:
> >>> >> > > > > >
> >>> >> > > > > > > The JDK 8 issues will likely require more time to harden
> >>> for
> >>> >> it
> >>> >> > to
> >>> >> > > be
> >>> >> > > > > > > included in the 1.13 release. My recommendation would be
> >>> to
> >>> >> move
> >>> >> > > > ahead
> >>> >> > > > > > with
> >>> >> > > > > > > the 1.13 release now and address these issues right.
> >>> >> > > > > > >
> >>> >> > > > > > > Pritesh
> >>> >> > > > > > >
> >>> >> > > > > > > -----Original Message-----
> >>> >> > > > > > > From: Parth Chandra <pa...@apache.org>
> >>> >> > > > > > > Sent: March 7, 2018 3:34 AM
> >>> >> > > > > > > To: dev <de...@drill.apache.org>
> >>> >> > > > > > > Subject: Re: [DISCUSS] 1.13.0 release
> >>> >> > > > > > >
> >>> >> > > > > > > My mistake Volodymyr.
> >>> >> > > > > > >
> >>> >> > > > > > > Found some other JDK 8 issues in JIRA not tracked in
> >>> >> DRILL-1491
> >>> >> > > > > > >
> >>> >> > > > > > > DRILL-4547 Javadoc fails with Java8
> >>> >> > > > > > > DRILL-6163 Switch Travis To Java 8
> >>> >> > > > > > >
> >>> >> > > > > > > The following are tracked in DRILL-1491, but it doesn't
> >>> look
> >>> >> like
> >>> >> > > > we're
> >>> >> > > > > > > addressing these. Are we?
> >>> >> > > > > > >
> >>> >> > > > > > > DRILL-4329 13 Unit tests are failing with JDK 8
> >>> >> > > > > > > DRILL-4333 DRILL-4329 tests in
> >>> >> > > > > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in
> Java
> >>> 8
> >>> >> > > > > > > DRILL-5120 Upgrade JDBC Driver for new Java 8
> methods
> >>> >> > > > > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in
> >>> Eclipse
> >>> >> with
> >>> >> > > > Java
> >>> >> > > > > 8
> >>> >> > > > > > >
> >>> >> > > > > > >
> >>> >> > > > > > > *DRILL-4547 is a showstopper*. The release build
> >>> >> > (-Papache-release)
> >>> >> > > > > fails
> >>> >> > > > > > > with far too many Javadoc errors even with doc lint
> turned
> >>> >> off.
> >>> >> > > > > > >
> >>> >> > > > > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related
> which
> >>> is a
> >>> >> > > > project
> >>> >> > > > > by
> >>> >> > > > > > > itself.
> >>> >> > > > > > >
> >>> >> > > > > > > Note that fixing JDBC related issues and adding the
> >>> command
> >>> >> line
> >>> >> > > > option
> >>> >> > > > > > to
> >>> >> > > > > > > turn doc lint off will likely break Java 7 builds.
> >>> >> > > > > > >
> >>> >> > > > > > >
> >>> >> > > > > > > Folks who voted to get JDK 8 into this release, what is
> >>> the
> >>> >> > > consensus
> >>> >> > > > > on
> >>> >> > > > > > > JDBC/Java8 ?
> >>> >> > > > > > > Also, any volunteers on helping debug
> >>> >> > > > > > > TestDrillbitResilience.cancelAfterAllResultsProduced
> >>> >> > > > > > > ?
> >>> >> > > > > > >
> >>> >> > > > > > >
> >>> >> > > > > > >
> >>> >> > > > > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <
> >>> >> > > > vovatkach75@gmail.com
> >>> >> > > > > >
> >>> >> > > > > > > wrote:
> >>> >> > > > > > >
> >>> >> > > > > > > > Addition to my last message:
> >>> >> > > > > > > > The link with PR for DRILL-1491
> >>> >> https://urldefense.proofpoint
> >>> >> > .
> >>> >> > > > > > > com/v2/url?u=https-3A__github.
> >>> com_apache_drill_pull_1143&d=
> >>> >> > > DwIBaQ&c=
> >>> >> > > > > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=
> >>> >> > > oTnKwfjj5hFBosMrq_
> >>> >> > > > > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3
> >>> uialdRhgnf3XRY22R4SWXGZIq66a
> >>> >> > > > > > Pijuy-Ms0J_-4&e=
> >>> >> > > > > > > > on which the we can see TestDrillbitResilience.
> >>> >> > > > > > > > cancelAfterAllResultsProduced
> >>> >> > > > > > > > failure.
> >>> >> > > > > > > >
> >>> >> > > > > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <
> >>> >> > > vovatkach75@gmail.com
> >>> >> > > > >:
> >>> >> > > > > > > >
> >>> >> > > > > > > > > *To Parth:*
> >>> >> > > > > > > > > The failure can only be seen if run on DRILL-1491
> >>> branch,
> >>> >> > > because
> >>> >> > > > > it
> >>> >> > > > > > > uses
> >>> >> > > > > > > > > jdk 1.8 in pom.xml
> >>> >> > > > > > > > >
> >>> >> > > > > > > > > <source>1.8</source>
> >>> >> > > > > > > > > <target>1.8</target>
> >>> >> > > > > > > > >
> >>> >> > > > > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <
> >>> >> > > > shamirwasia@mapr.com
> >>> >> > > > > >:
> >>> >> > > > > > > > >
> >>> >> > > > > > > > >> Just sent an email on RCA of DRILL-6216 to discuss
> >>> next
> >>> >> > steps.
> >>> >> > > > > > > > >>
> >>> >> > > > > > > > >>
> >>> >> > > > > > > > >> Thanks,
> >>> >> > > > > > > > >> Sorabh
> >>> >> > > > > > > > >>
> >>> >> > > > > > > > >> ________________________________
> >>> >> > > > > > > > >> From: Parth Chandra <pa...@apache.org>
> >>> >> > > > > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
> >>> >> > > > > > > > >> To: dev
> >>> >> > > > > > > > >> Subject: Re: [DISCUSS] 1.13.0 release
> >>> >> > > > > > > > >>
> >>> >> > > > > > > > >> We have two items remaining -
> >>> >> > > > > > > > >>
> >>> >> > > > > > > > >> DRILL-1491 - Ideally, I would like to make sure
> that
> >>> >> CANCEL
> >>> >> > is
> >>> >> > > > > > handled
> >>> >> > > > > > > > >> correctly with JDK 8. If the failure of the unit
> >>> test is
> >>> >> > > because
> >>> >> > > > > the
> >>> >> > > > > > > > >> cancel
> >>> >> > > > > > > > >> is received after the query is completed, then the
> >>> issue
> >>> >> is
> >>> >> > > less
> >>> >> > > > > > > severe,
> >>> >> > > > > > > > >> but I would like to be sure that this is the case.
> >>> >> > > > > > > > >> Are there others who see the DrillbitResilience
> tests
> >>> >> > failing
> >>> >> > > > for
> >>> >> > > > > > > them?
> >>> >> > > > > > > > >> Can
> >>> >> > > > > > > > >> we try to assist Volodymyr? I don't see the
> failures
> >>> >> myself.
> >>> >> > > > > > > > >>
> >>> >> > > > > > > > >> DRILL-6216 - this is a showstopper.
> >>> >> > > > > > > > >>
> >>> >> > > > > > > > >>
> >>> >> > > > > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
> >>> >> > > > > kunalkhatua@gmail.com
> >>> >> > > > > > >
> >>> >> > > > > > > > >> wrote:
> >>> >> > > > > > > > >>
> >>> >> > > > > > > > >> > Hi Parth
> >>> >> > > > > > > > >> >
> >>> >> > > > > > > > >> > DRILL-6216 is a release blocker that is being
> >>> currently
> >>> >> > > looked
> >>> >> > > > > > into.
> >>> >> > > > > > > > >> >
> >>> >> > > > > > > > >> > Ref:
> >>> >> > > > > > > > >> > DRILL-6216: Metadata mismatch when connecting to
> a
> >>> >> Drill
> >>> >> > > > 1.12.0
> >>> >> > > > > > > with a
> >>> >> > > > > > > > >> > Drill-1.13.0-SNAPSHOT driver
> >>> >> > > > > > > > >> > https://urldefense.proofpoint.
> >>> >> > com/v2/url?u=https-3A__issues
> >>> >> > > .
> >>> >> > > > > > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=
> >>> >> > cskdkSMqhcnjZ
> >>> >> > > > > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOt
> >>> >> > hF2jlYxvhTlQg&m=xu
> >>> >> > > > > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz
> >>> >> > 3sxpP5-wHPs&s=txeKaKzF67flAi48
> >>> >> > > > > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> >>> >> > > > > > > > >> >
> >>> >> > > > > > > > >> > Please add it to the list of required commits as
> >>> well.
> >>> >> > > > > > > > >> >
> >>> >> > > > > > > > >> > Thanks
> >>> >> > > > > > > > >> > ~ Kunal
> >>> >> > > > > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <
> >>> >> > > > vovatkach75@gmail.com>
> >>> >> > > > > > > > wrote:
> >>> >> > > > > > > > >> > Right now i haven't found the reason of
> >>> >> > > > > > > > >> > TestDrillbitResilience.cancelA
> >>> fterAllResultsProduced
> >>> >> > > failure,
> >>> >> > > > > > most
> >>> >> > > > > > > > >> likely
> >>> >> > > > > > > > >> > the cause of the failure is that the query is
> able
> >>> to
> >>> >> have
> >>> >> > > > been
> >>> >> > > > > > > > >> completed
> >>> >> > > > > > > > >> > before cancellation request is processed.
> >>> >> > > > > > > > >> > This test not only the case, there is one more
> >>> ignored
> >>> >> > test
> >>> >> > > > > > > > >> > TestDrillbitResilience.cancelA
> >>> >> fterEverythingIsCompleted
> >>> >> > and
> >>> >> > > > > jira
> >>> >> > > > > > > > >> > DRILL-3967
> >>> >> > > > > > > > >> > created, although the environment is AWS.
> >>> >> > > > > > > > >> >
> >>> >> > > > > > > > >> > Maybe it makes sense to ignore this test to
> >>> unblock the
> >>> >> > > > release
> >>> >> > > > > > and
> >>> >> > > > > > > > >> merge
> >>> >> > > > > > > > >> > JDK8 changes?
> >>> >> > > > > > > > >> >
> >>> >> > > > > > > > >>
> >>> >> > > > > > > > >
> >>> >> > > > > > > > >
> >>> >> > > > > > > >
> >>> >> > > > > > >
> >>> >> > > > > >
> >>> >> > > > >
> >>> >> > > >
> >>> >> > >
> >>> >> >
> >>> >>
> >>> >
> >>> >
> >>>
> >>
> >>
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Jyothsna Reddy <jy...@gmail.com>.
When I disabled doclint and there were around 322 java doc warnings. I'm
not sure if we can fix all of them and I'm attaching a link to the log file
that contains all the warnings. Please let me know your thoughts on this.
https://drive.google.com/file/d/1GQ6xORr0HkoEmorzvTcl0XZS4NeLXu83/view?usp=sharing
On Thu, Mar 8, 2018 at 10:39 AM, Jyothsna Reddy <jy...@gmail.com>
wrote:
> I tried to disable it using -Dadditionalparam=-Xdoclint:none. Somewhere I
> have read that -DXdoclint:none is for older versions of maven. But I'm not
> sure of it.
>
>
>
>
> On Thu, Mar 8, 2018 at 10:18 AM, Jyothsna Reddy <jy...@gmail.com>
> wrote:
>
>> Following branch has changed related to disabling doclint in pom.xml.
>> Please check it out.
>>
>> https://github.com/dvjyothsna/drill.git DRILL-4547
>>
>>
>>
>>
>>
>>
>> On Thu, Mar 8, 2018 at 4:19 AM, Parth Chandra <pa...@apache.org> wrote:
>>
>>> The issue with the failing test in TestDrillbitResilience.
>>> cancelAfterAllResultsProduced is similar to DRILL-3967 (
>>> TestDrillbitResilience.cancelAfterEverythingIsCompleted failure).
>>>
>>> In both this test case, and DRILL-3967, the query is paused (in different
>>> places) and a cancel is sent. The query is then resumed and the resulting
>>> state is checked. The problem is that the tests have a race condition
>>> between the cancellation and the resuming of the query. Sometimes the
>>> resume reaches first and sometimes the cancel reaches first. The failure
>>> described by Volodymyr is caused by this race condition. I don't know why
>>> the test was done like this, but this is an existing problem and
>>> shouldn't
>>> hold up the release.
>>>
>>> However, I also see a failure where we encounter an illegal state
>>> transition (the query state is CANCELLATION_REQUESTED and the Foreman
>>> tries
>>> to move to an ENQUEUED state). This happens once in about twenty-five
>>> executions. The Foreman only requests ENQUEUE once, when the query is
>>> about
>>> to start, so this means the cancellation request reached before the query
>>> start request. How this happens in the unit test I have not been able to
>>> determine yet (because it really shouldn't be possible).
>>>
>>> To recreate the problem I simply added a repeat rule in the class and
>>> set a
>>> repeat count of 1000. The problem occurs easily if the test is run from
>>> the
>>> command line. When running in debug, I was unable to see the problem.
>>>
>>> I'll spend some more time on this, but just in case someone wants to
>>> investigate further, feel free ...
>>>
>>>
>>> On Thu, Mar 8, 2018 at 10:59 AM, Parth Chandra <pa...@apache.org>
>>> wrote:
>>>
>>> > Not sure if that would work. The release build does not allow
>>> uncommitted
>>> > files, so I have to commit pom.xml changes to at least the local repo,
>>> > which will get pushed to my public repo when the release is done. Not
>>> > committing this to Apache master would be cheating would leave us with
>>> a
>>> > build that does not match any source in Apache master? Javadoc
>>> generated is
>>> > never committed to any repo. It is part of the src release jars, AFAIK.
>>> >
>>> > Also, I'm not sure where in the pom Jyothsna made the change; I added
>>> the
>>> > -Dxoclint:none to the build section of the apache-release profile and
>>> java
>>> > exec still gives over 100 javadoc errors.
>>> >
>>> > We have to fix these one of these days. Might as well do it now.
>>> Knowing
>>> > how it works, if we don't fix these now, someone will be scrambling to
>>> fix
>>> > these just before the 1.14.0 release :(
>>> >
>>> >
>>> >
>>> > On Thu, Mar 8, 2018 at 10:06 AM, Aman Sinha <am...@apache.org>
>>> wrote:
>>> >
>>> >> Parth, would it work if you made the pom.xml changes locally in your
>>> >> branch, generated the javadoc but only commit the javadoc jar files
>>> to the
>>> >> release branch, not the pom.xml changes ?
>>> >> Anyone downloading Drill source code to build should not run into this
>>> >> since typically they won't be building javadoc.
>>> >>
>>> >> -Aman
>>> >>
>>> >> On Wed, Mar 7, 2018 at 6:37 PM, Parth Chandra <pa...@apache.org>
>>> wrote:
>>> >>
>>> >> > Unfortunately, we cannot do that since we also want to be able to
>>> build
>>> >> > with JDK 7 for at least a couple of releases to allow for a
>>> reasonable
>>> >> > transition time. doclint was introduced in JDK 8 so JDK 7 fails
>>> >> because it
>>> >> > doesn't recognize the parameter.
>>> >> >
>>> >> >
>>> >> >
>>> >> > On Thu, Mar 8, 2018 at 7:03 AM, Jyothsna Reddy <
>>> jyothsna.dvj@gmail.com>
>>> >> > wrote:
>>> >> >
>>> >> > > Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491) and
>>> added
>>> >> > > following lines to pom.xml to disable doc lint. The javadoc
>>> doesn't
>>> >> throw
>>> >> > > any errors and the build is successful.
>>> >> > >
>>> >> > > <activation>
>>> >> > >
>>> >> > > <jdk>[1.8,)</jdk>
>>> >> > >
>>> >> > > </activation>
>>> >> > >
>>> >> > > <properties>
>>> >> > >
>>> >> > > <additionalparam>-Xdoclint:none</additionalparam>
>>> >> > >
>>> >> > > </properties>
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > > On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri <
>>> >> hanu.ncr@gmail.com>
>>> >> > > wrote:
>>> >> > >
>>> >> > > > On my machine I couldn't repro the issue related to
>>> >> > > TestDrillbitResilience.
>>> >> > > > cancelAfterAllResultsProduced.
>>> >> > > > I used the vladimir's branch (i.e DRILL-1491).
>>> >> > > > Used the maven test command for testing it.
>>> >> > > >
>>> >> > > > output of the test run.
>>> >> > > > ... 4 common frames omitted
>>> >> > > > Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time elapsed:
>>> >> > 124.187
>>> >> > > > sec - in org.apache.drill.exec.server.TestDrillbitResilience
>>> >> > > >
>>> >> > > >
>>> >> > > >
>>> >> > > >
>>> >> > > > On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <
>>> parthc@apache.org>
>>> >> > > wrote:
>>> >> > > >
>>> >> > > > > Yes I agree. JDBC would be a new feature that we can defer to
>>> >> 1.14.0.
>>> >> > > > > I'm hoping we can resolve the other three in the next few
>>> days.
>>> >> > Target
>>> >> > > > date
>>> >> > > > > for starting release process - Friday Mar 9th
>>> >> > > > >
>>> >> > > > > Once these are resolved, I will create a branch for the
>>> release so
>>> >> > that
>>> >> > > > > Apache master remains open for commits. If any issues are
>>> found in
>>> >> > the
>>> >> > > > > release branch, we will fix them in master and I will
>>> cherry-pick
>>> >> the
>>> >> > > > into
>>> >> > > > > the release branch. Once the release is finalized I will add a
>>> >> > release
>>> >> > > > tag
>>> >> > > > > and remove the branch.
>>> >> > > > >
>>> >> > > > > Also note if QA folks want to get started on testing the
>>> release,
>>> >> the
>>> >> > > > > current head of Apache master is close to final. Javadoc
>>> >> generation
>>> >> > is
>>> >> > > > only
>>> >> > > > > a release build issue, and the other issues are localized to
>>> >> > specific
>>> >> > > > > cases.
>>> >> > > > >
>>> >> > > > > Note: to reproduce the javadoc issues:
>>> >> > > > > # set JAVA_HOME to JDK 8
>>> >> > > > > mvn javadoc:javadoc -Papache-release
>>> >> > > > >
>>> >> > > > >
>>> >> > > > >
>>> >> > > > > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <
>>> amansinha@apache.org
>>> >> >
>>> >> > > > wrote:
>>> >> > > > >
>>> >> > > > > > It seems to me the main blockers are:
>>> >> > > > > >
>>> >> > > > > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we
>>> split up
>>> >> the
>>> >> > > > work
>>> >> > > > > > among few people to resolve these ?
>>> >> > > > > > 2. DRILL-6216 Metadata mismatch.. <-- Agreement
>>> was
>>> >> to
>>> >> > > > revert
>>> >> > > > > > one small piece of code and it appears Sorabh is looking
>>> into it
>>> >> > > > > > 3. TestDrillbitResilience.cancelAfterAllResultsProduced
>>> <--
>>> >> need
>>> >> > > > > someone
>>> >> > > > > > to look into this
>>> >> > > > > >
>>> >> > > > > > Regarding the JDBC issues that Parth mentioned, looking at
>>> the
>>> >> > JIRAs,
>>> >> > > > it
>>> >> > > > > > seems they are not showstoppers...Parth do you agree ?
>>> >> > > > > >
>>> >> > > > > > Since we are close to the finish line for JDK 8, IMO we
>>> should
>>> >> try
>>> >> > > and
>>> >> > > > > see
>>> >> > > > > > if in another day or two we can get over these hurdles.
>>> >> > > > > >
>>> >> > > > > > -Aman
>>> >> > > > > >
>>> >> > > > > >
>>> >> > > > > >
>>> >> > > > > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <
>>> pmaker@mapr.com>
>>> >> > > wrote:
>>> >> > > > > >
>>> >> > > > > > > The JDK 8 issues will likely require more time to harden
>>> for
>>> >> it
>>> >> > to
>>> >> > > be
>>> >> > > > > > > included in the 1.13 release. My recommendation would be
>>> to
>>> >> move
>>> >> > > > ahead
>>> >> > > > > > with
>>> >> > > > > > > the 1.13 release now and address these issues right.
>>> >> > > > > > >
>>> >> > > > > > > Pritesh
>>> >> > > > > > >
>>> >> > > > > > > -----Original Message-----
>>> >> > > > > > > From: Parth Chandra <pa...@apache.org>
>>> >> > > > > > > Sent: March 7, 2018 3:34 AM
>>> >> > > > > > > To: dev <de...@drill.apache.org>
>>> >> > > > > > > Subject: Re: [DISCUSS] 1.13.0 release
>>> >> > > > > > >
>>> >> > > > > > > My mistake Volodymyr.
>>> >> > > > > > >
>>> >> > > > > > > Found some other JDK 8 issues in JIRA not tracked in
>>> >> DRILL-1491
>>> >> > > > > > >
>>> >> > > > > > > DRILL-4547 Javadoc fails with Java8
>>> >> > > > > > > DRILL-6163 Switch Travis To Java 8
>>> >> > > > > > >
>>> >> > > > > > > The following are tracked in DRILL-1491, but it doesn't
>>> look
>>> >> like
>>> >> > > > we're
>>> >> > > > > > > addressing these. Are we?
>>> >> > > > > > >
>>> >> > > > > > > DRILL-4329 13 Unit tests are failing with JDK 8
>>> >> > > > > > > DRILL-4333 DRILL-4329 tests in
>>> >> > > > > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in Java
>>> 8
>>> >> > > > > > > DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
>>> >> > > > > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in
>>> Eclipse
>>> >> with
>>> >> > > > Java
>>> >> > > > > 8
>>> >> > > > > > >
>>> >> > > > > > >
>>> >> > > > > > > *DRILL-4547 is a showstopper*. The release build
>>> >> > (-Papache-release)
>>> >> > > > > fails
>>> >> > > > > > > with far too many Javadoc errors even with doc lint turned
>>> >> off.
>>> >> > > > > > >
>>> >> > > > > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which
>>> is a
>>> >> > > > project
>>> >> > > > > by
>>> >> > > > > > > itself.
>>> >> > > > > > >
>>> >> > > > > > > Note that fixing JDBC related issues and adding the
>>> command
>>> >> line
>>> >> > > > option
>>> >> > > > > > to
>>> >> > > > > > > turn doc lint off will likely break Java 7 builds.
>>> >> > > > > > >
>>> >> > > > > > >
>>> >> > > > > > > Folks who voted to get JDK 8 into this release, what is
>>> the
>>> >> > > consensus
>>> >> > > > > on
>>> >> > > > > > > JDBC/Java8 ?
>>> >> > > > > > > Also, any volunteers on helping debug
>>> >> > > > > > > TestDrillbitResilience.cancelAfterAllResultsProduced
>>> >> > > > > > > ?
>>> >> > > > > > >
>>> >> > > > > > >
>>> >> > > > > > >
>>> >> > > > > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <
>>> >> > > > vovatkach75@gmail.com
>>> >> > > > > >
>>> >> > > > > > > wrote:
>>> >> > > > > > >
>>> >> > > > > > > > Addition to my last message:
>>> >> > > > > > > > The link with PR for DRILL-1491
>>> >> https://urldefense.proofpoint
>>> >> > .
>>> >> > > > > > > com/v2/url?u=https-3A__github.
>>> com_apache_drill_pull_1143&d=
>>> >> > > DwIBaQ&c=
>>> >> > > > > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=
>>> >> > > oTnKwfjj5hFBosMrq_
>>> >> > > > > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3
>>> uialdRhgnf3XRY22R4SWXGZIq66a
>>> >> > > > > > Pijuy-Ms0J_-4&e=
>>> >> > > > > > > > on which the we can see TestDrillbitResilience.
>>> >> > > > > > > > cancelAfterAllResultsProduced
>>> >> > > > > > > > failure.
>>> >> > > > > > > >
>>> >> > > > > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <
>>> >> > > vovatkach75@gmail.com
>>> >> > > > >:
>>> >> > > > > > > >
>>> >> > > > > > > > > *To Parth:*
>>> >> > > > > > > > > The failure can only be seen if run on DRILL-1491
>>> branch,
>>> >> > > because
>>> >> > > > > it
>>> >> > > > > > > uses
>>> >> > > > > > > > > jdk 1.8 in pom.xml
>>> >> > > > > > > > >
>>> >> > > > > > > > > <source>1.8</source>
>>> >> > > > > > > > > <target>1.8</target>
>>> >> > > > > > > > >
>>> >> > > > > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <
>>> >> > > > shamirwasia@mapr.com
>>> >> > > > > >:
>>> >> > > > > > > > >
>>> >> > > > > > > > >> Just sent an email on RCA of DRILL-6216 to discuss
>>> next
>>> >> > steps.
>>> >> > > > > > > > >>
>>> >> > > > > > > > >>
>>> >> > > > > > > > >> Thanks,
>>> >> > > > > > > > >> Sorabh
>>> >> > > > > > > > >>
>>> >> > > > > > > > >> ________________________________
>>> >> > > > > > > > >> From: Parth Chandra <pa...@apache.org>
>>> >> > > > > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
>>> >> > > > > > > > >> To: dev
>>> >> > > > > > > > >> Subject: Re: [DISCUSS] 1.13.0 release
>>> >> > > > > > > > >>
>>> >> > > > > > > > >> We have two items remaining -
>>> >> > > > > > > > >>
>>> >> > > > > > > > >> DRILL-1491 - Ideally, I would like to make sure that
>>> >> CANCEL
>>> >> > is
>>> >> > > > > > handled
>>> >> > > > > > > > >> correctly with JDK 8. If the failure of the unit
>>> test is
>>> >> > > because
>>> >> > > > > the
>>> >> > > > > > > > >> cancel
>>> >> > > > > > > > >> is received after the query is completed, then the
>>> issue
>>> >> is
>>> >> > > less
>>> >> > > > > > > severe,
>>> >> > > > > > > > >> but I would like to be sure that this is the case.
>>> >> > > > > > > > >> Are there others who see the DrillbitResilience tests
>>> >> > failing
>>> >> > > > for
>>> >> > > > > > > them?
>>> >> > > > > > > > >> Can
>>> >> > > > > > > > >> we try to assist Volodymyr? I don't see the failures
>>> >> myself.
>>> >> > > > > > > > >>
>>> >> > > > > > > > >> DRILL-6216 - this is a showstopper.
>>> >> > > > > > > > >>
>>> >> > > > > > > > >>
>>> >> > > > > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
>>> >> > > > > kunalkhatua@gmail.com
>>> >> > > > > > >
>>> >> > > > > > > > >> wrote:
>>> >> > > > > > > > >>
>>> >> > > > > > > > >> > Hi Parth
>>> >> > > > > > > > >> >
>>> >> > > > > > > > >> > DRILL-6216 is a release blocker that is being
>>> currently
>>> >> > > looked
>>> >> > > > > > into.
>>> >> > > > > > > > >> >
>>> >> > > > > > > > >> > Ref:
>>> >> > > > > > > > >> > DRILL-6216: Metadata mismatch when connecting to a
>>> >> Drill
>>> >> > > > 1.12.0
>>> >> > > > > > > with a
>>> >> > > > > > > > >> > Drill-1.13.0-SNAPSHOT driver
>>> >> > > > > > > > >> > https://urldefense.proofpoint.
>>> >> > com/v2/url?u=https-3A__issues
>>> >> > > .
>>> >> > > > > > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=
>>> >> > cskdkSMqhcnjZ
>>> >> > > > > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOt
>>> >> > hF2jlYxvhTlQg&m=xu
>>> >> > > > > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz
>>> >> > 3sxpP5-wHPs&s=txeKaKzF67flAi48
>>> >> > > > > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
>>> >> > > > > > > > >> >
>>> >> > > > > > > > >> > Please add it to the list of required commits as
>>> well.
>>> >> > > > > > > > >> >
>>> >> > > > > > > > >> > Thanks
>>> >> > > > > > > > >> > ~ Kunal
>>> >> > > > > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <
>>> >> > > > vovatkach75@gmail.com>
>>> >> > > > > > > > wrote:
>>> >> > > > > > > > >> > Right now i haven't found the reason of
>>> >> > > > > > > > >> > TestDrillbitResilience.cancelA
>>> fterAllResultsProduced
>>> >> > > failure,
>>> >> > > > > > most
>>> >> > > > > > > > >> likely
>>> >> > > > > > > > >> > the cause of the failure is that the query is able
>>> to
>>> >> have
>>> >> > > > been
>>> >> > > > > > > > >> completed
>>> >> > > > > > > > >> > before cancellation request is processed.
>>> >> > > > > > > > >> > This test not only the case, there is one more
>>> ignored
>>> >> > test
>>> >> > > > > > > > >> > TestDrillbitResilience.cancelA
>>> >> fterEverythingIsCompleted
>>> >> > and
>>> >> > > > > jira
>>> >> > > > > > > > >> > DRILL-3967
>>> >> > > > > > > > >> > created, although the environment is AWS.
>>> >> > > > > > > > >> >
>>> >> > > > > > > > >> > Maybe it makes sense to ignore this test to
>>> unblock the
>>> >> > > > release
>>> >> > > > > > and
>>> >> > > > > > > > >> merge
>>> >> > > > > > > > >> > JDK8 changes?
>>> >> > > > > > > > >> >
>>> >> > > > > > > > >>
>>> >> > > > > > > > >
>>> >> > > > > > > > >
>>> >> > > > > > > >
>>> >> > > > > > >
>>> >> > > > > >
>>> >> > > > >
>>> >> > > >
>>> >> > >
>>> >> >
>>> >>
>>> >
>>> >
>>>
>>
>>
>
Re: [DISCUSS] 1.13.0 release
Posted by Jyothsna Reddy <jy...@gmail.com>.
I tried to disable it using -Dadditionalparam=-Xdoclint:none. Somewhere I
have read that -DXdoclint:none is for older versions of maven. But I'm not
sure of it.
On Thu, Mar 8, 2018 at 10:18 AM, Jyothsna Reddy <jy...@gmail.com>
wrote:
> Following branch has changed related to disabling doclint in pom.xml.
> Please check it out.
>
> https://github.com/dvjyothsna/drill.git DRILL-4547
>
>
>
>
>
>
> On Thu, Mar 8, 2018 at 4:19 AM, Parth Chandra <pa...@apache.org> wrote:
>
>> The issue with the failing test in TestDrillbitResilience.
>> cancelAfterAllResultsProduced is similar to DRILL-3967 (
>> TestDrillbitResilience.cancelAfterEverythingIsCompleted failure).
>>
>> In both this test case, and DRILL-3967, the query is paused (in different
>> places) and a cancel is sent. The query is then resumed and the resulting
>> state is checked. The problem is that the tests have a race condition
>> between the cancellation and the resuming of the query. Sometimes the
>> resume reaches first and sometimes the cancel reaches first. The failure
>> described by Volodymyr is caused by this race condition. I don't know why
>> the test was done like this, but this is an existing problem and shouldn't
>> hold up the release.
>>
>> However, I also see a failure where we encounter an illegal state
>> transition (the query state is CANCELLATION_REQUESTED and the Foreman
>> tries
>> to move to an ENQUEUED state). This happens once in about twenty-five
>> executions. The Foreman only requests ENQUEUE once, when the query is
>> about
>> to start, so this means the cancellation request reached before the query
>> start request. How this happens in the unit test I have not been able to
>> determine yet (because it really shouldn't be possible).
>>
>> To recreate the problem I simply added a repeat rule in the class and set
>> a
>> repeat count of 1000. The problem occurs easily if the test is run from
>> the
>> command line. When running in debug, I was unable to see the problem.
>>
>> I'll spend some more time on this, but just in case someone wants to
>> investigate further, feel free ...
>>
>>
>> On Thu, Mar 8, 2018 at 10:59 AM, Parth Chandra <pa...@apache.org> wrote:
>>
>> > Not sure if that would work. The release build does not allow
>> uncommitted
>> > files, so I have to commit pom.xml changes to at least the local repo,
>> > which will get pushed to my public repo when the release is done. Not
>> > committing this to Apache master would be cheating would leave us with a
>> > build that does not match any source in Apache master? Javadoc
>> generated is
>> > never committed to any repo. It is part of the src release jars, AFAIK.
>> >
>> > Also, I'm not sure where in the pom Jyothsna made the change; I added
>> the
>> > -Dxoclint:none to the build section of the apache-release profile and
>> java
>> > exec still gives over 100 javadoc errors.
>> >
>> > We have to fix these one of these days. Might as well do it now. Knowing
>> > how it works, if we don't fix these now, someone will be scrambling to
>> fix
>> > these just before the 1.14.0 release :(
>> >
>> >
>> >
>> > On Thu, Mar 8, 2018 at 10:06 AM, Aman Sinha <am...@apache.org>
>> wrote:
>> >
>> >> Parth, would it work if you made the pom.xml changes locally in your
>> >> branch, generated the javadoc but only commit the javadoc jar files to
>> the
>> >> release branch, not the pom.xml changes ?
>> >> Anyone downloading Drill source code to build should not run into this
>> >> since typically they won't be building javadoc.
>> >>
>> >> -Aman
>> >>
>> >> On Wed, Mar 7, 2018 at 6:37 PM, Parth Chandra <pa...@apache.org>
>> wrote:
>> >>
>> >> > Unfortunately, we cannot do that since we also want to be able to
>> build
>> >> > with JDK 7 for at least a couple of releases to allow for a
>> reasonable
>> >> > transition time. doclint was introduced in JDK 8 so JDK 7 fails
>> >> because it
>> >> > doesn't recognize the parameter.
>> >> >
>> >> >
>> >> >
>> >> > On Thu, Mar 8, 2018 at 7:03 AM, Jyothsna Reddy <
>> jyothsna.dvj@gmail.com>
>> >> > wrote:
>> >> >
>> >> > > Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491) and
>> added
>> >> > > following lines to pom.xml to disable doc lint. The javadoc doesn't
>> >> throw
>> >> > > any errors and the build is successful.
>> >> > >
>> >> > > <activation>
>> >> > >
>> >> > > <jdk>[1.8,)</jdk>
>> >> > >
>> >> > > </activation>
>> >> > >
>> >> > > <properties>
>> >> > >
>> >> > > <additionalparam>-Xdoclint:none</additionalparam>
>> >> > >
>> >> > > </properties>
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > > On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri <
>> >> hanu.ncr@gmail.com>
>> >> > > wrote:
>> >> > >
>> >> > > > On my machine I couldn't repro the issue related to
>> >> > > TestDrillbitResilience.
>> >> > > > cancelAfterAllResultsProduced.
>> >> > > > I used the vladimir's branch (i.e DRILL-1491).
>> >> > > > Used the maven test command for testing it.
>> >> > > >
>> >> > > > output of the test run.
>> >> > > > ... 4 common frames omitted
>> >> > > > Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time elapsed:
>> >> > 124.187
>> >> > > > sec - in org.apache.drill.exec.server.TestDrillbitResilience
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <
>> parthc@apache.org>
>> >> > > wrote:
>> >> > > >
>> >> > > > > Yes I agree. JDBC would be a new feature that we can defer to
>> >> 1.14.0.
>> >> > > > > I'm hoping we can resolve the other three in the next few days.
>> >> > Target
>> >> > > > date
>> >> > > > > for starting release process - Friday Mar 9th
>> >> > > > >
>> >> > > > > Once these are resolved, I will create a branch for the
>> release so
>> >> > that
>> >> > > > > Apache master remains open for commits. If any issues are
>> found in
>> >> > the
>> >> > > > > release branch, we will fix them in master and I will
>> cherry-pick
>> >> the
>> >> > > > into
>> >> > > > > the release branch. Once the release is finalized I will add a
>> >> > release
>> >> > > > tag
>> >> > > > > and remove the branch.
>> >> > > > >
>> >> > > > > Also note if QA folks want to get started on testing the
>> release,
>> >> the
>> >> > > > > current head of Apache master is close to final. Javadoc
>> >> generation
>> >> > is
>> >> > > > only
>> >> > > > > a release build issue, and the other issues are localized to
>> >> > specific
>> >> > > > > cases.
>> >> > > > >
>> >> > > > > Note: to reproduce the javadoc issues:
>> >> > > > > # set JAVA_HOME to JDK 8
>> >> > > > > mvn javadoc:javadoc -Papache-release
>> >> > > > >
>> >> > > > >
>> >> > > > >
>> >> > > > > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <
>> amansinha@apache.org
>> >> >
>> >> > > > wrote:
>> >> > > > >
>> >> > > > > > It seems to me the main blockers are:
>> >> > > > > >
>> >> > > > > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we split
>> up
>> >> the
>> >> > > > work
>> >> > > > > > among few people to resolve these ?
>> >> > > > > > 2. DRILL-6216 Metadata mismatch.. <-- Agreement
>> was
>> >> to
>> >> > > > revert
>> >> > > > > > one small piece of code and it appears Sorabh is looking
>> into it
>> >> > > > > > 3. TestDrillbitResilience.cancelAfterAllResultsProduced <--
>> >> need
>> >> > > > > someone
>> >> > > > > > to look into this
>> >> > > > > >
>> >> > > > > > Regarding the JDBC issues that Parth mentioned, looking at
>> the
>> >> > JIRAs,
>> >> > > > it
>> >> > > > > > seems they are not showstoppers...Parth do you agree ?
>> >> > > > > >
>> >> > > > > > Since we are close to the finish line for JDK 8, IMO we
>> should
>> >> try
>> >> > > and
>> >> > > > > see
>> >> > > > > > if in another day or two we can get over these hurdles.
>> >> > > > > >
>> >> > > > > > -Aman
>> >> > > > > >
>> >> > > > > >
>> >> > > > > >
>> >> > > > > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <
>> pmaker@mapr.com>
>> >> > > wrote:
>> >> > > > > >
>> >> > > > > > > The JDK 8 issues will likely require more time to harden
>> for
>> >> it
>> >> > to
>> >> > > be
>> >> > > > > > > included in the 1.13 release. My recommendation would be to
>> >> move
>> >> > > > ahead
>> >> > > > > > with
>> >> > > > > > > the 1.13 release now and address these issues right.
>> >> > > > > > >
>> >> > > > > > > Pritesh
>> >> > > > > > >
>> >> > > > > > > -----Original Message-----
>> >> > > > > > > From: Parth Chandra <pa...@apache.org>
>> >> > > > > > > Sent: March 7, 2018 3:34 AM
>> >> > > > > > > To: dev <de...@drill.apache.org>
>> >> > > > > > > Subject: Re: [DISCUSS] 1.13.0 release
>> >> > > > > > >
>> >> > > > > > > My mistake Volodymyr.
>> >> > > > > > >
>> >> > > > > > > Found some other JDK 8 issues in JIRA not tracked in
>> >> DRILL-1491
>> >> > > > > > >
>> >> > > > > > > DRILL-4547 Javadoc fails with Java8
>> >> > > > > > > DRILL-6163 Switch Travis To Java 8
>> >> > > > > > >
>> >> > > > > > > The following are tracked in DRILL-1491, but it doesn't
>> look
>> >> like
>> >> > > > we're
>> >> > > > > > > addressing these. Are we?
>> >> > > > > > >
>> >> > > > > > > DRILL-4329 13 Unit tests are failing with JDK 8
>> >> > > > > > > DRILL-4333 DRILL-4329 tests in
>> >> > > > > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
>> >> > > > > > > DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
>> >> > > > > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in
>> Eclipse
>> >> with
>> >> > > > Java
>> >> > > > > 8
>> >> > > > > > >
>> >> > > > > > >
>> >> > > > > > > *DRILL-4547 is a showstopper*. The release build
>> >> > (-Papache-release)
>> >> > > > > fails
>> >> > > > > > > with far too many Javadoc errors even with doc lint turned
>> >> off.
>> >> > > > > > >
>> >> > > > > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which
>> is a
>> >> > > > project
>> >> > > > > by
>> >> > > > > > > itself.
>> >> > > > > > >
>> >> > > > > > > Note that fixing JDBC related issues and adding the command
>> >> line
>> >> > > > option
>> >> > > > > > to
>> >> > > > > > > turn doc lint off will likely break Java 7 builds.
>> >> > > > > > >
>> >> > > > > > >
>> >> > > > > > > Folks who voted to get JDK 8 into this release, what is the
>> >> > > consensus
>> >> > > > > on
>> >> > > > > > > JDBC/Java8 ?
>> >> > > > > > > Also, any volunteers on helping debug
>> >> > > > > > > TestDrillbitResilience.cancelAfterAllResultsProduced
>> >> > > > > > > ?
>> >> > > > > > >
>> >> > > > > > >
>> >> > > > > > >
>> >> > > > > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <
>> >> > > > vovatkach75@gmail.com
>> >> > > > > >
>> >> > > > > > > wrote:
>> >> > > > > > >
>> >> > > > > > > > Addition to my last message:
>> >> > > > > > > > The link with PR for DRILL-1491
>> >> https://urldefense.proofpoint
>> >> > .
>> >> > > > > > > com/v2/url?u=https-3A__github.
>> com_apache_drill_pull_1143&d=
>> >> > > DwIBaQ&c=
>> >> > > > > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=
>> >> > > oTnKwfjj5hFBosMrq_
>> >> > > > > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3uialdRhgnf3XRY22R4SWXGZIq66a
>> >> > > > > > Pijuy-Ms0J_-4&e=
>> >> > > > > > > > on which the we can see TestDrillbitResilience.
>> >> > > > > > > > cancelAfterAllResultsProduced
>> >> > > > > > > > failure.
>> >> > > > > > > >
>> >> > > > > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <
>> >> > > vovatkach75@gmail.com
>> >> > > > >:
>> >> > > > > > > >
>> >> > > > > > > > > *To Parth:*
>> >> > > > > > > > > The failure can only be seen if run on DRILL-1491
>> branch,
>> >> > > because
>> >> > > > > it
>> >> > > > > > > uses
>> >> > > > > > > > > jdk 1.8 in pom.xml
>> >> > > > > > > > >
>> >> > > > > > > > > <source>1.8</source>
>> >> > > > > > > > > <target>1.8</target>
>> >> > > > > > > > >
>> >> > > > > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <
>> >> > > > shamirwasia@mapr.com
>> >> > > > > >:
>> >> > > > > > > > >
>> >> > > > > > > > >> Just sent an email on RCA of DRILL-6216 to discuss
>> next
>> >> > steps.
>> >> > > > > > > > >>
>> >> > > > > > > > >>
>> >> > > > > > > > >> Thanks,
>> >> > > > > > > > >> Sorabh
>> >> > > > > > > > >>
>> >> > > > > > > > >> ________________________________
>> >> > > > > > > > >> From: Parth Chandra <pa...@apache.org>
>> >> > > > > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
>> >> > > > > > > > >> To: dev
>> >> > > > > > > > >> Subject: Re: [DISCUSS] 1.13.0 release
>> >> > > > > > > > >>
>> >> > > > > > > > >> We have two items remaining -
>> >> > > > > > > > >>
>> >> > > > > > > > >> DRILL-1491 - Ideally, I would like to make sure that
>> >> CANCEL
>> >> > is
>> >> > > > > > handled
>> >> > > > > > > > >> correctly with JDK 8. If the failure of the unit test
>> is
>> >> > > because
>> >> > > > > the
>> >> > > > > > > > >> cancel
>> >> > > > > > > > >> is received after the query is completed, then the
>> issue
>> >> is
>> >> > > less
>> >> > > > > > > severe,
>> >> > > > > > > > >> but I would like to be sure that this is the case.
>> >> > > > > > > > >> Are there others who see the DrillbitResilience tests
>> >> > failing
>> >> > > > for
>> >> > > > > > > them?
>> >> > > > > > > > >> Can
>> >> > > > > > > > >> we try to assist Volodymyr? I don't see the failures
>> >> myself.
>> >> > > > > > > > >>
>> >> > > > > > > > >> DRILL-6216 - this is a showstopper.
>> >> > > > > > > > >>
>> >> > > > > > > > >>
>> >> > > > > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
>> >> > > > > kunalkhatua@gmail.com
>> >> > > > > > >
>> >> > > > > > > > >> wrote:
>> >> > > > > > > > >>
>> >> > > > > > > > >> > Hi Parth
>> >> > > > > > > > >> >
>> >> > > > > > > > >> > DRILL-6216 is a release blocker that is being
>> currently
>> >> > > looked
>> >> > > > > > into.
>> >> > > > > > > > >> >
>> >> > > > > > > > >> > Ref:
>> >> > > > > > > > >> > DRILL-6216: Metadata mismatch when connecting to a
>> >> Drill
>> >> > > > 1.12.0
>> >> > > > > > > with a
>> >> > > > > > > > >> > Drill-1.13.0-SNAPSHOT driver
>> >> > > > > > > > >> > https://urldefense.proofpoint.
>> >> > com/v2/url?u=https-3A__issues
>> >> > > .
>> >> > > > > > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=
>> >> > cskdkSMqhcnjZ
>> >> > > > > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOt
>> >> > hF2jlYxvhTlQg&m=xu
>> >> > > > > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz
>> >> > 3sxpP5-wHPs&s=txeKaKzF67flAi48
>> >> > > > > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
>> >> > > > > > > > >> >
>> >> > > > > > > > >> > Please add it to the list of required commits as
>> well.
>> >> > > > > > > > >> >
>> >> > > > > > > > >> > Thanks
>> >> > > > > > > > >> > ~ Kunal
>> >> > > > > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <
>> >> > > > vovatkach75@gmail.com>
>> >> > > > > > > > wrote:
>> >> > > > > > > > >> > Right now i haven't found the reason of
>> >> > > > > > > > >> > TestDrillbitResilience.cancelA
>> fterAllResultsProduced
>> >> > > failure,
>> >> > > > > > most
>> >> > > > > > > > >> likely
>> >> > > > > > > > >> > the cause of the failure is that the query is able
>> to
>> >> have
>> >> > > > been
>> >> > > > > > > > >> completed
>> >> > > > > > > > >> > before cancellation request is processed.
>> >> > > > > > > > >> > This test not only the case, there is one more
>> ignored
>> >> > test
>> >> > > > > > > > >> > TestDrillbitResilience.cancelA
>> >> fterEverythingIsCompleted
>> >> > and
>> >> > > > > jira
>> >> > > > > > > > >> > DRILL-3967
>> >> > > > > > > > >> > created, although the environment is AWS.
>> >> > > > > > > > >> >
>> >> > > > > > > > >> > Maybe it makes sense to ignore this test to unblock
>> the
>> >> > > > release
>> >> > > > > > and
>> >> > > > > > > > >> merge
>> >> > > > > > > > >> > JDK8 changes?
>> >> > > > > > > > >> >
>> >> > > > > > > > >>
>> >> > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >
>> >
>>
>
>
Re: [DISCUSS] 1.13.0 release
Posted by Jyothsna Reddy <jy...@gmail.com>.
Following branch has changed related to disabling doclint in pom.xml.
Please check it out.
https://github.com/dvjyothsna/drill.git DRILL-4547
On Thu, Mar 8, 2018 at 4:19 AM, Parth Chandra <pa...@apache.org> wrote:
> The issue with the failing test in TestDrillbitResilience.
> cancelAfterAllResultsProduced is similar to DRILL-3967 (
> TestDrillbitResilience.cancelAfterEverythingIsCompleted failure).
>
> In both this test case, and DRILL-3967, the query is paused (in different
> places) and a cancel is sent. The query is then resumed and the resulting
> state is checked. The problem is that the tests have a race condition
> between the cancellation and the resuming of the query. Sometimes the
> resume reaches first and sometimes the cancel reaches first. The failure
> described by Volodymyr is caused by this race condition. I don't know why
> the test was done like this, but this is an existing problem and shouldn't
> hold up the release.
>
> However, I also see a failure where we encounter an illegal state
> transition (the query state is CANCELLATION_REQUESTED and the Foreman tries
> to move to an ENQUEUED state). This happens once in about twenty-five
> executions. The Foreman only requests ENQUEUE once, when the query is about
> to start, so this means the cancellation request reached before the query
> start request. How this happens in the unit test I have not been able to
> determine yet (because it really shouldn't be possible).
>
> To recreate the problem I simply added a repeat rule in the class and set a
> repeat count of 1000. The problem occurs easily if the test is run from the
> command line. When running in debug, I was unable to see the problem.
>
> I'll spend some more time on this, but just in case someone wants to
> investigate further, feel free ...
>
>
> On Thu, Mar 8, 2018 at 10:59 AM, Parth Chandra <pa...@apache.org> wrote:
>
> > Not sure if that would work. The release build does not allow uncommitted
> > files, so I have to commit pom.xml changes to at least the local repo,
> > which will get pushed to my public repo when the release is done. Not
> > committing this to Apache master would be cheating would leave us with a
> > build that does not match any source in Apache master? Javadoc generated
> is
> > never committed to any repo. It is part of the src release jars, AFAIK.
> >
> > Also, I'm not sure where in the pom Jyothsna made the change; I added the
> > -Dxoclint:none to the build section of the apache-release profile and
> java
> > exec still gives over 100 javadoc errors.
> >
> > We have to fix these one of these days. Might as well do it now. Knowing
> > how it works, if we don't fix these now, someone will be scrambling to
> fix
> > these just before the 1.14.0 release :(
> >
> >
> >
> > On Thu, Mar 8, 2018 at 10:06 AM, Aman Sinha <am...@apache.org>
> wrote:
> >
> >> Parth, would it work if you made the pom.xml changes locally in your
> >> branch, generated the javadoc but only commit the javadoc jar files to
> the
> >> release branch, not the pom.xml changes ?
> >> Anyone downloading Drill source code to build should not run into this
> >> since typically they won't be building javadoc.
> >>
> >> -Aman
> >>
> >> On Wed, Mar 7, 2018 at 6:37 PM, Parth Chandra <pa...@apache.org>
> wrote:
> >>
> >> > Unfortunately, we cannot do that since we also want to be able to
> build
> >> > with JDK 7 for at least a couple of releases to allow for a reasonable
> >> > transition time. doclint was introduced in JDK 8 so JDK 7 fails
> >> because it
> >> > doesn't recognize the parameter.
> >> >
> >> >
> >> >
> >> > On Thu, Mar 8, 2018 at 7:03 AM, Jyothsna Reddy <
> jyothsna.dvj@gmail.com>
> >> > wrote:
> >> >
> >> > > Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491) and added
> >> > > following lines to pom.xml to disable doc lint. The javadoc doesn't
> >> throw
> >> > > any errors and the build is successful.
> >> > >
> >> > > <activation>
> >> > >
> >> > > <jdk>[1.8,)</jdk>
> >> > >
> >> > > </activation>
> >> > >
> >> > > <properties>
> >> > >
> >> > > <additionalparam>-Xdoclint:none</additionalparam>
> >> > >
> >> > > </properties>
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri <
> >> hanu.ncr@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > On my machine I couldn't repro the issue related to
> >> > > TestDrillbitResilience.
> >> > > > cancelAfterAllResultsProduced.
> >> > > > I used the vladimir's branch (i.e DRILL-1491).
> >> > > > Used the maven test command for testing it.
> >> > > >
> >> > > > output of the test run.
> >> > > > ... 4 common frames omitted
> >> > > > Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time elapsed:
> >> > 124.187
> >> > > > sec - in org.apache.drill.exec.server.TestDrillbitResilience
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <parthc@apache.org
> >
> >> > > wrote:
> >> > > >
> >> > > > > Yes I agree. JDBC would be a new feature that we can defer to
> >> 1.14.0.
> >> > > > > I'm hoping we can resolve the other three in the next few days.
> >> > Target
> >> > > > date
> >> > > > > for starting release process - Friday Mar 9th
> >> > > > >
> >> > > > > Once these are resolved, I will create a branch for the release
> so
> >> > that
> >> > > > > Apache master remains open for commits. If any issues are found
> in
> >> > the
> >> > > > > release branch, we will fix them in master and I will
> cherry-pick
> >> the
> >> > > > into
> >> > > > > the release branch. Once the release is finalized I will add a
> >> > release
> >> > > > tag
> >> > > > > and remove the branch.
> >> > > > >
> >> > > > > Also note if QA folks want to get started on testing the
> release,
> >> the
> >> > > > > current head of Apache master is close to final. Javadoc
> >> generation
> >> > is
> >> > > > only
> >> > > > > a release build issue, and the other issues are localized to
> >> > specific
> >> > > > > cases.
> >> > > > >
> >> > > > > Note: to reproduce the javadoc issues:
> >> > > > > # set JAVA_HOME to JDK 8
> >> > > > > mvn javadoc:javadoc -Papache-release
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <
> amansinha@apache.org
> >> >
> >> > > > wrote:
> >> > > > >
> >> > > > > > It seems to me the main blockers are:
> >> > > > > >
> >> > > > > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we split
> up
> >> the
> >> > > > work
> >> > > > > > among few people to resolve these ?
> >> > > > > > 2. DRILL-6216 Metadata mismatch.. <-- Agreement was
> >> to
> >> > > > revert
> >> > > > > > one small piece of code and it appears Sorabh is looking into
> it
> >> > > > > > 3. TestDrillbitResilience.cancelAfterAllResultsProduced <--
> >> need
> >> > > > > someone
> >> > > > > > to look into this
> >> > > > > >
> >> > > > > > Regarding the JDBC issues that Parth mentioned, looking at the
> >> > JIRAs,
> >> > > > it
> >> > > > > > seems they are not showstoppers...Parth do you agree ?
> >> > > > > >
> >> > > > > > Since we are close to the finish line for JDK 8, IMO we should
> >> try
> >> > > and
> >> > > > > see
> >> > > > > > if in another day or two we can get over these hurdles.
> >> > > > > >
> >> > > > > > -Aman
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <
> pmaker@mapr.com>
> >> > > wrote:
> >> > > > > >
> >> > > > > > > The JDK 8 issues will likely require more time to harden for
> >> it
> >> > to
> >> > > be
> >> > > > > > > included in the 1.13 release. My recommendation would be to
> >> move
> >> > > > ahead
> >> > > > > > with
> >> > > > > > > the 1.13 release now and address these issues right.
> >> > > > > > >
> >> > > > > > > Pritesh
> >> > > > > > >
> >> > > > > > > -----Original Message-----
> >> > > > > > > From: Parth Chandra <pa...@apache.org>
> >> > > > > > > Sent: March 7, 2018 3:34 AM
> >> > > > > > > To: dev <de...@drill.apache.org>
> >> > > > > > > Subject: Re: [DISCUSS] 1.13.0 release
> >> > > > > > >
> >> > > > > > > My mistake Volodymyr.
> >> > > > > > >
> >> > > > > > > Found some other JDK 8 issues in JIRA not tracked in
> >> DRILL-1491
> >> > > > > > >
> >> > > > > > > DRILL-4547 Javadoc fails with Java8
> >> > > > > > > DRILL-6163 Switch Travis To Java 8
> >> > > > > > >
> >> > > > > > > The following are tracked in DRILL-1491, but it doesn't look
> >> like
> >> > > > we're
> >> > > > > > > addressing these. Are we?
> >> > > > > > >
> >> > > > > > > DRILL-4329 13 Unit tests are failing with JDK 8
> >> > > > > > > DRILL-4333 DRILL-4329 tests in
> >> > > > > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
> >> > > > > > > DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
> >> > > > > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in Eclipse
> >> with
> >> > > > Java
> >> > > > > 8
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > *DRILL-4547 is a showstopper*. The release build
> >> > (-Papache-release)
> >> > > > > fails
> >> > > > > > > with far too many Javadoc errors even with doc lint turned
> >> off.
> >> > > > > > >
> >> > > > > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which
> is a
> >> > > > project
> >> > > > > by
> >> > > > > > > itself.
> >> > > > > > >
> >> > > > > > > Note that fixing JDBC related issues and adding the command
> >> line
> >> > > > option
> >> > > > > > to
> >> > > > > > > turn doc lint off will likely break Java 7 builds.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > Folks who voted to get JDK 8 into this release, what is the
> >> > > consensus
> >> > > > > on
> >> > > > > > > JDBC/Java8 ?
> >> > > > > > > Also, any volunteers on helping debug
> >> > > > > > > TestDrillbitResilience.cancelAfterAllResultsProduced
> >> > > > > > > ?
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <
> >> > > > vovatkach75@gmail.com
> >> > > > > >
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > Addition to my last message:
> >> > > > > > > > The link with PR for DRILL-1491
> >> https://urldefense.proofpoint
> >> > .
> >> > > > > > > com/v2/url?u=https-3A__github.com_apache_drill_pull_1143&d=
> >> > > DwIBaQ&c=
> >> > > > > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=
> >> > > oTnKwfjj5hFBosMrq_
> >> > > > > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3uialdRhgnf3XRY22R4SWXGZIq66a
> >> > > > > > Pijuy-Ms0J_-4&e=
> >> > > > > > > > on which the we can see TestDrillbitResilience.
> >> > > > > > > > cancelAfterAllResultsProduced
> >> > > > > > > > failure.
> >> > > > > > > >
> >> > > > > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <
> >> > > vovatkach75@gmail.com
> >> > > > >:
> >> > > > > > > >
> >> > > > > > > > > *To Parth:*
> >> > > > > > > > > The failure can only be seen if run on DRILL-1491
> branch,
> >> > > because
> >> > > > > it
> >> > > > > > > uses
> >> > > > > > > > > jdk 1.8 in pom.xml
> >> > > > > > > > >
> >> > > > > > > > > <source>1.8</source>
> >> > > > > > > > > <target>1.8</target>
> >> > > > > > > > >
> >> > > > > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <
> >> > > > shamirwasia@mapr.com
> >> > > > > >:
> >> > > > > > > > >
> >> > > > > > > > >> Just sent an email on RCA of DRILL-6216 to discuss next
> >> > steps.
> >> > > > > > > > >>
> >> > > > > > > > >>
> >> > > > > > > > >> Thanks,
> >> > > > > > > > >> Sorabh
> >> > > > > > > > >>
> >> > > > > > > > >> ________________________________
> >> > > > > > > > >> From: Parth Chandra <pa...@apache.org>
> >> > > > > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
> >> > > > > > > > >> To: dev
> >> > > > > > > > >> Subject: Re: [DISCUSS] 1.13.0 release
> >> > > > > > > > >>
> >> > > > > > > > >> We have two items remaining -
> >> > > > > > > > >>
> >> > > > > > > > >> DRILL-1491 - Ideally, I would like to make sure that
> >> CANCEL
> >> > is
> >> > > > > > handled
> >> > > > > > > > >> correctly with JDK 8. If the failure of the unit test
> is
> >> > > because
> >> > > > > the
> >> > > > > > > > >> cancel
> >> > > > > > > > >> is received after the query is completed, then the
> issue
> >> is
> >> > > less
> >> > > > > > > severe,
> >> > > > > > > > >> but I would like to be sure that this is the case.
> >> > > > > > > > >> Are there others who see the DrillbitResilience tests
> >> > failing
> >> > > > for
> >> > > > > > > them?
> >> > > > > > > > >> Can
> >> > > > > > > > >> we try to assist Volodymyr? I don't see the failures
> >> myself.
> >> > > > > > > > >>
> >> > > > > > > > >> DRILL-6216 - this is a showstopper.
> >> > > > > > > > >>
> >> > > > > > > > >>
> >> > > > > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
> >> > > > > kunalkhatua@gmail.com
> >> > > > > > >
> >> > > > > > > > >> wrote:
> >> > > > > > > > >>
> >> > > > > > > > >> > Hi Parth
> >> > > > > > > > >> >
> >> > > > > > > > >> > DRILL-6216 is a release blocker that is being
> currently
> >> > > looked
> >> > > > > > into.
> >> > > > > > > > >> >
> >> > > > > > > > >> > Ref:
> >> > > > > > > > >> > DRILL-6216: Metadata mismatch when connecting to a
> >> Drill
> >> > > > 1.12.0
> >> > > > > > > with a
> >> > > > > > > > >> > Drill-1.13.0-SNAPSHOT driver
> >> > > > > > > > >> > https://urldefense.proofpoint.
> >> > com/v2/url?u=https-3A__issues
> >> > > .
> >> > > > > > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=
> >> > cskdkSMqhcnjZ
> >> > > > > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOt
> >> > hF2jlYxvhTlQg&m=xu
> >> > > > > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz
> >> > 3sxpP5-wHPs&s=txeKaKzF67flAi48
> >> > > > > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> >> > > > > > > > >> >
> >> > > > > > > > >> > Please add it to the list of required commits as
> well.
> >> > > > > > > > >> >
> >> > > > > > > > >> > Thanks
> >> > > > > > > > >> > ~ Kunal
> >> > > > > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <
> >> > > > vovatkach75@gmail.com>
> >> > > > > > > > wrote:
> >> > > > > > > > >> > Right now i haven't found the reason of
> >> > > > > > > > >> > TestDrillbitResilience.cancelAfterAllResultsProduced
> >> > > failure,
> >> > > > > > most
> >> > > > > > > > >> likely
> >> > > > > > > > >> > the cause of the failure is that the query is able to
> >> have
> >> > > > been
> >> > > > > > > > >> completed
> >> > > > > > > > >> > before cancellation request is processed.
> >> > > > > > > > >> > This test not only the case, there is one more
> ignored
> >> > test
> >> > > > > > > > >> > TestDrillbitResilience.cancelA
> >> fterEverythingIsCompleted
> >> > and
> >> > > > > jira
> >> > > > > > > > >> > DRILL-3967
> >> > > > > > > > >> > created, although the environment is AWS.
> >> > > > > > > > >> >
> >> > > > > > > > >> > Maybe it makes sense to ignore this test to unblock
> the
> >> > > > release
> >> > > > > > and
> >> > > > > > > > >> merge
> >> > > > > > > > >> > JDK8 changes?
> >> > > > > > > > >> >
> >> > > > > > > > >>
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
The issue with the failing test in TestDrillbitResilience.
cancelAfterAllResultsProduced is similar to DRILL-3967 (
TestDrillbitResilience.cancelAfterEverythingIsCompleted failure).
In both this test case, and DRILL-3967, the query is paused (in different
places) and a cancel is sent. The query is then resumed and the resulting
state is checked. The problem is that the tests have a race condition
between the cancellation and the resuming of the query. Sometimes the
resume reaches first and sometimes the cancel reaches first. The failure
described by Volodymyr is caused by this race condition. I don't know why
the test was done like this, but this is an existing problem and shouldn't
hold up the release.
However, I also see a failure where we encounter an illegal state
transition (the query state is CANCELLATION_REQUESTED and the Foreman tries
to move to an ENQUEUED state). This happens once in about twenty-five
executions. The Foreman only requests ENQUEUE once, when the query is about
to start, so this means the cancellation request reached before the query
start request. How this happens in the unit test I have not been able to
determine yet (because it really shouldn't be possible).
To recreate the problem I simply added a repeat rule in the class and set a
repeat count of 1000. The problem occurs easily if the test is run from the
command line. When running in debug, I was unable to see the problem.
I'll spend some more time on this, but just in case someone wants to
investigate further, feel free ...
On Thu, Mar 8, 2018 at 10:59 AM, Parth Chandra <pa...@apache.org> wrote:
> Not sure if that would work. The release build does not allow uncommitted
> files, so I have to commit pom.xml changes to at least the local repo,
> which will get pushed to my public repo when the release is done. Not
> committing this to Apache master would be cheating would leave us with a
> build that does not match any source in Apache master? Javadoc generated is
> never committed to any repo. It is part of the src release jars, AFAIK.
>
> Also, I'm not sure where in the pom Jyothsna made the change; I added the
> -Dxoclint:none to the build section of the apache-release profile and java
> exec still gives over 100 javadoc errors.
>
> We have to fix these one of these days. Might as well do it now. Knowing
> how it works, if we don't fix these now, someone will be scrambling to fix
> these just before the 1.14.0 release :(
>
>
>
> On Thu, Mar 8, 2018 at 10:06 AM, Aman Sinha <am...@apache.org> wrote:
>
>> Parth, would it work if you made the pom.xml changes locally in your
>> branch, generated the javadoc but only commit the javadoc jar files to the
>> release branch, not the pom.xml changes ?
>> Anyone downloading Drill source code to build should not run into this
>> since typically they won't be building javadoc.
>>
>> -Aman
>>
>> On Wed, Mar 7, 2018 at 6:37 PM, Parth Chandra <pa...@apache.org> wrote:
>>
>> > Unfortunately, we cannot do that since we also want to be able to build
>> > with JDK 7 for at least a couple of releases to allow for a reasonable
>> > transition time. doclint was introduced in JDK 8 so JDK 7 fails
>> because it
>> > doesn't recognize the parameter.
>> >
>> >
>> >
>> > On Thu, Mar 8, 2018 at 7:03 AM, Jyothsna Reddy <jy...@gmail.com>
>> > wrote:
>> >
>> > > Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491) and added
>> > > following lines to pom.xml to disable doc lint. The javadoc doesn't
>> throw
>> > > any errors and the build is successful.
>> > >
>> > > <activation>
>> > >
>> > > <jdk>[1.8,)</jdk>
>> > >
>> > > </activation>
>> > >
>> > > <properties>
>> > >
>> > > <additionalparam>-Xdoclint:none</additionalparam>
>> > >
>> > > </properties>
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri <
>> hanu.ncr@gmail.com>
>> > > wrote:
>> > >
>> > > > On my machine I couldn't repro the issue related to
>> > > TestDrillbitResilience.
>> > > > cancelAfterAllResultsProduced.
>> > > > I used the vladimir's branch (i.e DRILL-1491).
>> > > > Used the maven test command for testing it.
>> > > >
>> > > > output of the test run.
>> > > > ... 4 common frames omitted
>> > > > Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time elapsed:
>> > 124.187
>> > > > sec - in org.apache.drill.exec.server.TestDrillbitResilience
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <pa...@apache.org>
>> > > wrote:
>> > > >
>> > > > > Yes I agree. JDBC would be a new feature that we can defer to
>> 1.14.0.
>> > > > > I'm hoping we can resolve the other three in the next few days.
>> > Target
>> > > > date
>> > > > > for starting release process - Friday Mar 9th
>> > > > >
>> > > > > Once these are resolved, I will create a branch for the release so
>> > that
>> > > > > Apache master remains open for commits. If any issues are found in
>> > the
>> > > > > release branch, we will fix them in master and I will cherry-pick
>> the
>> > > > into
>> > > > > the release branch. Once the release is finalized I will add a
>> > release
>> > > > tag
>> > > > > and remove the branch.
>> > > > >
>> > > > > Also note if QA folks want to get started on testing the release,
>> the
>> > > > > current head of Apache master is close to final. Javadoc
>> generation
>> > is
>> > > > only
>> > > > > a release build issue, and the other issues are localized to
>> > specific
>> > > > > cases.
>> > > > >
>> > > > > Note: to reproduce the javadoc issues:
>> > > > > # set JAVA_HOME to JDK 8
>> > > > > mvn javadoc:javadoc -Papache-release
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <amansinha@apache.org
>> >
>> > > > wrote:
>> > > > >
>> > > > > > It seems to me the main blockers are:
>> > > > > >
>> > > > > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we split up
>> the
>> > > > work
>> > > > > > among few people to resolve these ?
>> > > > > > 2. DRILL-6216 Metadata mismatch.. <-- Agreement was
>> to
>> > > > revert
>> > > > > > one small piece of code and it appears Sorabh is looking into it
>> > > > > > 3. TestDrillbitResilience.cancelAfterAllResultsProduced <--
>> need
>> > > > > someone
>> > > > > > to look into this
>> > > > > >
>> > > > > > Regarding the JDBC issues that Parth mentioned, looking at the
>> > JIRAs,
>> > > > it
>> > > > > > seems they are not showstoppers...Parth do you agree ?
>> > > > > >
>> > > > > > Since we are close to the finish line for JDK 8, IMO we should
>> try
>> > > and
>> > > > > see
>> > > > > > if in another day or two we can get over these hurdles.
>> > > > > >
>> > > > > > -Aman
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <pm...@mapr.com>
>> > > wrote:
>> > > > > >
>> > > > > > > The JDK 8 issues will likely require more time to harden for
>> it
>> > to
>> > > be
>> > > > > > > included in the 1.13 release. My recommendation would be to
>> move
>> > > > ahead
>> > > > > > with
>> > > > > > > the 1.13 release now and address these issues right.
>> > > > > > >
>> > > > > > > Pritesh
>> > > > > > >
>> > > > > > > -----Original Message-----
>> > > > > > > From: Parth Chandra <pa...@apache.org>
>> > > > > > > Sent: March 7, 2018 3:34 AM
>> > > > > > > To: dev <de...@drill.apache.org>
>> > > > > > > Subject: Re: [DISCUSS] 1.13.0 release
>> > > > > > >
>> > > > > > > My mistake Volodymyr.
>> > > > > > >
>> > > > > > > Found some other JDK 8 issues in JIRA not tracked in
>> DRILL-1491
>> > > > > > >
>> > > > > > > DRILL-4547 Javadoc fails with Java8
>> > > > > > > DRILL-6163 Switch Travis To Java 8
>> > > > > > >
>> > > > > > > The following are tracked in DRILL-1491, but it doesn't look
>> like
>> > > > we're
>> > > > > > > addressing these. Are we?
>> > > > > > >
>> > > > > > > DRILL-4329 13 Unit tests are failing with JDK 8
>> > > > > > > DRILL-4333 DRILL-4329 tests in
>> > > > > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
>> > > > > > > DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
>> > > > > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in Eclipse
>> with
>> > > > Java
>> > > > > 8
>> > > > > > >
>> > > > > > >
>> > > > > > > *DRILL-4547 is a showstopper*. The release build
>> > (-Papache-release)
>> > > > > fails
>> > > > > > > with far too many Javadoc errors even with doc lint turned
>> off.
>> > > > > > >
>> > > > > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which is a
>> > > > project
>> > > > > by
>> > > > > > > itself.
>> > > > > > >
>> > > > > > > Note that fixing JDBC related issues and adding the command
>> line
>> > > > option
>> > > > > > to
>> > > > > > > turn doc lint off will likely break Java 7 builds.
>> > > > > > >
>> > > > > > >
>> > > > > > > Folks who voted to get JDK 8 into this release, what is the
>> > > consensus
>> > > > > on
>> > > > > > > JDBC/Java8 ?
>> > > > > > > Also, any volunteers on helping debug
>> > > > > > > TestDrillbitResilience.cancelAfterAllResultsProduced
>> > > > > > > ?
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <
>> > > > vovatkach75@gmail.com
>> > > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Addition to my last message:
>> > > > > > > > The link with PR for DRILL-1491
>> https://urldefense.proofpoint
>> > .
>> > > > > > > com/v2/url?u=https-3A__github.com_apache_drill_pull_1143&d=
>> > > DwIBaQ&c=
>> > > > > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=
>> > > oTnKwfjj5hFBosMrq_
>> > > > > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3uialdRhgnf3XRY22R4SWXGZIq66a
>> > > > > > Pijuy-Ms0J_-4&e=
>> > > > > > > > on which the we can see TestDrillbitResilience.
>> > > > > > > > cancelAfterAllResultsProduced
>> > > > > > > > failure.
>> > > > > > > >
>> > > > > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <
>> > > vovatkach75@gmail.com
>> > > > >:
>> > > > > > > >
>> > > > > > > > > *To Parth:*
>> > > > > > > > > The failure can only be seen if run on DRILL-1491 branch,
>> > > because
>> > > > > it
>> > > > > > > uses
>> > > > > > > > > jdk 1.8 in pom.xml
>> > > > > > > > >
>> > > > > > > > > <source>1.8</source>
>> > > > > > > > > <target>1.8</target>
>> > > > > > > > >
>> > > > > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <
>> > > > shamirwasia@mapr.com
>> > > > > >:
>> > > > > > > > >
>> > > > > > > > >> Just sent an email on RCA of DRILL-6216 to discuss next
>> > steps.
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > > >> Thanks,
>> > > > > > > > >> Sorabh
>> > > > > > > > >>
>> > > > > > > > >> ________________________________
>> > > > > > > > >> From: Parth Chandra <pa...@apache.org>
>> > > > > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
>> > > > > > > > >> To: dev
>> > > > > > > > >> Subject: Re: [DISCUSS] 1.13.0 release
>> > > > > > > > >>
>> > > > > > > > >> We have two items remaining -
>> > > > > > > > >>
>> > > > > > > > >> DRILL-1491 - Ideally, I would like to make sure that
>> CANCEL
>> > is
>> > > > > > handled
>> > > > > > > > >> correctly with JDK 8. If the failure of the unit test is
>> > > because
>> > > > > the
>> > > > > > > > >> cancel
>> > > > > > > > >> is received after the query is completed, then the issue
>> is
>> > > less
>> > > > > > > severe,
>> > > > > > > > >> but I would like to be sure that this is the case.
>> > > > > > > > >> Are there others who see the DrillbitResilience tests
>> > failing
>> > > > for
>> > > > > > > them?
>> > > > > > > > >> Can
>> > > > > > > > >> we try to assist Volodymyr? I don't see the failures
>> myself.
>> > > > > > > > >>
>> > > > > > > > >> DRILL-6216 - this is a showstopper.
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
>> > > > > kunalkhatua@gmail.com
>> > > > > > >
>> > > > > > > > >> wrote:
>> > > > > > > > >>
>> > > > > > > > >> > Hi Parth
>> > > > > > > > >> >
>> > > > > > > > >> > DRILL-6216 is a release blocker that is being currently
>> > > looked
>> > > > > > into.
>> > > > > > > > >> >
>> > > > > > > > >> > Ref:
>> > > > > > > > >> > DRILL-6216: Metadata mismatch when connecting to a
>> Drill
>> > > > 1.12.0
>> > > > > > > with a
>> > > > > > > > >> > Drill-1.13.0-SNAPSHOT driver
>> > > > > > > > >> > https://urldefense.proofpoint.
>> > com/v2/url?u=https-3A__issues
>> > > .
>> > > > > > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=
>> > cskdkSMqhcnjZ
>> > > > > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOt
>> > hF2jlYxvhTlQg&m=xu
>> > > > > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz
>> > 3sxpP5-wHPs&s=txeKaKzF67flAi48
>> > > > > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
>> > > > > > > > >> >
>> > > > > > > > >> > Please add it to the list of required commits as well.
>> > > > > > > > >> >
>> > > > > > > > >> > Thanks
>> > > > > > > > >> > ~ Kunal
>> > > > > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <
>> > > > vovatkach75@gmail.com>
>> > > > > > > > wrote:
>> > > > > > > > >> > Right now i haven't found the reason of
>> > > > > > > > >> > TestDrillbitResilience.cancelAfterAllResultsProduced
>> > > failure,
>> > > > > > most
>> > > > > > > > >> likely
>> > > > > > > > >> > the cause of the failure is that the query is able to
>> have
>> > > > been
>> > > > > > > > >> completed
>> > > > > > > > >> > before cancellation request is processed.
>> > > > > > > > >> > This test not only the case, there is one more ignored
>> > test
>> > > > > > > > >> > TestDrillbitResilience.cancelA
>> fterEverythingIsCompleted
>> > and
>> > > > > jira
>> > > > > > > > >> > DRILL-3967
>> > > > > > > > >> > created, although the environment is AWS.
>> > > > > > > > >> >
>> > > > > > > > >> > Maybe it makes sense to ignore this test to unblock the
>> > > > release
>> > > > > > and
>> > > > > > > > >> merge
>> > > > > > > > >> > JDK8 changes?
>> > > > > > > > >> >
>> > > > > > > > >>
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
Not sure if that would work. The release build does not allow uncommitted
files, so I have to commit pom.xml changes to at least the local repo,
which will get pushed to my public repo when the release is done. Not
committing this to Apache master would be cheating would leave us with a
build that does not match any source in Apache master? Javadoc generated is
never committed to any repo. It is part of the src release jars, AFAIK.
Also, I'm not sure where in the pom Jyothsna made the change; I added the
-Dxoclint:none to the build section of the apache-release profile and java
exec still gives over 100 javadoc errors.
We have to fix these one of these days. Might as well do it now. Knowing
how it works, if we don't fix these now, someone will be scrambling to fix
these just before the 1.14.0 release :(
On Thu, Mar 8, 2018 at 10:06 AM, Aman Sinha <am...@apache.org> wrote:
> Parth, would it work if you made the pom.xml changes locally in your
> branch, generated the javadoc but only commit the javadoc jar files to the
> release branch, not the pom.xml changes ?
> Anyone downloading Drill source code to build should not run into this
> since typically they won't be building javadoc.
>
> -Aman
>
> On Wed, Mar 7, 2018 at 6:37 PM, Parth Chandra <pa...@apache.org> wrote:
>
> > Unfortunately, we cannot do that since we also want to be able to build
> > with JDK 7 for at least a couple of releases to allow for a reasonable
> > transition time. doclint was introduced in JDK 8 so JDK 7 fails because
> it
> > doesn't recognize the parameter.
> >
> >
> >
> > On Thu, Mar 8, 2018 at 7:03 AM, Jyothsna Reddy <jy...@gmail.com>
> > wrote:
> >
> > > Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491) and added
> > > following lines to pom.xml to disable doc lint. The javadoc doesn't
> throw
> > > any errors and the build is successful.
> > >
> > > <activation>
> > >
> > > <jdk>[1.8,)</jdk>
> > >
> > > </activation>
> > >
> > > <properties>
> > >
> > > <additionalparam>-Xdoclint:none</additionalparam>
> > >
> > > </properties>
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri <
> hanu.ncr@gmail.com>
> > > wrote:
> > >
> > > > On my machine I couldn't repro the issue related to
> > > TestDrillbitResilience.
> > > > cancelAfterAllResultsProduced.
> > > > I used the vladimir's branch (i.e DRILL-1491).
> > > > Used the maven test command for testing it.
> > > >
> > > > output of the test run.
> > > > ... 4 common frames omitted
> > > > Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time elapsed:
> > 124.187
> > > > sec - in org.apache.drill.exec.server.TestDrillbitResilience
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <pa...@apache.org>
> > > wrote:
> > > >
> > > > > Yes I agree. JDBC would be a new feature that we can defer to
> 1.14.0.
> > > > > I'm hoping we can resolve the other three in the next few days.
> > Target
> > > > date
> > > > > for starting release process - Friday Mar 9th
> > > > >
> > > > > Once these are resolved, I will create a branch for the release so
> > that
> > > > > Apache master remains open for commits. If any issues are found in
> > the
> > > > > release branch, we will fix them in master and I will cherry-pick
> the
> > > > into
> > > > > the release branch. Once the release is finalized I will add a
> > release
> > > > tag
> > > > > and remove the branch.
> > > > >
> > > > > Also note if QA folks want to get started on testing the release,
> the
> > > > > current head of Apache master is close to final. Javadoc generation
> > is
> > > > only
> > > > > a release build issue, and the other issues are localized to
> > specific
> > > > > cases.
> > > > >
> > > > > Note: to reproduce the javadoc issues:
> > > > > # set JAVA_HOME to JDK 8
> > > > > mvn javadoc:javadoc -Papache-release
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <am...@apache.org>
> > > > wrote:
> > > > >
> > > > > > It seems to me the main blockers are:
> > > > > >
> > > > > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we split up
> the
> > > > work
> > > > > > among few people to resolve these ?
> > > > > > 2. DRILL-6216 Metadata mismatch.. <-- Agreement was to
> > > > revert
> > > > > > one small piece of code and it appears Sorabh is looking into it
> > > > > > 3. TestDrillbitResilience.cancelAfterAllResultsProduced <--
> need
> > > > > someone
> > > > > > to look into this
> > > > > >
> > > > > > Regarding the JDBC issues that Parth mentioned, looking at the
> > JIRAs,
> > > > it
> > > > > > seems they are not showstoppers...Parth do you agree ?
> > > > > >
> > > > > > Since we are close to the finish line for JDK 8, IMO we should
> try
> > > and
> > > > > see
> > > > > > if in another day or two we can get over these hurdles.
> > > > > >
> > > > > > -Aman
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <pm...@mapr.com>
> > > wrote:
> > > > > >
> > > > > > > The JDK 8 issues will likely require more time to harden for it
> > to
> > > be
> > > > > > > included in the 1.13 release. My recommendation would be to
> move
> > > > ahead
> > > > > > with
> > > > > > > the 1.13 release now and address these issues right.
> > > > > > >
> > > > > > > Pritesh
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Parth Chandra <pa...@apache.org>
> > > > > > > Sent: March 7, 2018 3:34 AM
> > > > > > > To: dev <de...@drill.apache.org>
> > > > > > > Subject: Re: [DISCUSS] 1.13.0 release
> > > > > > >
> > > > > > > My mistake Volodymyr.
> > > > > > >
> > > > > > > Found some other JDK 8 issues in JIRA not tracked in DRILL-1491
> > > > > > >
> > > > > > > DRILL-4547 Javadoc fails with Java8
> > > > > > > DRILL-6163 Switch Travis To Java 8
> > > > > > >
> > > > > > > The following are tracked in DRILL-1491, but it doesn't look
> like
> > > > we're
> > > > > > > addressing these. Are we?
> > > > > > >
> > > > > > > DRILL-4329 13 Unit tests are failing with JDK 8
> > > > > > > DRILL-4333 DRILL-4329 tests in
> > > > > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
> > > > > > > DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
> > > > > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in Eclipse
> with
> > > > Java
> > > > > 8
> > > > > > >
> > > > > > >
> > > > > > > *DRILL-4547 is a showstopper*. The release build
> > (-Papache-release)
> > > > > fails
> > > > > > > with far too many Javadoc errors even with doc lint turned off.
> > > > > > >
> > > > > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which is a
> > > > project
> > > > > by
> > > > > > > itself.
> > > > > > >
> > > > > > > Note that fixing JDBC related issues and adding the command
> line
> > > > option
> > > > > > to
> > > > > > > turn doc lint off will likely break Java 7 builds.
> > > > > > >
> > > > > > >
> > > > > > > Folks who voted to get JDK 8 into this release, what is the
> > > consensus
> > > > > on
> > > > > > > JDBC/Java8 ?
> > > > > > > Also, any volunteers on helping debug
> > > > > > > TestDrillbitResilience.cancelAfterAllResultsProduced
> > > > > > > ?
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <
> > > > vovatkach75@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Addition to my last message:
> > > > > > > > The link with PR for DRILL-1491
> https://urldefense.proofpoint
> > .
> > > > > > > com/v2/url?u=https-3A__github.com_apache_drill_pull_1143&d=
> > > DwIBaQ&c=
> > > > > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=
> > > oTnKwfjj5hFBosMrq_
> > > > > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3uialdRhgnf3XRY22R4SWXGZIq66a
> > > > > > Pijuy-Ms0J_-4&e=
> > > > > > > > on which the we can see TestDrillbitResilience.
> > > > > > > > cancelAfterAllResultsProduced
> > > > > > > > failure.
> > > > > > > >
> > > > > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <
> > > vovatkach75@gmail.com
> > > > >:
> > > > > > > >
> > > > > > > > > *To Parth:*
> > > > > > > > > The failure can only be seen if run on DRILL-1491 branch,
> > > because
> > > > > it
> > > > > > > uses
> > > > > > > > > jdk 1.8 in pom.xml
> > > > > > > > >
> > > > > > > > > <source>1.8</source>
> > > > > > > > > <target>1.8</target>
> > > > > > > > >
> > > > > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <
> > > > shamirwasia@mapr.com
> > > > > >:
> > > > > > > > >
> > > > > > > > >> Just sent an email on RCA of DRILL-6216 to discuss next
> > steps.
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> Thanks,
> > > > > > > > >> Sorabh
> > > > > > > > >>
> > > > > > > > >> ________________________________
> > > > > > > > >> From: Parth Chandra <pa...@apache.org>
> > > > > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
> > > > > > > > >> To: dev
> > > > > > > > >> Subject: Re: [DISCUSS] 1.13.0 release
> > > > > > > > >>
> > > > > > > > >> We have two items remaining -
> > > > > > > > >>
> > > > > > > > >> DRILL-1491 - Ideally, I would like to make sure that
> CANCEL
> > is
> > > > > > handled
> > > > > > > > >> correctly with JDK 8. If the failure of the unit test is
> > > because
> > > > > the
> > > > > > > > >> cancel
> > > > > > > > >> is received after the query is completed, then the issue
> is
> > > less
> > > > > > > severe,
> > > > > > > > >> but I would like to be sure that this is the case.
> > > > > > > > >> Are there others who see the DrillbitResilience tests
> > failing
> > > > for
> > > > > > > them?
> > > > > > > > >> Can
> > > > > > > > >> we try to assist Volodymyr? I don't see the failures
> myself.
> > > > > > > > >>
> > > > > > > > >> DRILL-6216 - this is a showstopper.
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
> > > > > kunalkhatua@gmail.com
> > > > > > >
> > > > > > > > >> wrote:
> > > > > > > > >>
> > > > > > > > >> > Hi Parth
> > > > > > > > >> >
> > > > > > > > >> > DRILL-6216 is a release blocker that is being currently
> > > looked
> > > > > > into.
> > > > > > > > >> >
> > > > > > > > >> > Ref:
> > > > > > > > >> > DRILL-6216: Metadata mismatch when connecting to a Drill
> > > > 1.12.0
> > > > > > > with a
> > > > > > > > >> > Drill-1.13.0-SNAPSHOT driver
> > > > > > > > >> > https://urldefense.proofpoint.
> > com/v2/url?u=https-3A__issues
> > > .
> > > > > > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=
> > cskdkSMqhcnjZ
> > > > > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOt
> > hF2jlYxvhTlQg&m=xu
> > > > > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz
> > 3sxpP5-wHPs&s=txeKaKzF67flAi48
> > > > > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> > > > > > > > >> >
> > > > > > > > >> > Please add it to the list of required commits as well.
> > > > > > > > >> >
> > > > > > > > >> > Thanks
> > > > > > > > >> > ~ Kunal
> > > > > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <
> > > > vovatkach75@gmail.com>
> > > > > > > > wrote:
> > > > > > > > >> > Right now i haven't found the reason of
> > > > > > > > >> > TestDrillbitResilience.cancelAfterAllResultsProduced
> > > failure,
> > > > > > most
> > > > > > > > >> likely
> > > > > > > > >> > the cause of the failure is that the query is able to
> have
> > > > been
> > > > > > > > >> completed
> > > > > > > > >> > before cancellation request is processed.
> > > > > > > > >> > This test not only the case, there is one more ignored
> > test
> > > > > > > > >> > TestDrillbitResilience.cancelAfterEverythingIsCompleted
> > and
> > > > > jira
> > > > > > > > >> > DRILL-3967
> > > > > > > > >> > created, although the environment is AWS.
> > > > > > > > >> >
> > > > > > > > >> > Maybe it makes sense to ignore this test to unblock the
> > > > release
> > > > > > and
> > > > > > > > >> merge
> > > > > > > > >> > JDK8 changes?
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Aman Sinha <am...@apache.org>.
Parth, would it work if you made the pom.xml changes locally in your
branch, generated the javadoc but only commit the javadoc jar files to the
release branch, not the pom.xml changes ?
Anyone downloading Drill source code to build should not run into this
since typically they won't be building javadoc.
-Aman
On Wed, Mar 7, 2018 at 6:37 PM, Parth Chandra <pa...@apache.org> wrote:
> Unfortunately, we cannot do that since we also want to be able to build
> with JDK 7 for at least a couple of releases to allow for a reasonable
> transition time. doclint was introduced in JDK 8 so JDK 7 fails because it
> doesn't recognize the parameter.
>
>
>
> On Thu, Mar 8, 2018 at 7:03 AM, Jyothsna Reddy <jy...@gmail.com>
> wrote:
>
> > Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491) and added
> > following lines to pom.xml to disable doc lint. The javadoc doesn't throw
> > any errors and the build is successful.
> >
> > <activation>
> >
> > <jdk>[1.8,)</jdk>
> >
> > </activation>
> >
> > <properties>
> >
> > <additionalparam>-Xdoclint:none</additionalparam>
> >
> > </properties>
> >
> >
> >
> >
> >
> > On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri <ha...@gmail.com>
> > wrote:
> >
> > > On my machine I couldn't repro the issue related to
> > TestDrillbitResilience.
> > > cancelAfterAllResultsProduced.
> > > I used the vladimir's branch (i.e DRILL-1491).
> > > Used the maven test command for testing it.
> > >
> > > output of the test run.
> > > ... 4 common frames omitted
> > > Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time elapsed:
> 124.187
> > > sec - in org.apache.drill.exec.server.TestDrillbitResilience
> > >
> > >
> > >
> > >
> > > On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <pa...@apache.org>
> > wrote:
> > >
> > > > Yes I agree. JDBC would be a new feature that we can defer to 1.14.0.
> > > > I'm hoping we can resolve the other three in the next few days.
> Target
> > > date
> > > > for starting release process - Friday Mar 9th
> > > >
> > > > Once these are resolved, I will create a branch for the release so
> that
> > > > Apache master remains open for commits. If any issues are found in
> the
> > > > release branch, we will fix them in master and I will cherry-pick the
> > > into
> > > > the release branch. Once the release is finalized I will add a
> release
> > > tag
> > > > and remove the branch.
> > > >
> > > > Also note if QA folks want to get started on testing the release, the
> > > > current head of Apache master is close to final. Javadoc generation
> is
> > > only
> > > > a release build issue, and the other issues are localized to
> specific
> > > > cases.
> > > >
> > > > Note: to reproduce the javadoc issues:
> > > > # set JAVA_HOME to JDK 8
> > > > mvn javadoc:javadoc -Papache-release
> > > >
> > > >
> > > >
> > > > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <am...@apache.org>
> > > wrote:
> > > >
> > > > > It seems to me the main blockers are:
> > > > >
> > > > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we split up the
> > > work
> > > > > among few people to resolve these ?
> > > > > 2. DRILL-6216 Metadata mismatch.. <-- Agreement was to
> > > revert
> > > > > one small piece of code and it appears Sorabh is looking into it
> > > > > 3. TestDrillbitResilience.cancelAfterAllResultsProduced <-- need
> > > > someone
> > > > > to look into this
> > > > >
> > > > > Regarding the JDBC issues that Parth mentioned, looking at the
> JIRAs,
> > > it
> > > > > seems they are not showstoppers...Parth do you agree ?
> > > > >
> > > > > Since we are close to the finish line for JDK 8, IMO we should try
> > and
> > > > see
> > > > > if in another day or two we can get over these hurdles.
> > > > >
> > > > > -Aman
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <pm...@mapr.com>
> > wrote:
> > > > >
> > > > > > The JDK 8 issues will likely require more time to harden for it
> to
> > be
> > > > > > included in the 1.13 release. My recommendation would be to move
> > > ahead
> > > > > with
> > > > > > the 1.13 release now and address these issues right.
> > > > > >
> > > > > > Pritesh
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Parth Chandra <pa...@apache.org>
> > > > > > Sent: March 7, 2018 3:34 AM
> > > > > > To: dev <de...@drill.apache.org>
> > > > > > Subject: Re: [DISCUSS] 1.13.0 release
> > > > > >
> > > > > > My mistake Volodymyr.
> > > > > >
> > > > > > Found some other JDK 8 issues in JIRA not tracked in DRILL-1491
> > > > > >
> > > > > > DRILL-4547 Javadoc fails with Java8
> > > > > > DRILL-6163 Switch Travis To Java 8
> > > > > >
> > > > > > The following are tracked in DRILL-1491, but it doesn't look like
> > > we're
> > > > > > addressing these. Are we?
> > > > > >
> > > > > > DRILL-4329 13 Unit tests are failing with JDK 8
> > > > > > DRILL-4333 DRILL-4329 tests in
> > > > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
> > > > > > DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
> > > > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in Eclipse with
> > > Java
> > > > 8
> > > > > >
> > > > > >
> > > > > > *DRILL-4547 is a showstopper*. The release build
> (-Papache-release)
> > > > fails
> > > > > > with far too many Javadoc errors even with doc lint turned off.
> > > > > >
> > > > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which is a
> > > project
> > > > by
> > > > > > itself.
> > > > > >
> > > > > > Note that fixing JDBC related issues and adding the command line
> > > option
> > > > > to
> > > > > > turn doc lint off will likely break Java 7 builds.
> > > > > >
> > > > > >
> > > > > > Folks who voted to get JDK 8 into this release, what is the
> > consensus
> > > > on
> > > > > > JDBC/Java8 ?
> > > > > > Also, any volunteers on helping debug
> > > > > > TestDrillbitResilience.cancelAfterAllResultsProduced
> > > > > > ?
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <
> > > vovatkach75@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Addition to my last message:
> > > > > > > The link with PR for DRILL-1491 https://urldefense.proofpoint
> .
> > > > > > com/v2/url?u=https-3A__github.com_apache_drill_pull_1143&d=
> > DwIBaQ&c=
> > > > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=
> > oTnKwfjj5hFBosMrq_
> > > > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3uialdRhgnf3XRY22R4SWXGZIq66a
> > > > > Pijuy-Ms0J_-4&e=
> > > > > > > on which the we can see TestDrillbitResilience.
> > > > > > > cancelAfterAllResultsProduced
> > > > > > > failure.
> > > > > > >
> > > > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <
> > vovatkach75@gmail.com
> > > >:
> > > > > > >
> > > > > > > > *To Parth:*
> > > > > > > > The failure can only be seen if run on DRILL-1491 branch,
> > because
> > > > it
> > > > > > uses
> > > > > > > > jdk 1.8 in pom.xml
> > > > > > > >
> > > > > > > > <source>1.8</source>
> > > > > > > > <target>1.8</target>
> > > > > > > >
> > > > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <
> > > shamirwasia@mapr.com
> > > > >:
> > > > > > > >
> > > > > > > >> Just sent an email on RCA of DRILL-6216 to discuss next
> steps.
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Sorabh
> > > > > > > >>
> > > > > > > >> ________________________________
> > > > > > > >> From: Parth Chandra <pa...@apache.org>
> > > > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
> > > > > > > >> To: dev
> > > > > > > >> Subject: Re: [DISCUSS] 1.13.0 release
> > > > > > > >>
> > > > > > > >> We have two items remaining -
> > > > > > > >>
> > > > > > > >> DRILL-1491 - Ideally, I would like to make sure that CANCEL
> is
> > > > > handled
> > > > > > > >> correctly with JDK 8. If the failure of the unit test is
> > because
> > > > the
> > > > > > > >> cancel
> > > > > > > >> is received after the query is completed, then the issue is
> > less
> > > > > > severe,
> > > > > > > >> but I would like to be sure that this is the case.
> > > > > > > >> Are there others who see the DrillbitResilience tests
> failing
> > > for
> > > > > > them?
> > > > > > > >> Can
> > > > > > > >> we try to assist Volodymyr? I don't see the failures myself.
> > > > > > > >>
> > > > > > > >> DRILL-6216 - this is a showstopper.
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
> > > > kunalkhatua@gmail.com
> > > > > >
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >> > Hi Parth
> > > > > > > >> >
> > > > > > > >> > DRILL-6216 is a release blocker that is being currently
> > looked
> > > > > into.
> > > > > > > >> >
> > > > > > > >> > Ref:
> > > > > > > >> > DRILL-6216: Metadata mismatch when connecting to a Drill
> > > 1.12.0
> > > > > > with a
> > > > > > > >> > Drill-1.13.0-SNAPSHOT driver
> > > > > > > >> > https://urldefense.proofpoint.
> com/v2/url?u=https-3A__issues
> > .
> > > > > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=
> cskdkSMqhcnjZ
> > > > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOt
> hF2jlYxvhTlQg&m=xu
> > > > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz
> 3sxpP5-wHPs&s=txeKaKzF67flAi48
> > > > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> > > > > > > >> >
> > > > > > > >> > Please add it to the list of required commits as well.
> > > > > > > >> >
> > > > > > > >> > Thanks
> > > > > > > >> > ~ Kunal
> > > > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <
> > > vovatkach75@gmail.com>
> > > > > > > wrote:
> > > > > > > >> > Right now i haven't found the reason of
> > > > > > > >> > TestDrillbitResilience.cancelAfterAllResultsProduced
> > failure,
> > > > > most
> > > > > > > >> likely
> > > > > > > >> > the cause of the failure is that the query is able to have
> > > been
> > > > > > > >> completed
> > > > > > > >> > before cancellation request is processed.
> > > > > > > >> > This test not only the case, there is one more ignored
> test
> > > > > > > >> > TestDrillbitResilience.cancelAfterEverythingIsCompleted
> and
> > > > jira
> > > > > > > >> > DRILL-3967
> > > > > > > >> > created, although the environment is AWS.
> > > > > > > >> >
> > > > > > > >> > Maybe it makes sense to ignore this test to unblock the
> > > release
> > > > > and
> > > > > > > >> merge
> > > > > > > >> > JDK8 changes?
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
Unfortunately, we cannot do that since we also want to be able to build
with JDK 7 for at least a couple of releases to allow for a reasonable
transition time. doclint was introduced in JDK 8 so JDK 7 fails because it
doesn't recognize the parameter.
On Thu, Mar 8, 2018 at 7:03 AM, Jyothsna Reddy <jy...@gmail.com>
wrote:
> Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491) and added
> following lines to pom.xml to disable doc lint. The javadoc doesn't throw
> any errors and the build is successful.
>
> <activation>
>
> <jdk>[1.8,)</jdk>
>
> </activation>
>
> <properties>
>
> <additionalparam>-Xdoclint:none</additionalparam>
>
> </properties>
>
>
>
>
>
> On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri <ha...@gmail.com>
> wrote:
>
> > On my machine I couldn't repro the issue related to
> TestDrillbitResilience.
> > cancelAfterAllResultsProduced.
> > I used the vladimir's branch (i.e DRILL-1491).
> > Used the maven test command for testing it.
> >
> > output of the test run.
> > ... 4 common frames omitted
> > Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 124.187
> > sec - in org.apache.drill.exec.server.TestDrillbitResilience
> >
> >
> >
> >
> > On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <pa...@apache.org>
> wrote:
> >
> > > Yes I agree. JDBC would be a new feature that we can defer to 1.14.0.
> > > I'm hoping we can resolve the other three in the next few days. Target
> > date
> > > for starting release process - Friday Mar 9th
> > >
> > > Once these are resolved, I will create a branch for the release so that
> > > Apache master remains open for commits. If any issues are found in the
> > > release branch, we will fix them in master and I will cherry-pick the
> > into
> > > the release branch. Once the release is finalized I will add a release
> > tag
> > > and remove the branch.
> > >
> > > Also note if QA folks want to get started on testing the release, the
> > > current head of Apache master is close to final. Javadoc generation is
> > only
> > > a release build issue, and the other issues are localized to specific
> > > cases.
> > >
> > > Note: to reproduce the javadoc issues:
> > > # set JAVA_HOME to JDK 8
> > > mvn javadoc:javadoc -Papache-release
> > >
> > >
> > >
> > > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <am...@apache.org>
> > wrote:
> > >
> > > > It seems to me the main blockers are:
> > > >
> > > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we split up the
> > work
> > > > among few people to resolve these ?
> > > > 2. DRILL-6216 Metadata mismatch.. <-- Agreement was to
> > revert
> > > > one small piece of code and it appears Sorabh is looking into it
> > > > 3. TestDrillbitResilience.cancelAfterAllResultsProduced <-- need
> > > someone
> > > > to look into this
> > > >
> > > > Regarding the JDBC issues that Parth mentioned, looking at the JIRAs,
> > it
> > > > seems they are not showstoppers...Parth do you agree ?
> > > >
> > > > Since we are close to the finish line for JDK 8, IMO we should try
> and
> > > see
> > > > if in another day or two we can get over these hurdles.
> > > >
> > > > -Aman
> > > >
> > > >
> > > >
> > > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <pm...@mapr.com>
> wrote:
> > > >
> > > > > The JDK 8 issues will likely require more time to harden for it to
> be
> > > > > included in the 1.13 release. My recommendation would be to move
> > ahead
> > > > with
> > > > > the 1.13 release now and address these issues right.
> > > > >
> > > > > Pritesh
> > > > >
> > > > > -----Original Message-----
> > > > > From: Parth Chandra <pa...@apache.org>
> > > > > Sent: March 7, 2018 3:34 AM
> > > > > To: dev <de...@drill.apache.org>
> > > > > Subject: Re: [DISCUSS] 1.13.0 release
> > > > >
> > > > > My mistake Volodymyr.
> > > > >
> > > > > Found some other JDK 8 issues in JIRA not tracked in DRILL-1491
> > > > >
> > > > > DRILL-4547 Javadoc fails with Java8
> > > > > DRILL-6163 Switch Travis To Java 8
> > > > >
> > > > > The following are tracked in DRILL-1491, but it doesn't look like
> > we're
> > > > > addressing these. Are we?
> > > > >
> > > > > DRILL-4329 13 Unit tests are failing with JDK 8
> > > > > DRILL-4333 DRILL-4329 tests in
> > > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
> > > > > DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
> > > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in Eclipse with
> > Java
> > > 8
> > > > >
> > > > >
> > > > > *DRILL-4547 is a showstopper*. The release build (-Papache-release)
> > > fails
> > > > > with far too many Javadoc errors even with doc lint turned off.
> > > > >
> > > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which is a
> > project
> > > by
> > > > > itself.
> > > > >
> > > > > Note that fixing JDBC related issues and adding the command line
> > option
> > > > to
> > > > > turn doc lint off will likely break Java 7 builds.
> > > > >
> > > > >
> > > > > Folks who voted to get JDK 8 into this release, what is the
> consensus
> > > on
> > > > > JDBC/Java8 ?
> > > > > Also, any volunteers on helping debug
> > > > > TestDrillbitResilience.cancelAfterAllResultsProduced
> > > > > ?
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <
> > vovatkach75@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Addition to my last message:
> > > > > > The link with PR for DRILL-1491 https://urldefense.proofpoint.
> > > > > com/v2/url?u=https-3A__github.com_apache_drill_pull_1143&d=
> DwIBaQ&c=
> > > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=
> oTnKwfjj5hFBosMrq_
> > > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3uialdRhgnf3XRY22R4SWXGZIq66a
> > > > Pijuy-Ms0J_-4&e=
> > > > > > on which the we can see TestDrillbitResilience.
> > > > > > cancelAfterAllResultsProduced
> > > > > > failure.
> > > > > >
> > > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <
> vovatkach75@gmail.com
> > >:
> > > > > >
> > > > > > > *To Parth:*
> > > > > > > The failure can only be seen if run on DRILL-1491 branch,
> because
> > > it
> > > > > uses
> > > > > > > jdk 1.8 in pom.xml
> > > > > > >
> > > > > > > <source>1.8</source>
> > > > > > > <target>1.8</target>
> > > > > > >
> > > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <
> > shamirwasia@mapr.com
> > > >:
> > > > > > >
> > > > > > >> Just sent an email on RCA of DRILL-6216 to discuss next steps.
> > > > > > >>
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Sorabh
> > > > > > >>
> > > > > > >> ________________________________
> > > > > > >> From: Parth Chandra <pa...@apache.org>
> > > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
> > > > > > >> To: dev
> > > > > > >> Subject: Re: [DISCUSS] 1.13.0 release
> > > > > > >>
> > > > > > >> We have two items remaining -
> > > > > > >>
> > > > > > >> DRILL-1491 - Ideally, I would like to make sure that CANCEL is
> > > > handled
> > > > > > >> correctly with JDK 8. If the failure of the unit test is
> because
> > > the
> > > > > > >> cancel
> > > > > > >> is received after the query is completed, then the issue is
> less
> > > > > severe,
> > > > > > >> but I would like to be sure that this is the case.
> > > > > > >> Are there others who see the DrillbitResilience tests failing
> > for
> > > > > them?
> > > > > > >> Can
> > > > > > >> we try to assist Volodymyr? I don't see the failures myself.
> > > > > > >>
> > > > > > >> DRILL-6216 - this is a showstopper.
> > > > > > >>
> > > > > > >>
> > > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
> > > kunalkhatua@gmail.com
> > > > >
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Hi Parth
> > > > > > >> >
> > > > > > >> > DRILL-6216 is a release blocker that is being currently
> looked
> > > > into.
> > > > > > >> >
> > > > > > >> > Ref:
> > > > > > >> > DRILL-6216: Metadata mismatch when connecting to a Drill
> > 1.12.0
> > > > > with a
> > > > > > >> > Drill-1.13.0-SNAPSHOT driver
> > > > > > >> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues
> .
> > > > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=cskdkSMqhcnjZ
> > > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=xu
> > > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz3sxpP5-wHPs&s=txeKaKzF67flAi48
> > > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> > > > > > >> >
> > > > > > >> > Please add it to the list of required commits as well.
> > > > > > >> >
> > > > > > >> > Thanks
> > > > > > >> > ~ Kunal
> > > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <
> > vovatkach75@gmail.com>
> > > > > > wrote:
> > > > > > >> > Right now i haven't found the reason of
> > > > > > >> > TestDrillbitResilience.cancelAfterAllResultsProduced
> failure,
> > > > most
> > > > > > >> likely
> > > > > > >> > the cause of the failure is that the query is able to have
> > been
> > > > > > >> completed
> > > > > > >> > before cancellation request is processed.
> > > > > > >> > This test not only the case, there is one more ignored test
> > > > > > >> > TestDrillbitResilience.cancelAfterEverythingIsCompleted and
> > > jira
> > > > > > >> > DRILL-3967
> > > > > > >> > created, although the environment is AWS.
> > > > > > >> >
> > > > > > >> > Maybe it makes sense to ignore this test to unblock the
> > release
> > > > and
> > > > > > >> merge
> > > > > > >> > JDK8 changes?
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Jyothsna Reddy <jy...@gmail.com>.
Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491) and added
following lines to pom.xml to disable doc lint. The javadoc doesn't throw
any errors and the build is successful.
<activation>
<jdk>[1.8,)</jdk>
</activation>
<properties>
<additionalparam>-Xdoclint:none</additionalparam>
</properties>
On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri <ha...@gmail.com>
wrote:
> On my machine I couldn't repro the issue related to TestDrillbitResilience.
> cancelAfterAllResultsProduced.
> I used the vladimir's branch (i.e DRILL-1491).
> Used the maven test command for testing it.
>
> output of the test run.
> ... 4 common frames omitted
> Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 124.187
> sec - in org.apache.drill.exec.server.TestDrillbitResilience
>
>
>
>
> On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <pa...@apache.org> wrote:
>
> > Yes I agree. JDBC would be a new feature that we can defer to 1.14.0.
> > I'm hoping we can resolve the other three in the next few days. Target
> date
> > for starting release process - Friday Mar 9th
> >
> > Once these are resolved, I will create a branch for the release so that
> > Apache master remains open for commits. If any issues are found in the
> > release branch, we will fix them in master and I will cherry-pick the
> into
> > the release branch. Once the release is finalized I will add a release
> tag
> > and remove the branch.
> >
> > Also note if QA folks want to get started on testing the release, the
> > current head of Apache master is close to final. Javadoc generation is
> only
> > a release build issue, and the other issues are localized to specific
> > cases.
> >
> > Note: to reproduce the javadoc issues:
> > # set JAVA_HOME to JDK 8
> > mvn javadoc:javadoc -Papache-release
> >
> >
> >
> > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <am...@apache.org>
> wrote:
> >
> > > It seems to me the main blockers are:
> > >
> > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we split up the
> work
> > > among few people to resolve these ?
> > > 2. DRILL-6216 Metadata mismatch.. <-- Agreement was to
> revert
> > > one small piece of code and it appears Sorabh is looking into it
> > > 3. TestDrillbitResilience.cancelAfterAllResultsProduced <-- need
> > someone
> > > to look into this
> > >
> > > Regarding the JDBC issues that Parth mentioned, looking at the JIRAs,
> it
> > > seems they are not showstoppers...Parth do you agree ?
> > >
> > > Since we are close to the finish line for JDK 8, IMO we should try and
> > see
> > > if in another day or two we can get over these hurdles.
> > >
> > > -Aman
> > >
> > >
> > >
> > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <pm...@mapr.com> wrote:
> > >
> > > > The JDK 8 issues will likely require more time to harden for it to be
> > > > included in the 1.13 release. My recommendation would be to move
> ahead
> > > with
> > > > the 1.13 release now and address these issues right.
> > > >
> > > > Pritesh
> > > >
> > > > -----Original Message-----
> > > > From: Parth Chandra <pa...@apache.org>
> > > > Sent: March 7, 2018 3:34 AM
> > > > To: dev <de...@drill.apache.org>
> > > > Subject: Re: [DISCUSS] 1.13.0 release
> > > >
> > > > My mistake Volodymyr.
> > > >
> > > > Found some other JDK 8 issues in JIRA not tracked in DRILL-1491
> > > >
> > > > DRILL-4547 Javadoc fails with Java8
> > > > DRILL-6163 Switch Travis To Java 8
> > > >
> > > > The following are tracked in DRILL-1491, but it doesn't look like
> we're
> > > > addressing these. Are we?
> > > >
> > > > DRILL-4329 13 Unit tests are failing with JDK 8
> > > > DRILL-4333 DRILL-4329 tests in
> > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
> > > > DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
> > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in Eclipse with
> Java
> > 8
> > > >
> > > >
> > > > *DRILL-4547 is a showstopper*. The release build (-Papache-release)
> > fails
> > > > with far too many Javadoc errors even with doc lint turned off.
> > > >
> > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which is a
> project
> > by
> > > > itself.
> > > >
> > > > Note that fixing JDBC related issues and adding the command line
> option
> > > to
> > > > turn doc lint off will likely break Java 7 builds.
> > > >
> > > >
> > > > Folks who voted to get JDK 8 into this release, what is the consensus
> > on
> > > > JDBC/Java8 ?
> > > > Also, any volunteers on helping debug
> > > > TestDrillbitResilience.cancelAfterAllResultsProduced
> > > > ?
> > > >
> > > >
> > > >
> > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <
> vovatkach75@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Addition to my last message:
> > > > > The link with PR for DRILL-1491 https://urldefense.proofpoint.
> > > > com/v2/url?u=https-3A__github.com_apache_drill_pull_1143&d=DwIBaQ&c=
> > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=oTnKwfjj5hFBosMrq_
> > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3uialdRhgnf3XRY22R4SWXGZIq66a
> > > Pijuy-Ms0J_-4&e=
> > > > > on which the we can see TestDrillbitResilience.
> > > > > cancelAfterAllResultsProduced
> > > > > failure.
> > > > >
> > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <vovatkach75@gmail.com
> >:
> > > > >
> > > > > > *To Parth:*
> > > > > > The failure can only be seen if run on DRILL-1491 branch, because
> > it
> > > > uses
> > > > > > jdk 1.8 in pom.xml
> > > > > >
> > > > > > <source>1.8</source>
> > > > > > <target>1.8</target>
> > > > > >
> > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <
> shamirwasia@mapr.com
> > >:
> > > > > >
> > > > > >> Just sent an email on RCA of DRILL-6216 to discuss next steps.
> > > > > >>
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Sorabh
> > > > > >>
> > > > > >> ________________________________
> > > > > >> From: Parth Chandra <pa...@apache.org>
> > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
> > > > > >> To: dev
> > > > > >> Subject: Re: [DISCUSS] 1.13.0 release
> > > > > >>
> > > > > >> We have two items remaining -
> > > > > >>
> > > > > >> DRILL-1491 - Ideally, I would like to make sure that CANCEL is
> > > handled
> > > > > >> correctly with JDK 8. If the failure of the unit test is because
> > the
> > > > > >> cancel
> > > > > >> is received after the query is completed, then the issue is less
> > > > severe,
> > > > > >> but I would like to be sure that this is the case.
> > > > > >> Are there others who see the DrillbitResilience tests failing
> for
> > > > them?
> > > > > >> Can
> > > > > >> we try to assist Volodymyr? I don't see the failures myself.
> > > > > >>
> > > > > >> DRILL-6216 - this is a showstopper.
> > > > > >>
> > > > > >>
> > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
> > kunalkhatua@gmail.com
> > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi Parth
> > > > > >> >
> > > > > >> > DRILL-6216 is a release blocker that is being currently looked
> > > into.
> > > > > >> >
> > > > > >> > Ref:
> > > > > >> > DRILL-6216: Metadata mismatch when connecting to a Drill
> 1.12.0
> > > > with a
> > > > > >> > Drill-1.13.0-SNAPSHOT driver
> > > > > >> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=cskdkSMqhcnjZ
> > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=xu
> > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz3sxpP5-wHPs&s=txeKaKzF67flAi48
> > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> > > > > >> >
> > > > > >> > Please add it to the list of required commits as well.
> > > > > >> >
> > > > > >> > Thanks
> > > > > >> > ~ Kunal
> > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <
> vovatkach75@gmail.com>
> > > > > wrote:
> > > > > >> > Right now i haven't found the reason of
> > > > > >> > TestDrillbitResilience.cancelAfterAllResultsProduced failure,
> > > most
> > > > > >> likely
> > > > > >> > the cause of the failure is that the query is able to have
> been
> > > > > >> completed
> > > > > >> > before cancellation request is processed.
> > > > > >> > This test not only the case, there is one more ignored test
> > > > > >> > TestDrillbitResilience.cancelAfterEverythingIsCompleted and
> > jira
> > > > > >> > DRILL-3967
> > > > > >> > created, although the environment is AWS.
> > > > > >> >
> > > > > >> > Maybe it makes sense to ignore this test to unblock the
> release
> > > and
> > > > > >> merge
> > > > > >> > JDK8 changes?
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Hanumath Rao Maduri <ha...@gmail.com>.
On my machine I couldn't repro the issue related to TestDrillbitResilience.
cancelAfterAllResultsProduced.
I used the vladimir's branch (i.e DRILL-1491).
Used the maven test command for testing it.
output of the test run.
... 4 common frames omitted
Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 124.187
sec - in org.apache.drill.exec.server.TestDrillbitResilience
On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra <pa...@apache.org> wrote:
> Yes I agree. JDBC would be a new feature that we can defer to 1.14.0.
> I'm hoping we can resolve the other three in the next few days. Target date
> for starting release process - Friday Mar 9th
>
> Once these are resolved, I will create a branch for the release so that
> Apache master remains open for commits. If any issues are found in the
> release branch, we will fix them in master and I will cherry-pick the into
> the release branch. Once the release is finalized I will add a release tag
> and remove the branch.
>
> Also note if QA folks want to get started on testing the release, the
> current head of Apache master is close to final. Javadoc generation is only
> a release build issue, and the other issues are localized to specific
> cases.
>
> Note: to reproduce the javadoc issues:
> # set JAVA_HOME to JDK 8
> mvn javadoc:javadoc -Papache-release
>
>
>
> On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <am...@apache.org> wrote:
>
> > It seems to me the main blockers are:
> >
> > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we split up the work
> > among few people to resolve these ?
> > 2. DRILL-6216 Metadata mismatch.. <-- Agreement was to revert
> > one small piece of code and it appears Sorabh is looking into it
> > 3. TestDrillbitResilience.cancelAfterAllResultsProduced <-- need
> someone
> > to look into this
> >
> > Regarding the JDBC issues that Parth mentioned, looking at the JIRAs, it
> > seems they are not showstoppers...Parth do you agree ?
> >
> > Since we are close to the finish line for JDK 8, IMO we should try and
> see
> > if in another day or two we can get over these hurdles.
> >
> > -Aman
> >
> >
> >
> > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <pm...@mapr.com> wrote:
> >
> > > The JDK 8 issues will likely require more time to harden for it to be
> > > included in the 1.13 release. My recommendation would be to move ahead
> > with
> > > the 1.13 release now and address these issues right.
> > >
> > > Pritesh
> > >
> > > -----Original Message-----
> > > From: Parth Chandra <pa...@apache.org>
> > > Sent: March 7, 2018 3:34 AM
> > > To: dev <de...@drill.apache.org>
> > > Subject: Re: [DISCUSS] 1.13.0 release
> > >
> > > My mistake Volodymyr.
> > >
> > > Found some other JDK 8 issues in JIRA not tracked in DRILL-1491
> > >
> > > DRILL-4547 Javadoc fails with Java8
> > > DRILL-6163 Switch Travis To Java 8
> > >
> > > The following are tracked in DRILL-1491, but it doesn't look like we're
> > > addressing these. Are we?
> > >
> > > DRILL-4329 13 Unit tests are failing with JDK 8
> > > DRILL-4333 DRILL-4329 tests in
> > > Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
> > > DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
> > > DRILL-5680 BasicPhysicalOpUnitTest can't run in Eclipse with Java
> 8
> > >
> > >
> > > *DRILL-4547 is a showstopper*. The release build (-Papache-release)
> fails
> > > with far too many Javadoc errors even with doc lint turned off.
> > >
> > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which is a project
> by
> > > itself.
> > >
> > > Note that fixing JDBC related issues and adding the command line option
> > to
> > > turn doc lint off will likely break Java 7 builds.
> > >
> > >
> > > Folks who voted to get JDK 8 into this release, what is the consensus
> on
> > > JDBC/Java8 ?
> > > Also, any volunteers on helping debug
> > > TestDrillbitResilience.cancelAfterAllResultsProduced
> > > ?
> > >
> > >
> > >
> > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <vovatkach75@gmail.com
> >
> > > wrote:
> > >
> > > > Addition to my last message:
> > > > The link with PR for DRILL-1491 https://urldefense.proofpoint.
> > > com/v2/url?u=https-3A__github.com_apache_drill_pull_1143&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=oTnKwfjj5hFBosMrq_
> > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3uialdRhgnf3XRY22R4SWXGZIq66a
> > Pijuy-Ms0J_-4&e=
> > > > on which the we can see TestDrillbitResilience.
> > > > cancelAfterAllResultsProduced
> > > > failure.
> > > >
> > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <vo...@gmail.com>:
> > > >
> > > > > *To Parth:*
> > > > > The failure can only be seen if run on DRILL-1491 branch, because
> it
> > > uses
> > > > > jdk 1.8 in pom.xml
> > > > >
> > > > > <source>1.8</source>
> > > > > <target>1.8</target>
> > > > >
> > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <shamirwasia@mapr.com
> >:
> > > > >
> > > > >> Just sent an email on RCA of DRILL-6216 to discuss next steps.
> > > > >>
> > > > >>
> > > > >> Thanks,
> > > > >> Sorabh
> > > > >>
> > > > >> ________________________________
> > > > >> From: Parth Chandra <pa...@apache.org>
> > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
> > > > >> To: dev
> > > > >> Subject: Re: [DISCUSS] 1.13.0 release
> > > > >>
> > > > >> We have two items remaining -
> > > > >>
> > > > >> DRILL-1491 - Ideally, I would like to make sure that CANCEL is
> > handled
> > > > >> correctly with JDK 8. If the failure of the unit test is because
> the
> > > > >> cancel
> > > > >> is received after the query is completed, then the issue is less
> > > severe,
> > > > >> but I would like to be sure that this is the case.
> > > > >> Are there others who see the DrillbitResilience tests failing for
> > > them?
> > > > >> Can
> > > > >> we try to assist Volodymyr? I don't see the failures myself.
> > > > >>
> > > > >> DRILL-6216 - this is a showstopper.
> > > > >>
> > > > >>
> > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <
> kunalkhatua@gmail.com
> > >
> > > > >> wrote:
> > > > >>
> > > > >> > Hi Parth
> > > > >> >
> > > > >> > DRILL-6216 is a release blocker that is being currently looked
> > into.
> > > > >> >
> > > > >> > Ref:
> > > > >> > DRILL-6216: Metadata mismatch when connecting to a Drill 1.12.0
> > > with a
> > > > >> > Drill-1.13.0-SNAPSHOT driver
> > > > >> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=cskdkSMqhcnjZ
> > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=xu
> > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz3sxpP5-wHPs&s=txeKaKzF67flAi48
> > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> > > > >> >
> > > > >> > Please add it to the list of required commits as well.
> > > > >> >
> > > > >> > Thanks
> > > > >> > ~ Kunal
> > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <vo...@gmail.com>
> > > > wrote:
> > > > >> > Right now i haven't found the reason of
> > > > >> > TestDrillbitResilience.cancelAfterAllResultsProduced failure,
> > most
> > > > >> likely
> > > > >> > the cause of the failure is that the query is able to have been
> > > > >> completed
> > > > >> > before cancellation request is processed.
> > > > >> > This test not only the case, there is one more ignored test
> > > > >> > TestDrillbitResilience.cancelAfterEverythingIsCompleted and
> jira
> > > > >> > DRILL-3967
> > > > >> > created, although the environment is AWS.
> > > > >> >
> > > > >> > Maybe it makes sense to ignore this test to unblock the release
> > and
> > > > >> merge
> > > > >> > JDK8 changes?
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
Yes I agree. JDBC would be a new feature that we can defer to 1.14.0.
I'm hoping we can resolve the other three in the next few days. Target date
for starting release process - Friday Mar 9th
Once these are resolved, I will create a branch for the release so that
Apache master remains open for commits. If any issues are found in the
release branch, we will fix them in master and I will cherry-pick the into
the release branch. Once the release is finalized I will add a release tag
and remove the branch.
Also note if QA folks want to get started on testing the release, the
current head of Apache master is close to final. Javadoc generation is only
a release build issue, and the other issues are localized to specific
cases.
Note: to reproduce the javadoc issues:
# set JAVA_HOME to JDK 8
mvn javadoc:javadoc -Papache-release
On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha <am...@apache.org> wrote:
> It seems to me the main blockers are:
>
> 1. DRILL-4547 Javadoc fails with Java8 <-- Can we split up the work
> among few people to resolve these ?
> 2. DRILL-6216 Metadata mismatch.. <-- Agreement was to revert
> one small piece of code and it appears Sorabh is looking into it
> 3. TestDrillbitResilience.cancelAfterAllResultsProduced <-- need someone
> to look into this
>
> Regarding the JDBC issues that Parth mentioned, looking at the JIRAs, it
> seems they are not showstoppers...Parth do you agree ?
>
> Since we are close to the finish line for JDK 8, IMO we should try and see
> if in another day or two we can get over these hurdles.
>
> -Aman
>
>
>
> On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <pm...@mapr.com> wrote:
>
> > The JDK 8 issues will likely require more time to harden for it to be
> > included in the 1.13 release. My recommendation would be to move ahead
> with
> > the 1.13 release now and address these issues right.
> >
> > Pritesh
> >
> > -----Original Message-----
> > From: Parth Chandra <pa...@apache.org>
> > Sent: March 7, 2018 3:34 AM
> > To: dev <de...@drill.apache.org>
> > Subject: Re: [DISCUSS] 1.13.0 release
> >
> > My mistake Volodymyr.
> >
> > Found some other JDK 8 issues in JIRA not tracked in DRILL-1491
> >
> > DRILL-4547 Javadoc fails with Java8
> > DRILL-6163 Switch Travis To Java 8
> >
> > The following are tracked in DRILL-1491, but it doesn't look like we're
> > addressing these. Are we?
> >
> > DRILL-4329 13 Unit tests are failing with JDK 8
> > DRILL-4333 DRILL-4329 tests in
> > Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
> > DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
> > DRILL-5680 BasicPhysicalOpUnitTest can't run in Eclipse with Java 8
> >
> >
> > *DRILL-4547 is a showstopper*. The release build (-Papache-release) fails
> > with far too many Javadoc errors even with doc lint turned off.
> >
> > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which is a project by
> > itself.
> >
> > Note that fixing JDBC related issues and adding the command line option
> to
> > turn doc lint off will likely break Java 7 builds.
> >
> >
> > Folks who voted to get JDK 8 into this release, what is the consensus on
> > JDBC/Java8 ?
> > Also, any volunteers on helping debug
> > TestDrillbitResilience.cancelAfterAllResultsProduced
> > ?
> >
> >
> >
> > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <vo...@gmail.com>
> > wrote:
> >
> > > Addition to my last message:
> > > The link with PR for DRILL-1491 https://urldefense.proofpoint.
> > com/v2/url?u=https-3A__github.com_apache_drill_pull_1143&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=oTnKwfjj5hFBosMrq_
> > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3uialdRhgnf3XRY22R4SWXGZIq66a
> Pijuy-Ms0J_-4&e=
> > > on which the we can see TestDrillbitResilience.
> > > cancelAfterAllResultsProduced
> > > failure.
> > >
> > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <vo...@gmail.com>:
> > >
> > > > *To Parth:*
> > > > The failure can only be seen if run on DRILL-1491 branch, because it
> > uses
> > > > jdk 1.8 in pom.xml
> > > >
> > > > <source>1.8</source>
> > > > <target>1.8</target>
> > > >
> > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <sh...@mapr.com>:
> > > >
> > > >> Just sent an email on RCA of DRILL-6216 to discuss next steps.
> > > >>
> > > >>
> > > >> Thanks,
> > > >> Sorabh
> > > >>
> > > >> ________________________________
> > > >> From: Parth Chandra <pa...@apache.org>
> > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
> > > >> To: dev
> > > >> Subject: Re: [DISCUSS] 1.13.0 release
> > > >>
> > > >> We have two items remaining -
> > > >>
> > > >> DRILL-1491 - Ideally, I would like to make sure that CANCEL is
> handled
> > > >> correctly with JDK 8. If the failure of the unit test is because the
> > > >> cancel
> > > >> is received after the query is completed, then the issue is less
> > severe,
> > > >> but I would like to be sure that this is the case.
> > > >> Are there others who see the DrillbitResilience tests failing for
> > them?
> > > >> Can
> > > >> we try to assist Volodymyr? I don't see the failures myself.
> > > >>
> > > >> DRILL-6216 - this is a showstopper.
> > > >>
> > > >>
> > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <kunalkhatua@gmail.com
> >
> > > >> wrote:
> > > >>
> > > >> > Hi Parth
> > > >> >
> > > >> > DRILL-6216 is a release blocker that is being currently looked
> into.
> > > >> >
> > > >> > Ref:
> > > >> > DRILL-6216: Metadata mismatch when connecting to a Drill 1.12.0
> > with a
> > > >> > Drill-1.13.0-SNAPSHOT driver
> > > >> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=cskdkSMqhcnjZ
> > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=xu
> > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz3sxpP5-wHPs&s=txeKaKzF67flAi48
> > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> > > >> >
> > > >> > Please add it to the list of required commits as well.
> > > >> >
> > > >> > Thanks
> > > >> > ~ Kunal
> > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <vo...@gmail.com>
> > > wrote:
> > > >> > Right now i haven't found the reason of
> > > >> > TestDrillbitResilience.cancelAfterAllResultsProduced failure,
> most
> > > >> likely
> > > >> > the cause of the failure is that the query is able to have been
> > > >> completed
> > > >> > before cancellation request is processed.
> > > >> > This test not only the case, there is one more ignored test
> > > >> > TestDrillbitResilience.cancelAfterEverythingIsCompleted and jira
> > > >> > DRILL-3967
> > > >> > created, although the environment is AWS.
> > > >> >
> > > >> > Maybe it makes sense to ignore this test to unblock the release
> and
> > > >> merge
> > > >> > JDK8 changes?
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Aman Sinha <am...@apache.org>.
It seems to me the main blockers are:
1. DRILL-4547 Javadoc fails with Java8 <-- Can we split up the work
among few people to resolve these ?
2. DRILL-6216 Metadata mismatch.. <-- Agreement was to revert
one small piece of code and it appears Sorabh is looking into it
3. TestDrillbitResilience.cancelAfterAllResultsProduced <-- need someone
to look into this
Regarding the JDBC issues that Parth mentioned, looking at the JIRAs, it
seems they are not showstoppers...Parth do you agree ?
Since we are close to the finish line for JDK 8, IMO we should try and see
if in another day or two we can get over these hurdles.
-Aman
On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker <pm...@mapr.com> wrote:
> The JDK 8 issues will likely require more time to harden for it to be
> included in the 1.13 release. My recommendation would be to move ahead with
> the 1.13 release now and address these issues right.
>
> Pritesh
>
> -----Original Message-----
> From: Parth Chandra <pa...@apache.org>
> Sent: March 7, 2018 3:34 AM
> To: dev <de...@drill.apache.org>
> Subject: Re: [DISCUSS] 1.13.0 release
>
> My mistake Volodymyr.
>
> Found some other JDK 8 issues in JIRA not tracked in DRILL-1491
>
> DRILL-4547 Javadoc fails with Java8
> DRILL-6163 Switch Travis To Java 8
>
> The following are tracked in DRILL-1491, but it doesn't look like we're
> addressing these. Are we?
>
> DRILL-4329 13 Unit tests are failing with JDK 8
> DRILL-4333 DRILL-4329 tests in
> Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
> DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
> DRILL-5680 BasicPhysicalOpUnitTest can't run in Eclipse with Java 8
>
>
> *DRILL-4547 is a showstopper*. The release build (-Papache-release) fails
> with far too many Javadoc errors even with doc lint turned off.
>
> DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which is a project by
> itself.
>
> Note that fixing JDBC related issues and adding the command line option to
> turn doc lint off will likely break Java 7 builds.
>
>
> Folks who voted to get JDK 8 into this release, what is the consensus on
> JDBC/Java8 ?
> Also, any volunteers on helping debug
> TestDrillbitResilience.cancelAfterAllResultsProduced
> ?
>
>
>
> On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <vo...@gmail.com>
> wrote:
>
> > Addition to my last message:
> > The link with PR for DRILL-1491 https://urldefense.proofpoint.
> com/v2/url?u=https-3A__github.com_apache_drill_pull_1143&d=DwIBaQ&c=
> cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=oTnKwfjj5hFBosMrq_
> WWhazhGeoC2nGSKeMOPxU2_cM&s=p3uialdRhgnf3XRY22R4SWXGZIq66aPijuy-Ms0J_-4&e=
> > on which the we can see TestDrillbitResilience.
> > cancelAfterAllResultsProduced
> > failure.
> >
> > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <vo...@gmail.com>:
> >
> > > *To Parth:*
> > > The failure can only be seen if run on DRILL-1491 branch, because it
> uses
> > > jdk 1.8 in pom.xml
> > >
> > > <source>1.8</source>
> > > <target>1.8</target>
> > >
> > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <sh...@mapr.com>:
> > >
> > >> Just sent an email on RCA of DRILL-6216 to discuss next steps.
> > >>
> > >>
> > >> Thanks,
> > >> Sorabh
> > >>
> > >> ________________________________
> > >> From: Parth Chandra <pa...@apache.org>
> > >> Sent: Tuesday, March 6, 2018 6:48:21 PM
> > >> To: dev
> > >> Subject: Re: [DISCUSS] 1.13.0 release
> > >>
> > >> We have two items remaining -
> > >>
> > >> DRILL-1491 - Ideally, I would like to make sure that CANCEL is handled
> > >> correctly with JDK 8. If the failure of the unit test is because the
> > >> cancel
> > >> is received after the query is completed, then the issue is less
> severe,
> > >> but I would like to be sure that this is the case.
> > >> Are there others who see the DrillbitResilience tests failing for
> them?
> > >> Can
> > >> we try to assist Volodymyr? I don't see the failures myself.
> > >>
> > >> DRILL-6216 - this is a showstopper.
> > >>
> > >>
> > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <ku...@gmail.com>
> > >> wrote:
> > >>
> > >> > Hi Parth
> > >> >
> > >> > DRILL-6216 is a release blocker that is being currently looked into.
> > >> >
> > >> > Ref:
> > >> > DRILL-6216: Metadata mismatch when connecting to a Drill 1.12.0
> with a
> > >> > Drill-1.13.0-SNAPSHOT driver
> > >> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=cskdkSMqhcnjZ
> > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=xu
> > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz3sxpP5-wHPs&s=txeKaKzF67flAi48
> > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> > >> >
> > >> > Please add it to the list of required commits as well.
> > >> >
> > >> > Thanks
> > >> > ~ Kunal
> > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <vo...@gmail.com>
> > wrote:
> > >> > Right now i haven't found the reason of
> > >> > TestDrillbitResilience.cancelAfterAllResultsProduced failure, most
> > >> likely
> > >> > the cause of the failure is that the query is able to have been
> > >> completed
> > >> > before cancellation request is processed.
> > >> > This test not only the case, there is one more ignored test
> > >> > TestDrillbitResilience.cancelAfterEverythingIsCompleted and jira
> > >> > DRILL-3967
> > >> > created, although the environment is AWS.
> > >> >
> > >> > Maybe it makes sense to ignore this test to unblock the release and
> > >> merge
> > >> > JDK8 changes?
> > >> >
> > >>
> > >
> > >
> >
>
RE: [DISCUSS] 1.13.0 release
Posted by Pritesh Maker <pm...@mapr.com>.
The JDK 8 issues will likely require more time to harden for it to be included in the 1.13 release. My recommendation would be to move ahead with the 1.13 release now and address these issues right.
Pritesh
-----Original Message-----
From: Parth Chandra <pa...@apache.org>
Sent: March 7, 2018 3:34 AM
To: dev <de...@drill.apache.org>
Subject: Re: [DISCUSS] 1.13.0 release
My mistake Volodymyr.
Found some other JDK 8 issues in JIRA not tracked in DRILL-1491
DRILL-4547 Javadoc fails with Java8
DRILL-6163 Switch Travis To Java 8
The following are tracked in DRILL-1491, but it doesn't look like we're
addressing these. Are we?
DRILL-4329 13 Unit tests are failing with JDK 8
DRILL-4333 DRILL-4329 tests in
Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
DRILL-5680 BasicPhysicalOpUnitTest can't run in Eclipse with Java 8
*DRILL-4547 is a showstopper*. The release build (-Papache-release) fails
with far too many Javadoc errors even with doc lint turned off.
DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which is a project by
itself.
Note that fixing JDBC related issues and adding the command line option to
turn doc lint off will likely break Java 7 builds.
Folks who voted to get JDK 8 into this release, what is the consensus on
JDBC/Java8 ?
Also, any volunteers on helping debug
TestDrillbitResilience.cancelAfterAllResultsProduced
?
On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <vo...@gmail.com>
wrote:
> Addition to my last message:
> The link with PR for DRILL-1491 https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_drill_pull_1143&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=oTnKwfjj5hFBosMrq_WWhazhGeoC2nGSKeMOPxU2_cM&s=p3uialdRhgnf3XRY22R4SWXGZIq66aPijuy-Ms0J_-4&e=
> on which the we can see TestDrillbitResilience.
> cancelAfterAllResultsProduced
> failure.
>
> 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <vo...@gmail.com>:
>
> > *To Parth:*
> > The failure can only be seen if run on DRILL-1491 branch, because it uses
> > jdk 1.8 in pom.xml
> >
> > <source>1.8</source>
> > <target>1.8</target>
> >
> > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <sh...@mapr.com>:
> >
> >> Just sent an email on RCA of DRILL-6216 to discuss next steps.
> >>
> >>
> >> Thanks,
> >> Sorabh
> >>
> >> ________________________________
> >> From: Parth Chandra <pa...@apache.org>
> >> Sent: Tuesday, March 6, 2018 6:48:21 PM
> >> To: dev
> >> Subject: Re: [DISCUSS] 1.13.0 release
> >>
> >> We have two items remaining -
> >>
> >> DRILL-1491 - Ideally, I would like to make sure that CANCEL is handled
> >> correctly with JDK 8. If the failure of the unit test is because the
> >> cancel
> >> is received after the query is completed, then the issue is less severe,
> >> but I would like to be sure that this is the case.
> >> Are there others who see the DrillbitResilience tests failing for them?
> >> Can
> >> we try to assist Volodymyr? I don't see the failures myself.
> >>
> >> DRILL-6216 - this is a showstopper.
> >>
> >>
> >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <ku...@gmail.com>
> >> wrote:
> >>
> >> > Hi Parth
> >> >
> >> > DRILL-6216 is a release blocker that is being currently looked into.
> >> >
> >> > Ref:
> >> > DRILL-6216: Metadata mismatch when connecting to a Drill 1.12.0 with a
> >> > Drill-1.13.0-SNAPSHOT driver
> >> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=cskdkSMqhcnjZ
> >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=xu
> >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz3sxpP5-wHPs&s=txeKaKzF67flAi48
> >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> >> >
> >> > Please add it to the list of required commits as well.
> >> >
> >> > Thanks
> >> > ~ Kunal
> >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <vo...@gmail.com>
> wrote:
> >> > Right now i haven't found the reason of
> >> > TestDrillbitResilience.cancelAfterAllResultsProduced failure, most
> >> likely
> >> > the cause of the failure is that the query is able to have been
> >> completed
> >> > before cancellation request is processed.
> >> > This test not only the case, there is one more ignored test
> >> > TestDrillbitResilience.cancelAfterEverythingIsCompleted and jira
> >> > DRILL-3967
> >> > created, although the environment is AWS.
> >> >
> >> > Maybe it makes sense to ignore this test to unblock the release and
> >> merge
> >> > JDK8 changes?
> >> >
> >>
> >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
My mistake Volodymyr.
Found some other JDK 8 issues in JIRA not tracked in DRILL-1491
DRILL-4547 Javadoc fails with Java8
DRILL-6163 Switch Travis To Java 8
The following are tracked in DRILL-1491, but it doesn't look like we're
addressing these. Are we?
DRILL-4329 13 Unit tests are failing with JDK 8
DRILL-4333 DRILL-4329 tests in
Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
DRILL-5120 Upgrade JDBC Driver for new Java 8 methods
DRILL-5680 BasicPhysicalOpUnitTest can't run in Eclipse with Java 8
*DRILL-4547 is a showstopper*. The release build (-Papache-release) fails
with far too many Javadoc errors even with doc lint turned off.
DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which is a project by
itself.
Note that fixing JDBC related issues and adding the command line option to
turn doc lint off will likely break Java 7 builds.
Folks who voted to get JDK 8 into this release, what is the consensus on
JDBC/Java8 ?
Also, any volunteers on helping debug
TestDrillbitResilience.cancelAfterAllResultsProduced
?
On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach <vo...@gmail.com>
wrote:
> Addition to my last message:
> The link with PR for DRILL-1491 https://github.com/apache/drill/pull/1143
> on which the we can see TestDrillbitResilience.
> cancelAfterAllResultsProduced
> failure.
>
> 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <vo...@gmail.com>:
>
> > *To Parth:*
> > The failure can only be seen if run on DRILL-1491 branch, because it uses
> > jdk 1.8 in pom.xml
> >
> > <source>1.8</source>
> > <target>1.8</target>
> >
> > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <sh...@mapr.com>:
> >
> >> Just sent an email on RCA of DRILL-6216 to discuss next steps.
> >>
> >>
> >> Thanks,
> >> Sorabh
> >>
> >> ________________________________
> >> From: Parth Chandra <pa...@apache.org>
> >> Sent: Tuesday, March 6, 2018 6:48:21 PM
> >> To: dev
> >> Subject: Re: [DISCUSS] 1.13.0 release
> >>
> >> We have two items remaining -
> >>
> >> DRILL-1491 - Ideally, I would like to make sure that CANCEL is handled
> >> correctly with JDK 8. If the failure of the unit test is because the
> >> cancel
> >> is received after the query is completed, then the issue is less severe,
> >> but I would like to be sure that this is the case.
> >> Are there others who see the DrillbitResilience tests failing for them?
> >> Can
> >> we try to assist Volodymyr? I don't see the failures myself.
> >>
> >> DRILL-6216 - this is a showstopper.
> >>
> >>
> >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <ku...@gmail.com>
> >> wrote:
> >>
> >> > Hi Parth
> >> >
> >> > DRILL-6216 is a release blocker that is being currently looked into.
> >> >
> >> > Ref:
> >> > DRILL-6216: Metadata mismatch when connecting to a Drill 1.12.0 with a
> >> > Drill-1.13.0-SNAPSHOT driver
> >> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=cskdkSMqhcnjZ
> >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=xu
> >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz3sxpP5-wHPs&s=txeKaKzF67flAi48
> >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> >> >
> >> > Please add it to the list of required commits as well.
> >> >
> >> > Thanks
> >> > ~ Kunal
> >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <vo...@gmail.com>
> wrote:
> >> > Right now i haven't found the reason of
> >> > TestDrillbitResilience.cancelAfterAllResultsProduced failure, most
> >> likely
> >> > the cause of the failure is that the query is able to have been
> >> completed
> >> > before cancellation request is processed.
> >> > This test not only the case, there is one more ignored test
> >> > TestDrillbitResilience.cancelAfterEverythingIsCompleted and jira
> >> > DRILL-3967
> >> > created, although the environment is AWS.
> >> >
> >> > Maybe it makes sense to ignore this test to unblock the release and
> >> merge
> >> > JDK8 changes?
> >> >
> >>
> >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Volodymyr Tkach <vo...@gmail.com>.
Addition to my last message:
The link with PR for DRILL-1491 https://github.com/apache/drill/pull/1143
on which the we can see TestDrillbitResilience.cancelAfterAllResultsProduced
failure.
2018-03-07 11:45 GMT+02:00 Volodymyr Tkach <vo...@gmail.com>:
> *To Parth:*
> The failure can only be seen if run on DRILL-1491 branch, because it uses
> jdk 1.8 in pom.xml
>
> <source>1.8</source>
> <target>1.8</target>
>
> 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <sh...@mapr.com>:
>
>> Just sent an email on RCA of DRILL-6216 to discuss next steps.
>>
>>
>> Thanks,
>> Sorabh
>>
>> ________________________________
>> From: Parth Chandra <pa...@apache.org>
>> Sent: Tuesday, March 6, 2018 6:48:21 PM
>> To: dev
>> Subject: Re: [DISCUSS] 1.13.0 release
>>
>> We have two items remaining -
>>
>> DRILL-1491 - Ideally, I would like to make sure that CANCEL is handled
>> correctly with JDK 8. If the failure of the unit test is because the
>> cancel
>> is received after the query is completed, then the issue is less severe,
>> but I would like to be sure that this is the case.
>> Are there others who see the DrillbitResilience tests failing for them?
>> Can
>> we try to assist Volodymyr? I don't see the failures myself.
>>
>> DRILL-6216 - this is a showstopper.
>>
>>
>> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <ku...@gmail.com>
>> wrote:
>>
>> > Hi Parth
>> >
>> > DRILL-6216 is a release blocker that is being currently looked into.
>> >
>> > Ref:
>> > DRILL-6216: Metadata mismatch when connecting to a Drill 1.12.0 with a
>> > Drill-1.13.0-SNAPSHOT driver
>> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
>> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=cskdkSMqhcnjZ
>> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=xu
>> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz3sxpP5-wHPs&s=txeKaKzF67flAi48
>> DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
>> >
>> > Please add it to the list of required commits as well.
>> >
>> > Thanks
>> > ~ Kunal
>> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <vo...@gmail.com> wrote:
>> > Right now i haven't found the reason of
>> > TestDrillbitResilience.cancelAfterAllResultsProduced failure, most
>> likely
>> > the cause of the failure is that the query is able to have been
>> completed
>> > before cancellation request is processed.
>> > This test not only the case, there is one more ignored test
>> > TestDrillbitResilience.cancelAfterEverythingIsCompleted and jira
>> > DRILL-3967
>> > created, although the environment is AWS.
>> >
>> > Maybe it makes sense to ignore this test to unblock the release and
>> merge
>> > JDK8 changes?
>> >
>>
>
>
Re: [DISCUSS] 1.13.0 release
Posted by Volodymyr Tkach <vo...@gmail.com>.
*To Parth:*
The failure can only be seen if run on DRILL-1491 branch, because it uses
jdk 1.8 in pom.xml
<source>1.8</source>
<target>1.8</target>
2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia <sh...@mapr.com>:
> Just sent an email on RCA of DRILL-6216 to discuss next steps.
>
>
> Thanks,
> Sorabh
>
> ________________________________
> From: Parth Chandra <pa...@apache.org>
> Sent: Tuesday, March 6, 2018 6:48:21 PM
> To: dev
> Subject: Re: [DISCUSS] 1.13.0 release
>
> We have two items remaining -
>
> DRILL-1491 - Ideally, I would like to make sure that CANCEL is handled
> correctly with JDK 8. If the failure of the unit test is because the cancel
> is received after the query is completed, then the issue is less severe,
> but I would like to be sure that this is the case.
> Are there others who see the DrillbitResilience tests failing for them? Can
> we try to assist Volodymyr? I don't see the failures myself.
>
> DRILL-6216 - this is a showstopper.
>
>
> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <ku...@gmail.com>
> wrote:
>
> > Hi Parth
> >
> > DRILL-6216 is a release blocker that is being currently looked into.
> >
> > Ref:
> > DRILL-6216: Metadata mismatch when connecting to a Drill 1.12.0 with a
> > Drill-1.13.0-SNAPSHOT driver
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=
> gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=
> xuRz02Sbprxvbtw1OrBuDvlRbp2lh9mz3sxpP5-wHPs&s=
> txeKaKzF67flAi48DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
> >
> > Please add it to the list of required commits as well.
> >
> > Thanks
> > ~ Kunal
> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach <vo...@gmail.com> wrote:
> > Right now i haven't found the reason of
> > TestDrillbitResilience.cancelAfterAllResultsProduced failure, most
> likely
> > the cause of the failure is that the query is able to have been completed
> > before cancellation request is processed.
> > This test not only the case, there is one more ignored test
> > TestDrillbitResilience.cancelAfterEverythingIsCompleted and jira
> > DRILL-3967
> > created, although the environment is AWS.
> >
> > Maybe it makes sense to ignore this test to unblock the release and merge
> > JDK8 changes?
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Sorabh Hamirwasia <sh...@mapr.com>.
Just sent an email on RCA of DRILL-6216 to discuss next steps.
Thanks,
Sorabh
________________________________
From: Parth Chandra <pa...@apache.org>
Sent: Tuesday, March 6, 2018 6:48:21 PM
To: dev
Subject: Re: [DISCUSS] 1.13.0 release
We have two items remaining -
DRILL-1491 - Ideally, I would like to make sure that CANCEL is handled
correctly with JDK 8. If the failure of the unit test is because the cancel
is received after the query is completed, then the issue is less severe,
but I would like to be sure that this is the case.
Are there others who see the DrillbitResilience tests failing for them? Can
we try to assist Volodymyr? I don't see the failures myself.
DRILL-6216 - this is a showstopper.
On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <ku...@gmail.com> wrote:
> Hi Parth
>
> DRILL-6216 is a release blocker that is being currently looked into.
>
> Ref:
> DRILL-6216: Metadata mismatch when connecting to a Drill 1.12.0 with a
> Drill-1.13.0-SNAPSHOT driver
> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=xuRz02Sbprxvbtw1OrBuDvlRbp2lh9mz3sxpP5-wHPs&s=txeKaKzF67flAi48DUNLgMWbxje1GXWxfFpG6BEPXk0&e=
>
> Please add it to the list of required commits as well.
>
> Thanks
> ~ Kunal
> On 3/6/2018 9:53:06 AM, Volodymyr Tkach <vo...@gmail.com> wrote:
> Right now i haven't found the reason of
> TestDrillbitResilience.cancelAfterAllResultsProduced failure, most likely
> the cause of the failure is that the query is able to have been completed
> before cancellation request is processed.
> This test not only the case, there is one more ignored test
> TestDrillbitResilience.cancelAfterEverythingIsCompleted and jira
> DRILL-3967
> created, although the environment is AWS.
>
> Maybe it makes sense to ignore this test to unblock the release and merge
> JDK8 changes?
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
We have two items remaining -
DRILL-1491 - Ideally, I would like to make sure that CANCEL is handled
correctly with JDK 8. If the failure of the unit test is because the cancel
is received after the query is completed, then the issue is less severe,
but I would like to be sure that this is the case.
Are there others who see the DrillbitResilience tests failing for them? Can
we try to assist Volodymyr? I don't see the failures myself.
DRILL-6216 - this is a showstopper.
On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua <ku...@gmail.com> wrote:
> Hi Parth
>
> DRILL-6216 is a release blocker that is being currently looked into.
>
> Ref:
> DRILL-6216: Metadata mismatch when connecting to a Drill 1.12.0 with a
> Drill-1.13.0-SNAPSHOT driver
> https://issues.apache.org/jira/browse/DRILL-6216
>
> Please add it to the list of required commits as well.
>
> Thanks
> ~ Kunal
> On 3/6/2018 9:53:06 AM, Volodymyr Tkach <vo...@gmail.com> wrote:
> Right now i haven't found the reason of
> TestDrillbitResilience.cancelAfterAllResultsProduced failure, most likely
> the cause of the failure is that the query is able to have been completed
> before cancellation request is processed.
> This test not only the case, there is one more ignored test
> TestDrillbitResilience.cancelAfterEverythingIsCompleted and jira
> DRILL-3967
> created, although the environment is AWS.
>
> Maybe it makes sense to ignore this test to unblock the release and merge
> JDK8 changes?
>
Re: [DISCUSS] 1.13.0 release
Posted by Kunal Khatua <ku...@gmail.com>.
Hi Parth
DRILL-6216 is a release blocker that is being currently looked into.
Ref:
DRILL-6216: Metadata mismatch when connecting to a Drill 1.12.0 with a Drill-1.13.0-SNAPSHOT driver
https://issues.apache.org/jira/browse/DRILL-6216
Please add it to the list of required commits as well.
Thanks
~ Kunal
On 3/6/2018 9:53:06 AM, Volodymyr Tkach <vo...@gmail.com> wrote:
Right now i haven't found the reason of
TestDrillbitResilience.cancelAfterAllResultsProduced failure, most likely
the cause of the failure is that the query is able to have been completed
before cancellation request is processed.
This test not only the case, there is one more ignored test
TestDrillbitResilience.cancelAfterEverythingIsCompleted and jira DRILL-3967
created, although the environment is AWS.
Maybe it makes sense to ignore this test to unblock the release and merge
JDK8 changes?
Re: [DISCUSS] 1.13.0 release
Posted by Volodymyr Tkach <vo...@gmail.com>.
Right now i haven't found the reason of
TestDrillbitResilience.cancelAfterAllResultsProduced failure, most likely
the cause of the failure is that the query is able to have been completed
before cancellation request is processed.
This test not only the case, there is one more ignored test
TestDrillbitResilience.cancelAfterEverythingIsCompleted and jira DRILL-3967
created, although the environment is AWS.
Maybe it makes sense to ignore this test to unblock the release and merge
JDK8 changes?
Re: [DISCUSS] 1.13.0 release
Posted by Volodymyr Tkach <vo...@gmail.com>.
*Update for:* DRILL-1491: Support for JDK 8
Currently we have 1 unit tests fail on test cluster. If ignored, all other
tests succeed.
Failed tests:
TestDrillbitResilience.cancelAfterAllResultsProduced:672->assertCancelledWithoutException:564->assertCancelledWithoutException:557->assertStateCompleted:543
Query state is incorrect (expected: CANCELED, actual: COMPLETED) AND/OR
Exception thrown: none.
We run tests from TestDrillbitResilience class only (mvn
-Dtest=TestDrillbitResilience -pl exec/java-exec test) there might be
random failures of: TestDrillbitResilience#cancelAfterAllResultsProduced
and cancelInMiddleOfFetchingResults
2018-03-04 19:21 GMT+02:00 Arina Yelchiyeva <ar...@gmail.com>:
> Merged Drill on Yarn. Paul thanks for making the changes.
>
> Regarding JDK8 there is an open PR (
> https://github.com/apache/drill/pull/1143) which needs to be merged.
> As Volodymyr mentioned there were some issues with unit tests and Travis.
> Padma and I have addressed unit tests issues but we still need to check if
> problems with Travis are solved.
>
> @Volodymyr please rebase on the latest master and check.
>
> Kind regards
> Arina
>
> On Sun, Mar 4, 2018 at 3:46 PM, Parth Chandra <pa...@apache.org> wrote:
>
> > Sounds good.
> >
> > On Sun, Mar 4, 2018 at 1:18 PM, Paul Rogers <pa...@yahoo.com.invalid>
> > wrote:
> >
> > > Hi Parth,
> > > Issues with DRILL-1170 are resolved. Needs one final review by Arina,
> > then
> > > we should be good to do.
> > > Thanks to everyone for getting the other two "batch size" PRs committed
> > > recently.
> > > Thanks,
> > > - Paul
> > >
> > >
> > >
> > > On Saturday, March 3, 2018, 10:57:10 PM PST, Parth Chandra <
> > > parthc@apache.org> wrote:
> > >
> > > Thank you Arina !
> > >
> > > Build and unit tests with JDK 8 passed for me with the latest master.
> I'm
> > > guessing there is still something more to be addressed, since
> DRILL-1491
> > is
> > > still open?
> > >
> > > Updated list:
> > >
> > > DRILL-1491: Support for JDK 8 --* In progress.*
> > > DRILL-1170: YARN support for Drill -- Needs Committer +1 and Travis
> fix.
> > > DRILL-6027: Implement spill to disk for the Hash Join --- No PR and
> is a
> > > major feature that should be reviewed (properly!).
> > > DRILL-6173: Support transitive closure during filter push down and
> > > partition pruning. -- No PR and depends on 3 Apache Calcite issues
> that
> > > are open.
> > > DRILL-6023: Graceful shutdown improvements -- Marked as 1.14.0 (?)
> > >
> > >
> > >
> > > On Sun, Mar 4, 2018 at 12:35 AM, Arina Ielchiieva <ar...@apache.org>
> > > wrote:
> > >
> > > > I will take care of FunctionInitializerTest#
> > > > testConcurrentFunctionBodyLoad
> > > > in DRILL-6208.
> > > >
> > > > Also I have merged all Jiras with ready-to-commit label.
> > > > Drill on Yarn was NOT merged due to banned dependencies issue.
> > > >
> > > > Kind regards
> > > > Arina
> > > >
> > > >
> > > > On Sat, Mar 3, 2018 at 12:11 PM, Parth Chandra <pa...@apache.org>
> > > wrote:
> > > >
> > > > > Thanks for the updates guys. Please keep updating the status in
> this
> > > > > thread.
> > > > >
> > > > > On Fri, Mar 2, 2018 at 11:18 PM, Padma Penumarthy <
> > > ppenumarthy@mapr.com>
> > > > > wrote:
> > > > >
> > > > > > I can look at failure in testFlattenUpperLimit. I added this test
> > > > > recently.
> > > > > >
> > > > > > Thanks
> > > > > > Padma
> > > > > >
> > > > > >
> > > > > > On Mar 2, 2018, at 8:19 AM, Volodymyr Tkach <
> vovatkach75@gmail.com
> > > > > <mailto:
> > > > > > vovatkach75@gmail.com>> wrote:
> > > > > >
> > > > > > testFlattenUpperLimit
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Arina Yelchiyeva <ar...@gmail.com>.
Merged Drill on Yarn. Paul thanks for making the changes.
Regarding JDK8 there is an open PR (
https://github.com/apache/drill/pull/1143) which needs to be merged.
As Volodymyr mentioned there were some issues with unit tests and Travis.
Padma and I have addressed unit tests issues but we still need to check if
problems with Travis are solved.
@Volodymyr please rebase on the latest master and check.
Kind regards
Arina
On Sun, Mar 4, 2018 at 3:46 PM, Parth Chandra <pa...@apache.org> wrote:
> Sounds good.
>
> On Sun, Mar 4, 2018 at 1:18 PM, Paul Rogers <pa...@yahoo.com.invalid>
> wrote:
>
> > Hi Parth,
> > Issues with DRILL-1170 are resolved. Needs one final review by Arina,
> then
> > we should be good to do.
> > Thanks to everyone for getting the other two "batch size" PRs committed
> > recently.
> > Thanks,
> > - Paul
> >
> >
> >
> > On Saturday, March 3, 2018, 10:57:10 PM PST, Parth Chandra <
> > parthc@apache.org> wrote:
> >
> > Thank you Arina !
> >
> > Build and unit tests with JDK 8 passed for me with the latest master. I'm
> > guessing there is still something more to be addressed, since DRILL-1491
> is
> > still open?
> >
> > Updated list:
> >
> > DRILL-1491: Support for JDK 8 --* In progress.*
> > DRILL-1170: YARN support for Drill -- Needs Committer +1 and Travis fix.
> > DRILL-6027: Implement spill to disk for the Hash Join --- No PR and is a
> > major feature that should be reviewed (properly!).
> > DRILL-6173: Support transitive closure during filter push down and
> > partition pruning. -- No PR and depends on 3 Apache Calcite issues that
> > are open.
> > DRILL-6023: Graceful shutdown improvements -- Marked as 1.14.0 (?)
> >
> >
> >
> > On Sun, Mar 4, 2018 at 12:35 AM, Arina Ielchiieva <ar...@apache.org>
> > wrote:
> >
> > > I will take care of FunctionInitializerTest#
> > > testConcurrentFunctionBodyLoad
> > > in DRILL-6208.
> > >
> > > Also I have merged all Jiras with ready-to-commit label.
> > > Drill on Yarn was NOT merged due to banned dependencies issue.
> > >
> > > Kind regards
> > > Arina
> > >
> > >
> > > On Sat, Mar 3, 2018 at 12:11 PM, Parth Chandra <pa...@apache.org>
> > wrote:
> > >
> > > > Thanks for the updates guys. Please keep updating the status in this
> > > > thread.
> > > >
> > > > On Fri, Mar 2, 2018 at 11:18 PM, Padma Penumarthy <
> > ppenumarthy@mapr.com>
> > > > wrote:
> > > >
> > > > > I can look at failure in testFlattenUpperLimit. I added this test
> > > > recently.
> > > > >
> > > > > Thanks
> > > > > Padma
> > > > >
> > > > >
> > > > > On Mar 2, 2018, at 8:19 AM, Volodymyr Tkach <vovatkach75@gmail.com
> > > > <mailto:
> > > > > vovatkach75@gmail.com>> wrote:
> > > > >
> > > > > testFlattenUpperLimit
> > > > >
> > > > >
> > > >
> > >
> >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
Sounds good.
On Sun, Mar 4, 2018 at 1:18 PM, Paul Rogers <pa...@yahoo.com.invalid>
wrote:
> Hi Parth,
> Issues with DRILL-1170 are resolved. Needs one final review by Arina, then
> we should be good to do.
> Thanks to everyone for getting the other two "batch size" PRs committed
> recently.
> Thanks,
> - Paul
>
>
>
> On Saturday, March 3, 2018, 10:57:10 PM PST, Parth Chandra <
> parthc@apache.org> wrote:
>
> Thank you Arina !
>
> Build and unit tests with JDK 8 passed for me with the latest master. I'm
> guessing there is still something more to be addressed, since DRILL-1491 is
> still open?
>
> Updated list:
>
> DRILL-1491: Support for JDK 8 --* In progress.*
> DRILL-1170: YARN support for Drill -- Needs Committer +1 and Travis fix.
> DRILL-6027: Implement spill to disk for the Hash Join --- No PR and is a
> major feature that should be reviewed (properly!).
> DRILL-6173: Support transitive closure during filter push down and
> partition pruning. -- No PR and depends on 3 Apache Calcite issues that
> are open.
> DRILL-6023: Graceful shutdown improvements -- Marked as 1.14.0 (?)
>
>
>
> On Sun, Mar 4, 2018 at 12:35 AM, Arina Ielchiieva <ar...@apache.org>
> wrote:
>
> > I will take care of FunctionInitializerTest#
> > testConcurrentFunctionBodyLoad
> > in DRILL-6208.
> >
> > Also I have merged all Jiras with ready-to-commit label.
> > Drill on Yarn was NOT merged due to banned dependencies issue.
> >
> > Kind regards
> > Arina
> >
> >
> > On Sat, Mar 3, 2018 at 12:11 PM, Parth Chandra <pa...@apache.org>
> wrote:
> >
> > > Thanks for the updates guys. Please keep updating the status in this
> > > thread.
> > >
> > > On Fri, Mar 2, 2018 at 11:18 PM, Padma Penumarthy <
> ppenumarthy@mapr.com>
> > > wrote:
> > >
> > > > I can look at failure in testFlattenUpperLimit. I added this test
> > > recently.
> > > >
> > > > Thanks
> > > > Padma
> > > >
> > > >
> > > > On Mar 2, 2018, at 8:19 AM, Volodymyr Tkach <vovatkach75@gmail.com
> > > <mailto:
> > > > vovatkach75@gmail.com>> wrote:
> > > >
> > > > testFlattenUpperLimit
> > > >
> > > >
> > >
> >
>
>
Re: [DISCUSS] 1.13.0 release
Posted by Paul Rogers <pa...@yahoo.com.INVALID>.
Hi Parth,
Issues with DRILL-1170 are resolved. Needs one final review by Arina, then we should be good to do.
Thanks to everyone for getting the other two "batch size" PRs committed recently.
Thanks,
- Paul
On Saturday, March 3, 2018, 10:57:10 PM PST, Parth Chandra <pa...@apache.org> wrote:
Thank you Arina !
Build and unit tests with JDK 8 passed for me with the latest master. I'm
guessing there is still something more to be addressed, since DRILL-1491 is
still open?
Updated list:
DRILL-1491: Support for JDK 8 --* In progress.*
DRILL-1170: YARN support for Drill -- Needs Committer +1 and Travis fix.
DRILL-6027: Implement spill to disk for the Hash Join --- No PR and is a
major feature that should be reviewed (properly!).
DRILL-6173: Support transitive closure during filter push down and
partition pruning. -- No PR and depends on 3 Apache Calcite issues that
are open.
DRILL-6023: Graceful shutdown improvements -- Marked as 1.14.0 (?)
On Sun, Mar 4, 2018 at 12:35 AM, Arina Ielchiieva <ar...@apache.org> wrote:
> I will take care of FunctionInitializerTest#
> testConcurrentFunctionBodyLoad
> in DRILL-6208.
>
> Also I have merged all Jiras with ready-to-commit label.
> Drill on Yarn was NOT merged due to banned dependencies issue.
>
> Kind regards
> Arina
>
>
> On Sat, Mar 3, 2018 at 12:11 PM, Parth Chandra <pa...@apache.org> wrote:
>
> > Thanks for the updates guys. Please keep updating the status in this
> > thread.
> >
> > On Fri, Mar 2, 2018 at 11:18 PM, Padma Penumarthy <pp...@mapr.com>
> > wrote:
> >
> > > I can look at failure in testFlattenUpperLimit. I added this test
> > recently.
> > >
> > > Thanks
> > > Padma
> > >
> > >
> > > On Mar 2, 2018, at 8:19 AM, Volodymyr Tkach <vovatkach75@gmail.com
> > <mailto:
> > > vovatkach75@gmail.com>> wrote:
> > >
> > > testFlattenUpperLimit
> > >
> > >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
Thank you Arina !
Build and unit tests with JDK 8 passed for me with the latest master. I'm
guessing there is still something more to be addressed, since DRILL-1491 is
still open?
Updated list:
DRILL-1491: Support for JDK 8 --* In progress.*
DRILL-1170: YARN support for Drill -- Needs Committer +1 and Travis fix.
DRILL-6027: Implement spill to disk for the Hash Join --- No PR and is a
major feature that should be reviewed (properly!).
DRILL-6173: Support transitive closure during filter push down and
partition pruning. -- No PR and depends on 3 Apache Calcite issues that
are open.
DRILL-6023: Graceful shutdown improvements -- Marked as 1.14.0 (?)
On Sun, Mar 4, 2018 at 12:35 AM, Arina Ielchiieva <ar...@apache.org> wrote:
> I will take care of FunctionInitializerTest#
> testConcurrentFunctionBodyLoad
> in DRILL-6208.
>
> Also I have merged all Jiras with ready-to-commit label.
> Drill on Yarn was NOT merged due to banned dependencies issue.
>
> Kind regards
> Arina
>
>
> On Sat, Mar 3, 2018 at 12:11 PM, Parth Chandra <pa...@apache.org> wrote:
>
> > Thanks for the updates guys. Please keep updating the status in this
> > thread.
> >
> > On Fri, Mar 2, 2018 at 11:18 PM, Padma Penumarthy <pp...@mapr.com>
> > wrote:
> >
> > > I can look at failure in testFlattenUpperLimit. I added this test
> > recently.
> > >
> > > Thanks
> > > Padma
> > >
> > >
> > > On Mar 2, 2018, at 8:19 AM, Volodymyr Tkach <vovatkach75@gmail.com
> > <mailto:
> > > vovatkach75@gmail.com>> wrote:
> > >
> > > testFlattenUpperLimit
> > >
> > >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Arina Ielchiieva <ar...@apache.org>.
I will take care of FunctionInitializerTest#testConcurrentFunctionBodyLoad
in DRILL-6208.
Also I have merged all Jiras with ready-to-commit label.
Drill on Yarn was NOT merged due to banned dependencies issue.
Kind regards
Arina
On Sat, Mar 3, 2018 at 12:11 PM, Parth Chandra <pa...@apache.org> wrote:
> Thanks for the updates guys. Please keep updating the status in this
> thread.
>
> On Fri, Mar 2, 2018 at 11:18 PM, Padma Penumarthy <pp...@mapr.com>
> wrote:
>
> > I can look at failure in testFlattenUpperLimit. I added this test
> recently.
> >
> > Thanks
> > Padma
> >
> >
> > On Mar 2, 2018, at 8:19 AM, Volodymyr Tkach <vovatkach75@gmail.com
> <mailto:
> > vovatkach75@gmail.com>> wrote:
> >
> > testFlattenUpperLimit
> >
> >
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
Thanks for the updates guys. Please keep updating the status in this thread.
On Fri, Mar 2, 2018 at 11:18 PM, Padma Penumarthy <pp...@mapr.com>
wrote:
> I can look at failure in testFlattenUpperLimit. I added this test recently.
>
> Thanks
> Padma
>
>
> On Mar 2, 2018, at 8:19 AM, Volodymyr Tkach <vovatkach75@gmail.com<mailto:
> vovatkach75@gmail.com>> wrote:
>
> testFlattenUpperLimit
>
>
Re: [DISCUSS] 1.13.0 release
Posted by Padma Penumarthy <pp...@mapr.com>.
I can look at failure in testFlattenUpperLimit. I added this test recently.
Thanks
Padma
On Mar 2, 2018, at 8:19 AM, Volodymyr Tkach <vo...@gmail.com>> wrote:
testFlattenUpperLimit
Re: [DISCUSS] 1.13.0 release
Posted by Volodymyr Tkach <vo...@gmail.com>.
*Update for:* DRILL-1491: Support for JDK 8
There is 1 failing
test FunctionInitializerTest#testConcurrentFunctionBodyLoad caused by
Jmokit library. Looking for replacement for mocking private method.
*Travis:*
Currenlty fails due to
testFlattenUpperLimit(org.apache.drill.exec.physical.unit.TestOutputBatchSize)
Time elapsed: 260.527 sec <<< ERROR!
java.lang.OutOfMemoryError: GC overhead limit exceeded
Tried to increase *-DdirectMemoryMb* to 6 and 8 gb, but it then fails due
to timeouts.
Functional, advanced and unit tests (except 1 described above) are fine.
2018-03-01 16:01 GMT+02:00 Parth Chandra <pa...@apache.org>:
> Good show.
>
> Updated list:
>
> DRILL-6185: Error is displaying while accessing query profiles via the
> Web-UI -- Ready to commit
> DRILL-6174: Parquet pushdown planning improvements -- Ready to commit
> DRILL-6191: Need more information on TCP flags -- Ready to commit
> DRILL-6190: Packets can be bigger than strictly legal -- Ready to commit
> DRILL-6188: Fix C++ client build on Centos 7 and OS X -- Ready to commit
>
> DRILL-1491: Support for JDK 8 --* In progress.*
>
> DRILL-1170: YARN support for Drill -- Needs Committer +1 and Travis fix.
>
> DRILL-6027: Implement spill to disk for the Hash Join --- No PR and is a
> major feature that should be reviewed (properly!).
>
> DRILL-6173: Support transitive closure during filter push down and
> partition pruning. -- No PR and depends on 3 Apache Calcite issues that
> are open.
>
> DRILL-6023: Graceful shutdown improvements -- No PR. Consists of 6 sub
> JIra's none of which have PRs.
>
Re: [DISCUSS] 1.13.0 release
Posted by Parth Chandra <pa...@apache.org>.
Good show.
Updated list:
DRILL-6185: Error is displaying while accessing query profiles via the
Web-UI -- Ready to commit
DRILL-6174: Parquet pushdown planning improvements -- Ready to commit
DRILL-6191: Need more information on TCP flags -- Ready to commit
DRILL-6190: Packets can be bigger than strictly legal -- Ready to commit
DRILL-6188: Fix C++ client build on Centos 7 and OS X -- Ready to commit
DRILL-1491: Support for JDK 8 --* In progress.*
DRILL-1170: YARN support for Drill -- Needs Committer +1 and Travis fix.
DRILL-6027: Implement spill to disk for the Hash Join --- No PR and is a
major feature that should be reviewed (properly!).
DRILL-6173: Support transitive closure during filter push down and
partition pruning. -- No PR and depends on 3 Apache Calcite issues that
are open.
DRILL-6023: Graceful shutdown improvements -- No PR. Consists of 6 sub
JIra's none of which have PRs.
Re: [DISCUSS] 1.13.0 release
Posted by Ted Dunning <te...@gmail.com>.
DRILL-6190 and DRILL-6191 are ready to merge to master for release.
Code review and unit tests all pass.
On Wed, Feb 28, 2018 at 11:16 PM, Parth Chandra <pa...@apache.org> wrote:
> Moved Ted's PR's down in the list. Let's see where we are at the end of the
> week.
> Arina, Volodymyr, ank ETA on JDK 8 work? It's the gating factor for the
> release.
> Meanwhile, people, feel free to commit your work as usual.
>
> Updated list:
>
> DRILL-6185: Error is displaying while accessing query profiles via the
> Web-UI -- Ready to commit
> DRILL-6174: Parquet pushdown planning improvements -- Ready to commit
> DRILL-6188: Fix C++ client build on Centos 7 and OS X -- Ready to commit
>
> DRILL-1491: Support for JDK 8 --* In progress.*
>
> DRILL-6191: Need more information on TCP flags -- *In progress*
>
> DRILL-6190: Packets can be bigger than strictly legal -- *In progress*
>
> DRILL-1170: YARN support for Drill -- Needs Committer +1 and Travis fix.
>
> DRILL-6027: Implement spill to disk for the Hash Join --- No PR and is a
> major feature that should be reviewed (properly!).
>
> DRILL-6173: Support transitive closure during filter push down and
> partition pruning. -- No PR and depends on 3 Apache Calcite issues that
> are open.
>
> DRILL-6023: Graceful shutdown improvements -- No PR. Consists of 6 sub
> JIra's none of which have PRs.
>
> On Wed, Feb 28, 2018 at 5:45 PM, Ted Dunning <te...@gmail.com>
> wrote:
>
> > 6190 and/or 6191 cause test failures that I have been unable to spend
> time
> > on yet. I don't think that they are ready to commit.
> >
> > At least one of these is likely to be something very simple like a test
> > that didn't clean up after itself. The other should be as simple, but I
> > can't understand it yet. It may be a memory pressure thing rather than a
> > real problem with the test.
> >
> >
> > On Wed, Feb 28, 2018 at 3:18 AM, Parth Chandra <pa...@apache.org>
> wrote:
> >
> > > OK. So let's try to get as many of the following as we can without
> > breaking
> > > anything. As far as I can see none of the open items below are show
> > > stoppers for a release, but I'm happy to give in to popular demand for
> > JDK
> > > 8 :).
> > >
> > > Note that the last three appear to be big ticket items that have no PR
> > yet.
> > > Usually, it is a mistake to rush these into a release (one advantage of
> > > frequent, predictable releases is that they won't have to wait too long
> > for
> > > the next release).
> > >
> > > Here's what I'm tracking :
> > >
> > > DRILL-6185: Error is displaying while accessing query profiles via the
> > > Web-UI -- Ready to commit
> > > DRILL-6174: Parquet pushdown planning improvements -- Ready to commit
> > > DRILL-6191: Need more information on TCP flags -- Ready to commit
> > > DRILL-6190: Packets can be bigger than strictly legal -- Ready to
> commit
> > >
> > > DRILL-6188: Fix C++ client build on Centos 7 and OS X -- Needs
> > committer
> > > +1
> > >
> > > DRILL-1491: Support for JDK 8 --* In progress.*
> > >
> > > DRILL-1170: YARN support for Drill -- Needs Committer +1 and Travis
> fix.
> > >
> > > DRILL-6027: Implement spill to disk for the Hash Join --- No PR and
> is
> > a
> > > major feature that should be reviewed (properly!).
> > >
> > > DRILL-6173: Support transitive closure during filter push down and
> > > partition pruning. -- No PR and depends on 3 Apache Calcite issues
> that
> > > are open.
> > >
> > > DRILL-6023: Graceful shutdown improvements -- No PR. Consists of 6 sub
> > > JIra's none of which have PRs.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Feb 28, 2018 at 12:32 AM, Ted Dunning <te...@gmail.com>
> > > wrote:
> > >
> > > > I have two very small improvements to PCAP support with DRILL-6190
> and
> > > > DRILL-6191 that I would like to get in.
> > > >
> > > > I think that PCAP-NG support is too far from ready.
> > > >
> > > >
> > > >
> > > > On Tue, Feb 27, 2018 at 10:52 AM, Pritesh Maker <pm...@mapr.com>
> > wrote:
> > > >
> > > > > I see a few more issues that are in review and worth including for
> > the
> > > > > 1.13 release (maybe give another week to resolve this before the
> 1st
> > RC
> > > > is
> > > > > created?)
> > > > >
> > > > > DRILL-6027 Implement spill to disk for the Hash Join -- Boaz and
> Tim
> > > > > DRILL-6173 Support transitive closure during filter push down and
> > > > > partition pruning - Vitalii
> > > > > DRILL-6023 Graceful shutdown improvements -- Jyothsna
> > > > >
> > > > > There are several other bugs/ improvements that are marked in
> > progress
> > > -
> > > > > https://issues.apache.org/jira/secure/Dashboard.jspa?
> > > > selectPageId=12332152
> > > > > - if folks are not working on them, we should remove the fixVersion
> > for
> > > > > 1.13.
> > > > >
> > > > > Pritesh
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Abhishek Girish <ag...@apache.org>
> > > > > Sent: February 27, 2018 10:44 AM
> > > > > To: dev@drill.apache.org
> > > > > Subject: Re: [DISCUSS] 1.13.0 release
> > > > >
> > > > > For JDK 8, we've run through both unit tests & regression tests
> from
> > > [1]
> > > > > and have observed no issues - so i think we should get the fixes
> into
> > > > > 1.13.0 and claim support for JDK 8.
> > > > >
> > > > >
> > > > > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.
> > > > > com_mapr_drill-2Dtest-2Dframework&d=DwIBaQ&c=
> > cskdkSMqhcnjZxdQVpwTXg&r=
> > > > > zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > > 1fr4O7tpGKl1SddY0&s=
> > > > > J6FGSG0CSPNSx1abvNe53qFNBwCr3UaO8ILqmUYfuWg&e=
> > > > >
> > > > > On Tue, Feb 27, 2018 at 9:46 AM, Aman Sinha <am...@apache.org>
> > > > wrote:
> > > > >
> > > > > > Agree with Arina on JDK8 support..let's get through the last
> > > remaining
> > > > > > hurdles for this for 1.13 release.
> > > > > > Calcite in fact is dropping support for JDK 7 in their next
> > release.
> > > > > > They are even looking ahead to JDK 9.
> > > > > >
> > > > > > For Drill-on-Yarn, I think ultimately it is at the discretion of
> > the
> > > > > > release manager. My personal opinion is as long as a committer
> (in
> > > > > > this case Arina) feels comfortable with it, and given that the
> > > > > > functionality has been well-tested, I am okay to include it.
> > > > > >
> > > > > > -Aman
> > > > > >
> > > > > > On Tue, Feb 27, 2018 at 8:51 AM, Arina Yelchiyeva <
> > > > > > arina.yelchiyeva@gmail.com> wrote:
> > > > > >
> > > > > > > I want to include DRILL-6174. It has already passed code
> review.
> > > > > > >
> > > > > > > Regarding JDK8 support. Volodymyr Tkach is working on this
> issue.
> > > > > > Currently
> > > > > > > all unit tests have passed. Now he is working on enforcing
> Java 8
> > > > > > (changes
> > > > > > > in travis.yml, drill-config.sh, pom.xml etc).
> > > > > > >
> > > > > > > Regarding Drill on Yarn, Salim has done code review. Failing
> > Travis
> > > > > > > check is easy to fix. Tim has already proposed the solution
> (Paul
> > > > > > > just needs to add the dependency).
> > > > > > > I think its safe to include these changes in this release. They
> > go
> > > > > > > in separate module and have no impact any existing
> functionality.
> > > > > > > Even if there are some flows, users will be able to give it a
> try
> > > > > > > and feedback in case of issues.
> > > > > > >
> > > > > > > On Tue, Feb 27, 2018 at 8:53 AM, Parth Chandra <
> > parthc@apache.org>
> > > > > > wrote:
> > > > > > >
> > > > > > > > There are two issues marked as blockers for 1.13.0 - TPatch
> > > > > > > > InfoKeySummaryReporterPStatusResolutionCreatedUpdatedDueFix
> > > > > > > > Version/s
> > > > > > > > Assignee
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_browse_DRILL-2D6185&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > > 1fr4O7tpGKl1Sd
> > > > > > > > dY0&s=ZFTj2m_MKC_o4_s9du-VuNWvzjx4ab8CJLGZMzhGkGg&e=>[image:
> > > Bug]
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_browse_DRILL-2D6185&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > > 1fr4O7tpGKl1Sd
> > > > > > > > dY0&s=ZFTj2m_MKC_o4_s9du-VuNWvzjx4ab8CJLGZMzhGkGg&e=>
> > DRILL-6185
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_browse_DRILL-2D6185&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > > 1fr4O7tpGKl1Sd
> > > > > > > > dY0&s=ZFTj2m_MKC_o4_s9du-VuNWvzjx4ab8CJLGZMzhGkGg&e=>
> > > > > > > >
> > > > > > > > Error is displaying while accessing query profiles via the
> > Web-UI
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_browse_DRILL-2D6185&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > > 1fr4O7tpGKl1Sd
> > > > > > > > dY0&s=ZFTj2m_MKC_o4_s9du-VuNWvzjx4ab8CJLGZMzhGkGg&e=>
> > > > > > > > Anton Gozhiy
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_secure_ViewProfile.jspa-3Fname-3Dangozhiy&d=
> > > DwIBaQ&c=cs
> > > > > > > > kdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=DWH_
> > > joEaRjyZJAvkuL
> > > > > > > > xyCK5ln-1fr4O7tpGKl1SddY0&s=RgS2wqYAu_
> > > VHlOHCeLA1RDhSkXdeB4iBFDoE5k
> > > > > > > > xlbAA&e=>
> > > > > > > > [image:
> > > > > > > > Blocker] OPEN *Unresolved* 26/Feb/18 26/Feb/18 1.13.0
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_issues_-3Fjql-3Dproject-2B-253D-2B&d=DwIBaQ&
> > > c=cskdkSMqh
> > > > > > > > cnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=DWH_
> > > joEaRjyZJAvkuLxyCK5ln
> > > > > > > > -1fr4O7tpGKl1SddY0&s=wWbDmtFEWvWuGsnOIe1NCWslFGraA6
> > > kS2EYn1NKpYsA&e
> > > > > > > > =
> > > > > > > > DRILL+AND+fixVersion+%3D+1.13.0>
> > > > > > > > Kunal
> > > > > > > > Khatua
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_secure_ViewProfile.jspa-3F&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQV
> > > > > > > > pwTXg&r=zySISmkmM4WNViCKijENtQ&m=DWH_
> > > joEaRjyZJAvkuLxyCK5ln-1fr4O7t
> > > > > > > > pGKl1SddY0&s=roeA-sW1VokgfvGn5dw37qWNIjBku2uKnpjCKJqaeg0&e=
> > > > > > > > name=kkhatua>
> > > > > > > > Actions
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_rest_api_1.0_issues_&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQVpwTXg&
> > > > > > > > r=zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > > 1fr4O7tpGKl1S
> > > > > > > > ddY0&s=4YiyewNsaFHgEIQkn3y3tfUgczHbSpkn-9WCR0EJjCM&e=
> > > > > > > > 13140915/ActionsAndOperations?atl_token=A5KQ-2QAV-T4JA-FDED|
> > > > > > > > 582202a248949006c997dc02e0361c023d2c491d|lin>
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_browse_DRILL-2D1491&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > > 1fr4O7tpGKl1Sd
> > > > > > > > dY0&s=WnrFj6sDbesjt-UWX1QEMR0fnZxNDREzvU6YtHaavvs&e=>[image:
> > > Task]
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_browse_DRILL-2D1491&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > > 1fr4O7tpGKl1Sd
> > > > > > > > dY0&s=WnrFj6sDbesjt-UWX1QEMR0fnZxNDREzvU6YtHaavvs&e=>
> > DRILL-1491
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_browse_DRILL-2D1491&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > > 1fr4O7tpGKl1Sd
> > > > > > > > dY0&s=WnrFj6sDbesjt-UWX1QEMR0fnZxNDREzvU6YtHaavvs&e=>
> > > > > > > >
> > > > > > > > Support for JDK 8
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_browse_DRILL-2D1491&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQVpwTXg&r
> > > > > > > > =zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > > 1fr4O7tpGKl1Sd
> > > > > > > > dY0&s=WnrFj6sDbesjt-UWX1QEMR0fnZxNDREzvU6YtHaavvs&e=>
> > > > > > > > Aditya Kishore
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_secure_ViewProfile.jspa-3F&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQV
> > > > > > > > pwTXg&r=zySISmkmM4WNViCKijENtQ&m=DWH_
> > > joEaRjyZJAvkuLxyCK5ln-1fr4O7t
> > > > > > > > pGKl1SddY0&s=roeA-sW1VokgfvGn5dw37qWNIjBku2uKnpjCKJqaeg0&e=
> > > > > > > name=adityakishore
> > > > > > > > >
> > > > > > > > [image:
> > > > > > > > Blocker] OPEN *Unresolved* 03/Oct/14 22/Feb/18 1.13.0
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_issues_-3Fjql-3Dproject-2B-253D-2B&d=DwIBaQ&
> > > c=cskdkSMqh
> > > > > > > > cnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m=DWH_
> > > joEaRjyZJAvkuLxyCK5ln
> > > > > > > > -1fr4O7tpGKl1SddY0&s=wWbDmtFEWvWuGsnOIe1NCWslFGraA6
> > > kS2EYn1NKpYsA&e
> > > > > > > > =
> > > > > > > > DRILL+AND+fixVersion+%3D+1.13.0>
> > > > > > > > Volodymyr
> > > > > > > > Tkach
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_secure_ViewProfile.jspa-3F&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQV
> > > > > > > > pwTXg&r=zySISmkmM4WNViCKijENtQ&m=DWH_
> > > joEaRjyZJAvkuLxyCK5ln-1fr4O7t
> > > > > > > > pGKl1SddY0&s=roeA-sW1VokgfvGn5dw37qWNIjBku2uKnpjCKJqaeg0&e=
> > > > > > > > name=volodymyr.tkach>
> > > > > > > > Actions
> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__issues.apach
> > > > > > > > e.org_jira_rest_api_1.0_issues_&d=DwIBaQ&c=
> > > cskdkSMqhcnjZxdQVpwTXg&
> > > > > > > > r=zySISmkmM4WNViCKijENtQ&m=DWH_joEaRjyZJAvkuLxyCK5ln-
> > > 1fr4O7tpGKl1S
> > > > > > > > ddY0&s=4YiyewNsaFHgEIQkn3y3tfUgczHbSpkn-9WCR0EJjCM&e=
> > > > > > > > 12745818/ActionsAndOperations?atl_token=A5KQ-2QAV-T4JA-FDED|
> > > > > > > > 582202a248949006c997dc02e0361c023d2c491d|lin>
> > > > > > > >
> > > > > > > > DRILL-6185 is definitely a blocker. We cannot do a release
> > with a
> > > > > > > > UI regression.
> > > > > > > >
> > > > > > > > From a quick perusal of DRILL-1491, JDK-8 support has issues
> > that
> > > > > > > > no
> > > > > > one
> > > > > > > is
> > > > > > > > looking at. Is someone addressing these and JIRA has not been
> > > > > updated?
> > > > > > > >
> > > > > > > > Also, - Drill 1170: Drill-on-YARN #1011, is failing Travis
> > > checks.
> > > > > > Call
> > > > > > > me
> > > > > > > > paranoid but a 21K line PR with one review comment doesn't
> > sound
> > > > > > > > like
> > > > > > the
> > > > > > > > review is completed. Any committer looking at it?
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Feb 27, 2018 at 9:28 AM, Ted Dunning
> > > > > > > > <te...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > There is a pcap improvement about to be ready. And a bug
> > (with
> > > a
> > > > > > > > > fix)
> > > > > > > > that
> > > > > > > > > just turned up. I will push for a PR
> > > > > > > > >
> > > > > > > > > On Feb 23, 2018 00:14, "Parth Chandra" <pa...@apache.org>
> > > > wrote:
> > > > > > > > >
> > > > > > > > > > Bit of a tepid response from dev; but Aman's approval is
> > all
> > > > > > > > > > the encouragement I need to roll out a release :)
> > > > > > > > > >
> > > > > > > > > > Thoughts on pending PRs?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Thu, Feb 22, 2018 at 9:54 PM, Aman Sinha
> > > > > > > > > > <am...@apache.org>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Agreed...it would be good to get the ball rolling on
> the
> > > > > > > > > > > 1.13.0
> > > > > > > > > release.
> > > > > > > > > > > Among other things, this release has the long pending
> > > > > > > > > > > Calcite rebase changes and the sooner we get
> > > > > > > it
> > > > > > > > it
> > > > > > > > > > out
> > > > > > > > > > > for users, the better.
> > > > > > > > > > >
> > > > > > > > > > > Thanks for volunteering !
> > > > > > > > > > >
> > > > > > > > > > > -Aman
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Feb 21, 2018 at 9:03 PM, Parth Chandra <
> > > > > > parthc@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hello Drillers,
> > > > > > > > > > > >
> > > > > > > > > > > > I feel we might benefit from a early release for
> > > 1.13.0.
> > > > > > > > > > > > We
> > > > > > > took
> > > > > > > > > > longer
> > > > > > > > > > > > to do the previous release so it would be nice to
> bring
> > > > > > > > > > > > the
> > > > > > > release
> > > > > > > > > > train
> > > > > > > > > > > > back on track.
> > > > > > > > > > > >
> > > > > > > > > > > > I'll volunteer (!) to manage the release :)
> > > > > > > > > > > >
> > > > > > > > > > > > What do you guys think?
> > > > > > > > > > > >
> > > > > > > > > > > > If we are in agreement on starting the release
> cycle
> > > and
> > > > > > there
> > > > > > > > are
> > > > > > > > > > any
> > > > > > > > > > > > issues on which work is in progress, that you feel we
> > > > > > > > > > > > *must*
> > > > > > > > include
> > > > > > > > > in
> > > > > > > > > > > the
> > > > > > > > > > > > release, please post in reply to this thread. Let's
> at
> > > > > > > > > > > > least
> > > > > > get
> > > > > > > a
> > > > > > > > > head
> > > > > > > > > > > > start on closing pending PRs since these are usually
> > what
> > > > > > delays
> > > > > > > > > > > releases.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks
> > > > > > > > > > > >
> > > > > > > > > > > > Parth
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>