You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@drill.apache.org by Jason Altekruse <al...@gmail.com> on 2016/01/04 22:48:37 UTC

Time for a 1.5 release?

Hello All,

With the allocator changes merged and about a month since the last release
I think it would be good to start a vote soon. I would like to volunteer to
be release manager.

I know that there were some issues that were identified after the transfer
patch was merged. I think that these issues should be fixed before we cut a
release candidate.

>From looking at the associated JIRAs it looked like there was a possible
short term fix just adjusting the max_query_memory_per_node option, and
some more involved work to change how we determine the correct time to
spill during external sort. I believe it makes sense to make external sort
work well with the newly improved memory accounting before cutting a
release, but I'm not sure how much work is left to be done there. [1]

Please respond with your thoughts on a release soon and any JIRAs you would
like to include in the release.

[1] - https://issues.apache.org/jira/browse/DRILL-4243

Thanks,
Jason

Re: Time for a 1.5 release?

Posted by Abdel Hakim Deneche <ad...@maprtech.com>.

I just noticed that we never fixed DRILL-4174 [1], it's a one line fix and
would like to get it in 1.5 release.

[1] https://issues.apache.org/jira/browse/DRILL-4174

On Mon, Jan 4, 2016 at 1:59 PM, Abdel Hakim Deneche <ad...@maprtech.com>
wrote:

> We are seeing an IllegalStateException in one of functional tests
> (flatten), I already reported it in DRILL-4246, I think this should be
> fixed as part of 1.5 release [1]
>
> [1] https://issues.apache.org/jira/browse/DRILL-4246
>
> On Mon, Jan 4, 2016 at 1:52 PM, Amit Hadke <am...@gmail.com> wrote:
>
>> https://issues.apache.org/jira/browse/DRILL-4190
>> https://issues.apache.org/jira/browse/DRILL-4196
>>
>> I'm working on a fix for both of these.
>>
>> On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <altekrusejason@gmail.com
>> >
>> wrote:
>>
>> > Hello All,
>> >
>> > With the allocator changes merged and about a month since the last
>> release
>> > I think it would be good to start a vote soon. I would like to
>> volunteer to
>> > be release manager.
>> >
>> > I know that there were some issues that were identified after the
>> transfer
>> > patch was merged. I think that these issues should be fixed before we
>> cut a
>> > release candidate.
>> >
>> > From looking at the associated JIRAs it looked like there was a possible
>> > short term fix just adjusting the max_query_memory_per_node option, and
>> > some more involved work to change how we determine the correct time to
>> > spill during external sort. I believe it makes sense to make external
>> sort
>> > work well with the newly improved memory accounting before cutting a
>> > release, but I'm not sure how much work is left to be done there. [1]
>> >
>> > Please respond with your thoughts on a release soon and any JIRAs you
>> would
>> > like to include in the release.
>> >
>> > [1] - https://issues.apache.org/jira/browse/DRILL-4243
>> >
>> > Thanks,
>> > Jason
>> >
>>
>
>
>
> --
>
> Abdelhakim Deneche
>
> Software Engineer
>
>   <http://www.mapr.com/>
>
>
> Now Available - Free Hadoop On-Demand Training
> <http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>
>



-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>

Re: Time for a 1.5 release?

Posted by Jason Altekruse <al...@gmail.com>.

I am working on a fix for Drill-4203 that I would like to have included in
1.5. A user trying to read dates out of a Drill produced parquet file from
spark was getting back incorrect data. Unfortunately we have been writing a
non-standard format for dates that we have just been reading correctly in
our implementation. I am working on correcting the dates automatically if
we know that the file was written by Drill. Will be opening a discussion on
the parquet list as well to see if they would be okay with including the
workaround in parquet-mr so other libraries could read the old files
correctly as well.

https://issues.apache.org/jira/browse/DRILL-4203


On Mon, Jan 4, 2016 at 3:59 PM, Abdel Hakim Deneche <ad...@maprtech.com>
wrote:

> We are seeing an IllegalStateException in one of functional tests
> (flatten), I already reported it in DRILL-4246, I think this should be
> fixed as part of 1.5 release [1]
>
> [1] https://issues.apache.org/jira/browse/DRILL-4246
>
> On Mon, Jan 4, 2016 at 1:52 PM, Amit Hadke <am...@gmail.com> wrote:
>
> > https://issues.apache.org/jira/browse/DRILL-4190
> > https://issues.apache.org/jira/browse/DRILL-4196
> >
> > I'm working on a fix for both of these.
> >
> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> altekrusejason@gmail.com>
> > wrote:
> >
> > > Hello All,
> > >
> > > With the allocator changes merged and about a month since the last
> > release
> > > I think it would be good to start a vote soon. I would like to
> volunteer
> > to
> > > be release manager.
> > >
> > > I know that there were some issues that were identified after the
> > transfer
> > > patch was merged. I think that these issues should be fixed before we
> > cut a
> > > release candidate.
> > >
> > > From looking at the associated JIRAs it looked like there was a
> possible
> > > short term fix just adjusting the max_query_memory_per_node option, and
> > > some more involved work to change how we determine the correct time to
> > > spill during external sort. I believe it makes sense to make external
> > sort
> > > work well with the newly improved memory accounting before cutting a
> > > release, but I'm not sure how much work is left to be done there. [1]
> > >
> > > Please respond with your thoughts on a release soon and any JIRAs you
> > would
> > > like to include in the release.
> > >
> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> > >
> > > Thanks,
> > > Jason
> > >
> >
>
>
>
> --
>
> Abdelhakim Deneche
>
> Software Engineer
>
>   <http://www.mapr.com/>
>
>
> Now Available - Free Hadoop On-Demand Training
> <
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >
>

Re: Time for a 1.5 release?

Posted by Abdel Hakim Deneche <ad...@maprtech.com>.

We are seeing an IllegalStateException in one of functional tests
(flatten), I already reported it in DRILL-4246, I think this should be
fixed as part of 1.5 release [1]

[1] https://issues.apache.org/jira/browse/DRILL-4246

On Mon, Jan 4, 2016 at 1:52 PM, Amit Hadke <am...@gmail.com> wrote:

> https://issues.apache.org/jira/browse/DRILL-4190
> https://issues.apache.org/jira/browse/DRILL-4196
>
> I'm working on a fix for both of these.
>
> On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <al...@gmail.com>
> wrote:
>
> > Hello All,
> >
> > With the allocator changes merged and about a month since the last
> release
> > I think it would be good to start a vote soon. I would like to volunteer
> to
> > be release manager.
> >
> > I know that there were some issues that were identified after the
> transfer
> > patch was merged. I think that these issues should be fixed before we
> cut a
> > release candidate.
> >
> > From looking at the associated JIRAs it looked like there was a possible
> > short term fix just adjusting the max_query_memory_per_node option, and
> > some more involved work to change how we determine the correct time to
> > spill during external sort. I believe it makes sense to make external
> sort
> > work well with the newly improved memory accounting before cutting a
> > release, but I'm not sure how much work is left to be done there. [1]
> >
> > Please respond with your thoughts on a release soon and any JIRAs you
> would
> > like to include in the release.
> >
> > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> >
> > Thanks,
> > Jason
> >
>



-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>

Re: Time for a 1.5 release?

Posted by Amit Hadke <am...@gmail.com>.

https://issues.apache.org/jira/browse/DRILL-4190
https://issues.apache.org/jira/browse/DRILL-4196

I'm working on a fix for both of these.

On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <al...@gmail.com>
wrote:

> Hello All,
>
> With the allocator changes merged and about a month since the last release
> I think it would be good to start a vote soon. I would like to volunteer to
> be release manager.
>
> I know that there were some issues that were identified after the transfer
> patch was merged. I think that these issues should be fixed before we cut a
> release candidate.
>
> From looking at the associated JIRAs it looked like there was a possible
> short term fix just adjusting the max_query_memory_per_node option, and
> some more involved work to change how we determine the correct time to
> spill during external sort. I believe it makes sense to make external sort
> work well with the newly improved memory accounting before cutting a
> release, but I'm not sure how much work is left to be done there. [1]
>
> Please respond with your thoughts on a release soon and any JIRAs you would
> like to include in the release.
>
> [1] - https://issues.apache.org/jira/browse/DRILL-4243
>
> Thanks,
> Jason
>

Re: Time for a 1.5 release?

Posted by Jason Altekruse <al...@gmail.com>.

I think 4247 is definitely worth trying to get into the release as it has
performance implications for a typical use case, partitioning by date. I
will be looking at this today to try to figure out what is causing it, I
don't anticipate it being difficult to fix.

I also agree 4256 should be fixed if we can. It is currently unassigned,
but Venki was asking about the environment setup. Venki, are you available
to take a closer look at this?

With Hakim's fixes on the external sort memory issues, Amit can you connect
with Vicky about testing your merge join fixes rebased on the tip of master?

I had some travel difficulties that kind of killed most of the end of last
week for me, so I haven't completed my fix for Drill-4203 the parquet date
issue. Anticipating a complete patch posted for review sometime today.

- Jason

On Mon, Jan 11, 2016 at 9:05 AM, Jason Altekruse <al...@gmail.com>
wrote:

> Kristine,
>
> I think that this would be a good thing to get fixed for the release, but
> it looks like we will need to mess with building or packaging different
> windows binaries to fix this issue. As far as I know Parth has been
> carrying the majority of this type of work in the past and had a bit of a
> hard time with it. I can try to see if I can get my hands on a 32 bit
> windows machine to try to get a better idea of the time required to get it
> fixed.
>
> - Jason
>
>
> On Tue, Jan 5, 2016 at 2:20 PM, Kristine Hahn <kh...@maprtech.com> wrote:
>
>> https://issues.apache.org/jira/browse/DRILL-4239 (Drill reported not to
>> work on 32-bit Windows) has been opened as a doc bug, but maybe we're
>> actually talking about a product bug. If so, how about a fix in 1.5?
>>
>> Kristine Hahn
>> Sr. Technical Writer
>> 415-497-8107 @krishahn skype:krishahn
>>
>>
>> On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <altekrusejason@gmail.com
>> >
>> wrote:
>>
>> > Hello All,
>> >
>> > With the allocator changes merged and about a month since the last
>> release
>> > I think it would be good to start a vote soon. I would like to
>> volunteer to
>> > be release manager.
>> >
>> > I know that there were some issues that were identified after the
>> transfer
>> > patch was merged. I think that these issues should be fixed before we
>> cut a
>> > release candidate.
>> >
>> > From looking at the associated JIRAs it looked like there was a possible
>> > short term fix just adjusting the max_query_memory_per_node option, and
>> > some more involved work to change how we determine the correct time to
>> > spill during external sort. I believe it makes sense to make external
>> sort
>> > work well with the newly improved memory accounting before cutting a
>> > release, but I'm not sure how much work is left to be done there. [1]
>> >
>> > Please respond with your thoughts on a release soon and any JIRAs you
>> would
>> > like to include in the release.
>> >
>> > [1] - https://issues.apache.org/jira/browse/DRILL-4243
>> >
>> > Thanks,
>> > Jason
>> >
>>
>
>

Re: Time for a 1.5 release?

Posted by Jason Altekruse <al...@gmail.com>.

Kristine,

I think that this would be a good thing to get fixed for the release, but
it looks like we will need to mess with building or packaging different
windows binaries to fix this issue. As far as I know Parth has been
carrying the majority of this type of work in the past and had a bit of a
hard time with it. I can try to see if I can get my hands on a 32 bit
windows machine to try to get a better idea of the time required to get it
fixed.

- Jason

On Tue, Jan 5, 2016 at 2:20 PM, Kristine Hahn <kh...@maprtech.com> wrote:

> https://issues.apache.org/jira/browse/DRILL-4239 (Drill reported not to
> work on 32-bit Windows) has been opened as a doc bug, but maybe we're
> actually talking about a product bug. If so, how about a fix in 1.5?
>
> Kristine Hahn
> Sr. Technical Writer
> 415-497-8107 @krishahn skype:krishahn
>
>
> On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <al...@gmail.com>
> wrote:
>
> > Hello All,
> >
> > With the allocator changes merged and about a month since the last
> release
> > I think it would be good to start a vote soon. I would like to volunteer
> to
> > be release manager.
> >
> > I know that there were some issues that were identified after the
> transfer
> > patch was merged. I think that these issues should be fixed before we
> cut a
> > release candidate.
> >
> > From looking at the associated JIRAs it looked like there was a possible
> > short term fix just adjusting the max_query_memory_per_node option, and
> > some more involved work to change how we determine the correct time to
> > spill during external sort. I believe it makes sense to make external
> sort
> > work well with the newly improved memory accounting before cutting a
> > release, but I'm not sure how much work is left to be done there. [1]
> >
> > Please respond with your thoughts on a release soon and any JIRAs you
> would
> > like to include in the release.
> >
> > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> >
> > Thanks,
> > Jason
> >
>

Re: Time for a 1.5 release?

Posted by Zelaine Fong <zf...@maprtech.com>.

What about https://issues.apache.org/jira/browse/DRILL-4247 and
https://issues.apache.org/jira/browse/DRILL-4256?  4256 appears to be a
regression.  Not sure about 4247.

-- Zelaine

On Tue, Jan 5, 2016 at 12:20 PM, Kristine Hahn <kh...@maprtech.com> wrote:

> https://issues.apache.org/jira/browse/DRILL-4239 (Drill reported not to
> work on 32-bit Windows) has been opened as a doc bug, but maybe we're
> actually talking about a product bug. If so, how about a fix in 1.5?
>
> Kristine Hahn
> Sr. Technical Writer
> 415-497-8107 @krishahn skype:krishahn
>
>
> On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <al...@gmail.com>
> wrote:
>
> > Hello All,
> >
> > With the allocator changes merged and about a month since the last
> release
> > I think it would be good to start a vote soon. I would like to volunteer
> to
> > be release manager.
> >
> > I know that there were some issues that were identified after the
> transfer
> > patch was merged. I think that these issues should be fixed before we
> cut a
> > release candidate.
> >
> > From looking at the associated JIRAs it looked like there was a possible
> > short term fix just adjusting the max_query_memory_per_node option, and
> > some more involved work to change how we determine the correct time to
> > spill during external sort. I believe it makes sense to make external
> sort
> > work well with the newly improved memory accounting before cutting a
> > release, but I'm not sure how much work is left to be done there. [1]
> >
> > Please respond with your thoughts on a release soon and any JIRAs you
> would
> > like to include in the release.
> >
> > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> >
> > Thanks,
> > Jason
> >
>

Re: Time for a 1.5 release?

Posted by Kristine Hahn <kh...@maprtech.com>.

https://issues.apache.org/jira/browse/DRILL-4239 (Drill reported not to
work on 32-bit Windows) has been opened as a doc bug, but maybe we're
actually talking about a product bug. If so, how about a fix in 1.5?

Kristine Hahn
Sr. Technical Writer
415-497-8107 @krishahn skype:krishahn


On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <al...@gmail.com>
wrote:

> Hello All,
>
> With the allocator changes merged and about a month since the last release
> I think it would be good to start a vote soon. I would like to volunteer to
> be release manager.
>
> I know that there were some issues that were identified after the transfer
> patch was merged. I think that these issues should be fixed before we cut a
> release candidate.
>
> From looking at the associated JIRAs it looked like there was a possible
> short term fix just adjusting the max_query_memory_per_node option, and
> some more involved work to change how we determine the correct time to
> spill during external sort. I believe it makes sense to make external sort
> work well with the newly improved memory accounting before cutting a
> release, but I'm not sure how much work is left to be done there. [1]
>
> Please respond with your thoughts on a release soon and any JIRAs you would
> like to include in the release.
>
> [1] - https://issues.apache.org/jira/browse/DRILL-4243
>
> Thanks,
> Jason
>

Re: Time for a 1.5 release?

Posted by Abhishek Girish <ag...@mapr.com>.

Jacques, I've scheduled Functional and Advanced regression runs against the
specified branch. Will let you know the results, once completed.

Regards,
Abhishek

On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com> wrote:

> 4196 was merged today. I have an updated patch for 4291 that is ready.
> Unfortunately, it seems that something isn't working with our extended
> tests so I haven't been able to run an extended regression. Unit tests
> pass. Is someone else possibly able to run a regression suite against this
> branch [1] so we can confirm things look good and start the release
> process?
>
> thanks,
> Jacques
>
> [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <ja...@dremio.com>
> wrote:
>
> > I think the main things are 4196 and 4291 should be completed. I know
> Amit
> > was able to reproduce 4196 locally this weekend so I think we're close on
> > that.
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <zf...@maprtech.com>
> wrote:
> >
> >> Any updates on this?  What's blocking us from taking this to a vote --
> the
> >> sort merge join issues?
> >>
> >> -- Zelaine
> >>
> >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <ja...@dremio.com>
> >> wrote:
> >>
> >> > Bumping this thread...
> >> >
> >> > Here are the issues that were mentioned in this thread along with a
> >> > proposed categorization:
> >> >
> >> > Release Blockers
> >> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4190
> >> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4196
> >> > Ready to merge Jacques
> https://issues.apache.org/jira/browse/DRILL-4246
> >> > In-review Jinfeng https://issues.apache.org/jira/browse/DRILL-4256
> >> > In-progress Jacques https://issues.apache.org/jira/browse/DRILL-4278
> >> > Ready to merge Laurent
> https://issues.apache.org/jira/browse/DRILL-4285
> >> > Nice to Have
> >> > Open Jason/Hakim https://issues.apache.org/jira/browse/DRILL-4247
> >> > In-progress Jason https://issues.apache.org/jira/browse/DRILL-4203
> >> > Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
> >> > Ready to merge Jacques
> https://issues.apache.org/jira/browse/DRILL-4131
> >> >
> >> > What do others think? Let's try to get the blockers wrapped up in the
> >> next
> >> > day or two and start a release vote...
> >> >
> >> >
> >> >
> >> > --
> >> > Jacques Nadeau
> >> > CTO and Co-Founder, Dremio
> >> >
> >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> >> altekrusejason@gmail.com>
> >> > wrote:
> >> >
> >> > > Hello All,
> >> > >
> >> > > With the allocator changes merged and about a month since the last
> >> > release
> >> > > I think it would be good to start a vote soon. I would like to
> >> volunteer
> >> > to
> >> > > be release manager.
> >> > >
> >> > > I know that there were some issues that were identified after the
> >> > transfer
> >> > > patch was merged. I think that these issues should be fixed before
> we
> >> > cut a
> >> > > release candidate.
> >> > >
> >> > > From looking at the associated JIRAs it looked like there was a
> >> possible
> >> > > short term fix just adjusting the max_query_memory_per_node option,
> >> and
> >> > > some more involved work to change how we determine the correct time
> to
> >> > > spill during external sort. I believe it makes sense to make
> external
> >> > sort
> >> > > work well with the newly improved memory accounting before cutting a
> >> > > release, but I'm not sure how much work is left to be done there.
> [1]
> >> > >
> >> > > Please respond with your thoughts on a release soon and any JIRAs
> you
> >> > would
> >> > > like to include in the release.
> >> > >
> >> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> >> > >
> >> > > Thanks,
> >> > > Jason
> >> > >
> >> >
> >>
> >
> >
>

Re: Time for a 1.5 release?

Posted by Venki Korukanti <ve...@gmail.com>.

Removing updates from StoragePlugin seems like a good approach. Will update
the patch soon.

On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse <al...@gmail.com>
wrote:

> We could revert the change to the StoragePlugin interface to add the
> getLogicalOptimizerRules
> and getPhysicalOptimizerRules, and only put those in AbstractStoragePlugin
> with default implementations.
>
> We could then update the docs to tell users to not implement the interface
> directly, and instead extend the abstract class, which I believe would
> allow us to make changes like this in the future without breaking plugins.
> At a major release version we could then just pull up all of the new
> methods we added to the abstract class into the interface, or just get rid
> of it all-together.
>
> On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
> venki.korukanti@gmail.com>
> wrote:
>
> > Here is the partial fix:
> >
> >
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> >
> > If the existing StragePlugins are implementing AbstractStoragePlugin
> class
> > this fix should resolve the backward compat issue. If they are
> implementing
> > StoragePlugin interface directly then it won't resolve the issue. Not
> sure
> > how to handle it in that case. Let me know if there any ways to resolve
> > this issue completely.
> >
> > Thanks
> > Venki
> >
> > On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <as...@maprtech.com> wrote:
> >
> > > I was able to do the build with maven 3.3.9 (after doing 'brew install
> > > maven').  Here are my env variables related to maven:
> > > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> > > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> > > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> > >
> > >
> > > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <ja...@dremio.com>
> > > wrote:
> > >
> > > > Aman, for reference on the Maven version issue. You can see the
> output
> > of
> > > > the shade plugin between the two Apache Builds. If you search for
> > > > maven-shade-plugin you'll see that build 647 (maven 3.0.5) fails
> > because
> > > it
> > > > includes a bunch of hadoop (and other libraries that are listed as
> > being
> > > > excluded). If you look at build 648 (maven 3.3.3), those items are
> > > included
> > > > so that the build completes successfully.
> > > >
> > > > https://builds.apache.org/job/drill-scm/647/consoleText
> > > > https://builds.apache.org/job/drill-scm/648/consoleText
> > > >
> > > > --
> > > > Jacques Nadeau
> > > > CTO and Co-Founder, Dremio
> > > >
> > > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
> > > > challapallirahul@gmail.com> wrote:
> > > >
> > > > > Jason,
> > > > >
> > > > > As of now I haven't narrowed down the exact commit which caused
> > > > DRILL-4323
> > > > > and I do not know the scope of the fix.
> > > > >
> > > > > - Rahul
> > > > >
> > > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
> > > > altekrusejason@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > I think it makes sense to wait for Venki's fix of the API, as
> > > > releasing a
> > > > > > version with the change would make reverting/refactoring the API
> > > > breaking
> > > > > > change less useful.
> > > > > >
> > > > > > Do you have any idea about the scope of fixing the hive native
> > > reader,
> > > > or
> > > > > > which change since the 1.4 release broke it? Is someone available
> > to
> > > > look
> > > > > > at it today?
> > > > > >
> > > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> > > > > > challapallirahul@gmail.com> wrote:
> > > > > >
> > > > > > > Just found an issue with hive native parquet reader
> (DRILL-4323).
> > > > This
> > > > > > is a
> > > > > > > regression from 1.4 and in my opinion should be treated as a
> > > blocker.
> > > > > > >
> > > > > > > - Rahul
> > > > > > >
> > > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> > > > > > > venki.korukanti@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Sorry I didn't realize I was breaking the public interface
> API.
> > > Let
> > > > > me
> > > > > > > work
> > > > > > > > on a patch to make it backward compatible.
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Venki
> > > > > > > >
> > > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <
> > > > steven@dremio.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > I just wanted to bring up an issue that I just now
> > discovered,
> > > > that
> > > > > > has
> > > > > > > > > caused me a fair amount of grief.
> > > > > > > > >
> > > > > > > > > https://github.com/apache/drill/pull/300/commits
> > > > > > > > >
> > > > > > > > > DRILL-4198 changes a user-facing API, and causes
> > StoragePlugins
> > > > > that
> > > > > > > were
> > > > > > > > > compiled against currently released versions of Drill to no
> > > > longer
> > > > > > > > > functional properly. I would prefer that this breaking
> change
> > > be
> > > > > > > modified
> > > > > > > > > to be backward compatible if possible.
> > > > > > > > >
> > > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> > > > > > > > > altekrusejason@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Aman,
> > > > > > > > > >
> > > > > > > > > > This is the failure that he was seeing. He figured out
> that
> > > the
> > > > > new
> > > > > > > > > > exclusions in jdbc-all were not being respected when the
> > > build
> > > > > was
> > > > > > > run
> > > > > > > > > with
> > > > > > > > > > an older Maven version, causing the jar size to increase
> > > > > > > significantly.
> > > > > > > > > He
> > > > > > > > > > added an enforcer to make sure the JAR didn't grow
> > > > unexpectedly.
> > > > > > Can
> > > > > > > > you
> > > > > > > > > > try to update your maven version and re-run the build?
> > > > > > > > > >
> > > > > > > > > > - Jason
> > > > > > > > > >
> > > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
> > > > > asinha@maprtech.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Jacques, I am getting the following build failure on
> the
> > > > latest
> > > > > > > > master
> > > > > > > > > > > branch...is this what you saw for the Apache build ?
> My
> > > mvn
> > > > > > > version
> > > > > > > > > > output
> > > > > > > > > > > is shown below.  Should we all be upgrading to a newer
> > mvn
> > > ?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > > > > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> > > > > > > > > > > [WARNING] Rule 0:
> > > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
> > > > > > > > > > failed
> > > > > > > > > > > with message:
> > > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside
> the
> > > > > > expected
> > > > > > > > size
> > > > > > > > > > > range.
> > > > > > > > > > >
> > > > > > > > > > >                   This is likely due to you adding new
> > > > > > dependencies
> > > > > > > > to
> > > > > > > > > a
> > > > > > > > > > > java-exec and not updating the excludes in this module.
> > > This
> > > > is
> > > > > > > > > important
> > > > > > > > > > > as it minimizes the size of the dependency of Drill
> > > > application
> > > > > > > > users.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > > > > > > > size (44664121) too large. Max. is
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill asinha$
> > mvn
> > > > > > > --version
> > > > > > > > > > > Apache Maven 3.0.5
> > > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > > > > > > > > 2013-02-19
> > > > > > > > > > > 05:51:28-0800)
> > > > > > > > > > > Maven home: /opt/local/share/java/maven3
> > > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle Corporation
> > > > > > > > > > > Java home:
> > > > > > > > > > >
> > > > > >
> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > > > > > > > > > > Default locale: en_US, platform encoding: UTF-8
> > > > > > > > > > > OS name: "mac os x", version: "10.9.5", arch: "x86_64",
> > > > family:
> > > > > > > "mac"
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <
> > > > > > > jacques@dremio.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Build back to normal. It looks like the Apache server
> > was
> > > > > using
> > > > > > > an
> > > > > > > > > old
> > > > > > > > > > > > version of Maven. Once I switched to something more
> > > recent,
> > > > > the
> > > > > > > > build
> > > > > > > > > > > > passed.
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Jacques Nadeau
> > > > > > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <
> > > > > > > > jacques@dremio.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hmm... this merge caused the Apache build to fail.
> > > > > > > > Investigating...
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > Jacques Nadeau
> > > > > > > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <
> > > > > > > > > jacques@dremio.com>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > >> I got clean regression runs as well. I've merged
> the
> > > > > patch.
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Jason, you want to start the release process?
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> --
> > > > > > > > > > > > >> Jacques Nadeau
> > > > > > > > > > > > >> CTO and Co-Founder, Dremio
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish
> <
> > > > > > > > > agirish@mapr.com
> > > > > > > > > > >
> > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > >>
> > > > > > > > > > > > >>> Had two clean Functional runs. TPC-H SF100 was
> also
> > > > > > > successful.
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul
> > challapalli <
> > > > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> > Kicked off a functional run with your branch.
> > Will
> > > > let
> > > > > > you
> > > > > > > > know
> > > > > > > > > > > once
> > > > > > > > > > > > it
> > > > > > > > > > > > >>> > finishes
> > > > > > > > > > > > >>> >
> > > > > > > > > > > > >>> > - Rahul
> > > > > > > > > > > > >>> >
> > > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques
> Nadeau <
> > > > > > > > > > > jacques@dremio.com>
> > > > > > > > > > > > >>> > wrote:
> > > > > > > > > > > > >>> >
> > > > > > > > > > > > >>> > > 4196 was merged today. I have an updated
> patch
> > > for
> > > > > 4291
> > > > > > > > that
> > > > > > > > > is
> > > > > > > > > > > > >>> ready.
> > > > > > > > > > > > >>> > > Unfortunately, it seems that something isn't
> > > > working
> > > > > > with
> > > > > > > > our
> > > > > > > > > > > > >>> extended
> > > > > > > > > > > > >>> > > tests so I haven't been able to run an
> extended
> > > > > > > regression.
> > > > > > > > > > Unit
> > > > > > > > > > > > >>> tests
> > > > > > > > > > > > >>> > > pass. Is someone else possibly able to run a
> > > > > regression
> > > > > > > > suite
> > > > > > > > > > > > against
> > > > > > > > > > > > >>> > this
> > > > > > > > > > > > >>> > > branch [1] so we can confirm things look good
> > and
> > > > > start
> > > > > > > the
> > > > > > > > > > > release
> > > > > > > > > > > > >>> > > process?
> > > > > > > > > > > > >>> > >
> > > > > > > > > > > > >>> > > thanks,
> > > > > > > > > > > > >>> > > Jacques
> > > > > > > > > > > > >>> > >
> > > > > > > > > > > > >>> > > [1]
> > > > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > > > > > > > > > > >>> > >
> > > > > > > > > > > > >>> > > --
> > > > > > > > > > > > >>> > > Jacques Nadeau
> > > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
> > > > > > > > > > > > >>> > >
> > > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques
> > Nadeau
> > > <
> > > > > > > > > > > > jacques@dremio.com
> > > > > > > > > > > > >>> >
> > > > > > > > > > > > >>> > > wrote:
> > > > > > > > > > > > >>> > >
> > > > > > > > > > > > >>> > > > I think the main things are 4196 and 4291
> > > should
> > > > be
> > > > > > > > > > completed.
> > > > > > > > > > > I
> > > > > > > > > > > > >>> know
> > > > > > > > > > > > >>> > > Amit
> > > > > > > > > > > > >>> > > > was able to reproduce 4196 locally this
> > weekend
> > > > so
> > > > > I
> > > > > > > > think
> > > > > > > > > > > we're
> > > > > > > > > > > > >>> close
> > > > > > > > > > > > >>> > on
> > > > > > > > > > > > >>> > > > that.
> > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > >>> > > > --
> > > > > > > > > > > > >>> > > > Jacques Nadeau
> > > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
> > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine
> > Fong
> > > <
> > > > > > > > > > > > zfong@maprtech.com
> > > > > > > > > > > > >>> >
> > > > > > > > > > > > >>> > > wrote:
> > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > >>> > > >> Any updates on this?  What's blocking us
> > from
> > > > > taking
> > > > > > > > this
> > > > > > > > > > to a
> > > > > > > > > > > > >>> vote --
> > > > > > > > > > > > >>> > > the
> > > > > > > > > > > > >>> > > >> sort merge join issues?
> > > > > > > > > > > > >>> > > >>
> > > > > > > > > > > > >>> > > >> -- Zelaine
> > > > > > > > > > > > >>> > > >>
> > > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques
> > > Nadeau
> > > > <
> > > > > > > > > > > > >>> jacques@dremio.com>
> > > > > > > > > > > > >>> > > >> wrote:
> > > > > > > > > > > > >>> > > >>
> > > > > > > > > > > > >>> > > >> > Bumping this thread...
> > > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > > >>> > > >> > Here are the issues that were mentioned
> in
> > > > this
> > > > > > > thread
> > > > > > > > > > along
> > > > > > > > > > > > >>> with a
> > > > > > > > > > > > >>> > > >> > proposed categorization:
> > > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > > >>> > > >> > Release Blockers
> > > > > > > > > > > > >>> > > >> > In-progress Amit
> > > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4190
> > > > > > > > > > > > >>> > > >> > In-progress Amit
> > > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4196
> > > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > > > > > > > >>> > >
> > https://issues.apache.org/jira/browse/DRILL-4246
> > > > > > > > > > > > >>> > > >> > In-review Jinfeng
> > > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4256
> > > > > > > > > > > > >>> > > >> > In-progress Jacques
> > > > > > > > > > > > >>> >
> https://issues.apache.org/jira/browse/DRILL-4278
> > > > > > > > > > > > >>> > > >> > Ready to merge Laurent
> > > > > > > > > > > > >>> > >
> > https://issues.apache.org/jira/browse/DRILL-4285
> > > > > > > > > > > > >>> > > >> > Nice to Have
> > > > > > > > > > > > >>> > > >> > Open Jason/Hakim
> > > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4247
> > > > > > > > > > > > >>> > > >> > In-progress Jason
> > > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4203
> > > > > > > > > > > > >>> > > >> > Open Jacques
> > > > > > > > > > > https://issues.apache.org/jira/browse/DRILL-4266
> > > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > > > > > > > >>> > >
> > https://issues.apache.org/jira/browse/DRILL-4131
> > > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > > >>> > > >> > What do others think? Let's try to get
> the
> > > > > > blockers
> > > > > > > > > > wrapped
> > > > > > > > > > > up
> > > > > > > > > > > > >>> in
> > > > > > > > > > > > >>> > the
> > > > > > > > > > > > >>> > > >> next
> > > > > > > > > > > > >>> > > >> > day or two and start a release vote...
> > > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > > >>> > > >> > --
> > > > > > > > > > > > >>> > > >> > Jacques Nadeau
> > > > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> > > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason
> > > > Altekruse
> > > > > <
> > > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
> > > > > > > > > > > > >>> > > >> > wrote:
> > > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > > >>> > > >> > > Hello All,
> > > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > > >>> > > >> > > With the allocator changes merged and
> > > about
> > > > a
> > > > > > > month
> > > > > > > > > > since
> > > > > > > > > > > > the
> > > > > > > > > > > > >>> last
> > > > > > > > > > > > >>> > > >> > release
> > > > > > > > > > > > >>> > > >> > > I think it would be good to start a
> vote
> > > > > soon. I
> > > > > > > > would
> > > > > > > > > > > like
> > > > > > > > > > > > to
> > > > > > > > > > > > >>> > > >> volunteer
> > > > > > > > > > > > >>> > > >> > to
> > > > > > > > > > > > >>> > > >> > > be release manager.
> > > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > > >>> > > >> > > I know that there were some issues
> that
> > > were
> > > > > > > > > identified
> > > > > > > > > > > > after
> > > > > > > > > > > > >>> the
> > > > > > > > > > > > >>> > > >> > transfer
> > > > > > > > > > > > >>> > > >> > > patch was merged. I think that these
> > > issues
> > > > > > should
> > > > > > > > be
> > > > > > > > > > > fixed
> > > > > > > > > > > > >>> before
> > > > > > > > > > > > >>> > > we
> > > > > > > > > > > > >>> > > >> > cut a
> > > > > > > > > > > > >>> > > >> > > release candidate.
> > > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > > >>> > > >> > > From looking at the associated JIRAs
> it
> > > > looked
> > > > > > > like
> > > > > > > > > > there
> > > > > > > > > > > > was
> > > > > > > > > > > > >>> a
> > > > > > > > > > > > >>> > > >> possible
> > > > > > > > > > > > >>> > > >> > > short term fix just adjusting the
> > > > > > > > > > > max_query_memory_per_node
> > > > > > > > > > > > >>> > option,
> > > > > > > > > > > > >>> > > >> and
> > > > > > > > > > > > >>> > > >> > > some more involved work to change how
> we
> > > > > > determine
> > > > > > > > the
> > > > > > > > > > > > correct
> > > > > > > > > > > > >>> > time
> > > > > > > > > > > > >>> > > to
> > > > > > > > > > > > >>> > > >> > > spill during external sort. I believe
> it
> > > > makes
> > > > > > > sense
> > > > > > > > > to
> > > > > > > > > > > make
> > > > > > > > > > > > >>> > > external
> > > > > > > > > > > > >>> > > >> > sort
> > > > > > > > > > > > >>> > > >> > > work well with the newly improved
> memory
> > > > > > > accounting
> > > > > > > > > > before
> > > > > > > > > > > > >>> > cutting a
> > > > > > > > > > > > >>> > > >> > > release, but I'm not sure how much
> work
> > is
> > > > > left
> > > > > > to
> > > > > > > > be
> > > > > > > > > > done
> > > > > > > > > > > > >>> there.
> > > > > > > > > > > > >>> > > [1]
> > > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > > >>> > > >> > > Please respond with your thoughts on a
> > > > release
> > > > > > > soon
> > > > > > > > > and
> > > > > > > > > > > any
> > > > > > > > > > > > >>> JIRAs
> > > > > > > > > > > > >>> > > you
> > > > > > > > > > > > >>> > > >> > would
> > > > > > > > > > > > >>> > > >> > > like to include in the release.
> > > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > > >>> > > >> > > [1] -
> > > > > > > > > https://issues.apache.org/jira/browse/DRILL-4243
> > > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > > >>> > > >> > > Thanks,
> > > > > > > > > > > > >>> > > >> > > Jason
> > > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > > >>> > > >>
> > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > >>> > >
> > > > > > > > > > > > >>> >
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>
> > > > > > > > > > > > >>
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Abdel Hakim Deneche <ad...@maprtech.com>.

Oh, I see. I didn't have to do it.

On Mon, Feb 1, 2016 at 1:42 PM, Jason Altekruse <al...@gmail.com>
wrote:

> Fair enough. I meant having my public GPG key signed by someone already in
> the Apache web of trust.
>
> On Mon, Feb 1, 2016 at 1:39 PM, Abdel Hakim Deneche <adeneche@maprtech.com
> > wrote:
>
>> I have trouble understanding the meaning of "GPG signed" =P
>>
>> On Mon, Feb 1, 2016 at 1:27 PM, Jason Altekruse <altekrusejason@gmail.com
>> > wrote:
>>
>>> A question for previous release managers. Jacques was thinking I may
>>> need to have my GPG signed to get linked into the Apache Web of Trust. Is
>>> this necessary for making a release?
>>>
>>> On Mon, Feb 1, 2016 at 11:24 AM, Jason Altekruse <
>>> altekrusejason@gmail.com> wrote:
>>>
>>>> The only solution to this issue I have found is disabling the new
>>>> plugin to automatically enforce the size of the jdbc-all plugin. I don't
>>>> know why it is behaving differently when performing a release than it did
>>>> during a 'mvn install', but I have verified the correct size of the JAR
>>>> manually and am planning on pushing a candidate soon with the small change
>>>> to disable the automatic jar size enforcement.
>>>>
>>>> I will open a JIRA for getting this resolved after the release.
>>>>
>>>> On Sun, Jan 31, 2016 at 10:46 PM, Jinfeng Ni <ji...@gmail.com>
>>>> wrote:
>>>>
>>>>> Good to hear this, Jacques & Jason. That makes feel relieved. :-)
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Jan 31, 2016 at 10:36 PM, Jason Altekruse
>>>>> <al...@gmail.com> wrote:
>>>>> > I don't think there was any issue with the merge either Jinfeng, I
>>>>> don't
>>>>> > think we need to close master. I had to step out for a few hours,
>>>>> but I am
>>>>> > working on finishing up the candidate right now.
>>>>> >
>>>>> > Having a weird RAT issue running the actual maven release command,
>>>>> it's
>>>>> > complaining about a dependency-reduced-pom.xml in the target
>>>>> directory of
>>>>> > jdbc-all lacking a license header (although target directories
>>>>> should be
>>>>> > ignored). Checking out the commit before the recent changes to the
>>>>> jdbc-all
>>>>> > exclusions seems to get rid of the issue, but the changes in the pom
>>>>> file
>>>>> > don't seem like they should be causing any problems.
>>>>> >
>>>>> > Still trying to debug and hoping to have a vote up tonight.
>>>>> >
>>>>> > On Sun, Jan 31, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com>
>>>>> wrote:
>>>>> >
>>>>> >> Given how easy it is to create a branch off a random commit, I
>>>>> generally
>>>>> >> don't see a point in closing the master branch. I figure the release
>>>>> >> manager can propose where they'd like to fork if this type of thing
>>>>> comes
>>>>> >> up.
>>>>> >>
>>>>> >> Thanks for the feedback Jinfeng. Of course you're right about
>>>>> Venki's
>>>>> >> commit. My bad.
>>>>> >>
>>>>> >> --
>>>>> >> Jacques Nadeau
>>>>> >> CTO and Co-Founder, Dremio
>>>>> >>
>>>>> >> On Sun, Jan 31, 2016 at 9:21 PM, Jinfeng Ni <ji...@gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> > Venki's backwards compatibility fix (commit id:
>>>>> >> >    03197d0) is before the partition pruning commit. So, it seems
>>>>> to be
>>>>> >> > fine to create a release candidate from 03197d0; no need to
>>>>> >> > cherry-pick.
>>>>> >> >
>>>>> >> > As for whether the partition pruning changes are risky, the
>>>>> pre-commit
>>>>> >> > run did not show any serious problems in the first run; all the
>>>>> plan
>>>>> >> > changes are expected, or expose some existing problem
>>>>> (drill-4279).
>>>>> >> > However, I agree that it impacts the main code path of query
>>>>> planning,
>>>>> >> > and I'm fine that we exclude the partition change from 1.5.0
>>>>> release,
>>>>> >> > since it's merged right before we start a voting.
>>>>> >> >
>>>>> >> > I did not see any one mentioned that the master branch is closed,
>>>>> and
>>>>> >> > therefore assume it's still open for commits.  I probably should
>>>>> have
>>>>> >> > asked whether the master was open for new patches before I merged
>>>>> the
>>>>> >> > commits.
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> > On Sun, Jan 31, 2016 at 8:43 PM, Jacques Nadeau <
>>>>> jacques@dremio.com>
>>>>> >> > wrote:
>>>>> >> > > Jinfeng, do you agree that the changes are risky? I was going
>>>>> mostly by
>>>>> >> > > commit messages.
>>>>> >> > >
>>>>> >> > > Also just realized that the backwards compatibility fix to the
>>>>> storage
>>>>> >> > > plugin interface is after these. Could create release branch
>>>>> from
>>>>> >> > 03197d...
>>>>> >> > > and then cherry pick the Venki change. Or could keep them all.
>>>>> Or
>>>>> >> > > go without the compatibility fix....
>>>>> >> > >
>>>>> >> > > --
>>>>> >> > > Jacques Nadeau
>>>>> >> > > CTO and Co-Founder, Dremio
>>>>> >> > >
>>>>> >> > > On Sun, Jan 31, 2016 at 7:50 PM, Jinfeng Ni <
>>>>> jinfengni99@gmail.com>
>>>>> >> > wrote:
>>>>> >> > >
>>>>> >> > >> Hi Jason,
>>>>> >> > >>
>>>>> >> > >> I'm fine that release 1.5.0 starts from 03197d0f2.  One thing
>>>>> I need
>>>>> >> > >> point out is that the pre-commit test suite has some
>>>>> testcase's plan
>>>>> >> > >> output modified because of the partition pruning change.  If
>>>>> we run
>>>>> >> > >> pre-commit against the candidate release for 1.5.0, we have to
>>>>> use a
>>>>> >> > >> prior version of pre-commit suite.
>>>>> >> > >>
>>>>> >> > >>
>>>>> >> > >>
>>>>> >> > >> On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse
>>>>> >> > >> <al...@gmail.com> wrote:
>>>>> >> > >> > That sounds reasonable to me. I'll start preparing a release
>>>>> >> > candidate.
>>>>> >> > >> >
>>>>> >> > >> > Jinfeng, are you okay waiting for the next release to
>>>>> include this
>>>>> >> > >> change?
>>>>> >> > >> >
>>>>> >> > >> > On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <
>>>>> jacques@dremio.com
>>>>> >> >
>>>>> >> > >> wrote:
>>>>> >> > >> >
>>>>> >> > >> >> Hey Jason,
>>>>> >> > >> >>
>>>>> >> > >> >> Can you start the release vote? We've let this drag on too
>>>>> long.
>>>>> >> > >> >>
>>>>> >> > >> >> I suggest we start from 03197d0f2. It seems like the
>>>>> partition
>>>>> >> > pruning
>>>>> >> > >> >> changes are pretty complicated and on a pretty main path to
>>>>> add
>>>>> >> right
>>>>> >> > >> >> before the vote. Since Hsuan's fix (pr 349) hasn't been
>>>>> reviewed, I
>>>>> >> > >> think
>>>>> >> > >> >> we should start the vote without it.
>>>>> >> > >> >>
>>>>> >> > >> >> Jacques
>>>>> >> > >> >>
>>>>> >> > >> >>
>>>>> >> > >> >> --
>>>>> >> > >> >> Jacques Nadeau
>>>>> >> > >> >> CTO and Co-Founder, Dremio
>>>>> >> > >> >>
>>>>> >> > >> >> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <
>>>>> hyichu@maprtech.com
>>>>> >> >
>>>>> >> > >> wrote:
>>>>> >> > >> >>
>>>>> >> > >> >> > Yes, just send a pull request.
>>>>> >> > >> >> > https://github.com/apache/drill/pull/349
>>>>> >> > >> >> >
>>>>> >> > >> >> > Can Jinfeng review it?
>>>>> >> > >> >> >
>>>>> >> > >> >> > Thanks,
>>>>> >> > >> >> >
>>>>> >> > >> >> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <
>>>>> >> jinfengni99@gmail.com
>>>>> >> > >
>>>>> >> > >> >> wrote:
>>>>> >> > >> >> >
>>>>> >> > >> >> > > Sean probably is quite close to find a fix for this
>>>>> issue.
>>>>> >> Let's
>>>>> >> > see
>>>>> >> > >> >> > > whether he can post a patch soon.
>>>>> >> > >> >> > >
>>>>> >> > >> >> > >
>>>>> >> > >> >> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
>>>>> >> > >> >> > > <ch...@gmail.com> wrote:
>>>>> >> > >> >> > > > DRILL-4323 only happens when assertions are turned on
>>>>> (my
>>>>> >> > install
>>>>> >> > >> >> > script
>>>>> >> > >> >> > > > automatically enables assertions).
>>>>> >> > >> >> > > >
>>>>> >> > >> >> > > > - Rahul
>>>>> >> > >> >> > > >
>>>>> >> > >> >> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <
>>>>> >> > >> zfong@maprtech.com>
>>>>> >> > >> >> > > wrote:
>>>>> >> > >> >> > > >
>>>>> >> > >> >> > > >> Rahul,
>>>>> >> > >> >> > > >>
>>>>> >> > >> >> > > >> Let us know if you can reproduce the issue with
>>>>> assertions
>>>>> >> > turned
>>>>> >> > >> >> OFF.
>>>>> >> > >> >> > > If
>>>>> >> > >> >> > > >> you cannot, then I think this should not be a
>>>>> blocker for
>>>>> >> 1.5,
>>>>> >> > >> and
>>>>> >> > >> >> > Hsuan
>>>>> >> > >> >> > > >> will continue to look into fixing this.
>>>>> >> > >> >> > > >>
>>>>> >> > >> >> > > >> -- Zelaine
>>>>> >> > >> >> > > >>
>>>>> >> > >> >> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
>>>>> >> > >> >> > knguyen@maprtech.com>
>>>>> >> > >> >> > > >> wrote:
>>>>> >> > >> >> > > >>
>>>>> >> > >> >> > > >> > I do see the same error with assertion turned on.
>>>>> >> > >> >> > > >> >
>>>>> >> > >> >> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <
>>>>> >> > >> >> jinfengni99@gmail.com>
>>>>> >> > >> >> > > >> wrote:
>>>>> >> > >> >> > > >> >
>>>>> >> > >> >> > > >> > > Krystal, by any chance, did you turn on
>>>>> assertion when
>>>>> >> you
>>>>> >> > >> start
>>>>> >> > >> >> > > >> > drillbit?
>>>>> >> > >> >> > > >> > >
>>>>> >> > >> >> > > >> > > I think the error in drill-4323 will be raised
>>>>> when
>>>>> >> > >> assertion =
>>>>> >> > >> >> > on.
>>>>> >> > >> >> > > >> > >
>>>>> >> > >> >> > > >> > >
>>>>> >> > >> >> > > >> > >
>>>>> >> > >> >> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
>>>>> >> > >> >> > > knguyen@maprtech.com>
>>>>> >> > >> >> > > >> > > wrote:
>>>>> >> > >> >> > > >> > > > I am running drill-1.4 and is able to run
>>>>> count(*)
>>>>> >> > >> >> successfully.
>>>>> >> > >> >> > > >> > > >
>>>>> >> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100>
>>>>> alter
>>>>> >> > >> session
>>>>> >> > >> >> > set
>>>>> >> > >> >> > > >> > > > `store.hive.optimize_scan_with_native_readers`
>>>>> = true;
>>>>> >> > >> >> > > >> > > >
>>>>> >> > >> >> >
>>>>> >> +-------+--------------------------------------------------------+
>>>>> >> > >> >> > > >> > > > |  ok   |                        summary
>>>>> >> > >> >> >  |
>>>>> >> > >> >> > > >> > > >
>>>>> >> > >> >> >
>>>>> >> +-------+--------------------------------------------------------+
>>>>> >> > >> >> > > >> > > > | true  |
>>>>> store.hive.optimize_scan_with_native_readers
>>>>> >> > >> >> > updated.  |
>>>>> >> > >> >> > > >> > > >
>>>>> >> > >> >> >
>>>>> >> +-------+--------------------------------------------------------+
>>>>> >> > >> >> > > >> > > > 1 row selected (0.374 seconds)
>>>>> >> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100>
>>>>> select
>>>>> >> > >> >> count(*)
>>>>> >> > >> >> > > from
>>>>> >> > >> >> > > >> > > > lineitem_parquet_partitioned_hive_string;
>>>>> >> > >> >> > > >> > > > +---------+
>>>>> >> > >> >> > > >> > > > | EXPR$0  |
>>>>> >> > >> >> > > >> > > > +---------+
>>>>> >> > >> >> > > >> > > > | 60175   |
>>>>> >> > >> >> > > >> > > > +---------+
>>>>> >> > >> >> > > >> > > > 1 row selected (2.986 seconds)
>>>>> >> > >> >> > > >> > > >
>>>>> >> > >> >> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
>>>>> >> > >> >> > > jinfengni99@gmail.com>
>>>>> >> > >> >> > > >> > > wrote:
>>>>> >> > >> >> > > >> > > >
>>>>> >> > >> >> > > >> > > >> Venki and I did some investigation for
>>>>> DRILL-4323.
>>>>> >> The
>>>>> >> > >> issue
>>>>> >> > >> >> > > >> reported
>>>>> >> > >> >> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0
>>>>> release as
>>>>> >> well.
>>>>> >> > >> Seems
>>>>> >> > >> >> > to
>>>>> >> > >> >> > > us
>>>>> >> > >> >> > > >> > > >> this is not a regression from 1.4.0; it's a
>>>>> >> regression
>>>>> >> > >> from
>>>>> >> > >> >> > 1.3.0
>>>>> >> > >> >> > > >> > > >> probably.
>>>>> >> > >> >> > > >> > > >>
>>>>> >> > >> >> > > >> > > >> DRILL-4083 makes the planner to use
>>>>> >> > DrillHiveNativeReader
>>>>> >> > >> in
>>>>> >> > >> >> > > stead
>>>>> >> > >> >> > > >> of
>>>>> >> > >> >> > > >> > > >> HiveReader for "select count(*) from
>>>>> hive_table"
>>>>> >> query.
>>>>> >> > >> >> > However,
>>>>> >> > >> >> > > >> the
>>>>> >> > >> >> > > >> > > >> Project after the scan produces empty
>>>>> schema.  Before
>>>>> >> > >> >> > DRILL-4083,
>>>>> >> > >> >> > > >> > > >> Drill uses HiveScan, which works fine.
>>>>> >> > >> >> > > >> > > >>
>>>>> >> > >> >> > > >> > > >>
>>>>> >> > >> >> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason
>>>>> Altekruse
>>>>> >> > >> >> > > >> > > >> <al...@gmail.com> wrote:
>>>>> >> > >> >> > > >> > > >> > We could revert the change to the
>>>>> StoragePlugin
>>>>> >> > >> interface
>>>>> >> > >> >> to
>>>>> >> > >> >> > > add
>>>>> >> > >> >> > > >> the
>>>>> >> > >> >> > > >> > > >> > getLogicalOptimizerRules
>>>>> >> > >> >> > > >> > > >> > and getPhysicalOptimizerRules, and only put
>>>>> those
>>>>> >> in
>>>>> >> > >> >> > > >> > > >> AbstractStoragePlugin
>>>>> >> > >> >> > > >> > > >> > with default implementations.
>>>>> >> > >> >> > > >> > > >> >
>>>>> >> > >> >> > > >> > > >> > We could then update the docs to tell users
>>>>> to not
>>>>> >> > >> >> implement
>>>>> >> > >> >> > > the
>>>>> >> > >> >> > > >> > > >> interface
>>>>> >> > >> >> > > >> > > >> > directly, and instead extend the abstract
>>>>> class,
>>>>> >> > which I
>>>>> >> > >> >> > > believe
>>>>> >> > >> >> > > >> > would
>>>>> >> > >> >> > > >> > > >> > allow us to make changes like this in the
>>>>> future
>>>>> >> > without
>>>>> >> > >> >> > > breaking
>>>>> >> > >> >> > > >> > > >> plugins.
>>>>> >> > >> >> > > >> > > >> > At a major release version we could then
>>>>> just pull
>>>>> >> up
>>>>> >> > >> all
>>>>> >> > >> >> of
>>>>> >> > >> >> > > the
>>>>> >> > >> >> > > >> new
>>>>> >> > >> >> > > >> > > >> > methods we added to the abstract class into
>>>>> the
>>>>> >> > >> interface,
>>>>> >> > >> >> or
>>>>> >> > >> >> > > just
>>>>> >> > >> >> > > >> > get
>>>>> >> > >> >> > > >> > > >> rid
>>>>> >> > >> >> > > >> > > >> > of it all-together.
>>>>> >> > >> >> > > >> > > >> >
>>>>> >> > >> >> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki
>>>>> Korukanti <
>>>>> >> > >> >> > > >> > > >> venki.korukanti@gmail.com>
>>>>> >> > >> >> > > >> > > >> > wrote:
>>>>> >> > >> >> > > >> > > >> >
>>>>> >> > >> >> > > >> > > >> >> Here is the partial fix:
>>>>> >> > >> >> > > >> > > >> >>
>>>>> >> > >> >> > > >> > > >> >>
>>>>> >> > >> >> > > >> > > >>
>>>>> >> > >> >> > > >> > >
>>>>> >> > >> >> > > >> >
>>>>> >> > >> >> > > >>
>>>>> >> > >> >> > >
>>>>> >> > >> >> >
>>>>> >> > >> >>
>>>>> >> > >>
>>>>> >> >
>>>>> >>
>>>>> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
>>>>> >> > >> >> > > >> > > >> >>
>>>>> >> > >> >> > > >> > > >> >> If the existing StragePlugins are
>>>>> implementing
>>>>> >> > >> >> > > >> > AbstractStoragePlugin
>>>>> >> > >> >> > > >> > > >> class
>>>>> >> > >> >> > > >> > > >> >> this fix should resolve the backward
>>>>> compat issue.
>>>>> >> > If
>>>>> >> > >> they
>>>>> >> > >> >> > are
>>>>> >> > >> >> > > >> > > >> implementing
>>>>> >> > >> >> > > >> > > >> >> StoragePlugin interface directly then it
>>>>> won't
>>>>> >> > resolve
>>>>> >> > >> the
>>>>> >> > >> >> > > issue.
>>>>> >> > >> >> > > >> > Not
>>>>> >> > >> >> > > >> > > >> sure
>>>>> >> > >> >> > > >> > > >> >> how to handle it in that case. Let me know
>>>>> if
>>>>> >> there
>>>>> >> > any
>>>>> >> > >> >> ways
>>>>> >> > >> >> > > to
>>>>> >> > >> >> > > >> > > resolve
>>>>> >> > >> >> > > >> > > >> >> this issue completely.
>>>>> >> > >> >> > > >> > > >> >>
>>>>> >> > >> >> > > >> > > >> >> Thanks
>>>>> >> > >> >> > > >> > > >> >> Venki
>>>>> >> > >> >> > > >> > > >> >>
>>>>> >> > >> >> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman
>>>>> Sinha <
>>>>> >> > >> >> > > asinha@maprtech.com
>>>>> >> > >> >> > > >> >
>>>>> >> > >> >> > > >> > > >> wrote:
>>>>> >> > >> >> > > >> > > >> >>
>>>>> >> > >> >> > > >> > > >> >> > I was able to do the build with maven
>>>>> 3.3.9
>>>>> >> (after
>>>>> >> > >> doing
>>>>> >> > >> >> > > 'brew
>>>>> >> > >> >> > > >> > > install
>>>>> >> > >> >> > > >> > > >> >> > maven').  Here are my env variables
>>>>> related to
>>>>> >> > maven:
>>>>> >> > >> >> > > >> > > >> >> >
>>>>> M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
>>>>> >> > >> >> > > >> > > >> >> >
>>>>> M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
>>>>> >> > >> >> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
>>>>> >> > >> >> > > >> > > >> >> >
>>>>> >> > >> >> > > >> > > >> >> >
>>>>> >> > >> >> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques
>>>>> Nadeau
>>>>> >> <
>>>>> >> > >> >> > > >> > > jacques@dremio.com>
>>>>> >> > >> >> > > >> > > >> >> > wrote:
>>>>> >> > >> >> > > >> > > >> >> >
>>>>> >> > >> >> > > >> > > >> >> > > Aman, for reference on the Maven
>>>>> version
>>>>> >> issue.
>>>>> >> > You
>>>>> >> > >> >> can
>>>>> >> > >> >> > > see
>>>>> >> > >> >> > > >> the
>>>>> >> > >> >> > > >> > > >> output
>>>>> >> > >> >> > > >> > > >> >> of
>>>>> >> > >> >> > > >> > > >> >> > > the shade plugin between the two Apache
>>>>> >> Builds.
>>>>> >> > If
>>>>> >> > >> you
>>>>> >> > >> >> > > search
>>>>> >> > >> >> > > >> > for
>>>>> >> > >> >> > > >> > > >> >> > > maven-shade-plugin you'll see that
>>>>> build 647
>>>>> >> > (maven
>>>>> >> > >> >> > 3.0.5)
>>>>> >> > >> >> > > >> > fails
>>>>> >> > >> >> > > >> > > >> >> because
>>>>> >> > >> >> > > >> > > >> >> > it
>>>>> >> > >> >> > > >> > > >> >> > > includes a bunch of hadoop (and other
>>>>> >> libraries
>>>>> >> > >> that
>>>>> >> > >> >> are
>>>>> >> > >> >> > > >> listed
>>>>> >> > >> >> > > >> > > as
>>>>> >> > >> >> > > >> > > >> >> being
>>>>> >> > >> >> > > >> > > >> >> > > excluded). If you look at build 648
>>>>> (maven
>>>>> >> > 3.3.3),
>>>>> >> > >> >> those
>>>>> >> > >> >> > > >> items
>>>>> >> > >> >> > > >> > > are
>>>>> >> > >> >> > > >> > > >> >> > included
>>>>> >> > >> >> > > >> > > >> >> > > so that the build completes
>>>>> successfully.
>>>>> >> > >> >> > > >> > > >> >> > >
>>>>> >> > >> >> > > >> > > >> >> > >
>>>>> >> > >> >> https://builds.apache.org/job/drill-scm/647/consoleText
>>>>> >> > >> >> > > >> > > >> >> > >
>>>>> >> > >> >> https://builds.apache.org/job/drill-scm/648/consoleText
>>>>> >> > >> >> > > >> > > >> >> > >
>>>>> >> > >> >> > > >> > > >> >> > > --
>>>>> >> > >> >> > > >> > > >> >> > > Jacques Nadeau
>>>>> >> > >> >> > > >> > > >> >> > > CTO and Co-Founder, Dremio
>>>>> >> > >> >> > > >> > > >> >> > >
>>>>> >> > >> >> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul
>>>>> >> > challapalli
>>>>> >> > >> <
>>>>> >> > >> >> > > >> > > >> >> > > challapallirahul@gmail.com> wrote:
>>>>> >> > >> >> > > >> > > >> >> > >
>>>>> >> > >> >> > > >> > > >> >> > > > Jason,
>>>>> >> > >> >> > > >> > > >> >> > > >
>>>>> >> > >> >> > > >> > > >> >> > > > As of now I haven't narrowed down
>>>>> the exact
>>>>> >> > >> commit
>>>>> >> > >> >> > which
>>>>> >> > >> >> > > >> > caused
>>>>> >> > >> >> > > >> > > >> >> > > DRILL-4323
>>>>> >> > >> >> > > >> > > >> >> > > > and I do not know the scope of the
>>>>> fix.
>>>>> >> > >> >> > > >> > > >> >> > > >
>>>>> >> > >> >> > > >> > > >> >> > > > - Rahul
>>>>> >> > >> >> > > >> > > >> >> > > >
>>>>> >> > >> >> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM,
>>>>> Jason
>>>>> >> > Altekruse
>>>>> >> > >> <
>>>>> >> > >> >> > > >> > > >> >> > > altekrusejason@gmail.com
>>>>> >> > >> >> > > >> > > >> >> > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > wrote:
>>>>> >> > >> >> > > >> > > >> >> > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > I think it makes sense to wait for
>>>>> Venki's
>>>>> >> > fix
>>>>> >> > >> of
>>>>> >> > >> >> > the
>>>>> >> > >> >> > > >> API,
>>>>> >> > >> >> > > >> > as
>>>>> >> > >> >> > > >> > > >> >> > > releasing a
>>>>> >> > >> >> > > >> > > >> >> > > > > version with the change would make
>>>>> >> > >> >> > > reverting/refactoring
>>>>> >> > >> >> > > >> > the
>>>>> >> > >> >> > > >> > > API
>>>>> >> > >> >> > > >> > > >> >> > > breaking
>>>>> >> > >> >> > > >> > > >> >> > > > > change less useful.
>>>>> >> > >> >> > > >> > > >> >> > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > Do you have any idea about the
>>>>> scope of
>>>>> >> > fixing
>>>>> >> > >> the
>>>>> >> > >> >> > > hive
>>>>> >> > >> >> > > >> > > native
>>>>> >> > >> >> > > >> > > >> >> > reader,
>>>>> >> > >> >> > > >> > > >> >> > > or
>>>>> >> > >> >> > > >> > > >> >> > > > > which change since the 1.4 release
>>>>> broke
>>>>> >> > it? Is
>>>>> >> > >> >> > > someone
>>>>> >> > >> >> > > >> > > >> available
>>>>> >> > >> >> > > >> > > >> >> to
>>>>> >> > >> >> > > >> > > >> >> > > look
>>>>> >> > >> >> > > >> > > >> >> > > > > at it today?
>>>>> >> > >> >> > > >> > > >> >> > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM,
>>>>> rahul
>>>>> >> > >> >> challapalli <
>>>>> >> > >> >> > > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > Just found an issue with hive
>>>>> native
>>>>> >> > parquet
>>>>> >> > >> >> > reader
>>>>> >> > >> >> > > >> > > >> (DRILL-4323).
>>>>> >> > >> >> > > >> > > >> >> > > This
>>>>> >> > >> >> > > >> > > >> >> > > > > is a
>>>>> >> > >> >> > > >> > > >> >> > > > > > regression from 1.4 and in my
>>>>> opinion
>>>>> >> > should
>>>>> >> > >> be
>>>>> >> > >> >> > > treated
>>>>> >> > >> >> > > >> > as
>>>>> >> > >> >> > > >> > > a
>>>>> >> > >> >> > > >> > > >> >> > blocker.
>>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > - Rahul
>>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM,
>>>>> Venki
>>>>> >> > >> >> Korukanti <
>>>>> >> > >> >> > > >> > > >> >> > > > > > venki.korukanti@gmail.com>
>>>>> >> > >> >> > > >> > > >> >> > > > > > wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > Sorry I didn't realize I was
>>>>> breaking
>>>>> >> > the
>>>>> >> > >> >> public
>>>>> >> > >> >> > > >> > > interface
>>>>> >> > >> >> > > >> > > >> API.
>>>>> >> > >> >> > > >> > > >> >> > Let
>>>>> >> > >> >> > > >> > > >> >> > > > me
>>>>> >> > >> >> > > >> > > >> >> > > > > > work
>>>>> >> > >> >> > > >> > > >> >> > > > > > > on a patch to make it backward
>>>>> >> > compatible.
>>>>> >> > >> >> > > >> > > >> >> > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > Thanks
>>>>> >> > >> >> > > >> > > >> >> > > > > > > Venki
>>>>> >> > >> >> > > >> > > >> >> > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54
>>>>> PM,
>>>>> >> Steven
>>>>> >> > >> >> > Phillips
>>>>> >> > >> >> > > <
>>>>> >> > >> >> > > >> > > >> >> > > steven@dremio.com
>>>>> >> > >> >> > > >> > > >> >> > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > I just wanted to bring up an
>>>>> issue
>>>>> >> > that I
>>>>> >> > >> >> just
>>>>> >> > >> >> > > now
>>>>> >> > >> >> > > >> > > >> >> discovered,
>>>>> >> > >> >> > > >> > > >> >> > > that
>>>>> >> > >> >> > > >> > > >> >> > > > > has
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > caused me a fair amount of
>>>>> grief.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>>> >> > >> >> > > https://github.com/apache/drill/pull/300/commits
>>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > DRILL-4198 changes a
>>>>> user-facing
>>>>> >> API,
>>>>> >> > and
>>>>> >> > >> >> > causes
>>>>> >> > >> >> > > >> > > >> >> StoragePlugins
>>>>> >> > >> >> > > >> > > >> >> > > > that
>>>>> >> > >> >> > > >> > > >> >> > > > > > were
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > compiled against currently
>>>>> released
>>>>> >> > >> versions
>>>>> >> > >> >> > of
>>>>> >> > >> >> > > >> Drill
>>>>> >> > >> >> > > >> > > to
>>>>> >> > >> >> > > >> > > >> no
>>>>> >> > >> >> > > >> > > >> >> > > longer
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > functional properly. I would
>>>>> prefer
>>>>> >> > that
>>>>> >> > >> >> this
>>>>> >> > >> >> > > >> > breaking
>>>>> >> > >> >> > > >> > > >> change
>>>>> >> > >> >> > > >> > > >> >> > be
>>>>> >> > >> >> > > >> > > >> >> > > > > > modified
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > to be backward compatible if
>>>>> >> possible.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at
>>>>> 11:23 AM,
>>>>> >> > Jason
>>>>> >> > >> >> > > Altekruse <
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > Hi Aman,
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > This is the failure that
>>>>> he was
>>>>> >> > >> seeing. He
>>>>> >> > >> >> > > >> figured
>>>>> >> > >> >> > > >> > > out
>>>>> >> > >> >> > > >> > > >> that
>>>>> >> > >> >> > > >> > > >> >> > the
>>>>> >> > >> >> > > >> > > >> >> > > > new
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all
>>>>> were not
>>>>> >> > being
>>>>> >> > >> >> > > respected
>>>>> >> > >> >> > > >> > when
>>>>> >> > >> >> > > >> > > the
>>>>> >> > >> >> > > >> > > >> >> > build
>>>>> >> > >> >> > > >> > > >> >> > > > was
>>>>> >> > >> >> > > >> > > >> >> > > > > > run
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > with
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > an older Maven version,
>>>>> causing
>>>>> >> the
>>>>> >> > jar
>>>>> >> > >> >> size
>>>>> >> > >> >> > > to
>>>>> >> > >> >> > > >> > > increase
>>>>> >> > >> >> > > >> > > >> >> > > > > > significantly.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > He
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > added an enforcer to make
>>>>> sure the
>>>>> >> > JAR
>>>>> >> > >> >> > didn't
>>>>> >> > >> >> > > >> grow
>>>>> >> > >> >> > > >> > > >> >> > > unexpectedly.
>>>>> >> > >> >> > > >> > > >> >> > > > > Can
>>>>> >> > >> >> > > >> > > >> >> > > > > > > you
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > try to update your maven
>>>>> version
>>>>> >> and
>>>>> >> > >> >> re-run
>>>>> >> > >> >> > > the
>>>>> >> > >> >> > > >> > > build?
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > - Jason
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at
>>>>> 11:18 AM,
>>>>> >> > Aman
>>>>> >> > >> >> > Sinha <
>>>>> >> > >> >> > > >> > > >> >> > > > asinha@maprtech.com>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the
>>>>> >> > following
>>>>> >> > >> >> build
>>>>> >> > >> >> > > >> failure
>>>>> >> > >> >> > > >> > > on
>>>>> >> > >> >> > > >> > > >> the
>>>>> >> > >> >> > > >> > > >> >> > > latest
>>>>> >> > >> >> > > >> > > >> >> > > > > > > master
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > branch...is this what
>>>>> you saw
>>>>> >> for
>>>>> >> > the
>>>>> >> > >> >> > Apache
>>>>> >> > >> >> > > >> > build
>>>>> >> > >> >> > > >> > > ?
>>>>> >> > >> >> > > >> > > >> My
>>>>> >> > >> >> > > >> > > >> >> > mvn
>>>>> >> > >> >> > > >> > > >> >> > > > > > version
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > output
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > is shown below.  Should
>>>>> we all
>>>>> >> be
>>>>> >> > >> >> > upgrading
>>>>> >> > >> >> > > to
>>>>> >> > >> >> > > >> a
>>>>> >> > >> >> > > >> > > newer
>>>>> >> > >> >> > > >> > > >> >> mvn
>>>>> >> > >> >> > > >> > > >> >> > ?
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > [INFO] ---
>>>>> >> > >> >> > > maven-enforcer-plugin:1.3.1:enforce
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> (enforce-jdbc-jar-compactness) @
>>>>> >> > >> >> > > drill-jdbc-all
>>>>> >> > >> >> > > >> > ---
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
>>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>>> >> > >> >> org.apache.maven.plugins.enforcer.RequireFilesSize
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > failed
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > with message:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > The file
>>>>> >> > >> >> drill-jdbc-all-1.5.0-SNAPSHOT.jar
>>>>> >> > >> >> > > is
>>>>> >> > >> >> > > >> > > outside
>>>>> >> > >> >> > > >> > > >> the
>>>>> >> > >> >> > > >> > > >> >> > > > > expected
>>>>> >> > >> >> > > >> > > >> >> > > > > > > size
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > range.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >                   This
>>>>> is likely
>>>>> >> > due
>>>>> >> > >> to
>>>>> >> > >> >> > you
>>>>> >> > >> >> > > >> > adding
>>>>> >> > >> >> > > >> > > new
>>>>> >> > >> >> > > >> > > >> >> > > > > dependencies
>>>>> >> > >> >> > > >> > > >> >> > > > > > > to
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > a
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > java-exec and not
>>>>> updating the
>>>>> >> > >> excludes
>>>>> >> > >> >> in
>>>>> >> > >> >> > > this
>>>>> >> > >> >> > > >> > > >> module.
>>>>> >> > >> >> > > >> > > >> >> > This
>>>>> >> > >> >> > > >> > > >> >> > > is
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > important
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > as it minimizes the size
>>>>> of the
>>>>> >> > >> >> dependency
>>>>> >> > >> >> > > of
>>>>> >> > >> >> > > >> > Drill
>>>>> >> > >> >> > > >> > > >> >> > > application
>>>>> >> > >> >> > > >> > > >> >> > > > > > > users.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > >
>>>>> >> > >> >> > > >> > > >> >> > > >
>>>>> >> > >> >> > > >> > > >> >> > >
>>>>> >> > >> >> > > >> > > >> >> >
>>>>> >> > >> >> > > >> > > >> >>
>>>>> >> > >> >> > > >> > > >>
>>>>> >> > >> >> > > >> > >
>>>>> >> > >> >> > > >> >
>>>>> >> > >> >> > > >>
>>>>> >> > >> >> > >
>>>>> >> > >> >> >
>>>>> >> > >> >>
>>>>> >> > >>
>>>>> >> >
>>>>> >>
>>>>> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > size (44664121) too
>>>>> large. Max.
>>>>> >> is
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > >
>>>>> >> > >> >> > > >> > > >> >> > > >
>>>>> >> > >> >> > > >> > > >> >> > >
>>>>> >> > >> >> > > >> > > >> >> >
>>>>> >> > >> >> > > >> > > >> >>
>>>>> >> > >> >> > > >> > > >>
>>>>> >> > >> >> > > >> > >
>>>>> >> > >> >> > > >> >
>>>>> >> > >> >> > > >>
>>>>> >> > >> >> > >
>>>>> >> > >> >> >
>>>>> >> > >> >>
>>>>> >> > >>
>>>>> >> >
>>>>> >>
>>>>> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > Administrators-MacBook-Pro-144:incubator-drill
>>>>> >> > >> >> > > >> > > asinha$
>>>>> >> > >> >> > > >> > > >> >> mvn
>>>>> >> > >> >> > > >> > > >> >> > > > > > --version
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
>>>>> >> > >> >> > > >> > > >> >> > >
>>>>> (r01de14724cdef164cd33c7c8c2fe155faf9602da;
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > 2013-02-19
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > 05:51:28-0800)
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Maven home:
>>>>> >> > >> /opt/local/share/java/maven3
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45,
>>>>> vendor:
>>>>> >> > >> Oracle
>>>>> >> > >> >> > > >> > Corporation
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Java home:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > >
>>>>> >> > >> >> > > >> > > >>
>>>>> >> > >> >> > >
>>>>> >> >
>>>>> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Default locale: en_US,
>>>>> platform
>>>>> >> > >> >> encoding:
>>>>> >> > >> >> > > UTF-8
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > OS name: "mac os x",
>>>>> version:
>>>>> >> > >> "10.9.5",
>>>>> >> > >> >> > > arch:
>>>>> >> > >> >> > > >> > > >> "x86_64",
>>>>> >> > >> >> > > >> > > >> >> > > family:
>>>>> >> > >> >> > > >> > > >> >> > > > > > "mac"
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at
>>>>> 8:20 AM,
>>>>> >> > >> Jacques
>>>>> >> > >> >> > > >> Nadeau <
>>>>> >> > >> >> > > >> > > >> >> > > > > > jacques@dremio.com>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > Build back to normal.
>>>>> It looks
>>>>> >> > like
>>>>> >> > >> >> the
>>>>> >> > >> >> > > >> Apache
>>>>> >> > >> >> > > >> > > >> server
>>>>> >> > >> >> > > >> > > >> >> was
>>>>> >> > >> >> > > >> > > >> >> > > > using
>>>>> >> > >> >> > > >> > > >> >> > > > > > an
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > old
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > version of Maven. Once
>>>>> I
>>>>> >> > switched
>>>>> >> > >> to
>>>>> >> > >> >> > > >> something
>>>>> >> > >> >> > > >> > > more
>>>>> >> > >> >> > > >> > > >> >> > recent,
>>>>> >> > >> >> > > >> > > >> >> > > > the
>>>>> >> > >> >> > > >> > > >> >> > > > > > > build
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > passed.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > --
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder,
>>>>> Dremio
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016
>>>>> at 7:02
>>>>> >> AM,
>>>>> >> > >> >> Jacques
>>>>> >> > >> >> > > >> > Nadeau <
>>>>> >> > >> >> > > >> > > >> >> > > > > > > jacques@dremio.com>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge
>>>>> caused the
>>>>> >> > >> Apache
>>>>> >> > >> >> > > build
>>>>> >> > >> >> > > >> to
>>>>> >> > >> >> > > >> > > fail.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > Investigating...
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > --
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder,
>>>>> Dremio
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016
>>>>> at 6:31
>>>>> >> > AM,
>>>>> >> > >> >> > Jacques
>>>>> >> > >> >> > > >> > > Nadeau <
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > jacques@dremio.com>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> I got clean
>>>>> regression runs
>>>>> >> > as
>>>>> >> > >> >> well.
>>>>> >> > >> >> > > I've
>>>>> >> > >> >> > > >> > > merged
>>>>> >> > >> >> > > >> > > >> the
>>>>> >> > >> >> > > >> > > >> >> > > > patch.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to
>>>>> start
>>>>> >> the
>>>>> >> > >> >> release
>>>>> >> > >> >> > > >> > process?
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> --
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder,
>>>>> Dremio
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27,
>>>>> 2016 at
>>>>> >> 10:42
>>>>> >> > >> PM,
>>>>> >> > >> >> > > Abhishek
>>>>> >> > >> >> > > >> > > >> Girish <
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > agirish@mapr.com
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> Had two clean
>>>>> Functional
>>>>> >> > runs.
>>>>> >> > >> >> TPC-H
>>>>> >> > >> >> > > >> SF100
>>>>> >> > >> >> > > >> > > was
>>>>> >> > >> >> > > >> > > >> also
>>>>> >> > >> >> > > >> > > >> >> > > > > > successful.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27,
>>>>> 2016 at
>>>>> >> > 10:07
>>>>> >> > >> PM,
>>>>> >> > >> >> > > rahul
>>>>> >> > >> >> > > >> > > >> >> challapalli <
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>>> >> challapallirahul@gmail.com>
>>>>> >> > >> >> wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a
>>>>> functional
>>>>> >> > run
>>>>> >> > >> with
>>>>> >> > >> >> > > your
>>>>> >> > >> >> > > >> > > branch.
>>>>> >> > >> >> > > >> > > >> >> Will
>>>>> >> > >> >> > > >> > > >> >> > > let
>>>>> >> > >> >> > > >> > > >> >> > > > > you
>>>>> >> > >> >> > > >> > > >> >> > > > > > > know
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > once
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > it
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > finishes
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27,
>>>>> 2016 at
>>>>> >> > 9:56
>>>>> >> > >> PM,
>>>>> >> > >> >> > > >> Jacques
>>>>> >> > >> >> > > >> > > >> Nadeau <
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > jacques@dremio.com>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged
>>>>> >> today. I
>>>>> >> > >> have
>>>>> >> > >> >> an
>>>>> >> > >> >> > > >> > updated
>>>>> >> > >> >> > > >> > > >> patch
>>>>> >> > >> >> > > >> > > >> >> > for
>>>>> >> > >> >> > > >> > > >> >> > > > 4291
>>>>> >> > >> >> > > >> > > >> >> > > > > > > that
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > is
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> ready.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately,
>>>>> it
>>>>> >> seems
>>>>> >> > >> that
>>>>> >> > >> >> > > >> something
>>>>> >> > >> >> > > >> > > isn't
>>>>> >> > >> >> > > >> > > >> >> > > working
>>>>> >> > >> >> > > >> > > >> >> > > > > with
>>>>> >> > >> >> > > >> > > >> >> > > > > > > our
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> extended
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I
>>>>> haven't
>>>>> >> been
>>>>> >> > >> able
>>>>> >> > >> >> to
>>>>> >> > >> >> > > run
>>>>> >> > >> >> > > >> an
>>>>> >> > >> >> > > >> > > >> extended
>>>>> >> > >> >> > > >> > > >> >> > > > > > regression.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > Unit
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> tests
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is
>>>>> someone else
>>>>> >> > >> possibly
>>>>> >> > >> >> > > able
>>>>> >> > >> >> > > >> to
>>>>> >> > >> >> > > >> > > run a
>>>>> >> > >> >> > > >> > > >> >> > > > regression
>>>>> >> > >> >> > > >> > > >> >> > > > > > > suite
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > against
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > this
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so
>>>>> we can
>>>>> >> > >> confirm
>>>>> >> > >> >> > > things
>>>>> >> > >> >> > > >> > look
>>>>> >> > >> >> > > >> > > >> good
>>>>> >> > >> >> > > >> > > >> >> and
>>>>> >> > >> >> > > >> > > >> >> > > > start
>>>>> >> > >> >> > > >> > > >> >> > > > > > the
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > release
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > process?
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > thanks,
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>>>>> >> > >> >> > > >> > > >> >> > > > >
>>>>> >> > >> >> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > --
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and
>>>>> Co-Founder,
>>>>> >> > Dremio
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan
>>>>> 25, 2016
>>>>> >> at
>>>>> >> > >> 11:20
>>>>> >> > >> >> > AM,
>>>>> >> > >> >> > > >> > Jacques
>>>>> >> > >> >> > > >> > > >> >> Nadeau
>>>>> >> > >> >> > > >> > > >> >> > <
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > jacques@dremio.com
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the
>>>>> main
>>>>> >> > things
>>>>> >> > >> are
>>>>> >> > >> >> > 4196
>>>>> >> > >> >> > > >> and
>>>>> >> > >> >> > > >> > > 4291
>>>>> >> > >> >> > > >> > > >> >> > should
>>>>> >> > >> >> > > >> > > >> >> > > be
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > completed.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > I
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> know
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Amit
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to
>>>>> >> reproduce
>>>>> >> > >> 4196
>>>>> >> > >> >> > > locally
>>>>> >> > >> >> > > >> > this
>>>>> >> > >> >> > > >> > > >> >> weekend
>>>>> >> > >> >> > > >> > > >> >> > > so
>>>>> >> > >> >> > > >> > > >> >> > > > I
>>>>> >> > >> >> > > >> > > >> >> > > > > > > think
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > we're
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> close
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > on
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > that.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > --
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques
>>>>> Nadeau
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and
>>>>> Co-Founder,
>>>>> >> > >> Dremio
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan
>>>>> 25, 2016
>>>>> >> > at
>>>>> >> > >> >> 10:14
>>>>> >> > >> >> > > AM,
>>>>> >> > >> >> > > >> > > Zelaine
>>>>> >> > >> >> > > >> > > >> >> Fong
>>>>> >> > >> >> > > >> > > >> >> > <
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates
>>>>> on
>>>>> >> this?
>>>>> >> > >> >> What's
>>>>> >> > >> >> > > >> > blocking
>>>>> >> > >> >> > > >> > > us
>>>>> >> > >> >> > > >> > > >> >> from
>>>>> >> > >> >> > > >> > > >> >> > > > taking
>>>>> >> > >> >> > > >> > > >> >> > > > > > > this
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > to a
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> vote --
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > the
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge
>>>>> join
>>>>> >> > issues?
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan
>>>>> 19,
>>>>> >> 2016
>>>>> >> > at
>>>>> >> > >> >> 4:18
>>>>> >> > >> >> > > PM,
>>>>> >> > >> >> > > >> > > Jacques
>>>>> >> > >> >> > > >> > > >> >> > Nadeau
>>>>> >> > >> >> > > >> > > >> >> > > <
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com
>>>>> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping
>>>>> this
>>>>> >> > thread...
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are
>>>>> the
>>>>> >> issues
>>>>> >> > >> that
>>>>> >> > >> >> > were
>>>>> >> > >> >> > > >> > > >> mentioned in
>>>>> >> > >> >> > > >> > > >> >> > > this
>>>>> >> > >> >> > > >> > > >> >> > > > > > thread
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > along
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> with a
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed
>>>>> >> > >> categorization:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release
>>>>> Blockers
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> In-progress Amit
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>>> >> > >> >> > > >> > > >>
>>>>> https://issues.apache.org/jira/browse/DRILL-4190
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> In-progress Amit
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>>> >> > >> >> > > >> > > >>
>>>>> https://issues.apache.org/jira/browse/DRILL-4196
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to
>>>>> merge
>>>>> >> > Jacques
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>>> >> > >> >> > > >> > > >> >>
>>>>> https://issues.apache.org/jira/browse/DRILL-4246
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review
>>>>> Jinfeng
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>>> >> > >> >> > > >> > > >>
>>>>> https://issues.apache.org/jira/browse/DRILL-4256
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> In-progress
>>>>> >> Jacques
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>>> >> > >> >> > > >> > > >>
>>>>> https://issues.apache.org/jira/browse/DRILL-4278
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to
>>>>> merge
>>>>> >> > Laurent
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>>> >> > >> >> > > >> > > >> >>
>>>>> https://issues.apache.org/jira/browse/DRILL-4285
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to
>>>>> Have
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open
>>>>> Jason/Hakim
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>>> >> > >> >> > > >> > > >>
>>>>> https://issues.apache.org/jira/browse/DRILL-4247
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> In-progress Jason
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>>> >> > >> >> > > >> > > >>
>>>>> https://issues.apache.org/jira/browse/DRILL-4203
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open
>>>>> Jacques
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> https://issues.apache.org/jira/browse/DRILL-4266
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to
>>>>> merge
>>>>> >> > Jacques
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>>> >> > >> >> > > >> > > >> >>
>>>>> https://issues.apache.org/jira/browse/DRILL-4131
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do
>>>>> others
>>>>> >> > think?
>>>>> >> > >> >> Let's
>>>>> >> > >> >> > > try
>>>>> >> > >> >> > > >> to
>>>>> >> > >> >> > > >> > > get
>>>>> >> > >> >> > > >> > > >> the
>>>>> >> > >> >> > > >> > > >> >> > > > > blockers
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > wrapped
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > up
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> in
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > the
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> next
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or
>>>>> two and
>>>>> >> > start a
>>>>> >> > >> >> > > release
>>>>> >> > >> >> > > >> > > vote...
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > --
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques
>>>>> Nadeau
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and
>>>>> >> Co-Founder,
>>>>> >> > >> >> Dremio
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon,
>>>>> Jan 4,
>>>>> >> > 2016 at
>>>>> >> > >> >> 1:48
>>>>> >> > >> >> > > PM,
>>>>> >> > >> >> > > >> > > Jason
>>>>> >> > >> >> > > >> > > >> >> > > Altekruse
>>>>> >> > >> >> > > >> > > >> >> > > > <
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>>>> >> > >> altekrusejason@gmail.com>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello
>>>>> All,
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the
>>>>> >> > allocator
>>>>> >> > >> >> > changes
>>>>> >> > >> >> > > >> > merged
>>>>> >> > >> >> > > >> > > and
>>>>> >> > >> >> > > >> > > >> >> > about
>>>>> >> > >> >> > > >> > > >> >> > > a
>>>>> >> > >> >> > > >> > > >> >> > > > > > month
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > since
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > the
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> last
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think
>>>>> it
>>>>> >> would
>>>>> >> > be
>>>>> >> > >> >> good
>>>>> >> > >> >> > to
>>>>> >> > >> >> > > >> > start
>>>>> >> > >> >> > > >> > > a
>>>>> >> > >> >> > > >> > > >> vote
>>>>> >> > >> >> > > >> > > >> >> > > > soon. I
>>>>> >> > >> >> > > >> > > >> >> > > > > > > would
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > like
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > to
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be
>>>>> release
>>>>> >> > manager.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know
>>>>> that
>>>>> >> there
>>>>> >> > >> were
>>>>> >> > >> >> > some
>>>>> >> > >> >> > > >> > issues
>>>>> >> > >> >> > > >> > > >> that
>>>>> >> > >> >> > > >> > > >> >> > were
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > identified
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > after
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> the
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch
>>>>> was
>>>>> >> > merged. I
>>>>> >> > >> >> think
>>>>> >> > >> >> > > that
>>>>> >> > >> >> > > >> > > these
>>>>> >> > >> >> > > >> > > >> >> > issues
>>>>> >> > >> >> > > >> > > >> >> > > > > should
>>>>> >> > >> >> > > >> > > >> >> > > > > > > be
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > fixed
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> before
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > we
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release
>>>>> >> > candidate.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From
>>>>> looking at
>>>>> >> > the
>>>>> >> > >> >> > > associated
>>>>> >> > >> >> > > >> > > JIRAs
>>>>> >> > >> >> > > >> > > >> it
>>>>> >> > >> >> > > >> > > >> >> > > looked
>>>>> >> > >> >> > > >> > > >> >> > > > > > like
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > there
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > was
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> a
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short
>>>>> term fix
>>>>> >> > just
>>>>> >> > >> >> > > adjusting
>>>>> >> > >> >> > > >> > the
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > option,
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> and
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some
>>>>> more
>>>>> >> > involved
>>>>> >> > >> work
>>>>> >> > >> >> > to
>>>>> >> > >> >> > > >> > change
>>>>> >> > >> >> > > >> > > >> how we
>>>>> >> > >> >> > > >> > > >> >> > > > > determine
>>>>> >> > >> >> > > >> > > >> >> > > > > > > the
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > correct
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > time
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > to
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill
>>>>> during
>>>>> >> > >> external
>>>>> >> > >> >> > > sort. I
>>>>> >> > >> >> > > >> > > >> believe it
>>>>> >> > >> >> > > >> > > >> >> > > makes
>>>>> >> > >> >> > > >> > > >> >> > > > > > sense
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > to
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > make
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > external
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work
>>>>> well with
>>>>> >> > the
>>>>> >> > >> >> newly
>>>>> >> > >> >> > > >> > improved
>>>>> >> > >> >> > > >> > > >> memory
>>>>> >> > >> >> > > >> > > >> >> > > > > > accounting
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > before
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > cutting a
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>>> release, but
>>>>> >> I'm
>>>>> >> > not
>>>>> >> > >> >> sure
>>>>> >> > >> >> > > how
>>>>> >> > >> >> > > >> > much
>>>>> >> > >> >> > > >> > > >> work
>>>>> >> > >> >> > > >> > > >> >> is
>>>>> >> > >> >> > > >> > > >> >> > > > left
>>>>> >> > >> >> > > >> > > >> >> > > > > to
>>>>> >> > >> >> > > >> > > >> >> > > > > > > be
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > done
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> there.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please
>>>>> respond
>>>>> >> > with
>>>>> >> > >> >> your
>>>>> >> > >> >> > > >> > thoughts
>>>>> >> > >> >> > > >> > > on
>>>>> >> > >> >> > > >> > > >> a
>>>>> >> > >> >> > > >> > > >> >> > > release
>>>>> >> > >> >> > > >> > > >> >> > > > > > soon
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > and
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > any
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> JIRAs
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > you
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to
>>>>> include
>>>>> >> > in
>>>>> >> > >> the
>>>>> >> > >> >> > > >> release.
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
>>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>>> >> > >> >> > > https://issues.apache.org/jira/browse/DRILL-4243
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>>> >> > >> >> > > >> > > >> >> > > > >
>>>>> >> > >> >> > > >> > > >> >> > > >
>>>>> >> > >> >> > > >> > > >> >> > >
>>>>> >> > >> >> > > >> > > >> >> >
>>>>> >> > >> >> > > >> > > >> >>
>>>>> >> > >> >> > > >> > > >>
>>>>> >> > >> >> > > >> > >
>>>>> >> > >> >> > > >> >
>>>>> >> > >> >> > > >>
>>>>> >> > >> >> > >
>>>>> >> > >> >> >
>>>>> >> > >> >>
>>>>> >> > >>
>>>>> >> > >>
>>>>> >> >
>>>>> >> >
>>>>> >>
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>>
>> Abdelhakim Deneche
>>
>> Software Engineer
>>
>>   <http://www.mapr.com/>
>>
>>
>> Now Available - Free Hadoop On-Demand Training
>> <http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>
>>
>
>


-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>

Re: Time for a 1.5 release?

Posted by Jason Altekruse <al...@gmail.com>.

Fair enough. I meant having my public GPG key signed by someone already in
the Apache web of trust.

On Mon, Feb 1, 2016 at 1:39 PM, Abdel Hakim Deneche <ad...@maprtech.com>
wrote:

> I have trouble understanding the meaning of "GPG signed" =P
>
> On Mon, Feb 1, 2016 at 1:27 PM, Jason Altekruse <al...@gmail.com>
> wrote:
>
>> A question for previous release managers. Jacques was thinking I may need
>> to have my GPG signed to get linked into the Apache Web of Trust. Is this
>> necessary for making a release?
>>
>> On Mon, Feb 1, 2016 at 11:24 AM, Jason Altekruse <
>> altekrusejason@gmail.com> wrote:
>>
>>> The only solution to this issue I have found is disabling the new plugin
>>> to automatically enforce the size of the jdbc-all plugin. I don't know why
>>> it is behaving differently when performing a release than it did during a
>>> 'mvn install', but I have verified the correct size of the JAR manually and
>>> am planning on pushing a candidate soon with the small change to disable
>>> the automatic jar size enforcement.
>>>
>>> I will open a JIRA for getting this resolved after the release.
>>>
>>> On Sun, Jan 31, 2016 at 10:46 PM, Jinfeng Ni <ji...@gmail.com>
>>> wrote:
>>>
>>>> Good to hear this, Jacques & Jason. That makes feel relieved. :-)
>>>>
>>>>
>>>>
>>>> On Sun, Jan 31, 2016 at 10:36 PM, Jason Altekruse
>>>> <al...@gmail.com> wrote:
>>>> > I don't think there was any issue with the merge either Jinfeng, I
>>>> don't
>>>> > think we need to close master. I had to step out for a few hours, but
>>>> I am
>>>> > working on finishing up the candidate right now.
>>>> >
>>>> > Having a weird RAT issue running the actual maven release command,
>>>> it's
>>>> > complaining about a dependency-reduced-pom.xml in the target
>>>> directory of
>>>> > jdbc-all lacking a license header (although target directories should
>>>> be
>>>> > ignored). Checking out the commit before the recent changes to the
>>>> jdbc-all
>>>> > exclusions seems to get rid of the issue, but the changes in the pom
>>>> file
>>>> > don't seem like they should be causing any problems.
>>>> >
>>>> > Still trying to debug and hoping to have a vote up tonight.
>>>> >
>>>> > On Sun, Jan 31, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com>
>>>> wrote:
>>>> >
>>>> >> Given how easy it is to create a branch off a random commit, I
>>>> generally
>>>> >> don't see a point in closing the master branch. I figure the release
>>>> >> manager can propose where they'd like to fork if this type of thing
>>>> comes
>>>> >> up.
>>>> >>
>>>> >> Thanks for the feedback Jinfeng. Of course you're right about Venki's
>>>> >> commit. My bad.
>>>> >>
>>>> >> --
>>>> >> Jacques Nadeau
>>>> >> CTO and Co-Founder, Dremio
>>>> >>
>>>> >> On Sun, Jan 31, 2016 at 9:21 PM, Jinfeng Ni <ji...@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> > Venki's backwards compatibility fix (commit id:
>>>> >> >    03197d0) is before the partition pruning commit. So, it seems
>>>> to be
>>>> >> > fine to create a release candidate from 03197d0; no need to
>>>> >> > cherry-pick.
>>>> >> >
>>>> >> > As for whether the partition pruning changes are risky, the
>>>> pre-commit
>>>> >> > run did not show any serious problems in the first run; all the
>>>> plan
>>>> >> > changes are expected, or expose some existing problem (drill-4279).
>>>> >> > However, I agree that it impacts the main code path of query
>>>> planning,
>>>> >> > and I'm fine that we exclude the partition change from 1.5.0
>>>> release,
>>>> >> > since it's merged right before we start a voting.
>>>> >> >
>>>> >> > I did not see any one mentioned that the master branch is closed,
>>>> and
>>>> >> > therefore assume it's still open for commits.  I probably should
>>>> have
>>>> >> > asked whether the master was open for new patches before I merged
>>>> the
>>>> >> > commits.
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> > On Sun, Jan 31, 2016 at 8:43 PM, Jacques Nadeau <
>>>> jacques@dremio.com>
>>>> >> > wrote:
>>>> >> > > Jinfeng, do you agree that the changes are risky? I was going
>>>> mostly by
>>>> >> > > commit messages.
>>>> >> > >
>>>> >> > > Also just realized that the backwards compatibility fix to the
>>>> storage
>>>> >> > > plugin interface is after these. Could create release branch from
>>>> >> > 03197d...
>>>> >> > > and then cherry pick the Venki change. Or could keep them all. Or
>>>> >> > > go without the compatibility fix....
>>>> >> > >
>>>> >> > > --
>>>> >> > > Jacques Nadeau
>>>> >> > > CTO and Co-Founder, Dremio
>>>> >> > >
>>>> >> > > On Sun, Jan 31, 2016 at 7:50 PM, Jinfeng Ni <
>>>> jinfengni99@gmail.com>
>>>> >> > wrote:
>>>> >> > >
>>>> >> > >> Hi Jason,
>>>> >> > >>
>>>> >> > >> I'm fine that release 1.5.0 starts from 03197d0f2.  One thing I
>>>> need
>>>> >> > >> point out is that the pre-commit test suite has some testcase's
>>>> plan
>>>> >> > >> output modified because of the partition pruning change.  If we
>>>> run
>>>> >> > >> pre-commit against the candidate release for 1.5.0, we have to
>>>> use a
>>>> >> > >> prior version of pre-commit suite.
>>>> >> > >>
>>>> >> > >>
>>>> >> > >>
>>>> >> > >> On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse
>>>> >> > >> <al...@gmail.com> wrote:
>>>> >> > >> > That sounds reasonable to me. I'll start preparing a release
>>>> >> > candidate.
>>>> >> > >> >
>>>> >> > >> > Jinfeng, are you okay waiting for the next release to include
>>>> this
>>>> >> > >> change?
>>>> >> > >> >
>>>> >> > >> > On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <
>>>> jacques@dremio.com
>>>> >> >
>>>> >> > >> wrote:
>>>> >> > >> >
>>>> >> > >> >> Hey Jason,
>>>> >> > >> >>
>>>> >> > >> >> Can you start the release vote? We've let this drag on too
>>>> long.
>>>> >> > >> >>
>>>> >> > >> >> I suggest we start from 03197d0f2. It seems like the
>>>> partition
>>>> >> > pruning
>>>> >> > >> >> changes are pretty complicated and on a pretty main path to
>>>> add
>>>> >> right
>>>> >> > >> >> before the vote. Since Hsuan's fix (pr 349) hasn't been
>>>> reviewed, I
>>>> >> > >> think
>>>> >> > >> >> we should start the vote without it.
>>>> >> > >> >>
>>>> >> > >> >> Jacques
>>>> >> > >> >>
>>>> >> > >> >>
>>>> >> > >> >> --
>>>> >> > >> >> Jacques Nadeau
>>>> >> > >> >> CTO and Co-Founder, Dremio
>>>> >> > >> >>
>>>> >> > >> >> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <
>>>> hyichu@maprtech.com
>>>> >> >
>>>> >> > >> wrote:
>>>> >> > >> >>
>>>> >> > >> >> > Yes, just send a pull request.
>>>> >> > >> >> > https://github.com/apache/drill/pull/349
>>>> >> > >> >> >
>>>> >> > >> >> > Can Jinfeng review it?
>>>> >> > >> >> >
>>>> >> > >> >> > Thanks,
>>>> >> > >> >> >
>>>> >> > >> >> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <
>>>> >> jinfengni99@gmail.com
>>>> >> > >
>>>> >> > >> >> wrote:
>>>> >> > >> >> >
>>>> >> > >> >> > > Sean probably is quite close to find a fix for this
>>>> issue.
>>>> >> Let's
>>>> >> > see
>>>> >> > >> >> > > whether he can post a patch soon.
>>>> >> > >> >> > >
>>>> >> > >> >> > >
>>>> >> > >> >> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
>>>> >> > >> >> > > <ch...@gmail.com> wrote:
>>>> >> > >> >> > > > DRILL-4323 only happens when assertions are turned on
>>>> (my
>>>> >> > install
>>>> >> > >> >> > script
>>>> >> > >> >> > > > automatically enables assertions).
>>>> >> > >> >> > > >
>>>> >> > >> >> > > > - Rahul
>>>> >> > >> >> > > >
>>>> >> > >> >> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <
>>>> >> > >> zfong@maprtech.com>
>>>> >> > >> >> > > wrote:
>>>> >> > >> >> > > >
>>>> >> > >> >> > > >> Rahul,
>>>> >> > >> >> > > >>
>>>> >> > >> >> > > >> Let us know if you can reproduce the issue with
>>>> assertions
>>>> >> > turned
>>>> >> > >> >> OFF.
>>>> >> > >> >> > > If
>>>> >> > >> >> > > >> you cannot, then I think this should not be a blocker
>>>> for
>>>> >> 1.5,
>>>> >> > >> and
>>>> >> > >> >> > Hsuan
>>>> >> > >> >> > > >> will continue to look into fixing this.
>>>> >> > >> >> > > >>
>>>> >> > >> >> > > >> -- Zelaine
>>>> >> > >> >> > > >>
>>>> >> > >> >> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
>>>> >> > >> >> > knguyen@maprtech.com>
>>>> >> > >> >> > > >> wrote:
>>>> >> > >> >> > > >>
>>>> >> > >> >> > > >> > I do see the same error with assertion turned on.
>>>> >> > >> >> > > >> >
>>>> >> > >> >> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <
>>>> >> > >> >> jinfengni99@gmail.com>
>>>> >> > >> >> > > >> wrote:
>>>> >> > >> >> > > >> >
>>>> >> > >> >> > > >> > > Krystal, by any chance, did you turn on assertion
>>>> when
>>>> >> you
>>>> >> > >> start
>>>> >> > >> >> > > >> > drillbit?
>>>> >> > >> >> > > >> > >
>>>> >> > >> >> > > >> > > I think the error in drill-4323 will be raised
>>>> when
>>>> >> > >> assertion =
>>>> >> > >> >> > on.
>>>> >> > >> >> > > >> > >
>>>> >> > >> >> > > >> > >
>>>> >> > >> >> > > >> > >
>>>> >> > >> >> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
>>>> >> > >> >> > > knguyen@maprtech.com>
>>>> >> > >> >> > > >> > > wrote:
>>>> >> > >> >> > > >> > > > I am running drill-1.4 and is able to run
>>>> count(*)
>>>> >> > >> >> successfully.
>>>> >> > >> >> > > >> > > >
>>>> >> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100>
>>>> alter
>>>> >> > >> session
>>>> >> > >> >> > set
>>>> >> > >> >> > > >> > > > `store.hive.optimize_scan_with_native_readers`
>>>> = true;
>>>> >> > >> >> > > >> > > >
>>>> >> > >> >> >
>>>> >> +-------+--------------------------------------------------------+
>>>> >> > >> >> > > >> > > > |  ok   |                        summary
>>>> >> > >> >> >  |
>>>> >> > >> >> > > >> > > >
>>>> >> > >> >> >
>>>> >> +-------+--------------------------------------------------------+
>>>> >> > >> >> > > >> > > > | true  |
>>>> store.hive.optimize_scan_with_native_readers
>>>> >> > >> >> > updated.  |
>>>> >> > >> >> > > >> > > >
>>>> >> > >> >> >
>>>> >> +-------+--------------------------------------------------------+
>>>> >> > >> >> > > >> > > > 1 row selected (0.374 seconds)
>>>> >> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100>
>>>> select
>>>> >> > >> >> count(*)
>>>> >> > >> >> > > from
>>>> >> > >> >> > > >> > > > lineitem_parquet_partitioned_hive_string;
>>>> >> > >> >> > > >> > > > +---------+
>>>> >> > >> >> > > >> > > > | EXPR$0  |
>>>> >> > >> >> > > >> > > > +---------+
>>>> >> > >> >> > > >> > > > | 60175   |
>>>> >> > >> >> > > >> > > > +---------+
>>>> >> > >> >> > > >> > > > 1 row selected (2.986 seconds)
>>>> >> > >> >> > > >> > > >
>>>> >> > >> >> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
>>>> >> > >> >> > > jinfengni99@gmail.com>
>>>> >> > >> >> > > >> > > wrote:
>>>> >> > >> >> > > >> > > >
>>>> >> > >> >> > > >> > > >> Venki and I did some investigation for
>>>> DRILL-4323.
>>>> >> The
>>>> >> > >> issue
>>>> >> > >> >> > > >> reported
>>>> >> > >> >> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release
>>>> as
>>>> >> well.
>>>> >> > >> Seems
>>>> >> > >> >> > to
>>>> >> > >> >> > > us
>>>> >> > >> >> > > >> > > >> this is not a regression from 1.4.0; it's a
>>>> >> regression
>>>> >> > >> from
>>>> >> > >> >> > 1.3.0
>>>> >> > >> >> > > >> > > >> probably.
>>>> >> > >> >> > > >> > > >>
>>>> >> > >> >> > > >> > > >> DRILL-4083 makes the planner to use
>>>> >> > DrillHiveNativeReader
>>>> >> > >> in
>>>> >> > >> >> > > stead
>>>> >> > >> >> > > >> of
>>>> >> > >> >> > > >> > > >> HiveReader for "select count(*) from
>>>> hive_table"
>>>> >> query.
>>>> >> > >> >> > However,
>>>> >> > >> >> > > >> the
>>>> >> > >> >> > > >> > > >> Project after the scan produces empty schema.
>>>> Before
>>>> >> > >> >> > DRILL-4083,
>>>> >> > >> >> > > >> > > >> Drill uses HiveScan, which works fine.
>>>> >> > >> >> > > >> > > >>
>>>> >> > >> >> > > >> > > >>
>>>> >> > >> >> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason
>>>> Altekruse
>>>> >> > >> >> > > >> > > >> <al...@gmail.com> wrote:
>>>> >> > >> >> > > >> > > >> > We could revert the change to the
>>>> StoragePlugin
>>>> >> > >> interface
>>>> >> > >> >> to
>>>> >> > >> >> > > add
>>>> >> > >> >> > > >> the
>>>> >> > >> >> > > >> > > >> > getLogicalOptimizerRules
>>>> >> > >> >> > > >> > > >> > and getPhysicalOptimizerRules, and only put
>>>> those
>>>> >> in
>>>> >> > >> >> > > >> > > >> AbstractStoragePlugin
>>>> >> > >> >> > > >> > > >> > with default implementations.
>>>> >> > >> >> > > >> > > >> >
>>>> >> > >> >> > > >> > > >> > We could then update the docs to tell users
>>>> to not
>>>> >> > >> >> implement
>>>> >> > >> >> > > the
>>>> >> > >> >> > > >> > > >> interface
>>>> >> > >> >> > > >> > > >> > directly, and instead extend the abstract
>>>> class,
>>>> >> > which I
>>>> >> > >> >> > > believe
>>>> >> > >> >> > > >> > would
>>>> >> > >> >> > > >> > > >> > allow us to make changes like this in the
>>>> future
>>>> >> > without
>>>> >> > >> >> > > breaking
>>>> >> > >> >> > > >> > > >> plugins.
>>>> >> > >> >> > > >> > > >> > At a major release version we could then
>>>> just pull
>>>> >> up
>>>> >> > >> all
>>>> >> > >> >> of
>>>> >> > >> >> > > the
>>>> >> > >> >> > > >> new
>>>> >> > >> >> > > >> > > >> > methods we added to the abstract class into
>>>> the
>>>> >> > >> interface,
>>>> >> > >> >> or
>>>> >> > >> >> > > just
>>>> >> > >> >> > > >> > get
>>>> >> > >> >> > > >> > > >> rid
>>>> >> > >> >> > > >> > > >> > of it all-together.
>>>> >> > >> >> > > >> > > >> >
>>>> >> > >> >> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki
>>>> Korukanti <
>>>> >> > >> >> > > >> > > >> venki.korukanti@gmail.com>
>>>> >> > >> >> > > >> > > >> > wrote:
>>>> >> > >> >> > > >> > > >> >
>>>> >> > >> >> > > >> > > >> >> Here is the partial fix:
>>>> >> > >> >> > > >> > > >> >>
>>>> >> > >> >> > > >> > > >> >>
>>>> >> > >> >> > > >> > > >>
>>>> >> > >> >> > > >> > >
>>>> >> > >> >> > > >> >
>>>> >> > >> >> > > >>
>>>> >> > >> >> > >
>>>> >> > >> >> >
>>>> >> > >> >>
>>>> >> > >>
>>>> >> >
>>>> >>
>>>> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
>>>> >> > >> >> > > >> > > >> >>
>>>> >> > >> >> > > >> > > >> >> If the existing StragePlugins are
>>>> implementing
>>>> >> > >> >> > > >> > AbstractStoragePlugin
>>>> >> > >> >> > > >> > > >> class
>>>> >> > >> >> > > >> > > >> >> this fix should resolve the backward compat
>>>> issue.
>>>> >> > If
>>>> >> > >> they
>>>> >> > >> >> > are
>>>> >> > >> >> > > >> > > >> implementing
>>>> >> > >> >> > > >> > > >> >> StoragePlugin interface directly then it
>>>> won't
>>>> >> > resolve
>>>> >> > >> the
>>>> >> > >> >> > > issue.
>>>> >> > >> >> > > >> > Not
>>>> >> > >> >> > > >> > > >> sure
>>>> >> > >> >> > > >> > > >> >> how to handle it in that case. Let me know
>>>> if
>>>> >> there
>>>> >> > any
>>>> >> > >> >> ways
>>>> >> > >> >> > > to
>>>> >> > >> >> > > >> > > resolve
>>>> >> > >> >> > > >> > > >> >> this issue completely.
>>>> >> > >> >> > > >> > > >> >>
>>>> >> > >> >> > > >> > > >> >> Thanks
>>>> >> > >> >> > > >> > > >> >> Venki
>>>> >> > >> >> > > >> > > >> >>
>>>> >> > >> >> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha
>>>> <
>>>> >> > >> >> > > asinha@maprtech.com
>>>> >> > >> >> > > >> >
>>>> >> > >> >> > > >> > > >> wrote:
>>>> >> > >> >> > > >> > > >> >>
>>>> >> > >> >> > > >> > > >> >> > I was able to do the build with maven
>>>> 3.3.9
>>>> >> (after
>>>> >> > >> doing
>>>> >> > >> >> > > 'brew
>>>> >> > >> >> > > >> > > install
>>>> >> > >> >> > > >> > > >> >> > maven').  Here are my env variables
>>>> related to
>>>> >> > maven:
>>>> >> > >> >> > > >> > > >> >> >
>>>> M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
>>>> >> > >> >> > > >> > > >> >> >
>>>> M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
>>>> >> > >> >> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
>>>> >> > >> >> > > >> > > >> >> >
>>>> >> > >> >> > > >> > > >> >> >
>>>> >> > >> >> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques
>>>> Nadeau
>>>> >> <
>>>> >> > >> >> > > >> > > jacques@dremio.com>
>>>> >> > >> >> > > >> > > >> >> > wrote:
>>>> >> > >> >> > > >> > > >> >> >
>>>> >> > >> >> > > >> > > >> >> > > Aman, for reference on the Maven version
>>>> >> issue.
>>>> >> > You
>>>> >> > >> >> can
>>>> >> > >> >> > > see
>>>> >> > >> >> > > >> the
>>>> >> > >> >> > > >> > > >> output
>>>> >> > >> >> > > >> > > >> >> of
>>>> >> > >> >> > > >> > > >> >> > > the shade plugin between the two Apache
>>>> >> Builds.
>>>> >> > If
>>>> >> > >> you
>>>> >> > >> >> > > search
>>>> >> > >> >> > > >> > for
>>>> >> > >> >> > > >> > > >> >> > > maven-shade-plugin you'll see that
>>>> build 647
>>>> >> > (maven
>>>> >> > >> >> > 3.0.5)
>>>> >> > >> >> > > >> > fails
>>>> >> > >> >> > > >> > > >> >> because
>>>> >> > >> >> > > >> > > >> >> > it
>>>> >> > >> >> > > >> > > >> >> > > includes a bunch of hadoop (and other
>>>> >> libraries
>>>> >> > >> that
>>>> >> > >> >> are
>>>> >> > >> >> > > >> listed
>>>> >> > >> >> > > >> > > as
>>>> >> > >> >> > > >> > > >> >> being
>>>> >> > >> >> > > >> > > >> >> > > excluded). If you look at build 648
>>>> (maven
>>>> >> > 3.3.3),
>>>> >> > >> >> those
>>>> >> > >> >> > > >> items
>>>> >> > >> >> > > >> > > are
>>>> >> > >> >> > > >> > > >> >> > included
>>>> >> > >> >> > > >> > > >> >> > > so that the build completes
>>>> successfully.
>>>> >> > >> >> > > >> > > >> >> > >
>>>> >> > >> >> > > >> > > >> >> > >
>>>> >> > >> >> https://builds.apache.org/job/drill-scm/647/consoleText
>>>> >> > >> >> > > >> > > >> >> > >
>>>> >> > >> >> https://builds.apache.org/job/drill-scm/648/consoleText
>>>> >> > >> >> > > >> > > >> >> > >
>>>> >> > >> >> > > >> > > >> >> > > --
>>>> >> > >> >> > > >> > > >> >> > > Jacques Nadeau
>>>> >> > >> >> > > >> > > >> >> > > CTO and Co-Founder, Dremio
>>>> >> > >> >> > > >> > > >> >> > >
>>>> >> > >> >> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul
>>>> >> > challapalli
>>>> >> > >> <
>>>> >> > >> >> > > >> > > >> >> > > challapallirahul@gmail.com> wrote:
>>>> >> > >> >> > > >> > > >> >> > >
>>>> >> > >> >> > > >> > > >> >> > > > Jason,
>>>> >> > >> >> > > >> > > >> >> > > >
>>>> >> > >> >> > > >> > > >> >> > > > As of now I haven't narrowed down the
>>>> exact
>>>> >> > >> commit
>>>> >> > >> >> > which
>>>> >> > >> >> > > >> > caused
>>>> >> > >> >> > > >> > > >> >> > > DRILL-4323
>>>> >> > >> >> > > >> > > >> >> > > > and I do not know the scope of the
>>>> fix.
>>>> >> > >> >> > > >> > > >> >> > > >
>>>> >> > >> >> > > >> > > >> >> > > > - Rahul
>>>> >> > >> >> > > >> > > >> >> > > >
>>>> >> > >> >> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason
>>>> >> > Altekruse
>>>> >> > >> <
>>>> >> > >> >> > > >> > > >> >> > > altekrusejason@gmail.com
>>>> >> > >> >> > > >> > > >> >> > > > >
>>>> >> > >> >> > > >> > > >> >> > > > wrote:
>>>> >> > >> >> > > >> > > >> >> > > >
>>>> >> > >> >> > > >> > > >> >> > > > > I think it makes sense to wait for
>>>> Venki's
>>>> >> > fix
>>>> >> > >> of
>>>> >> > >> >> > the
>>>> >> > >> >> > > >> API,
>>>> >> > >> >> > > >> > as
>>>> >> > >> >> > > >> > > >> >> > > releasing a
>>>> >> > >> >> > > >> > > >> >> > > > > version with the change would make
>>>> >> > >> >> > > reverting/refactoring
>>>> >> > >> >> > > >> > the
>>>> >> > >> >> > > >> > > API
>>>> >> > >> >> > > >> > > >> >> > > breaking
>>>> >> > >> >> > > >> > > >> >> > > > > change less useful.
>>>> >> > >> >> > > >> > > >> >> > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > Do you have any idea about the
>>>> scope of
>>>> >> > fixing
>>>> >> > >> the
>>>> >> > >> >> > > hive
>>>> >> > >> >> > > >> > > native
>>>> >> > >> >> > > >> > > >> >> > reader,
>>>> >> > >> >> > > >> > > >> >> > > or
>>>> >> > >> >> > > >> > > >> >> > > > > which change since the 1.4 release
>>>> broke
>>>> >> > it? Is
>>>> >> > >> >> > > someone
>>>> >> > >> >> > > >> > > >> available
>>>> >> > >> >> > > >> > > >> >> to
>>>> >> > >> >> > > >> > > >> >> > > look
>>>> >> > >> >> > > >> > > >> >> > > > > at it today?
>>>> >> > >> >> > > >> > > >> >> > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM,
>>>> rahul
>>>> >> > >> >> challapalli <
>>>> >> > >> >> > > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
>>>> >> > >> >> > > >> > > >> >> > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > Just found an issue with hive
>>>> native
>>>> >> > parquet
>>>> >> > >> >> > reader
>>>> >> > >> >> > > >> > > >> (DRILL-4323).
>>>> >> > >> >> > > >> > > >> >> > > This
>>>> >> > >> >> > > >> > > >> >> > > > > is a
>>>> >> > >> >> > > >> > > >> >> > > > > > regression from 1.4 and in my
>>>> opinion
>>>> >> > should
>>>> >> > >> be
>>>> >> > >> >> > > treated
>>>> >> > >> >> > > >> > as
>>>> >> > >> >> > > >> > > a
>>>> >> > >> >> > > >> > > >> >> > blocker.
>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > - Rahul
>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM,
>>>> Venki
>>>> >> > >> >> Korukanti <
>>>> >> > >> >> > > >> > > >> >> > > > > > venki.korukanti@gmail.com>
>>>> >> > >> >> > > >> > > >> >> > > > > > wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > Sorry I didn't realize I was
>>>> breaking
>>>> >> > the
>>>> >> > >> >> public
>>>> >> > >> >> > > >> > > interface
>>>> >> > >> >> > > >> > > >> API.
>>>> >> > >> >> > > >> > > >> >> > Let
>>>> >> > >> >> > > >> > > >> >> > > > me
>>>> >> > >> >> > > >> > > >> >> > > > > > work
>>>> >> > >> >> > > >> > > >> >> > > > > > > on a patch to make it backward
>>>> >> > compatible.
>>>> >> > >> >> > > >> > > >> >> > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > Thanks
>>>> >> > >> >> > > >> > > >> >> > > > > > > Venki
>>>> >> > >> >> > > >> > > >> >> > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54
>>>> PM,
>>>> >> Steven
>>>> >> > >> >> > Phillips
>>>> >> > >> >> > > <
>>>> >> > >> >> > > >> > > >> >> > > steven@dremio.com
>>>> >> > >> >> > > >> > > >> >> > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > I just wanted to bring up an
>>>> issue
>>>> >> > that I
>>>> >> > >> >> just
>>>> >> > >> >> > > now
>>>> >> > >> >> > > >> > > >> >> discovered,
>>>> >> > >> >> > > >> > > >> >> > > that
>>>> >> > >> >> > > >> > > >> >> > > > > has
>>>> >> > >> >> > > >> > > >> >> > > > > > > > caused me a fair amount of
>>>> grief.
>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>> >> > >> >> > > https://github.com/apache/drill/pull/300/commits
>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > DRILL-4198 changes a
>>>> user-facing
>>>> >> API,
>>>> >> > and
>>>> >> > >> >> > causes
>>>> >> > >> >> > > >> > > >> >> StoragePlugins
>>>> >> > >> >> > > >> > > >> >> > > > that
>>>> >> > >> >> > > >> > > >> >> > > > > > were
>>>> >> > >> >> > > >> > > >> >> > > > > > > > compiled against currently
>>>> released
>>>> >> > >> versions
>>>> >> > >> >> > of
>>>> >> > >> >> > > >> Drill
>>>> >> > >> >> > > >> > > to
>>>> >> > >> >> > > >> > > >> no
>>>> >> > >> >> > > >> > > >> >> > > longer
>>>> >> > >> >> > > >> > > >> >> > > > > > > > functional properly. I would
>>>> prefer
>>>> >> > that
>>>> >> > >> >> this
>>>> >> > >> >> > > >> > breaking
>>>> >> > >> >> > > >> > > >> change
>>>> >> > >> >> > > >> > > >> >> > be
>>>> >> > >> >> > > >> > > >> >> > > > > > modified
>>>> >> > >> >> > > >> > > >> >> > > > > > > > to be backward compatible if
>>>> >> possible.
>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23
>>>> AM,
>>>> >> > Jason
>>>> >> > >> >> > > Altekruse <
>>>> >> > >> >> > > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > Hi Aman,
>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > This is the failure that he
>>>> was
>>>> >> > >> seeing. He
>>>> >> > >> >> > > >> figured
>>>> >> > >> >> > > >> > > out
>>>> >> > >> >> > > >> > > >> that
>>>> >> > >> >> > > >> > > >> >> > the
>>>> >> > >> >> > > >> > > >> >> > > > new
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were
>>>> not
>>>> >> > being
>>>> >> > >> >> > > respected
>>>> >> > >> >> > > >> > when
>>>> >> > >> >> > > >> > > the
>>>> >> > >> >> > > >> > > >> >> > build
>>>> >> > >> >> > > >> > > >> >> > > > was
>>>> >> > >> >> > > >> > > >> >> > > > > > run
>>>> >> > >> >> > > >> > > >> >> > > > > > > > with
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > an older Maven version,
>>>> causing
>>>> >> the
>>>> >> > jar
>>>> >> > >> >> size
>>>> >> > >> >> > > to
>>>> >> > >> >> > > >> > > increase
>>>> >> > >> >> > > >> > > >> >> > > > > > significantly.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > He
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > added an enforcer to make
>>>> sure the
>>>> >> > JAR
>>>> >> > >> >> > didn't
>>>> >> > >> >> > > >> grow
>>>> >> > >> >> > > >> > > >> >> > > unexpectedly.
>>>> >> > >> >> > > >> > > >> >> > > > > Can
>>>> >> > >> >> > > >> > > >> >> > > > > > > you
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > try to update your maven
>>>> version
>>>> >> and
>>>> >> > >> >> re-run
>>>> >> > >> >> > > the
>>>> >> > >> >> > > >> > > build?
>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > - Jason
>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at
>>>> 11:18 AM,
>>>> >> > Aman
>>>> >> > >> >> > Sinha <
>>>> >> > >> >> > > >> > > >> >> > > > asinha@maprtech.com>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the
>>>> >> > following
>>>> >> > >> >> build
>>>> >> > >> >> > > >> failure
>>>> >> > >> >> > > >> > > on
>>>> >> > >> >> > > >> > > >> the
>>>> >> > >> >> > > >> > > >> >> > > latest
>>>> >> > >> >> > > >> > > >> >> > > > > > > master
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > branch...is this what you
>>>> saw
>>>> >> for
>>>> >> > the
>>>> >> > >> >> > Apache
>>>> >> > >> >> > > >> > build
>>>> >> > >> >> > > >> > > ?
>>>> >> > >> >> > > >> > > >> My
>>>> >> > >> >> > > >> > > >> >> > mvn
>>>> >> > >> >> > > >> > > >> >> > > > > > version
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > output
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > is shown below.  Should
>>>> we all
>>>> >> be
>>>> >> > >> >> > upgrading
>>>> >> > >> >> > > to
>>>> >> > >> >> > > >> a
>>>> >> > >> >> > > >> > > newer
>>>> >> > >> >> > > >> > > >> >> mvn
>>>> >> > >> >> > > >> > > >> >> > ?
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > [INFO] ---
>>>> >> > >> >> > > maven-enforcer-plugin:1.3.1:enforce
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> (enforce-jdbc-jar-compactness) @
>>>> >> > >> >> > > drill-jdbc-all
>>>> >> > >> >> > > >> > ---
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>> >> > >> >> org.apache.maven.plugins.enforcer.RequireFilesSize
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > failed
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > with message:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > The file
>>>> >> > >> >> drill-jdbc-all-1.5.0-SNAPSHOT.jar
>>>> >> > >> >> > > is
>>>> >> > >> >> > > >> > > outside
>>>> >> > >> >> > > >> > > >> the
>>>> >> > >> >> > > >> > > >> >> > > > > expected
>>>> >> > >> >> > > >> > > >> >> > > > > > > size
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > range.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >                   This is
>>>> likely
>>>> >> > due
>>>> >> > >> to
>>>> >> > >> >> > you
>>>> >> > >> >> > > >> > adding
>>>> >> > >> >> > > >> > > new
>>>> >> > >> >> > > >> > > >> >> > > > > dependencies
>>>> >> > >> >> > > >> > > >> >> > > > > > > to
>>>> >> > >> >> > > >> > > >> >> > > > > > > > a
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > java-exec and not
>>>> updating the
>>>> >> > >> excludes
>>>> >> > >> >> in
>>>> >> > >> >> > > this
>>>> >> > >> >> > > >> > > >> module.
>>>> >> > >> >> > > >> > > >> >> > This
>>>> >> > >> >> > > >> > > >> >> > > is
>>>> >> > >> >> > > >> > > >> >> > > > > > > > important
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > as it minimizes the size
>>>> of the
>>>> >> > >> >> dependency
>>>> >> > >> >> > > of
>>>> >> > >> >> > > >> > Drill
>>>> >> > >> >> > > >> > > >> >> > > application
>>>> >> > >> >> > > >> > > >> >> > > > > > > users.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > >
>>>> >> > >> >> > > >> > > >> >> > > >
>>>> >> > >> >> > > >> > > >> >> > >
>>>> >> > >> >> > > >> > > >> >> >
>>>> >> > >> >> > > >> > > >> >>
>>>> >> > >> >> > > >> > > >>
>>>> >> > >> >> > > >> > >
>>>> >> > >> >> > > >> >
>>>> >> > >> >> > > >>
>>>> >> > >> >> > >
>>>> >> > >> >> >
>>>> >> > >> >>
>>>> >> > >>
>>>> >> >
>>>> >>
>>>> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > size (44664121) too
>>>> large. Max.
>>>> >> is
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > >
>>>> >> > >> >> > > >> > > >> >> > > >
>>>> >> > >> >> > > >> > > >> >> > >
>>>> >> > >> >> > > >> > > >> >> >
>>>> >> > >> >> > > >> > > >> >>
>>>> >> > >> >> > > >> > > >>
>>>> >> > >> >> > > >> > >
>>>> >> > >> >> > > >> >
>>>> >> > >> >> > > >>
>>>> >> > >> >> > >
>>>> >> > >> >> >
>>>> >> > >> >>
>>>> >> > >>
>>>> >> >
>>>> >>
>>>> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > Administrators-MacBook-Pro-144:incubator-drill
>>>> >> > >> >> > > >> > > asinha$
>>>> >> > >> >> > > >> > > >> >> mvn
>>>> >> > >> >> > > >> > > >> >> > > > > > --version
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
>>>> >> > >> >> > > >> > > >> >> > >
>>>> (r01de14724cdef164cd33c7c8c2fe155faf9602da;
>>>> >> > >> >> > > >> > > >> >> > > > > > > > 2013-02-19
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > 05:51:28-0800)
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Maven home:
>>>> >> > >> /opt/local/share/java/maven3
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45,
>>>> vendor:
>>>> >> > >> Oracle
>>>> >> > >> >> > > >> > Corporation
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Java home:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > >
>>>> >> > >> >> > > >> > > >>
>>>> >> > >> >> > >
>>>> >> > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Default locale: en_US,
>>>> platform
>>>> >> > >> >> encoding:
>>>> >> > >> >> > > UTF-8
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > OS name: "mac os x",
>>>> version:
>>>> >> > >> "10.9.5",
>>>> >> > >> >> > > arch:
>>>> >> > >> >> > > >> > > >> "x86_64",
>>>> >> > >> >> > > >> > > >> >> > > family:
>>>> >> > >> >> > > >> > > >> >> > > > > > "mac"
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at
>>>> 8:20 AM,
>>>> >> > >> Jacques
>>>> >> > >> >> > > >> Nadeau <
>>>> >> > >> >> > > >> > > >> >> > > > > > jacques@dremio.com>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > Build back to normal.
>>>> It looks
>>>> >> > like
>>>> >> > >> >> the
>>>> >> > >> >> > > >> Apache
>>>> >> > >> >> > > >> > > >> server
>>>> >> > >> >> > > >> > > >> >> was
>>>> >> > >> >> > > >> > > >> >> > > > using
>>>> >> > >> >> > > >> > > >> >> > > > > > an
>>>> >> > >> >> > > >> > > >> >> > > > > > > > old
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I
>>>> >> > switched
>>>> >> > >> to
>>>> >> > >> >> > > >> something
>>>> >> > >> >> > > >> > > more
>>>> >> > >> >> > > >> > > >> >> > recent,
>>>> >> > >> >> > > >> > > >> >> > > > the
>>>> >> > >> >> > > >> > > >> >> > > > > > > build
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > passed.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > --
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder,
>>>> Dremio
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at
>>>> 7:02
>>>> >> AM,
>>>> >> > >> >> Jacques
>>>> >> > >> >> > > >> > Nadeau <
>>>> >> > >> >> > > >> > > >> >> > > > > > > jacques@dremio.com>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge
>>>> caused the
>>>> >> > >> Apache
>>>> >> > >> >> > > build
>>>> >> > >> >> > > >> to
>>>> >> > >> >> > > >> > > fail.
>>>> >> > >> >> > > >> > > >> >> > > > > > > Investigating...
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > --
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder,
>>>> Dremio
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016
>>>> at 6:31
>>>> >> > AM,
>>>> >> > >> >> > Jacques
>>>> >> > >> >> > > >> > > Nadeau <
>>>> >> > >> >> > > >> > > >> >> > > > > > > > jacques@dremio.com>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> I got clean
>>>> regression runs
>>>> >> > as
>>>> >> > >> >> well.
>>>> >> > >> >> > > I've
>>>> >> > >> >> > > >> > > merged
>>>> >> > >> >> > > >> > > >> the
>>>> >> > >> >> > > >> > > >> >> > > > patch.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to
>>>> start
>>>> >> the
>>>> >> > >> >> release
>>>> >> > >> >> > > >> > process?
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> --
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder,
>>>> Dremio
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016
>>>> at
>>>> >> 10:42
>>>> >> > >> PM,
>>>> >> > >> >> > > Abhishek
>>>> >> > >> >> > > >> > > >> Girish <
>>>> >> > >> >> > > >> > > >> >> > > > > > > > agirish@mapr.com
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> Had two clean
>>>> Functional
>>>> >> > runs.
>>>> >> > >> >> TPC-H
>>>> >> > >> >> > > >> SF100
>>>> >> > >> >> > > >> > > was
>>>> >> > >> >> > > >> > > >> also
>>>> >> > >> >> > > >> > > >> >> > > > > > successful.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27,
>>>> 2016 at
>>>> >> > 10:07
>>>> >> > >> PM,
>>>> >> > >> >> > > rahul
>>>> >> > >> >> > > >> > > >> >> challapalli <
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>> >> challapallirahul@gmail.com>
>>>> >> > >> >> wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a
>>>> functional
>>>> >> > run
>>>> >> > >> with
>>>> >> > >> >> > > your
>>>> >> > >> >> > > >> > > branch.
>>>> >> > >> >> > > >> > > >> >> Will
>>>> >> > >> >> > > >> > > >> >> > > let
>>>> >> > >> >> > > >> > > >> >> > > > > you
>>>> >> > >> >> > > >> > > >> >> > > > > > > know
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > once
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > it
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > finishes
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27,
>>>> 2016 at
>>>> >> > 9:56
>>>> >> > >> PM,
>>>> >> > >> >> > > >> Jacques
>>>> >> > >> >> > > >> > > >> Nadeau <
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > jacques@dremio.com>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged
>>>> >> today. I
>>>> >> > >> have
>>>> >> > >> >> an
>>>> >> > >> >> > > >> > updated
>>>> >> > >> >> > > >> > > >> patch
>>>> >> > >> >> > > >> > > >> >> > for
>>>> >> > >> >> > > >> > > >> >> > > > 4291
>>>> >> > >> >> > > >> > > >> >> > > > > > > that
>>>> >> > >> >> > > >> > > >> >> > > > > > > > is
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> ready.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately,
>>>> it
>>>> >> seems
>>>> >> > >> that
>>>> >> > >> >> > > >> something
>>>> >> > >> >> > > >> > > isn't
>>>> >> > >> >> > > >> > > >> >> > > working
>>>> >> > >> >> > > >> > > >> >> > > > > with
>>>> >> > >> >> > > >> > > >> >> > > > > > > our
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> extended
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I
>>>> haven't
>>>> >> been
>>>> >> > >> able
>>>> >> > >> >> to
>>>> >> > >> >> > > run
>>>> >> > >> >> > > >> an
>>>> >> > >> >> > > >> > > >> extended
>>>> >> > >> >> > > >> > > >> >> > > > > > regression.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > Unit
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> tests
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is
>>>> someone else
>>>> >> > >> possibly
>>>> >> > >> >> > > able
>>>> >> > >> >> > > >> to
>>>> >> > >> >> > > >> > > run a
>>>> >> > >> >> > > >> > > >> >> > > > regression
>>>> >> > >> >> > > >> > > >> >> > > > > > > suite
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > against
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > this
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so
>>>> we can
>>>> >> > >> confirm
>>>> >> > >> >> > > things
>>>> >> > >> >> > > >> > look
>>>> >> > >> >> > > >> > > >> good
>>>> >> > >> >> > > >> > > >> >> and
>>>> >> > >> >> > > >> > > >> >> > > > start
>>>> >> > >> >> > > >> > > >> >> > > > > > the
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > release
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > process?
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > thanks,
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>>>> >> > >> >> > > >> > > >> >> > > > >
>>>> >> > >> >> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > --
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and
>>>> Co-Founder,
>>>> >> > Dremio
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25,
>>>> 2016
>>>> >> at
>>>> >> > >> 11:20
>>>> >> > >> >> > AM,
>>>> >> > >> >> > > >> > Jacques
>>>> >> > >> >> > > >> > > >> >> Nadeau
>>>> >> > >> >> > > >> > > >> >> > <
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > jacques@dremio.com
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the
>>>> main
>>>> >> > things
>>>> >> > >> are
>>>> >> > >> >> > 4196
>>>> >> > >> >> > > >> and
>>>> >> > >> >> > > >> > > 4291
>>>> >> > >> >> > > >> > > >> >> > should
>>>> >> > >> >> > > >> > > >> >> > > be
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > completed.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > I
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> know
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Amit
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to
>>>> >> reproduce
>>>> >> > >> 4196
>>>> >> > >> >> > > locally
>>>> >> > >> >> > > >> > this
>>>> >> > >> >> > > >> > > >> >> weekend
>>>> >> > >> >> > > >> > > >> >> > > so
>>>> >> > >> >> > > >> > > >> >> > > > I
>>>> >> > >> >> > > >> > > >> >> > > > > > > think
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > we're
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> close
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > on
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > that.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > --
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and
>>>> Co-Founder,
>>>> >> > >> Dremio
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan
>>>> 25, 2016
>>>> >> > at
>>>> >> > >> >> 10:14
>>>> >> > >> >> > > AM,
>>>> >> > >> >> > > >> > > Zelaine
>>>> >> > >> >> > > >> > > >> >> Fong
>>>> >> > >> >> > > >> > > >> >> > <
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates
>>>> on
>>>> >> this?
>>>> >> > >> >> What's
>>>> >> > >> >> > > >> > blocking
>>>> >> > >> >> > > >> > > us
>>>> >> > >> >> > > >> > > >> >> from
>>>> >> > >> >> > > >> > > >> >> > > > taking
>>>> >> > >> >> > > >> > > >> >> > > > > > > this
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > to a
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> vote --
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > the
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge
>>>> join
>>>> >> > issues?
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan
>>>> 19,
>>>> >> 2016
>>>> >> > at
>>>> >> > >> >> 4:18
>>>> >> > >> >> > > PM,
>>>> >> > >> >> > > >> > > Jacques
>>>> >> > >> >> > > >> > > >> >> > Nadeau
>>>> >> > >> >> > > >> > > >> >> > > <
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping
>>>> this
>>>> >> > thread...
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are
>>>> the
>>>> >> issues
>>>> >> > >> that
>>>> >> > >> >> > were
>>>> >> > >> >> > > >> > > >> mentioned in
>>>> >> > >> >> > > >> > > >> >> > > this
>>>> >> > >> >> > > >> > > >> >> > > > > > thread
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > along
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> with a
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed
>>>> >> > >> categorization:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release
>>>> Blockers
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>> In-progress Amit
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>> >> > >> >> > > >> > > >>
>>>> https://issues.apache.org/jira/browse/DRILL-4190
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>> In-progress Amit
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>> >> > >> >> > > >> > > >>
>>>> https://issues.apache.org/jira/browse/DRILL-4196
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to
>>>> merge
>>>> >> > Jacques
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>> >> > >> >> > > >> > > >> >>
>>>> https://issues.apache.org/jira/browse/DRILL-4246
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review
>>>> Jinfeng
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>> >> > >> >> > > >> > > >>
>>>> https://issues.apache.org/jira/browse/DRILL-4256
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
>>>> >> Jacques
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>> >> > >> >> > > >> > > >>
>>>> https://issues.apache.org/jira/browse/DRILL-4278
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to
>>>> merge
>>>> >> > Laurent
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>> >> > >> >> > > >> > > >> >>
>>>> https://issues.apache.org/jira/browse/DRILL-4285
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to
>>>> Have
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open
>>>> Jason/Hakim
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>> >> > >> >> > > >> > > >>
>>>> https://issues.apache.org/jira/browse/DRILL-4247
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>> In-progress Jason
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>> >> > >> >> > > >> > > >>
>>>> https://issues.apache.org/jira/browse/DRILL-4203
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open
>>>> Jacques
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> https://issues.apache.org/jira/browse/DRILL-4266
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to
>>>> merge
>>>> >> > Jacques
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>> >> > >> >> > > >> > > >> >>
>>>> https://issues.apache.org/jira/browse/DRILL-4131
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do
>>>> others
>>>> >> > think?
>>>> >> > >> >> Let's
>>>> >> > >> >> > > try
>>>> >> > >> >> > > >> to
>>>> >> > >> >> > > >> > > get
>>>> >> > >> >> > > >> > > >> the
>>>> >> > >> >> > > >> > > >> >> > > > > blockers
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > wrapped
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > up
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> in
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > the
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> next
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two
>>>> and
>>>> >> > start a
>>>> >> > >> >> > > release
>>>> >> > >> >> > > >> > > vote...
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > --
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques
>>>> Nadeau
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and
>>>> >> Co-Founder,
>>>> >> > >> >> Dremio
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon,
>>>> Jan 4,
>>>> >> > 2016 at
>>>> >> > >> >> 1:48
>>>> >> > >> >> > > PM,
>>>> >> > >> >> > > >> > > Jason
>>>> >> > >> >> > > >> > > >> >> > > Altekruse
>>>> >> > >> >> > > >> > > >> >> > > > <
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>>> >> > >> altekrusejason@gmail.com>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello
>>>> All,
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the
>>>> >> > allocator
>>>> >> > >> >> > changes
>>>> >> > >> >> > > >> > merged
>>>> >> > >> >> > > >> > > and
>>>> >> > >> >> > > >> > > >> >> > about
>>>> >> > >> >> > > >> > > >> >> > > a
>>>> >> > >> >> > > >> > > >> >> > > > > > month
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > since
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > the
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> last
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think
>>>> it
>>>> >> would
>>>> >> > be
>>>> >> > >> >> good
>>>> >> > >> >> > to
>>>> >> > >> >> > > >> > start
>>>> >> > >> >> > > >> > > a
>>>> >> > >> >> > > >> > > >> vote
>>>> >> > >> >> > > >> > > >> >> > > > soon. I
>>>> >> > >> >> > > >> > > >> >> > > > > > > would
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > like
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > to
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be
>>>> release
>>>> >> > manager.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know
>>>> that
>>>> >> there
>>>> >> > >> were
>>>> >> > >> >> > some
>>>> >> > >> >> > > >> > issues
>>>> >> > >> >> > > >> > > >> that
>>>> >> > >> >> > > >> > > >> >> > were
>>>> >> > >> >> > > >> > > >> >> > > > > > > > identified
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > after
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> the
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was
>>>> >> > merged. I
>>>> >> > >> >> think
>>>> >> > >> >> > > that
>>>> >> > >> >> > > >> > > these
>>>> >> > >> >> > > >> > > >> >> > issues
>>>> >> > >> >> > > >> > > >> >> > > > > should
>>>> >> > >> >> > > >> > > >> >> > > > > > > be
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > fixed
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> before
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > we
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release
>>>> >> > candidate.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From
>>>> looking at
>>>> >> > the
>>>> >> > >> >> > > associated
>>>> >> > >> >> > > >> > > JIRAs
>>>> >> > >> >> > > >> > > >> it
>>>> >> > >> >> > > >> > > >> >> > > looked
>>>> >> > >> >> > > >> > > >> >> > > > > > like
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > there
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > was
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> a
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short
>>>> term fix
>>>> >> > just
>>>> >> > >> >> > > adjusting
>>>> >> > >> >> > > >> > the
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > option,
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> and
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more
>>>> >> > involved
>>>> >> > >> work
>>>> >> > >> >> > to
>>>> >> > >> >> > > >> > change
>>>> >> > >> >> > > >> > > >> how we
>>>> >> > >> >> > > >> > > >> >> > > > > determine
>>>> >> > >> >> > > >> > > >> >> > > > > > > the
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > correct
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > time
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > to
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill
>>>> during
>>>> >> > >> external
>>>> >> > >> >> > > sort. I
>>>> >> > >> >> > > >> > > >> believe it
>>>> >> > >> >> > > >> > > >> >> > > makes
>>>> >> > >> >> > > >> > > >> >> > > > > > sense
>>>> >> > >> >> > > >> > > >> >> > > > > > > > to
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > make
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > external
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work
>>>> well with
>>>> >> > the
>>>> >> > >> >> newly
>>>> >> > >> >> > > >> > improved
>>>> >> > >> >> > > >> > > >> memory
>>>> >> > >> >> > > >> > > >> >> > > > > > accounting
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > before
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > cutting a
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release,
>>>> but
>>>> >> I'm
>>>> >> > not
>>>> >> > >> >> sure
>>>> >> > >> >> > > how
>>>> >> > >> >> > > >> > much
>>>> >> > >> >> > > >> > > >> work
>>>> >> > >> >> > > >> > > >> >> is
>>>> >> > >> >> > > >> > > >> >> > > > left
>>>> >> > >> >> > > >> > > >> >> > > > > to
>>>> >> > >> >> > > >> > > >> >> > > > > > > be
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > done
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> there.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please
>>>> respond
>>>> >> > with
>>>> >> > >> >> your
>>>> >> > >> >> > > >> > thoughts
>>>> >> > >> >> > > >> > > on
>>>> >> > >> >> > > >> > > >> a
>>>> >> > >> >> > > >> > > >> >> > > release
>>>> >> > >> >> > > >> > > >> >> > > > > > soon
>>>> >> > >> >> > > >> > > >> >> > > > > > > > and
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > any
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> JIRAs
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > you
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to
>>>> include
>>>> >> > in
>>>> >> > >> the
>>>> >> > >> >> > > >> release.
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>> >> > >> >> > > https://issues.apache.org/jira/browse/DRILL-4243
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > > >
>>>> >> > >> >> > > >> > > >> >> > > > >
>>>> >> > >> >> > > >> > > >> >> > > >
>>>> >> > >> >> > > >> > > >> >> > >
>>>> >> > >> >> > > >> > > >> >> >
>>>> >> > >> >> > > >> > > >> >>
>>>> >> > >> >> > > >> > > >>
>>>> >> > >> >> > > >> > >
>>>> >> > >> >> > > >> >
>>>> >> > >> >> > > >>
>>>> >> > >> >> > >
>>>> >> > >> >> >
>>>> >> > >> >>
>>>> >> > >>
>>>> >> > >>
>>>> >> >
>>>> >> >
>>>> >>
>>>>
>>>
>>>
>>
>
>
> --
>
> Abdelhakim Deneche
>
> Software Engineer
>
>   <http://www.mapr.com/>
>
>
> Now Available - Free Hadoop On-Demand Training
> <http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>
>

Re: Time for a 1.5 release?

Posted by Abdel Hakim Deneche <ad...@maprtech.com>.

I have trouble understanding the meaning of "GPG signed" =P

On Mon, Feb 1, 2016 at 1:27 PM, Jason Altekruse <al...@gmail.com>
wrote:

> A question for previous release managers. Jacques was thinking I may need
> to have my GPG signed to get linked into the Apache Web of Trust. Is this
> necessary for making a release?
>
> On Mon, Feb 1, 2016 at 11:24 AM, Jason Altekruse <altekrusejason@gmail.com
> > wrote:
>
>> The only solution to this issue I have found is disabling the new plugin
>> to automatically enforce the size of the jdbc-all plugin. I don't know why
>> it is behaving differently when performing a release than it did during a
>> 'mvn install', but I have verified the correct size of the JAR manually and
>> am planning on pushing a candidate soon with the small change to disable
>> the automatic jar size enforcement.
>>
>> I will open a JIRA for getting this resolved after the release.
>>
>> On Sun, Jan 31, 2016 at 10:46 PM, Jinfeng Ni <ji...@gmail.com>
>> wrote:
>>
>>> Good to hear this, Jacques & Jason. That makes feel relieved. :-)
>>>
>>>
>>>
>>> On Sun, Jan 31, 2016 at 10:36 PM, Jason Altekruse
>>> <al...@gmail.com> wrote:
>>> > I don't think there was any issue with the merge either Jinfeng, I
>>> don't
>>> > think we need to close master. I had to step out for a few hours, but
>>> I am
>>> > working on finishing up the candidate right now.
>>> >
>>> > Having a weird RAT issue running the actual maven release command, it's
>>> > complaining about a dependency-reduced-pom.xml in the target directory
>>> of
>>> > jdbc-all lacking a license header (although target directories should
>>> be
>>> > ignored). Checking out the commit before the recent changes to the
>>> jdbc-all
>>> > exclusions seems to get rid of the issue, but the changes in the pom
>>> file
>>> > don't seem like they should be causing any problems.
>>> >
>>> > Still trying to debug and hoping to have a vote up tonight.
>>> >
>>> > On Sun, Jan 31, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com>
>>> wrote:
>>> >
>>> >> Given how easy it is to create a branch off a random commit, I
>>> generally
>>> >> don't see a point in closing the master branch. I figure the release
>>> >> manager can propose where they'd like to fork if this type of thing
>>> comes
>>> >> up.
>>> >>
>>> >> Thanks for the feedback Jinfeng. Of course you're right about Venki's
>>> >> commit. My bad.
>>> >>
>>> >> --
>>> >> Jacques Nadeau
>>> >> CTO and Co-Founder, Dremio
>>> >>
>>> >> On Sun, Jan 31, 2016 at 9:21 PM, Jinfeng Ni <ji...@gmail.com>
>>> wrote:
>>> >>
>>> >> > Venki's backwards compatibility fix (commit id:
>>> >> >    03197d0) is before the partition pruning commit. So, it seems to
>>> be
>>> >> > fine to create a release candidate from 03197d0; no need to
>>> >> > cherry-pick.
>>> >> >
>>> >> > As for whether the partition pruning changes are risky, the
>>> pre-commit
>>> >> > run did not show any serious problems in the first run; all the plan
>>> >> > changes are expected, or expose some existing problem (drill-4279).
>>> >> > However, I agree that it impacts the main code path of query
>>> planning,
>>> >> > and I'm fine that we exclude the partition change from 1.5.0
>>> release,
>>> >> > since it's merged right before we start a voting.
>>> >> >
>>> >> > I did not see any one mentioned that the master branch is closed,
>>> and
>>> >> > therefore assume it's still open for commits.  I probably should
>>> have
>>> >> > asked whether the master was open for new patches before I merged
>>> the
>>> >> > commits.
>>> >> >
>>> >> >
>>> >> >
>>> >> > On Sun, Jan 31, 2016 at 8:43 PM, Jacques Nadeau <jacques@dremio.com
>>> >
>>> >> > wrote:
>>> >> > > Jinfeng, do you agree that the changes are risky? I was going
>>> mostly by
>>> >> > > commit messages.
>>> >> > >
>>> >> > > Also just realized that the backwards compatibility fix to the
>>> storage
>>> >> > > plugin interface is after these. Could create release branch from
>>> >> > 03197d...
>>> >> > > and then cherry pick the Venki change. Or could keep them all. Or
>>> >> > > go without the compatibility fix....
>>> >> > >
>>> >> > > --
>>> >> > > Jacques Nadeau
>>> >> > > CTO and Co-Founder, Dremio
>>> >> > >
>>> >> > > On Sun, Jan 31, 2016 at 7:50 PM, Jinfeng Ni <
>>> jinfengni99@gmail.com>
>>> >> > wrote:
>>> >> > >
>>> >> > >> Hi Jason,
>>> >> > >>
>>> >> > >> I'm fine that release 1.5.0 starts from 03197d0f2.  One thing I
>>> need
>>> >> > >> point out is that the pre-commit test suite has some testcase's
>>> plan
>>> >> > >> output modified because of the partition pruning change.  If we
>>> run
>>> >> > >> pre-commit against the candidate release for 1.5.0, we have to
>>> use a
>>> >> > >> prior version of pre-commit suite.
>>> >> > >>
>>> >> > >>
>>> >> > >>
>>> >> > >> On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse
>>> >> > >> <al...@gmail.com> wrote:
>>> >> > >> > That sounds reasonable to me. I'll start preparing a release
>>> >> > candidate.
>>> >> > >> >
>>> >> > >> > Jinfeng, are you okay waiting for the next release to include
>>> this
>>> >> > >> change?
>>> >> > >> >
>>> >> > >> > On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <
>>> jacques@dremio.com
>>> >> >
>>> >> > >> wrote:
>>> >> > >> >
>>> >> > >> >> Hey Jason,
>>> >> > >> >>
>>> >> > >> >> Can you start the release vote? We've let this drag on too
>>> long.
>>> >> > >> >>
>>> >> > >> >> I suggest we start from 03197d0f2. It seems like the partition
>>> >> > pruning
>>> >> > >> >> changes are pretty complicated and on a pretty main path to
>>> add
>>> >> right
>>> >> > >> >> before the vote. Since Hsuan's fix (pr 349) hasn't been
>>> reviewed, I
>>> >> > >> think
>>> >> > >> >> we should start the vote without it.
>>> >> > >> >>
>>> >> > >> >> Jacques
>>> >> > >> >>
>>> >> > >> >>
>>> >> > >> >> --
>>> >> > >> >> Jacques Nadeau
>>> >> > >> >> CTO and Co-Founder, Dremio
>>> >> > >> >>
>>> >> > >> >> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <
>>> hyichu@maprtech.com
>>> >> >
>>> >> > >> wrote:
>>> >> > >> >>
>>> >> > >> >> > Yes, just send a pull request.
>>> >> > >> >> > https://github.com/apache/drill/pull/349
>>> >> > >> >> >
>>> >> > >> >> > Can Jinfeng review it?
>>> >> > >> >> >
>>> >> > >> >> > Thanks,
>>> >> > >> >> >
>>> >> > >> >> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <
>>> >> jinfengni99@gmail.com
>>> >> > >
>>> >> > >> >> wrote:
>>> >> > >> >> >
>>> >> > >> >> > > Sean probably is quite close to find a fix for this issue.
>>> >> Let's
>>> >> > see
>>> >> > >> >> > > whether he can post a patch soon.
>>> >> > >> >> > >
>>> >> > >> >> > >
>>> >> > >> >> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
>>> >> > >> >> > > <ch...@gmail.com> wrote:
>>> >> > >> >> > > > DRILL-4323 only happens when assertions are turned on
>>> (my
>>> >> > install
>>> >> > >> >> > script
>>> >> > >> >> > > > automatically enables assertions).
>>> >> > >> >> > > >
>>> >> > >> >> > > > - Rahul
>>> >> > >> >> > > >
>>> >> > >> >> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <
>>> >> > >> zfong@maprtech.com>
>>> >> > >> >> > > wrote:
>>> >> > >> >> > > >
>>> >> > >> >> > > >> Rahul,
>>> >> > >> >> > > >>
>>> >> > >> >> > > >> Let us know if you can reproduce the issue with
>>> assertions
>>> >> > turned
>>> >> > >> >> OFF.
>>> >> > >> >> > > If
>>> >> > >> >> > > >> you cannot, then I think this should not be a blocker
>>> for
>>> >> 1.5,
>>> >> > >> and
>>> >> > >> >> > Hsuan
>>> >> > >> >> > > >> will continue to look into fixing this.
>>> >> > >> >> > > >>
>>> >> > >> >> > > >> -- Zelaine
>>> >> > >> >> > > >>
>>> >> > >> >> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
>>> >> > >> >> > knguyen@maprtech.com>
>>> >> > >> >> > > >> wrote:
>>> >> > >> >> > > >>
>>> >> > >> >> > > >> > I do see the same error with assertion turned on.
>>> >> > >> >> > > >> >
>>> >> > >> >> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <
>>> >> > >> >> jinfengni99@gmail.com>
>>> >> > >> >> > > >> wrote:
>>> >> > >> >> > > >> >
>>> >> > >> >> > > >> > > Krystal, by any chance, did you turn on assertion
>>> when
>>> >> you
>>> >> > >> start
>>> >> > >> >> > > >> > drillbit?
>>> >> > >> >> > > >> > >
>>> >> > >> >> > > >> > > I think the error in drill-4323 will be raised when
>>> >> > >> assertion =
>>> >> > >> >> > on.
>>> >> > >> >> > > >> > >
>>> >> > >> >> > > >> > >
>>> >> > >> >> > > >> > >
>>> >> > >> >> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
>>> >> > >> >> > > knguyen@maprtech.com>
>>> >> > >> >> > > >> > > wrote:
>>> >> > >> >> > > >> > > > I am running drill-1.4 and is able to run
>>> count(*)
>>> >> > >> >> successfully.
>>> >> > >> >> > > >> > > >
>>> >> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100>
>>> alter
>>> >> > >> session
>>> >> > >> >> > set
>>> >> > >> >> > > >> > > > `store.hive.optimize_scan_with_native_readers` =
>>> true;
>>> >> > >> >> > > >> > > >
>>> >> > >> >> >
>>> >> +-------+--------------------------------------------------------+
>>> >> > >> >> > > >> > > > |  ok   |                        summary
>>> >> > >> >> >  |
>>> >> > >> >> > > >> > > >
>>> >> > >> >> >
>>> >> +-------+--------------------------------------------------------+
>>> >> > >> >> > > >> > > > | true  |
>>> store.hive.optimize_scan_with_native_readers
>>> >> > >> >> > updated.  |
>>> >> > >> >> > > >> > > >
>>> >> > >> >> >
>>> >> +-------+--------------------------------------------------------+
>>> >> > >> >> > > >> > > > 1 row selected (0.374 seconds)
>>> >> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100>
>>> select
>>> >> > >> >> count(*)
>>> >> > >> >> > > from
>>> >> > >> >> > > >> > > > lineitem_parquet_partitioned_hive_string;
>>> >> > >> >> > > >> > > > +---------+
>>> >> > >> >> > > >> > > > | EXPR$0  |
>>> >> > >> >> > > >> > > > +---------+
>>> >> > >> >> > > >> > > > | 60175   |
>>> >> > >> >> > > >> > > > +---------+
>>> >> > >> >> > > >> > > > 1 row selected (2.986 seconds)
>>> >> > >> >> > > >> > > >
>>> >> > >> >> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
>>> >> > >> >> > > jinfengni99@gmail.com>
>>> >> > >> >> > > >> > > wrote:
>>> >> > >> >> > > >> > > >
>>> >> > >> >> > > >> > > >> Venki and I did some investigation for
>>> DRILL-4323.
>>> >> The
>>> >> > >> issue
>>> >> > >> >> > > >> reported
>>> >> > >> >> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release
>>> as
>>> >> well.
>>> >> > >> Seems
>>> >> > >> >> > to
>>> >> > >> >> > > us
>>> >> > >> >> > > >> > > >> this is not a regression from 1.4.0; it's a
>>> >> regression
>>> >> > >> from
>>> >> > >> >> > 1.3.0
>>> >> > >> >> > > >> > > >> probably.
>>> >> > >> >> > > >> > > >>
>>> >> > >> >> > > >> > > >> DRILL-4083 makes the planner to use
>>> >> > DrillHiveNativeReader
>>> >> > >> in
>>> >> > >> >> > > stead
>>> >> > >> >> > > >> of
>>> >> > >> >> > > >> > > >> HiveReader for "select count(*) from hive_table"
>>> >> query.
>>> >> > >> >> > However,
>>> >> > >> >> > > >> the
>>> >> > >> >> > > >> > > >> Project after the scan produces empty schema.
>>> Before
>>> >> > >> >> > DRILL-4083,
>>> >> > >> >> > > >> > > >> Drill uses HiveScan, which works fine.
>>> >> > >> >> > > >> > > >>
>>> >> > >> >> > > >> > > >>
>>> >> > >> >> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
>>> >> > >> >> > > >> > > >> <al...@gmail.com> wrote:
>>> >> > >> >> > > >> > > >> > We could revert the change to the
>>> StoragePlugin
>>> >> > >> interface
>>> >> > >> >> to
>>> >> > >> >> > > add
>>> >> > >> >> > > >> the
>>> >> > >> >> > > >> > > >> > getLogicalOptimizerRules
>>> >> > >> >> > > >> > > >> > and getPhysicalOptimizerRules, and only put
>>> those
>>> >> in
>>> >> > >> >> > > >> > > >> AbstractStoragePlugin
>>> >> > >> >> > > >> > > >> > with default implementations.
>>> >> > >> >> > > >> > > >> >
>>> >> > >> >> > > >> > > >> > We could then update the docs to tell users
>>> to not
>>> >> > >> >> implement
>>> >> > >> >> > > the
>>> >> > >> >> > > >> > > >> interface
>>> >> > >> >> > > >> > > >> > directly, and instead extend the abstract
>>> class,
>>> >> > which I
>>> >> > >> >> > > believe
>>> >> > >> >> > > >> > would
>>> >> > >> >> > > >> > > >> > allow us to make changes like this in the
>>> future
>>> >> > without
>>> >> > >> >> > > breaking
>>> >> > >> >> > > >> > > >> plugins.
>>> >> > >> >> > > >> > > >> > At a major release version we could then just
>>> pull
>>> >> up
>>> >> > >> all
>>> >> > >> >> of
>>> >> > >> >> > > the
>>> >> > >> >> > > >> new
>>> >> > >> >> > > >> > > >> > methods we added to the abstract class into
>>> the
>>> >> > >> interface,
>>> >> > >> >> or
>>> >> > >> >> > > just
>>> >> > >> >> > > >> > get
>>> >> > >> >> > > >> > > >> rid
>>> >> > >> >> > > >> > > >> > of it all-together.
>>> >> > >> >> > > >> > > >> >
>>> >> > >> >> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki
>>> Korukanti <
>>> >> > >> >> > > >> > > >> venki.korukanti@gmail.com>
>>> >> > >> >> > > >> > > >> > wrote:
>>> >> > >> >> > > >> > > >> >
>>> >> > >> >> > > >> > > >> >> Here is the partial fix:
>>> >> > >> >> > > >> > > >> >>
>>> >> > >> >> > > >> > > >> >>
>>> >> > >> >> > > >> > > >>
>>> >> > >> >> > > >> > >
>>> >> > >> >> > > >> >
>>> >> > >> >> > > >>
>>> >> > >> >> > >
>>> >> > >> >> >
>>> >> > >> >>
>>> >> > >>
>>> >> >
>>> >>
>>> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
>>> >> > >> >> > > >> > > >> >>
>>> >> > >> >> > > >> > > >> >> If the existing StragePlugins are
>>> implementing
>>> >> > >> >> > > >> > AbstractStoragePlugin
>>> >> > >> >> > > >> > > >> class
>>> >> > >> >> > > >> > > >> >> this fix should resolve the backward compat
>>> issue.
>>> >> > If
>>> >> > >> they
>>> >> > >> >> > are
>>> >> > >> >> > > >> > > >> implementing
>>> >> > >> >> > > >> > > >> >> StoragePlugin interface directly then it
>>> won't
>>> >> > resolve
>>> >> > >> the
>>> >> > >> >> > > issue.
>>> >> > >> >> > > >> > Not
>>> >> > >> >> > > >> > > >> sure
>>> >> > >> >> > > >> > > >> >> how to handle it in that case. Let me know if
>>> >> there
>>> >> > any
>>> >> > >> >> ways
>>> >> > >> >> > > to
>>> >> > >> >> > > >> > > resolve
>>> >> > >> >> > > >> > > >> >> this issue completely.
>>> >> > >> >> > > >> > > >> >>
>>> >> > >> >> > > >> > > >> >> Thanks
>>> >> > >> >> > > >> > > >> >> Venki
>>> >> > >> >> > > >> > > >> >>
>>> >> > >> >> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <
>>> >> > >> >> > > asinha@maprtech.com
>>> >> > >> >> > > >> >
>>> >> > >> >> > > >> > > >> wrote:
>>> >> > >> >> > > >> > > >> >>
>>> >> > >> >> > > >> > > >> >> > I was able to do the build with maven 3.3.9
>>> >> (after
>>> >> > >> doing
>>> >> > >> >> > > 'brew
>>> >> > >> >> > > >> > > install
>>> >> > >> >> > > >> > > >> >> > maven').  Here are my env variables
>>> related to
>>> >> > maven:
>>> >> > >> >> > > >> > > >> >> >
>>> M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
>>> >> > >> >> > > >> > > >> >> >
>>> M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
>>> >> > >> >> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
>>> >> > >> >> > > >> > > >> >> >
>>> >> > >> >> > > >> > > >> >> >
>>> >> > >> >> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques
>>> Nadeau
>>> >> <
>>> >> > >> >> > > >> > > jacques@dremio.com>
>>> >> > >> >> > > >> > > >> >> > wrote:
>>> >> > >> >> > > >> > > >> >> >
>>> >> > >> >> > > >> > > >> >> > > Aman, for reference on the Maven version
>>> >> issue.
>>> >> > You
>>> >> > >> >> can
>>> >> > >> >> > > see
>>> >> > >> >> > > >> the
>>> >> > >> >> > > >> > > >> output
>>> >> > >> >> > > >> > > >> >> of
>>> >> > >> >> > > >> > > >> >> > > the shade plugin between the two Apache
>>> >> Builds.
>>> >> > If
>>> >> > >> you
>>> >> > >> >> > > search
>>> >> > >> >> > > >> > for
>>> >> > >> >> > > >> > > >> >> > > maven-shade-plugin you'll see that build
>>> 647
>>> >> > (maven
>>> >> > >> >> > 3.0.5)
>>> >> > >> >> > > >> > fails
>>> >> > >> >> > > >> > > >> >> because
>>> >> > >> >> > > >> > > >> >> > it
>>> >> > >> >> > > >> > > >> >> > > includes a bunch of hadoop (and other
>>> >> libraries
>>> >> > >> that
>>> >> > >> >> are
>>> >> > >> >> > > >> listed
>>> >> > >> >> > > >> > > as
>>> >> > >> >> > > >> > > >> >> being
>>> >> > >> >> > > >> > > >> >> > > excluded). If you look at build 648
>>> (maven
>>> >> > 3.3.3),
>>> >> > >> >> those
>>> >> > >> >> > > >> items
>>> >> > >> >> > > >> > > are
>>> >> > >> >> > > >> > > >> >> > included
>>> >> > >> >> > > >> > > >> >> > > so that the build completes successfully.
>>> >> > >> >> > > >> > > >> >> > >
>>> >> > >> >> > > >> > > >> >> > >
>>> >> > >> >> https://builds.apache.org/job/drill-scm/647/consoleText
>>> >> > >> >> > > >> > > >> >> > >
>>> >> > >> >> https://builds.apache.org/job/drill-scm/648/consoleText
>>> >> > >> >> > > >> > > >> >> > >
>>> >> > >> >> > > >> > > >> >> > > --
>>> >> > >> >> > > >> > > >> >> > > Jacques Nadeau
>>> >> > >> >> > > >> > > >> >> > > CTO and Co-Founder, Dremio
>>> >> > >> >> > > >> > > >> >> > >
>>> >> > >> >> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul
>>> >> > challapalli
>>> >> > >> <
>>> >> > >> >> > > >> > > >> >> > > challapallirahul@gmail.com> wrote:
>>> >> > >> >> > > >> > > >> >> > >
>>> >> > >> >> > > >> > > >> >> > > > Jason,
>>> >> > >> >> > > >> > > >> >> > > >
>>> >> > >> >> > > >> > > >> >> > > > As of now I haven't narrowed down the
>>> exact
>>> >> > >> commit
>>> >> > >> >> > which
>>> >> > >> >> > > >> > caused
>>> >> > >> >> > > >> > > >> >> > > DRILL-4323
>>> >> > >> >> > > >> > > >> >> > > > and I do not know the scope of the fix.
>>> >> > >> >> > > >> > > >> >> > > >
>>> >> > >> >> > > >> > > >> >> > > > - Rahul
>>> >> > >> >> > > >> > > >> >> > > >
>>> >> > >> >> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason
>>> >> > Altekruse
>>> >> > >> <
>>> >> > >> >> > > >> > > >> >> > > altekrusejason@gmail.com
>>> >> > >> >> > > >> > > >> >> > > > >
>>> >> > >> >> > > >> > > >> >> > > > wrote:
>>> >> > >> >> > > >> > > >> >> > > >
>>> >> > >> >> > > >> > > >> >> > > > > I think it makes sense to wait for
>>> Venki's
>>> >> > fix
>>> >> > >> of
>>> >> > >> >> > the
>>> >> > >> >> > > >> API,
>>> >> > >> >> > > >> > as
>>> >> > >> >> > > >> > > >> >> > > releasing a
>>> >> > >> >> > > >> > > >> >> > > > > version with the change would make
>>> >> > >> >> > > reverting/refactoring
>>> >> > >> >> > > >> > the
>>> >> > >> >> > > >> > > API
>>> >> > >> >> > > >> > > >> >> > > breaking
>>> >> > >> >> > > >> > > >> >> > > > > change less useful.
>>> >> > >> >> > > >> > > >> >> > > > >
>>> >> > >> >> > > >> > > >> >> > > > > Do you have any idea about the scope
>>> of
>>> >> > fixing
>>> >> > >> the
>>> >> > >> >> > > hive
>>> >> > >> >> > > >> > > native
>>> >> > >> >> > > >> > > >> >> > reader,
>>> >> > >> >> > > >> > > >> >> > > or
>>> >> > >> >> > > >> > > >> >> > > > > which change since the 1.4 release
>>> broke
>>> >> > it? Is
>>> >> > >> >> > > someone
>>> >> > >> >> > > >> > > >> available
>>> >> > >> >> > > >> > > >> >> to
>>> >> > >> >> > > >> > > >> >> > > look
>>> >> > >> >> > > >> > > >> >> > > > > at it today?
>>> >> > >> >> > > >> > > >> >> > > > >
>>> >> > >> >> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM,
>>> rahul
>>> >> > >> >> challapalli <
>>> >> > >> >> > > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
>>> >> > >> >> > > >> > > >> >> > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > Just found an issue with hive
>>> native
>>> >> > parquet
>>> >> > >> >> > reader
>>> >> > >> >> > > >> > > >> (DRILL-4323).
>>> >> > >> >> > > >> > > >> >> > > This
>>> >> > >> >> > > >> > > >> >> > > > > is a
>>> >> > >> >> > > >> > > >> >> > > > > > regression from 1.4 and in my
>>> opinion
>>> >> > should
>>> >> > >> be
>>> >> > >> >> > > treated
>>> >> > >> >> > > >> > as
>>> >> > >> >> > > >> > > a
>>> >> > >> >> > > >> > > >> >> > blocker.
>>> >> > >> >> > > >> > > >> >> > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > - Rahul
>>> >> > >> >> > > >> > > >> >> > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM,
>>> Venki
>>> >> > >> >> Korukanti <
>>> >> > >> >> > > >> > > >> >> > > > > > venki.korukanti@gmail.com>
>>> >> > >> >> > > >> > > >> >> > > > > > wrote:
>>> >> > >> >> > > >> > > >> >> > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > Sorry I didn't realize I was
>>> breaking
>>> >> > the
>>> >> > >> >> public
>>> >> > >> >> > > >> > > interface
>>> >> > >> >> > > >> > > >> API.
>>> >> > >> >> > > >> > > >> >> > Let
>>> >> > >> >> > > >> > > >> >> > > > me
>>> >> > >> >> > > >> > > >> >> > > > > > work
>>> >> > >> >> > > >> > > >> >> > > > > > > on a patch to make it backward
>>> >> > compatible.
>>> >> > >> >> > > >> > > >> >> > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > Thanks
>>> >> > >> >> > > >> > > >> >> > > > > > > Venki
>>> >> > >> >> > > >> > > >> >> > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM,
>>> >> Steven
>>> >> > >> >> > Phillips
>>> >> > >> >> > > <
>>> >> > >> >> > > >> > > >> >> > > steven@dremio.com
>>> >> > >> >> > > >> > > >> >> > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > I just wanted to bring up an
>>> issue
>>> >> > that I
>>> >> > >> >> just
>>> >> > >> >> > > now
>>> >> > >> >> > > >> > > >> >> discovered,
>>> >> > >> >> > > >> > > >> >> > > that
>>> >> > >> >> > > >> > > >> >> > > > > has
>>> >> > >> >> > > >> > > >> >> > > > > > > > caused me a fair amount of
>>> grief.
>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>> >> > >> >> > > https://github.com/apache/drill/pull/300/commits
>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > DRILL-4198 changes a
>>> user-facing
>>> >> API,
>>> >> > and
>>> >> > >> >> > causes
>>> >> > >> >> > > >> > > >> >> StoragePlugins
>>> >> > >> >> > > >> > > >> >> > > > that
>>> >> > >> >> > > >> > > >> >> > > > > > were
>>> >> > >> >> > > >> > > >> >> > > > > > > > compiled against currently
>>> released
>>> >> > >> versions
>>> >> > >> >> > of
>>> >> > >> >> > > >> Drill
>>> >> > >> >> > > >> > > to
>>> >> > >> >> > > >> > > >> no
>>> >> > >> >> > > >> > > >> >> > > longer
>>> >> > >> >> > > >> > > >> >> > > > > > > > functional properly. I would
>>> prefer
>>> >> > that
>>> >> > >> >> this
>>> >> > >> >> > > >> > breaking
>>> >> > >> >> > > >> > > >> change
>>> >> > >> >> > > >> > > >> >> > be
>>> >> > >> >> > > >> > > >> >> > > > > > modified
>>> >> > >> >> > > >> > > >> >> > > > > > > > to be backward compatible if
>>> >> possible.
>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23
>>> AM,
>>> >> > Jason
>>> >> > >> >> > > Altekruse <
>>> >> > >> >> > > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
>>> >> > >> >> > > >> > > >> >> > > > > > > > wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > Hi Aman,
>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > This is the failure that he
>>> was
>>> >> > >> seeing. He
>>> >> > >> >> > > >> figured
>>> >> > >> >> > > >> > > out
>>> >> > >> >> > > >> > > >> that
>>> >> > >> >> > > >> > > >> >> > the
>>> >> > >> >> > > >> > > >> >> > > > new
>>> >> > >> >> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were
>>> not
>>> >> > being
>>> >> > >> >> > > respected
>>> >> > >> >> > > >> > when
>>> >> > >> >> > > >> > > the
>>> >> > >> >> > > >> > > >> >> > build
>>> >> > >> >> > > >> > > >> >> > > > was
>>> >> > >> >> > > >> > > >> >> > > > > > run
>>> >> > >> >> > > >> > > >> >> > > > > > > > with
>>> >> > >> >> > > >> > > >> >> > > > > > > > > an older Maven version,
>>> causing
>>> >> the
>>> >> > jar
>>> >> > >> >> size
>>> >> > >> >> > > to
>>> >> > >> >> > > >> > > increase
>>> >> > >> >> > > >> > > >> >> > > > > > significantly.
>>> >> > >> >> > > >> > > >> >> > > > > > > > He
>>> >> > >> >> > > >> > > >> >> > > > > > > > > added an enforcer to make
>>> sure the
>>> >> > JAR
>>> >> > >> >> > didn't
>>> >> > >> >> > > >> grow
>>> >> > >> >> > > >> > > >> >> > > unexpectedly.
>>> >> > >> >> > > >> > > >> >> > > > > Can
>>> >> > >> >> > > >> > > >> >> > > > > > > you
>>> >> > >> >> > > >> > > >> >> > > > > > > > > try to update your maven
>>> version
>>> >> and
>>> >> > >> >> re-run
>>> >> > >> >> > > the
>>> >> > >> >> > > >> > > build?
>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > - Jason
>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at
>>> 11:18 AM,
>>> >> > Aman
>>> >> > >> >> > Sinha <
>>> >> > >> >> > > >> > > >> >> > > > asinha@maprtech.com>
>>> >> > >> >> > > >> > > >> >> > > > > > > > wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the
>>> >> > following
>>> >> > >> >> build
>>> >> > >> >> > > >> failure
>>> >> > >> >> > > >> > > on
>>> >> > >> >> > > >> > > >> the
>>> >> > >> >> > > >> > > >> >> > > latest
>>> >> > >> >> > > >> > > >> >> > > > > > > master
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > branch...is this what you
>>> saw
>>> >> for
>>> >> > the
>>> >> > >> >> > Apache
>>> >> > >> >> > > >> > build
>>> >> > >> >> > > >> > > ?
>>> >> > >> >> > > >> > > >> My
>>> >> > >> >> > > >> > > >> >> > mvn
>>> >> > >> >> > > >> > > >> >> > > > > > version
>>> >> > >> >> > > >> > > >> >> > > > > > > > > output
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > is shown below.  Should we
>>> all
>>> >> be
>>> >> > >> >> > upgrading
>>> >> > >> >> > > to
>>> >> > >> >> > > >> a
>>> >> > >> >> > > >> > > newer
>>> >> > >> >> > > >> > > >> >> mvn
>>> >> > >> >> > > >> > > >> >> > ?
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > [INFO] ---
>>> >> > >> >> > > maven-enforcer-plugin:1.3.1:enforce
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> (enforce-jdbc-jar-compactness) @
>>> >> > >> >> > > drill-jdbc-all
>>> >> > >> >> > > >> > ---
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
>>> >> > >> >> > > >> > > >> >> > > > > >
>>> >> > >> >> org.apache.maven.plugins.enforcer.RequireFilesSize
>>> >> > >> >> > > >> > > >> >> > > > > > > > > failed
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > with message:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > The file
>>> >> > >> >> drill-jdbc-all-1.5.0-SNAPSHOT.jar
>>> >> > >> >> > > is
>>> >> > >> >> > > >> > > outside
>>> >> > >> >> > > >> > > >> the
>>> >> > >> >> > > >> > > >> >> > > > > expected
>>> >> > >> >> > > >> > > >> >> > > > > > > size
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > range.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >                   This is
>>> likely
>>> >> > due
>>> >> > >> to
>>> >> > >> >> > you
>>> >> > >> >> > > >> > adding
>>> >> > >> >> > > >> > > new
>>> >> > >> >> > > >> > > >> >> > > > > dependencies
>>> >> > >> >> > > >> > > >> >> > > > > > > to
>>> >> > >> >> > > >> > > >> >> > > > > > > > a
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > java-exec and not updating
>>> the
>>> >> > >> excludes
>>> >> > >> >> in
>>> >> > >> >> > > this
>>> >> > >> >> > > >> > > >> module.
>>> >> > >> >> > > >> > > >> >> > This
>>> >> > >> >> > > >> > > >> >> > > is
>>> >> > >> >> > > >> > > >> >> > > > > > > > important
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > as it minimizes the size
>>> of the
>>> >> > >> >> dependency
>>> >> > >> >> > > of
>>> >> > >> >> > > >> > Drill
>>> >> > >> >> > > >> > > >> >> > > application
>>> >> > >> >> > > >> > > >> >> > > > > > > users.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > >
>>> >> > >> >> > > >> > > >> >> > > > >
>>> >> > >> >> > > >> > > >> >> > > >
>>> >> > >> >> > > >> > > >> >> > >
>>> >> > >> >> > > >> > > >> >> >
>>> >> > >> >> > > >> > > >> >>
>>> >> > >> >> > > >> > > >>
>>> >> > >> >> > > >> > >
>>> >> > >> >> > > >> >
>>> >> > >> >> > > >>
>>> >> > >> >> > >
>>> >> > >> >> >
>>> >> > >> >>
>>> >> > >>
>>> >> >
>>> >>
>>> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > size (44664121) too large.
>>> Max.
>>> >> is
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > >
>>> >> > >> >> > > >> > > >> >> > > > >
>>> >> > >> >> > > >> > > >> >> > > >
>>> >> > >> >> > > >> > > >> >> > >
>>> >> > >> >> > > >> > > >> >> >
>>> >> > >> >> > > >> > > >> >>
>>> >> > >> >> > > >> > > >>
>>> >> > >> >> > > >> > >
>>> >> > >> >> > > >> >
>>> >> > >> >> > > >>
>>> >> > >> >> > >
>>> >> > >> >> >
>>> >> > >> >>
>>> >> > >>
>>> >> >
>>> >>
>>> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > Administrators-MacBook-Pro-144:incubator-drill
>>> >> > >> >> > > >> > > asinha$
>>> >> > >> >> > > >> > > >> >> mvn
>>> >> > >> >> > > >> > > >> >> > > > > > --version
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
>>> >> > >> >> > > >> > > >> >> > >
>>> (r01de14724cdef164cd33c7c8c2fe155faf9602da;
>>> >> > >> >> > > >> > > >> >> > > > > > > > 2013-02-19
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > 05:51:28-0800)
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Maven home:
>>> >> > >> /opt/local/share/java/maven3
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45,
>>> vendor:
>>> >> > >> Oracle
>>> >> > >> >> > > >> > Corporation
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Java home:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > >
>>> >> > >> >> > > >> > > >>
>>> >> > >> >> > >
>>> >> > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > Default locale: en_US,
>>> platform
>>> >> > >> >> encoding:
>>> >> > >> >> > > UTF-8
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > OS name: "mac os x",
>>> version:
>>> >> > >> "10.9.5",
>>> >> > >> >> > > arch:
>>> >> > >> >> > > >> > > >> "x86_64",
>>> >> > >> >> > > >> > > >> >> > > family:
>>> >> > >> >> > > >> > > >> >> > > > > > "mac"
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at
>>> 8:20 AM,
>>> >> > >> Jacques
>>> >> > >> >> > > >> Nadeau <
>>> >> > >> >> > > >> > > >> >> > > > > > jacques@dremio.com>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > Build back to normal. It
>>> looks
>>> >> > like
>>> >> > >> >> the
>>> >> > >> >> > > >> Apache
>>> >> > >> >> > > >> > > >> server
>>> >> > >> >> > > >> > > >> >> was
>>> >> > >> >> > > >> > > >> >> > > > using
>>> >> > >> >> > > >> > > >> >> > > > > > an
>>> >> > >> >> > > >> > > >> >> > > > > > > > old
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I
>>> >> > switched
>>> >> > >> to
>>> >> > >> >> > > >> something
>>> >> > >> >> > > >> > > more
>>> >> > >> >> > > >> > > >> >> > recent,
>>> >> > >> >> > > >> > > >> >> > > > the
>>> >> > >> >> > > >> > > >> >> > > > > > > build
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > passed.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > --
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder,
>>> Dremio
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at
>>> 7:02
>>> >> AM,
>>> >> > >> >> Jacques
>>> >> > >> >> > > >> > Nadeau <
>>> >> > >> >> > > >> > > >> >> > > > > > > jacques@dremio.com>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge
>>> caused the
>>> >> > >> Apache
>>> >> > >> >> > > build
>>> >> > >> >> > > >> to
>>> >> > >> >> > > >> > > fail.
>>> >> > >> >> > > >> > > >> >> > > > > > > Investigating...
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > --
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder,
>>> Dremio
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016
>>> at 6:31
>>> >> > AM,
>>> >> > >> >> > Jacques
>>> >> > >> >> > > >> > > Nadeau <
>>> >> > >> >> > > >> > > >> >> > > > > > > > jacques@dremio.com>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> I got clean
>>> regression runs
>>> >> > as
>>> >> > >> >> well.
>>> >> > >> >> > > I've
>>> >> > >> >> > > >> > > merged
>>> >> > >> >> > > >> > > >> the
>>> >> > >> >> > > >> > > >> >> > > > patch.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to
>>> start
>>> >> the
>>> >> > >> >> release
>>> >> > >> >> > > >> > process?
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> --
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder,
>>> Dremio
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016
>>> at
>>> >> 10:42
>>> >> > >> PM,
>>> >> > >> >> > > Abhishek
>>> >> > >> >> > > >> > > >> Girish <
>>> >> > >> >> > > >> > > >> >> > > > > > > > agirish@mapr.com
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> Had two clean
>>> Functional
>>> >> > runs.
>>> >> > >> >> TPC-H
>>> >> > >> >> > > >> SF100
>>> >> > >> >> > > >> > > was
>>> >> > >> >> > > >> > > >> also
>>> >> > >> >> > > >> > > >> >> > > > > > successful.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016
>>> at
>>> >> > 10:07
>>> >> > >> PM,
>>> >> > >> >> > > rahul
>>> >> > >> >> > > >> > > >> >> challapalli <
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>> >> challapallirahul@gmail.com>
>>> >> > >> >> wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a
>>> functional
>>> >> > run
>>> >> > >> with
>>> >> > >> >> > > your
>>> >> > >> >> > > >> > > branch.
>>> >> > >> >> > > >> > > >> >> Will
>>> >> > >> >> > > >> > > >> >> > > let
>>> >> > >> >> > > >> > > >> >> > > > > you
>>> >> > >> >> > > >> > > >> >> > > > > > > know
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > once
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > it
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > finishes
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27,
>>> 2016 at
>>> >> > 9:56
>>> >> > >> PM,
>>> >> > >> >> > > >> Jacques
>>> >> > >> >> > > >> > > >> Nadeau <
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > jacques@dremio.com>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged
>>> >> today. I
>>> >> > >> have
>>> >> > >> >> an
>>> >> > >> >> > > >> > updated
>>> >> > >> >> > > >> > > >> patch
>>> >> > >> >> > > >> > > >> >> > for
>>> >> > >> >> > > >> > > >> >> > > > 4291
>>> >> > >> >> > > >> > > >> >> > > > > > > that
>>> >> > >> >> > > >> > > >> >> > > > > > > > is
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> ready.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it
>>> >> seems
>>> >> > >> that
>>> >> > >> >> > > >> something
>>> >> > >> >> > > >> > > isn't
>>> >> > >> >> > > >> > > >> >> > > working
>>> >> > >> >> > > >> > > >> >> > > > > with
>>> >> > >> >> > > >> > > >> >> > > > > > > our
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> extended
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I
>>> haven't
>>> >> been
>>> >> > >> able
>>> >> > >> >> to
>>> >> > >> >> > > run
>>> >> > >> >> > > >> an
>>> >> > >> >> > > >> > > >> extended
>>> >> > >> >> > > >> > > >> >> > > > > > regression.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > Unit
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> tests
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone
>>> else
>>> >> > >> possibly
>>> >> > >> >> > > able
>>> >> > >> >> > > >> to
>>> >> > >> >> > > >> > > run a
>>> >> > >> >> > > >> > > >> >> > > > regression
>>> >> > >> >> > > >> > > >> >> > > > > > > suite
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > against
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > this
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we
>>> can
>>> >> > >> confirm
>>> >> > >> >> > > things
>>> >> > >> >> > > >> > look
>>> >> > >> >> > > >> > > >> good
>>> >> > >> >> > > >> > > >> >> and
>>> >> > >> >> > > >> > > >> >> > > > start
>>> >> > >> >> > > >> > > >> >> > > > > > the
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > release
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > process?
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > thanks,
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>>> >> > >> >> > > >> > > >> >> > > > >
>>> >> > >> >> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > --
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and
>>> Co-Founder,
>>> >> > Dremio
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25,
>>> 2016
>>> >> at
>>> >> > >> 11:20
>>> >> > >> >> > AM,
>>> >> > >> >> > > >> > Jacques
>>> >> > >> >> > > >> > > >> >> Nadeau
>>> >> > >> >> > > >> > > >> >> > <
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > jacques@dremio.com
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the
>>> main
>>> >> > things
>>> >> > >> are
>>> >> > >> >> > 4196
>>> >> > >> >> > > >> and
>>> >> > >> >> > > >> > > 4291
>>> >> > >> >> > > >> > > >> >> > should
>>> >> > >> >> > > >> > > >> >> > > be
>>> >> > >> >> > > >> > > >> >> > > > > > > > > completed.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > I
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> know
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Amit
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to
>>> >> reproduce
>>> >> > >> 4196
>>> >> > >> >> > > locally
>>> >> > >> >> > > >> > this
>>> >> > >> >> > > >> > > >> >> weekend
>>> >> > >> >> > > >> > > >> >> > > so
>>> >> > >> >> > > >> > > >> >> > > > I
>>> >> > >> >> > > >> > > >> >> > > > > > > think
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > we're
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> close
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > on
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > that.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > --
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and
>>> Co-Founder,
>>> >> > >> Dremio
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan
>>> 25, 2016
>>> >> > at
>>> >> > >> >> 10:14
>>> >> > >> >> > > AM,
>>> >> > >> >> > > >> > > Zelaine
>>> >> > >> >> > > >> > > >> >> Fong
>>> >> > >> >> > > >> > > >> >> > <
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on
>>> >> this?
>>> >> > >> >> What's
>>> >> > >> >> > > >> > blocking
>>> >> > >> >> > > >> > > us
>>> >> > >> >> > > >> > > >> >> from
>>> >> > >> >> > > >> > > >> >> > > > taking
>>> >> > >> >> > > >> > > >> >> > > > > > > this
>>> >> > >> >> > > >> > > >> >> > > > > > > > > to a
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> vote --
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > the
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge
>>> join
>>> >> > issues?
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan
>>> 19,
>>> >> 2016
>>> >> > at
>>> >> > >> >> 4:18
>>> >> > >> >> > > PM,
>>> >> > >> >> > > >> > > Jacques
>>> >> > >> >> > > >> > > >> >> > Nadeau
>>> >> > >> >> > > >> > > >> >> > > <
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this
>>> >> > thread...
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the
>>> >> issues
>>> >> > >> that
>>> >> > >> >> > were
>>> >> > >> >> > > >> > > >> mentioned in
>>> >> > >> >> > > >> > > >> >> > > this
>>> >> > >> >> > > >> > > >> >> > > > > > thread
>>> >> > >> >> > > >> > > >> >> > > > > > > > > along
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> with a
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed
>>> >> > >> categorization:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release
>>> Blockers
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
>>> Amit
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>> >> > >> >> > > >> > > >>
>>> https://issues.apache.org/jira/browse/DRILL-4190
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
>>> Amit
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>> >> > >> >> > > >> > > >>
>>> https://issues.apache.org/jira/browse/DRILL-4196
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to
>>> merge
>>> >> > Jacques
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>> >> > >> >> > > >> > > >> >>
>>> https://issues.apache.org/jira/browse/DRILL-4246
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review
>>> Jinfeng
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>> >> > >> >> > > >> > > >>
>>> https://issues.apache.org/jira/browse/DRILL-4256
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
>>> >> Jacques
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>> >> > >> >> > > >> > > >>
>>> https://issues.apache.org/jira/browse/DRILL-4278
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to
>>> merge
>>> >> > Laurent
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>> >> > >> >> > > >> > > >> >>
>>> https://issues.apache.org/jira/browse/DRILL-4285
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open
>>> Jason/Hakim
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>> >> > >> >> > > >> > > >>
>>> https://issues.apache.org/jira/browse/DRILL-4247
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
>>> Jason
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>> >> > >> >> > > >> > > >>
>>> https://issues.apache.org/jira/browse/DRILL-4203
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> https://issues.apache.org/jira/browse/DRILL-4266
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to
>>> merge
>>> >> > Jacques
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>> >> > >> >> > > >> > > >> >>
>>> https://issues.apache.org/jira/browse/DRILL-4131
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do
>>> others
>>> >> > think?
>>> >> > >> >> Let's
>>> >> > >> >> > > try
>>> >> > >> >> > > >> to
>>> >> > >> >> > > >> > > get
>>> >> > >> >> > > >> > > >> the
>>> >> > >> >> > > >> > > >> >> > > > > blockers
>>> >> > >> >> > > >> > > >> >> > > > > > > > > wrapped
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > up
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> in
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > the
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> next
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two
>>> and
>>> >> > start a
>>> >> > >> >> > > release
>>> >> > >> >> > > >> > > vote...
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > --
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques
>>> Nadeau
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and
>>> >> Co-Founder,
>>> >> > >> >> Dremio
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan
>>> 4,
>>> >> > 2016 at
>>> >> > >> >> 1:48
>>> >> > >> >> > > PM,
>>> >> > >> >> > > >> > > Jason
>>> >> > >> >> > > >> > > >> >> > > Altekruse
>>> >> > >> >> > > >> > > >> >> > > > <
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>> >> > >> altekrusejason@gmail.com>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the
>>> >> > allocator
>>> >> > >> >> > changes
>>> >> > >> >> > > >> > merged
>>> >> > >> >> > > >> > > and
>>> >> > >> >> > > >> > > >> >> > about
>>> >> > >> >> > > >> > > >> >> > > a
>>> >> > >> >> > > >> > > >> >> > > > > > month
>>> >> > >> >> > > >> > > >> >> > > > > > > > > since
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > the
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> last
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it
>>> >> would
>>> >> > be
>>> >> > >> >> good
>>> >> > >> >> > to
>>> >> > >> >> > > >> > start
>>> >> > >> >> > > >> > > a
>>> >> > >> >> > > >> > > >> vote
>>> >> > >> >> > > >> > > >> >> > > > soon. I
>>> >> > >> >> > > >> > > >> >> > > > > > > would
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > like
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > to
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release
>>> >> > manager.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know
>>> that
>>> >> there
>>> >> > >> were
>>> >> > >> >> > some
>>> >> > >> >> > > >> > issues
>>> >> > >> >> > > >> > > >> that
>>> >> > >> >> > > >> > > >> >> > were
>>> >> > >> >> > > >> > > >> >> > > > > > > > identified
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > after
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> the
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was
>>> >> > merged. I
>>> >> > >> >> think
>>> >> > >> >> > > that
>>> >> > >> >> > > >> > > these
>>> >> > >> >> > > >> > > >> >> > issues
>>> >> > >> >> > > >> > > >> >> > > > > should
>>> >> > >> >> > > >> > > >> >> > > > > > > be
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > fixed
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> before
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > we
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release
>>> >> > candidate.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From
>>> looking at
>>> >> > the
>>> >> > >> >> > > associated
>>> >> > >> >> > > >> > > JIRAs
>>> >> > >> >> > > >> > > >> it
>>> >> > >> >> > > >> > > >> >> > > looked
>>> >> > >> >> > > >> > > >> >> > > > > > like
>>> >> > >> >> > > >> > > >> >> > > > > > > > > there
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > was
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> a
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short
>>> term fix
>>> >> > just
>>> >> > >> >> > > adjusting
>>> >> > >> >> > > >> > the
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > option,
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> and
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more
>>> >> > involved
>>> >> > >> work
>>> >> > >> >> > to
>>> >> > >> >> > > >> > change
>>> >> > >> >> > > >> > > >> how we
>>> >> > >> >> > > >> > > >> >> > > > > determine
>>> >> > >> >> > > >> > > >> >> > > > > > > the
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > correct
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > time
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > to
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill
>>> during
>>> >> > >> external
>>> >> > >> >> > > sort. I
>>> >> > >> >> > > >> > > >> believe it
>>> >> > >> >> > > >> > > >> >> > > makes
>>> >> > >> >> > > >> > > >> >> > > > > > sense
>>> >> > >> >> > > >> > > >> >> > > > > > > > to
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > make
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > external
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well
>>> with
>>> >> > the
>>> >> > >> >> newly
>>> >> > >> >> > > >> > improved
>>> >> > >> >> > > >> > > >> memory
>>> >> > >> >> > > >> > > >> >> > > > > > accounting
>>> >> > >> >> > > >> > > >> >> > > > > > > > > before
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > cutting a
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release,
>>> but
>>> >> I'm
>>> >> > not
>>> >> > >> >> sure
>>> >> > >> >> > > how
>>> >> > >> >> > > >> > much
>>> >> > >> >> > > >> > > >> work
>>> >> > >> >> > > >> > > >> >> is
>>> >> > >> >> > > >> > > >> >> > > > left
>>> >> > >> >> > > >> > > >> >> > > > > to
>>> >> > >> >> > > >> > > >> >> > > > > > > be
>>> >> > >> >> > > >> > > >> >> > > > > > > > > done
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> there.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please
>>> respond
>>> >> > with
>>> >> > >> >> your
>>> >> > >> >> > > >> > thoughts
>>> >> > >> >> > > >> > > on
>>> >> > >> >> > > >> > > >> a
>>> >> > >> >> > > >> > > >> >> > > release
>>> >> > >> >> > > >> > > >> >> > > > > > soon
>>> >> > >> >> > > >> > > >> >> > > > > > > > and
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > any
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> JIRAs
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > you
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to
>>> include
>>> >> > in
>>> >> > >> the
>>> >> > >> >> > > >> release.
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>> >> > >> >> > > https://issues.apache.org/jira/browse/DRILL-4243
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > > >
>>> >> > >> >> > > >> > > >> >> > > > > >
>>> >> > >> >> > > >> > > >> >> > > > >
>>> >> > >> >> > > >> > > >> >> > > >
>>> >> > >> >> > > >> > > >> >> > >
>>> >> > >> >> > > >> > > >> >> >
>>> >> > >> >> > > >> > > >> >>
>>> >> > >> >> > > >> > > >>
>>> >> > >> >> > > >> > >
>>> >> > >> >> > > >> >
>>> >> > >> >> > > >>
>>> >> > >> >> > >
>>> >> > >> >> >
>>> >> > >> >>
>>> >> > >>
>>> >> > >>
>>> >> >
>>> >> >
>>> >>
>>>
>>
>>
>


-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>

Re: Time for a 1.5 release?

Posted by Jason Altekruse <al...@gmail.com>.

A question for previous release managers. Jacques was thinking I may need
to have my GPG signed to get linked into the Apache Web of Trust. Is this
necessary for making a release?

On Mon, Feb 1, 2016 at 11:24 AM, Jason Altekruse <al...@gmail.com>
wrote:

> The only solution to this issue I have found is disabling the new plugin
> to automatically enforce the size of the jdbc-all plugin. I don't know why
> it is behaving differently when performing a release than it did during a
> 'mvn install', but I have verified the correct size of the JAR manually and
> am planning on pushing a candidate soon with the small change to disable
> the automatic jar size enforcement.
>
> I will open a JIRA for getting this resolved after the release.
>
> On Sun, Jan 31, 2016 at 10:46 PM, Jinfeng Ni <ji...@gmail.com>
> wrote:
>
>> Good to hear this, Jacques & Jason. That makes feel relieved. :-)
>>
>>
>>
>> On Sun, Jan 31, 2016 at 10:36 PM, Jason Altekruse
>> <al...@gmail.com> wrote:
>> > I don't think there was any issue with the merge either Jinfeng, I don't
>> > think we need to close master. I had to step out for a few hours, but I
>> am
>> > working on finishing up the candidate right now.
>> >
>> > Having a weird RAT issue running the actual maven release command, it's
>> > complaining about a dependency-reduced-pom.xml in the target directory
>> of
>> > jdbc-all lacking a license header (although target directories should be
>> > ignored). Checking out the commit before the recent changes to the
>> jdbc-all
>> > exclusions seems to get rid of the issue, but the changes in the pom
>> file
>> > don't seem like they should be causing any problems.
>> >
>> > Still trying to debug and hoping to have a vote up tonight.
>> >
>> > On Sun, Jan 31, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com>
>> wrote:
>> >
>> >> Given how easy it is to create a branch off a random commit, I
>> generally
>> >> don't see a point in closing the master branch. I figure the release
>> >> manager can propose where they'd like to fork if this type of thing
>> comes
>> >> up.
>> >>
>> >> Thanks for the feedback Jinfeng. Of course you're right about Venki's
>> >> commit. My bad.
>> >>
>> >> --
>> >> Jacques Nadeau
>> >> CTO and Co-Founder, Dremio
>> >>
>> >> On Sun, Jan 31, 2016 at 9:21 PM, Jinfeng Ni <ji...@gmail.com>
>> wrote:
>> >>
>> >> > Venki's backwards compatibility fix (commit id:
>> >> >    03197d0) is before the partition pruning commit. So, it seems to
>> be
>> >> > fine to create a release candidate from 03197d0; no need to
>> >> > cherry-pick.
>> >> >
>> >> > As for whether the partition pruning changes are risky, the
>> pre-commit
>> >> > run did not show any serious problems in the first run; all the plan
>> >> > changes are expected, or expose some existing problem (drill-4279).
>> >> > However, I agree that it impacts the main code path of query
>> planning,
>> >> > and I'm fine that we exclude the partition change from 1.5.0 release,
>> >> > since it's merged right before we start a voting.
>> >> >
>> >> > I did not see any one mentioned that the master branch is closed, and
>> >> > therefore assume it's still open for commits.  I probably should have
>> >> > asked whether the master was open for new patches before I merged the
>> >> > commits.
>> >> >
>> >> >
>> >> >
>> >> > On Sun, Jan 31, 2016 at 8:43 PM, Jacques Nadeau <ja...@dremio.com>
>> >> > wrote:
>> >> > > Jinfeng, do you agree that the changes are risky? I was going
>> mostly by
>> >> > > commit messages.
>> >> > >
>> >> > > Also just realized that the backwards compatibility fix to the
>> storage
>> >> > > plugin interface is after these. Could create release branch from
>> >> > 03197d...
>> >> > > and then cherry pick the Venki change. Or could keep them all. Or
>> >> > > go without the compatibility fix....
>> >> > >
>> >> > > --
>> >> > > Jacques Nadeau
>> >> > > CTO and Co-Founder, Dremio
>> >> > >
>> >> > > On Sun, Jan 31, 2016 at 7:50 PM, Jinfeng Ni <jinfengni99@gmail.com
>> >
>> >> > wrote:
>> >> > >
>> >> > >> Hi Jason,
>> >> > >>
>> >> > >> I'm fine that release 1.5.0 starts from 03197d0f2.  One thing I
>> need
>> >> > >> point out is that the pre-commit test suite has some testcase's
>> plan
>> >> > >> output modified because of the partition pruning change.  If we
>> run
>> >> > >> pre-commit against the candidate release for 1.5.0, we have to
>> use a
>> >> > >> prior version of pre-commit suite.
>> >> > >>
>> >> > >>
>> >> > >>
>> >> > >> On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse
>> >> > >> <al...@gmail.com> wrote:
>> >> > >> > That sounds reasonable to me. I'll start preparing a release
>> >> > candidate.
>> >> > >> >
>> >> > >> > Jinfeng, are you okay waiting for the next release to include
>> this
>> >> > >> change?
>> >> > >> >
>> >> > >> > On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <
>> jacques@dremio.com
>> >> >
>> >> > >> wrote:
>> >> > >> >
>> >> > >> >> Hey Jason,
>> >> > >> >>
>> >> > >> >> Can you start the release vote? We've let this drag on too
>> long.
>> >> > >> >>
>> >> > >> >> I suggest we start from 03197d0f2. It seems like the partition
>> >> > pruning
>> >> > >> >> changes are pretty complicated and on a pretty main path to add
>> >> right
>> >> > >> >> before the vote. Since Hsuan's fix (pr 349) hasn't been
>> reviewed, I
>> >> > >> think
>> >> > >> >> we should start the vote without it.
>> >> > >> >>
>> >> > >> >> Jacques
>> >> > >> >>
>> >> > >> >>
>> >> > >> >> --
>> >> > >> >> Jacques Nadeau
>> >> > >> >> CTO and Co-Founder, Dremio
>> >> > >> >>
>> >> > >> >> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <
>> hyichu@maprtech.com
>> >> >
>> >> > >> wrote:
>> >> > >> >>
>> >> > >> >> > Yes, just send a pull request.
>> >> > >> >> > https://github.com/apache/drill/pull/349
>> >> > >> >> >
>> >> > >> >> > Can Jinfeng review it?
>> >> > >> >> >
>> >> > >> >> > Thanks,
>> >> > >> >> >
>> >> > >> >> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <
>> >> jinfengni99@gmail.com
>> >> > >
>> >> > >> >> wrote:
>> >> > >> >> >
>> >> > >> >> > > Sean probably is quite close to find a fix for this issue.
>> >> Let's
>> >> > see
>> >> > >> >> > > whether he can post a patch soon.
>> >> > >> >> > >
>> >> > >> >> > >
>> >> > >> >> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
>> >> > >> >> > > <ch...@gmail.com> wrote:
>> >> > >> >> > > > DRILL-4323 only happens when assertions are turned on (my
>> >> > install
>> >> > >> >> > script
>> >> > >> >> > > > automatically enables assertions).
>> >> > >> >> > > >
>> >> > >> >> > > > - Rahul
>> >> > >> >> > > >
>> >> > >> >> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <
>> >> > >> zfong@maprtech.com>
>> >> > >> >> > > wrote:
>> >> > >> >> > > >
>> >> > >> >> > > >> Rahul,
>> >> > >> >> > > >>
>> >> > >> >> > > >> Let us know if you can reproduce the issue with
>> assertions
>> >> > turned
>> >> > >> >> OFF.
>> >> > >> >> > > If
>> >> > >> >> > > >> you cannot, then I think this should not be a blocker
>> for
>> >> 1.5,
>> >> > >> and
>> >> > >> >> > Hsuan
>> >> > >> >> > > >> will continue to look into fixing this.
>> >> > >> >> > > >>
>> >> > >> >> > > >> -- Zelaine
>> >> > >> >> > > >>
>> >> > >> >> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
>> >> > >> >> > knguyen@maprtech.com>
>> >> > >> >> > > >> wrote:
>> >> > >> >> > > >>
>> >> > >> >> > > >> > I do see the same error with assertion turned on.
>> >> > >> >> > > >> >
>> >> > >> >> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <
>> >> > >> >> jinfengni99@gmail.com>
>> >> > >> >> > > >> wrote:
>> >> > >> >> > > >> >
>> >> > >> >> > > >> > > Krystal, by any chance, did you turn on assertion
>> when
>> >> you
>> >> > >> start
>> >> > >> >> > > >> > drillbit?
>> >> > >> >> > > >> > >
>> >> > >> >> > > >> > > I think the error in drill-4323 will be raised when
>> >> > >> assertion =
>> >> > >> >> > on.
>> >> > >> >> > > >> > >
>> >> > >> >> > > >> > >
>> >> > >> >> > > >> > >
>> >> > >> >> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
>> >> > >> >> > > knguyen@maprtech.com>
>> >> > >> >> > > >> > > wrote:
>> >> > >> >> > > >> > > > I am running drill-1.4 and is able to run count(*)
>> >> > >> >> successfully.
>> >> > >> >> > > >> > > >
>> >> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100>
>> alter
>> >> > >> session
>> >> > >> >> > set
>> >> > >> >> > > >> > > > `store.hive.optimize_scan_with_native_readers` =
>> true;
>> >> > >> >> > > >> > > >
>> >> > >> >> >
>> >> +-------+--------------------------------------------------------+
>> >> > >> >> > > >> > > > |  ok   |                        summary
>> >> > >> >> >  |
>> >> > >> >> > > >> > > >
>> >> > >> >> >
>> >> +-------+--------------------------------------------------------+
>> >> > >> >> > > >> > > > | true  |
>> store.hive.optimize_scan_with_native_readers
>> >> > >> >> > updated.  |
>> >> > >> >> > > >> > > >
>> >> > >> >> >
>> >> +-------+--------------------------------------------------------+
>> >> > >> >> > > >> > > > 1 row selected (0.374 seconds)
>> >> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100>
>> select
>> >> > >> >> count(*)
>> >> > >> >> > > from
>> >> > >> >> > > >> > > > lineitem_parquet_partitioned_hive_string;
>> >> > >> >> > > >> > > > +---------+
>> >> > >> >> > > >> > > > | EXPR$0  |
>> >> > >> >> > > >> > > > +---------+
>> >> > >> >> > > >> > > > | 60175   |
>> >> > >> >> > > >> > > > +---------+
>> >> > >> >> > > >> > > > 1 row selected (2.986 seconds)
>> >> > >> >> > > >> > > >
>> >> > >> >> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
>> >> > >> >> > > jinfengni99@gmail.com>
>> >> > >> >> > > >> > > wrote:
>> >> > >> >> > > >> > > >
>> >> > >> >> > > >> > > >> Venki and I did some investigation for
>> DRILL-4323.
>> >> The
>> >> > >> issue
>> >> > >> >> > > >> reported
>> >> > >> >> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as
>> >> well.
>> >> > >> Seems
>> >> > >> >> > to
>> >> > >> >> > > us
>> >> > >> >> > > >> > > >> this is not a regression from 1.4.0; it's a
>> >> regression
>> >> > >> from
>> >> > >> >> > 1.3.0
>> >> > >> >> > > >> > > >> probably.
>> >> > >> >> > > >> > > >>
>> >> > >> >> > > >> > > >> DRILL-4083 makes the planner to use
>> >> > DrillHiveNativeReader
>> >> > >> in
>> >> > >> >> > > stead
>> >> > >> >> > > >> of
>> >> > >> >> > > >> > > >> HiveReader for "select count(*) from hive_table"
>> >> query.
>> >> > >> >> > However,
>> >> > >> >> > > >> the
>> >> > >> >> > > >> > > >> Project after the scan produces empty schema.
>> Before
>> >> > >> >> > DRILL-4083,
>> >> > >> >> > > >> > > >> Drill uses HiveScan, which works fine.
>> >> > >> >> > > >> > > >>
>> >> > >> >> > > >> > > >>
>> >> > >> >> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
>> >> > >> >> > > >> > > >> <al...@gmail.com> wrote:
>> >> > >> >> > > >> > > >> > We could revert the change to the StoragePlugin
>> >> > >> interface
>> >> > >> >> to
>> >> > >> >> > > add
>> >> > >> >> > > >> the
>> >> > >> >> > > >> > > >> > getLogicalOptimizerRules
>> >> > >> >> > > >> > > >> > and getPhysicalOptimizerRules, and only put
>> those
>> >> in
>> >> > >> >> > > >> > > >> AbstractStoragePlugin
>> >> > >> >> > > >> > > >> > with default implementations.
>> >> > >> >> > > >> > > >> >
>> >> > >> >> > > >> > > >> > We could then update the docs to tell users to
>> not
>> >> > >> >> implement
>> >> > >> >> > > the
>> >> > >> >> > > >> > > >> interface
>> >> > >> >> > > >> > > >> > directly, and instead extend the abstract
>> class,
>> >> > which I
>> >> > >> >> > > believe
>> >> > >> >> > > >> > would
>> >> > >> >> > > >> > > >> > allow us to make changes like this in the
>> future
>> >> > without
>> >> > >> >> > > breaking
>> >> > >> >> > > >> > > >> plugins.
>> >> > >> >> > > >> > > >> > At a major release version we could then just
>> pull
>> >> up
>> >> > >> all
>> >> > >> >> of
>> >> > >> >> > > the
>> >> > >> >> > > >> new
>> >> > >> >> > > >> > > >> > methods we added to the abstract class into the
>> >> > >> interface,
>> >> > >> >> or
>> >> > >> >> > > just
>> >> > >> >> > > >> > get
>> >> > >> >> > > >> > > >> rid
>> >> > >> >> > > >> > > >> > of it all-together.
>> >> > >> >> > > >> > > >> >
>> >> > >> >> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki
>> Korukanti <
>> >> > >> >> > > >> > > >> venki.korukanti@gmail.com>
>> >> > >> >> > > >> > > >> > wrote:
>> >> > >> >> > > >> > > >> >
>> >> > >> >> > > >> > > >> >> Here is the partial fix:
>> >> > >> >> > > >> > > >> >>
>> >> > >> >> > > >> > > >> >>
>> >> > >> >> > > >> > > >>
>> >> > >> >> > > >> > >
>> >> > >> >> > > >> >
>> >> > >> >> > > >>
>> >> > >> >> > >
>> >> > >> >> >
>> >> > >> >>
>> >> > >>
>> >> >
>> >>
>> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
>> >> > >> >> > > >> > > >> >>
>> >> > >> >> > > >> > > >> >> If the existing StragePlugins are implementing
>> >> > >> >> > > >> > AbstractStoragePlugin
>> >> > >> >> > > >> > > >> class
>> >> > >> >> > > >> > > >> >> this fix should resolve the backward compat
>> issue.
>> >> > If
>> >> > >> they
>> >> > >> >> > are
>> >> > >> >> > > >> > > >> implementing
>> >> > >> >> > > >> > > >> >> StoragePlugin interface directly then it won't
>> >> > resolve
>> >> > >> the
>> >> > >> >> > > issue.
>> >> > >> >> > > >> > Not
>> >> > >> >> > > >> > > >> sure
>> >> > >> >> > > >> > > >> >> how to handle it in that case. Let me know if
>> >> there
>> >> > any
>> >> > >> >> ways
>> >> > >> >> > > to
>> >> > >> >> > > >> > > resolve
>> >> > >> >> > > >> > > >> >> this issue completely.
>> >> > >> >> > > >> > > >> >>
>> >> > >> >> > > >> > > >> >> Thanks
>> >> > >> >> > > >> > > >> >> Venki
>> >> > >> >> > > >> > > >> >>
>> >> > >> >> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <
>> >> > >> >> > > asinha@maprtech.com
>> >> > >> >> > > >> >
>> >> > >> >> > > >> > > >> wrote:
>> >> > >> >> > > >> > > >> >>
>> >> > >> >> > > >> > > >> >> > I was able to do the build with maven 3.3.9
>> >> (after
>> >> > >> doing
>> >> > >> >> > > 'brew
>> >> > >> >> > > >> > > install
>> >> > >> >> > > >> > > >> >> > maven').  Here are my env variables related
>> to
>> >> > maven:
>> >> > >> >> > > >> > > >> >> >
>> M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
>> >> > >> >> > > >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
>> >> > >> >> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
>> >> > >> >> > > >> > > >> >> >
>> >> > >> >> > > >> > > >> >> >
>> >> > >> >> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques
>> Nadeau
>> >> <
>> >> > >> >> > > >> > > jacques@dremio.com>
>> >> > >> >> > > >> > > >> >> > wrote:
>> >> > >> >> > > >> > > >> >> >
>> >> > >> >> > > >> > > >> >> > > Aman, for reference on the Maven version
>> >> issue.
>> >> > You
>> >> > >> >> can
>> >> > >> >> > > see
>> >> > >> >> > > >> the
>> >> > >> >> > > >> > > >> output
>> >> > >> >> > > >> > > >> >> of
>> >> > >> >> > > >> > > >> >> > > the shade plugin between the two Apache
>> >> Builds.
>> >> > If
>> >> > >> you
>> >> > >> >> > > search
>> >> > >> >> > > >> > for
>> >> > >> >> > > >> > > >> >> > > maven-shade-plugin you'll see that build
>> 647
>> >> > (maven
>> >> > >> >> > 3.0.5)
>> >> > >> >> > > >> > fails
>> >> > >> >> > > >> > > >> >> because
>> >> > >> >> > > >> > > >> >> > it
>> >> > >> >> > > >> > > >> >> > > includes a bunch of hadoop (and other
>> >> libraries
>> >> > >> that
>> >> > >> >> are
>> >> > >> >> > > >> listed
>> >> > >> >> > > >> > > as
>> >> > >> >> > > >> > > >> >> being
>> >> > >> >> > > >> > > >> >> > > excluded). If you look at build 648 (maven
>> >> > 3.3.3),
>> >> > >> >> those
>> >> > >> >> > > >> items
>> >> > >> >> > > >> > > are
>> >> > >> >> > > >> > > >> >> > included
>> >> > >> >> > > >> > > >> >> > > so that the build completes successfully.
>> >> > >> >> > > >> > > >> >> > >
>> >> > >> >> > > >> > > >> >> > >
>> >> > >> >> https://builds.apache.org/job/drill-scm/647/consoleText
>> >> > >> >> > > >> > > >> >> > >
>> >> > >> >> https://builds.apache.org/job/drill-scm/648/consoleText
>> >> > >> >> > > >> > > >> >> > >
>> >> > >> >> > > >> > > >> >> > > --
>> >> > >> >> > > >> > > >> >> > > Jacques Nadeau
>> >> > >> >> > > >> > > >> >> > > CTO and Co-Founder, Dremio
>> >> > >> >> > > >> > > >> >> > >
>> >> > >> >> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul
>> >> > challapalli
>> >> > >> <
>> >> > >> >> > > >> > > >> >> > > challapallirahul@gmail.com> wrote:
>> >> > >> >> > > >> > > >> >> > >
>> >> > >> >> > > >> > > >> >> > > > Jason,
>> >> > >> >> > > >> > > >> >> > > >
>> >> > >> >> > > >> > > >> >> > > > As of now I haven't narrowed down the
>> exact
>> >> > >> commit
>> >> > >> >> > which
>> >> > >> >> > > >> > caused
>> >> > >> >> > > >> > > >> >> > > DRILL-4323
>> >> > >> >> > > >> > > >> >> > > > and I do not know the scope of the fix.
>> >> > >> >> > > >> > > >> >> > > >
>> >> > >> >> > > >> > > >> >> > > > - Rahul
>> >> > >> >> > > >> > > >> >> > > >
>> >> > >> >> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason
>> >> > Altekruse
>> >> > >> <
>> >> > >> >> > > >> > > >> >> > > altekrusejason@gmail.com
>> >> > >> >> > > >> > > >> >> > > > >
>> >> > >> >> > > >> > > >> >> > > > wrote:
>> >> > >> >> > > >> > > >> >> > > >
>> >> > >> >> > > >> > > >> >> > > > > I think it makes sense to wait for
>> Venki's
>> >> > fix
>> >> > >> of
>> >> > >> >> > the
>> >> > >> >> > > >> API,
>> >> > >> >> > > >> > as
>> >> > >> >> > > >> > > >> >> > > releasing a
>> >> > >> >> > > >> > > >> >> > > > > version with the change would make
>> >> > >> >> > > reverting/refactoring
>> >> > >> >> > > >> > the
>> >> > >> >> > > >> > > API
>> >> > >> >> > > >> > > >> >> > > breaking
>> >> > >> >> > > >> > > >> >> > > > > change less useful.
>> >> > >> >> > > >> > > >> >> > > > >
>> >> > >> >> > > >> > > >> >> > > > > Do you have any idea about the scope
>> of
>> >> > fixing
>> >> > >> the
>> >> > >> >> > > hive
>> >> > >> >> > > >> > > native
>> >> > >> >> > > >> > > >> >> > reader,
>> >> > >> >> > > >> > > >> >> > > or
>> >> > >> >> > > >> > > >> >> > > > > which change since the 1.4 release
>> broke
>> >> > it? Is
>> >> > >> >> > > someone
>> >> > >> >> > > >> > > >> available
>> >> > >> >> > > >> > > >> >> to
>> >> > >> >> > > >> > > >> >> > > look
>> >> > >> >> > > >> > > >> >> > > > > at it today?
>> >> > >> >> > > >> > > >> >> > > > >
>> >> > >> >> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul
>> >> > >> >> challapalli <
>> >> > >> >> > > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
>> >> > >> >> > > >> > > >> >> > > > >
>> >> > >> >> > > >> > > >> >> > > > > > Just found an issue with hive native
>> >> > parquet
>> >> > >> >> > reader
>> >> > >> >> > > >> > > >> (DRILL-4323).
>> >> > >> >> > > >> > > >> >> > > This
>> >> > >> >> > > >> > > >> >> > > > > is a
>> >> > >> >> > > >> > > >> >> > > > > > regression from 1.4 and in my
>> opinion
>> >> > should
>> >> > >> be
>> >> > >> >> > > treated
>> >> > >> >> > > >> > as
>> >> > >> >> > > >> > > a
>> >> > >> >> > > >> > > >> >> > blocker.
>> >> > >> >> > > >> > > >> >> > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > - Rahul
>> >> > >> >> > > >> > > >> >> > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM,
>> Venki
>> >> > >> >> Korukanti <
>> >> > >> >> > > >> > > >> >> > > > > > venki.korukanti@gmail.com>
>> >> > >> >> > > >> > > >> >> > > > > > wrote:
>> >> > >> >> > > >> > > >> >> > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > Sorry I didn't realize I was
>> breaking
>> >> > the
>> >> > >> >> public
>> >> > >> >> > > >> > > interface
>> >> > >> >> > > >> > > >> API.
>> >> > >> >> > > >> > > >> >> > Let
>> >> > >> >> > > >> > > >> >> > > > me
>> >> > >> >> > > >> > > >> >> > > > > > work
>> >> > >> >> > > >> > > >> >> > > > > > > on a patch to make it backward
>> >> > compatible.
>> >> > >> >> > > >> > > >> >> > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > Thanks
>> >> > >> >> > > >> > > >> >> > > > > > > Venki
>> >> > >> >> > > >> > > >> >> > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM,
>> >> Steven
>> >> > >> >> > Phillips
>> >> > >> >> > > <
>> >> > >> >> > > >> > > >> >> > > steven@dremio.com
>> >> > >> >> > > >> > > >> >> > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > wrote:
>> >> > >> >> > > >> > > >> >> > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > I just wanted to bring up an
>> issue
>> >> > that I
>> >> > >> >> just
>> >> > >> >> > > now
>> >> > >> >> > > >> > > >> >> discovered,
>> >> > >> >> > > >> > > >> >> > > that
>> >> > >> >> > > >> > > >> >> > > > > has
>> >> > >> >> > > >> > > >> >> > > > > > > > caused me a fair amount of
>> grief.
>> >> > >> >> > > >> > > >> >> > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > >
>> >> > >> >> > > https://github.com/apache/drill/pull/300/commits
>> >> > >> >> > > >> > > >> >> > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing
>> >> API,
>> >> > and
>> >> > >> >> > causes
>> >> > >> >> > > >> > > >> >> StoragePlugins
>> >> > >> >> > > >> > > >> >> > > > that
>> >> > >> >> > > >> > > >> >> > > > > > were
>> >> > >> >> > > >> > > >> >> > > > > > > > compiled against currently
>> released
>> >> > >> versions
>> >> > >> >> > of
>> >> > >> >> > > >> Drill
>> >> > >> >> > > >> > > to
>> >> > >> >> > > >> > > >> no
>> >> > >> >> > > >> > > >> >> > > longer
>> >> > >> >> > > >> > > >> >> > > > > > > > functional properly. I would
>> prefer
>> >> > that
>> >> > >> >> this
>> >> > >> >> > > >> > breaking
>> >> > >> >> > > >> > > >> change
>> >> > >> >> > > >> > > >> >> > be
>> >> > >> >> > > >> > > >> >> > > > > > modified
>> >> > >> >> > > >> > > >> >> > > > > > > > to be backward compatible if
>> >> possible.
>> >> > >> >> > > >> > > >> >> > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23
>> AM,
>> >> > Jason
>> >> > >> >> > > Altekruse <
>> >> > >> >> > > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
>> >> > >> >> > > >> > > >> >> > > > > > > > wrote:
>> >> > >> >> > > >> > > >> >> > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > Hi Aman,
>> >> > >> >> > > >> > > >> >> > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > This is the failure that he
>> was
>> >> > >> seeing. He
>> >> > >> >> > > >> figured
>> >> > >> >> > > >> > > out
>> >> > >> >> > > >> > > >> that
>> >> > >> >> > > >> > > >> >> > the
>> >> > >> >> > > >> > > >> >> > > > new
>> >> > >> >> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were
>> not
>> >> > being
>> >> > >> >> > > respected
>> >> > >> >> > > >> > when
>> >> > >> >> > > >> > > the
>> >> > >> >> > > >> > > >> >> > build
>> >> > >> >> > > >> > > >> >> > > > was
>> >> > >> >> > > >> > > >> >> > > > > > run
>> >> > >> >> > > >> > > >> >> > > > > > > > with
>> >> > >> >> > > >> > > >> >> > > > > > > > > an older Maven version,
>> causing
>> >> the
>> >> > jar
>> >> > >> >> size
>> >> > >> >> > > to
>> >> > >> >> > > >> > > increase
>> >> > >> >> > > >> > > >> >> > > > > > significantly.
>> >> > >> >> > > >> > > >> >> > > > > > > > He
>> >> > >> >> > > >> > > >> >> > > > > > > > > added an enforcer to make
>> sure the
>> >> > JAR
>> >> > >> >> > didn't
>> >> > >> >> > > >> grow
>> >> > >> >> > > >> > > >> >> > > unexpectedly.
>> >> > >> >> > > >> > > >> >> > > > > Can
>> >> > >> >> > > >> > > >> >> > > > > > > you
>> >> > >> >> > > >> > > >> >> > > > > > > > > try to update your maven
>> version
>> >> and
>> >> > >> >> re-run
>> >> > >> >> > > the
>> >> > >> >> > > >> > > build?
>> >> > >> >> > > >> > > >> >> > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > - Jason
>> >> > >> >> > > >> > > >> >> > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18
>> AM,
>> >> > Aman
>> >> > >> >> > Sinha <
>> >> > >> >> > > >> > > >> >> > > > asinha@maprtech.com>
>> >> > >> >> > > >> > > >> >> > > > > > > > wrote:
>> >> > >> >> > > >> > > >> >> > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the
>> >> > following
>> >> > >> >> build
>> >> > >> >> > > >> failure
>> >> > >> >> > > >> > > on
>> >> > >> >> > > >> > > >> the
>> >> > >> >> > > >> > > >> >> > > latest
>> >> > >> >> > > >> > > >> >> > > > > > > master
>> >> > >> >> > > >> > > >> >> > > > > > > > > > branch...is this what you
>> saw
>> >> for
>> >> > the
>> >> > >> >> > Apache
>> >> > >> >> > > >> > build
>> >> > >> >> > > >> > > ?
>> >> > >> >> > > >> > > >> My
>> >> > >> >> > > >> > > >> >> > mvn
>> >> > >> >> > > >> > > >> >> > > > > > version
>> >> > >> >> > > >> > > >> >> > > > > > > > > output
>> >> > >> >> > > >> > > >> >> > > > > > > > > > is shown below.  Should we
>> all
>> >> be
>> >> > >> >> > upgrading
>> >> > >> >> > > to
>> >> > >> >> > > >> a
>> >> > >> >> > > >> > > newer
>> >> > >> >> > > >> > > >> >> mvn
>> >> > >> >> > > >> > > >> >> > ?
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > [INFO] ---
>> >> > >> >> > > maven-enforcer-plugin:1.3.1:enforce
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> (enforce-jdbc-jar-compactness) @
>> >> > >> >> > > drill-jdbc-all
>> >> > >> >> > > >> > ---
>> >> > >> >> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
>> >> > >> >> > > >> > > >> >> > > > > >
>> >> > >> >> org.apache.maven.plugins.enforcer.RequireFilesSize
>> >> > >> >> > > >> > > >> >> > > > > > > > > failed
>> >> > >> >> > > >> > > >> >> > > > > > > > > > with message:
>> >> > >> >> > > >> > > >> >> > > > > > > > > > The file
>> >> > >> >> drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> >> > >> >> > > is
>> >> > >> >> > > >> > > outside
>> >> > >> >> > > >> > > >> the
>> >> > >> >> > > >> > > >> >> > > > > expected
>> >> > >> >> > > >> > > >> >> > > > > > > size
>> >> > >> >> > > >> > > >> >> > > > > > > > > > range.
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > >                   This is
>> likely
>> >> > due
>> >> > >> to
>> >> > >> >> > you
>> >> > >> >> > > >> > adding
>> >> > >> >> > > >> > > new
>> >> > >> >> > > >> > > >> >> > > > > dependencies
>> >> > >> >> > > >> > > >> >> > > > > > > to
>> >> > >> >> > > >> > > >> >> > > > > > > > a
>> >> > >> >> > > >> > > >> >> > > > > > > > > > java-exec and not updating
>> the
>> >> > >> excludes
>> >> > >> >> in
>> >> > >> >> > > this
>> >> > >> >> > > >> > > >> module.
>> >> > >> >> > > >> > > >> >> > This
>> >> > >> >> > > >> > > >> >> > > is
>> >> > >> >> > > >> > > >> >> > > > > > > > important
>> >> > >> >> > > >> > > >> >> > > > > > > > > > as it minimizes the size of
>> the
>> >> > >> >> dependency
>> >> > >> >> > > of
>> >> > >> >> > > >> > Drill
>> >> > >> >> > > >> > > >> >> > > application
>> >> > >> >> > > >> > > >> >> > > > > > > users.
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > >
>> >> > >> >> > > >> > > >> >> > > > >
>> >> > >> >> > > >> > > >> >> > > >
>> >> > >> >> > > >> > > >> >> > >
>> >> > >> >> > > >> > > >> >> >
>> >> > >> >> > > >> > > >> >>
>> >> > >> >> > > >> > > >>
>> >> > >> >> > > >> > >
>> >> > >> >> > > >> >
>> >> > >> >> > > >>
>> >> > >> >> > >
>> >> > >> >> >
>> >> > >> >>
>> >> > >>
>> >> >
>> >>
>> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> >> > >> >> > > >> > > >> >> > > > > > > > > > size (44664121) too large.
>> Max.
>> >> is
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > >
>> >> > >> >> > > >> > > >> >> > > > >
>> >> > >> >> > > >> > > >> >> > > >
>> >> > >> >> > > >> > > >> >> > >
>> >> > >> >> > > >> > > >> >> >
>> >> > >> >> > > >> > > >> >>
>> >> > >> >> > > >> > > >>
>> >> > >> >> > > >> > >
>> >> > >> >> > > >> >
>> >> > >> >> > > >>
>> >> > >> >> > >
>> >> > >> >> >
>> >> > >> >>
>> >> > >>
>> >> >
>> >>
>> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > Administrators-MacBook-Pro-144:incubator-drill
>> >> > >> >> > > >> > > asinha$
>> >> > >> >> > > >> > > >> >> mvn
>> >> > >> >> > > >> > > >> >> > > > > > --version
>> >> > >> >> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
>> >> > >> >> > > >> > > >> >> > >
>> (r01de14724cdef164cd33c7c8c2fe155faf9602da;
>> >> > >> >> > > >> > > >> >> > > > > > > > 2013-02-19
>> >> > >> >> > > >> > > >> >> > > > > > > > > > 05:51:28-0800)
>> >> > >> >> > > >> > > >> >> > > > > > > > > > Maven home:
>> >> > >> /opt/local/share/java/maven3
>> >> > >> >> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45,
>> vendor:
>> >> > >> Oracle
>> >> > >> >> > > >> > Corporation
>> >> > >> >> > > >> > > >> >> > > > > > > > > > Java home:
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > >
>> >> > >> >> > > >> > > >>
>> >> > >> >> > >
>> >> > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
>> >> > >> >> > > >> > > >> >> > > > > > > > > > Default locale: en_US,
>> platform
>> >> > >> >> encoding:
>> >> > >> >> > > UTF-8
>> >> > >> >> > > >> > > >> >> > > > > > > > > > OS name: "mac os x",
>> version:
>> >> > >> "10.9.5",
>> >> > >> >> > > arch:
>> >> > >> >> > > >> > > >> "x86_64",
>> >> > >> >> > > >> > > >> >> > > family:
>> >> > >> >> > > >> > > >> >> > > > > > "mac"
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at
>> 8:20 AM,
>> >> > >> Jacques
>> >> > >> >> > > >> Nadeau <
>> >> > >> >> > > >> > > >> >> > > > > > jacques@dremio.com>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > wrote:
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > Build back to normal. It
>> looks
>> >> > like
>> >> > >> >> the
>> >> > >> >> > > >> Apache
>> >> > >> >> > > >> > > >> server
>> >> > >> >> > > >> > > >> >> was
>> >> > >> >> > > >> > > >> >> > > > using
>> >> > >> >> > > >> > > >> >> > > > > > an
>> >> > >> >> > > >> > > >> >> > > > > > > > old
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I
>> >> > switched
>> >> > >> to
>> >> > >> >> > > >> something
>> >> > >> >> > > >> > > more
>> >> > >> >> > > >> > > >> >> > recent,
>> >> > >> >> > > >> > > >> >> > > > the
>> >> > >> >> > > >> > > >> >> > > > > > > build
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > passed.
>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > --
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at
>> 7:02
>> >> AM,
>> >> > >> >> Jacques
>> >> > >> >> > > >> > Nadeau <
>> >> > >> >> > > >> > > >> >> > > > > > > jacques@dremio.com>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > wrote:
>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge
>> caused the
>> >> > >> Apache
>> >> > >> >> > > build
>> >> > >> >> > > >> to
>> >> > >> >> > > >> > > fail.
>> >> > >> >> > > >> > > >> >> > > > > > > Investigating...
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > --
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder,
>> Dremio
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at
>> 6:31
>> >> > AM,
>> >> > >> >> > Jacques
>> >> > >> >> > > >> > > Nadeau <
>> >> > >> >> > > >> > > >> >> > > > > > > > jacques@dremio.com>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > > wrote:
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> I got clean regression
>> runs
>> >> > as
>> >> > >> >> well.
>> >> > >> >> > > I've
>> >> > >> >> > > >> > > merged
>> >> > >> >> > > >> > > >> the
>> >> > >> >> > > >> > > >> >> > > > patch.
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to
>> start
>> >> the
>> >> > >> >> release
>> >> > >> >> > > >> > process?
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> --
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder,
>> Dremio
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at
>> >> 10:42
>> >> > >> PM,
>> >> > >> >> > > Abhishek
>> >> > >> >> > > >> > > >> Girish <
>> >> > >> >> > > >> > > >> >> > > > > > > > agirish@mapr.com
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> wrote:
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> Had two clean
>> Functional
>> >> > runs.
>> >> > >> >> TPC-H
>> >> > >> >> > > >> SF100
>> >> > >> >> > > >> > > was
>> >> > >> >> > > >> > > >> also
>> >> > >> >> > > >> > > >> >> > > > > > successful.
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016
>> at
>> >> > 10:07
>> >> > >> PM,
>> >> > >> >> > > rahul
>> >> > >> >> > > >> > > >> >> challapalli <
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> challapallirahul@gmail.com>
>> >> > >> >> wrote:
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a
>> functional
>> >> > run
>> >> > >> with
>> >> > >> >> > > your
>> >> > >> >> > > >> > > branch.
>> >> > >> >> > > >> > > >> >> Will
>> >> > >> >> > > >> > > >> >> > > let
>> >> > >> >> > > >> > > >> >> > > > > you
>> >> > >> >> > > >> > > >> >> > > > > > > know
>> >> > >> >> > > >> > > >> >> > > > > > > > > > once
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > it
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > finishes
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27,
>> 2016 at
>> >> > 9:56
>> >> > >> PM,
>> >> > >> >> > > >> Jacques
>> >> > >> >> > > >> > > >> Nadeau <
>> >> > >> >> > > >> > > >> >> > > > > > > > > > jacques@dremio.com>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > wrote:
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged
>> >> today. I
>> >> > >> have
>> >> > >> >> an
>> >> > >> >> > > >> > updated
>> >> > >> >> > > >> > > >> patch
>> >> > >> >> > > >> > > >> >> > for
>> >> > >> >> > > >> > > >> >> > > > 4291
>> >> > >> >> > > >> > > >> >> > > > > > > that
>> >> > >> >> > > >> > > >> >> > > > > > > > is
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> ready.
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it
>> >> seems
>> >> > >> that
>> >> > >> >> > > >> something
>> >> > >> >> > > >> > > isn't
>> >> > >> >> > > >> > > >> >> > > working
>> >> > >> >> > > >> > > >> >> > > > > with
>> >> > >> >> > > >> > > >> >> > > > > > > our
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> extended
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't
>> >> been
>> >> > >> able
>> >> > >> >> to
>> >> > >> >> > > run
>> >> > >> >> > > >> an
>> >> > >> >> > > >> > > >> extended
>> >> > >> >> > > >> > > >> >> > > > > > regression.
>> >> > >> >> > > >> > > >> >> > > > > > > > > Unit
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> tests
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone
>> else
>> >> > >> possibly
>> >> > >> >> > > able
>> >> > >> >> > > >> to
>> >> > >> >> > > >> > > run a
>> >> > >> >> > > >> > > >> >> > > > regression
>> >> > >> >> > > >> > > >> >> > > > > > > suite
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > against
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > this
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we
>> can
>> >> > >> confirm
>> >> > >> >> > > things
>> >> > >> >> > > >> > look
>> >> > >> >> > > >> > > >> good
>> >> > >> >> > > >> > > >> >> and
>> >> > >> >> > > >> > > >> >> > > > start
>> >> > >> >> > > >> > > >> >> > > > > > the
>> >> > >> >> > > >> > > >> >> > > > > > > > > > release
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > process?
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > thanks,
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>> >> > >> >> > > >> > > >> >> > > > >
>> >> > >> >> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > --
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and
>> Co-Founder,
>> >> > Dremio
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25,
>> 2016
>> >> at
>> >> > >> 11:20
>> >> > >> >> > AM,
>> >> > >> >> > > >> > Jacques
>> >> > >> >> > > >> > > >> >> Nadeau
>> >> > >> >> > > >> > > >> >> > <
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > jacques@dremio.com
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the main
>> >> > things
>> >> > >> are
>> >> > >> >> > 4196
>> >> > >> >> > > >> and
>> >> > >> >> > > >> > > 4291
>> >> > >> >> > > >> > > >> >> > should
>> >> > >> >> > > >> > > >> >> > > be
>> >> > >> >> > > >> > > >> >> > > > > > > > > completed.
>> >> > >> >> > > >> > > >> >> > > > > > > > > > I
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> know
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Amit
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to
>> >> reproduce
>> >> > >> 4196
>> >> > >> >> > > locally
>> >> > >> >> > > >> > this
>> >> > >> >> > > >> > > >> >> weekend
>> >> > >> >> > > >> > > >> >> > > so
>> >> > >> >> > > >> > > >> >> > > > I
>> >> > >> >> > > >> > > >> >> > > > > > > think
>> >> > >> >> > > >> > > >> >> > > > > > > > > > we're
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> close
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > on
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > that.
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > --
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and
>> Co-Founder,
>> >> > >> Dremio
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25,
>> 2016
>> >> > at
>> >> > >> >> 10:14
>> >> > >> >> > > AM,
>> >> > >> >> > > >> > > Zelaine
>> >> > >> >> > > >> > > >> >> Fong
>> >> > >> >> > > >> > > >> >> > <
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on
>> >> this?
>> >> > >> >> What's
>> >> > >> >> > > >> > blocking
>> >> > >> >> > > >> > > us
>> >> > >> >> > > >> > > >> >> from
>> >> > >> >> > > >> > > >> >> > > > taking
>> >> > >> >> > > >> > > >> >> > > > > > > this
>> >> > >> >> > > >> > > >> >> > > > > > > > > to a
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> vote --
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > the
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join
>> >> > issues?
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19,
>> >> 2016
>> >> > at
>> >> > >> >> 4:18
>> >> > >> >> > > PM,
>> >> > >> >> > > >> > > Jacques
>> >> > >> >> > > >> > > >> >> > Nadeau
>> >> > >> >> > > >> > > >> >> > > <
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this
>> >> > thread...
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the
>> >> issues
>> >> > >> that
>> >> > >> >> > were
>> >> > >> >> > > >> > > >> mentioned in
>> >> > >> >> > > >> > > >> >> > > this
>> >> > >> >> > > >> > > >> >> > > > > > thread
>> >> > >> >> > > >> > > >> >> > > > > > > > > along
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> with a
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed
>> >> > >> categorization:
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release
>> Blockers
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
>> Amit
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4190
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
>> Amit
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4196
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to
>> merge
>> >> > Jacques
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > >> >> > > >> > > >> >>
>> https://issues.apache.org/jira/browse/DRILL-4246
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review
>> Jinfeng
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4256
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
>> >> Jacques
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4278
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to
>> merge
>> >> > Laurent
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > >> >> > > >> > > >> >>
>> https://issues.apache.org/jira/browse/DRILL-4285
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open
>> Jason/Hakim
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4247
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
>> Jason
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4203
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> https://issues.apache.org/jira/browse/DRILL-4266
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to
>> merge
>> >> > Jacques
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > >> >> > > >> > > >> >>
>> https://issues.apache.org/jira/browse/DRILL-4131
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do
>> others
>> >> > think?
>> >> > >> >> Let's
>> >> > >> >> > > try
>> >> > >> >> > > >> to
>> >> > >> >> > > >> > > get
>> >> > >> >> > > >> > > >> the
>> >> > >> >> > > >> > > >> >> > > > > blockers
>> >> > >> >> > > >> > > >> >> > > > > > > > > wrapped
>> >> > >> >> > > >> > > >> >> > > > > > > > > > up
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> in
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > the
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> next
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two
>> and
>> >> > start a
>> >> > >> >> > > release
>> >> > >> >> > > >> > > vote...
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > --
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques
>> Nadeau
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and
>> >> Co-Founder,
>> >> > >> >> Dremio
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan
>> 4,
>> >> > 2016 at
>> >> > >> >> 1:48
>> >> > >> >> > > PM,
>> >> > >> >> > > >> > > Jason
>> >> > >> >> > > >> > > >> >> > > Altekruse
>> >> > >> >> > > >> > > >> >> > > > <
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> >> > >> altekrusejason@gmail.com>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the
>> >> > allocator
>> >> > >> >> > changes
>> >> > >> >> > > >> > merged
>> >> > >> >> > > >> > > and
>> >> > >> >> > > >> > > >> >> > about
>> >> > >> >> > > >> > > >> >> > > a
>> >> > >> >> > > >> > > >> >> > > > > > month
>> >> > >> >> > > >> > > >> >> > > > > > > > > since
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > the
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> last
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it
>> >> would
>> >> > be
>> >> > >> >> good
>> >> > >> >> > to
>> >> > >> >> > > >> > start
>> >> > >> >> > > >> > > a
>> >> > >> >> > > >> > > >> vote
>> >> > >> >> > > >> > > >> >> > > > soon. I
>> >> > >> >> > > >> > > >> >> > > > > > > would
>> >> > >> >> > > >> > > >> >> > > > > > > > > > like
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > to
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release
>> >> > manager.
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that
>> >> there
>> >> > >> were
>> >> > >> >> > some
>> >> > >> >> > > >> > issues
>> >> > >> >> > > >> > > >> that
>> >> > >> >> > > >> > > >> >> > were
>> >> > >> >> > > >> > > >> >> > > > > > > > identified
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > after
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> the
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was
>> >> > merged. I
>> >> > >> >> think
>> >> > >> >> > > that
>> >> > >> >> > > >> > > these
>> >> > >> >> > > >> > > >> >> > issues
>> >> > >> >> > > >> > > >> >> > > > > should
>> >> > >> >> > > >> > > >> >> > > > > > > be
>> >> > >> >> > > >> > > >> >> > > > > > > > > > fixed
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> before
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > we
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release
>> >> > candidate.
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From
>> looking at
>> >> > the
>> >> > >> >> > > associated
>> >> > >> >> > > >> > > JIRAs
>> >> > >> >> > > >> > > >> it
>> >> > >> >> > > >> > > >> >> > > looked
>> >> > >> >> > > >> > > >> >> > > > > > like
>> >> > >> >> > > >> > > >> >> > > > > > > > > there
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > was
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> a
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short term
>> fix
>> >> > just
>> >> > >> >> > > adjusting
>> >> > >> >> > > >> > the
>> >> > >> >> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > option,
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> and
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more
>> >> > involved
>> >> > >> work
>> >> > >> >> > to
>> >> > >> >> > > >> > change
>> >> > >> >> > > >> > > >> how we
>> >> > >> >> > > >> > > >> >> > > > > determine
>> >> > >> >> > > >> > > >> >> > > > > > > the
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > correct
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > time
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > to
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill
>> during
>> >> > >> external
>> >> > >> >> > > sort. I
>> >> > >> >> > > >> > > >> believe it
>> >> > >> >> > > >> > > >> >> > > makes
>> >> > >> >> > > >> > > >> >> > > > > > sense
>> >> > >> >> > > >> > > >> >> > > > > > > > to
>> >> > >> >> > > >> > > >> >> > > > > > > > > > make
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > external
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well
>> with
>> >> > the
>> >> > >> >> newly
>> >> > >> >> > > >> > improved
>> >> > >> >> > > >> > > >> memory
>> >> > >> >> > > >> > > >> >> > > > > > accounting
>> >> > >> >> > > >> > > >> >> > > > > > > > > before
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > cutting a
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release,
>> but
>> >> I'm
>> >> > not
>> >> > >> >> sure
>> >> > >> >> > > how
>> >> > >> >> > > >> > much
>> >> > >> >> > > >> > > >> work
>> >> > >> >> > > >> > > >> >> is
>> >> > >> >> > > >> > > >> >> > > > left
>> >> > >> >> > > >> > > >> >> > > > > to
>> >> > >> >> > > >> > > >> >> > > > > > > be
>> >> > >> >> > > >> > > >> >> > > > > > > > > done
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> there.
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please
>> respond
>> >> > with
>> >> > >> >> your
>> >> > >> >> > > >> > thoughts
>> >> > >> >> > > >> > > on
>> >> > >> >> > > >> > > >> a
>> >> > >> >> > > >> > > >> >> > > release
>> >> > >> >> > > >> > > >> >> > > > > > soon
>> >> > >> >> > > >> > > >> >> > > > > > > > and
>> >> > >> >> > > >> > > >> >> > > > > > > > > > any
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> JIRAs
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > you
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to
>> include
>> >> > in
>> >> > >> the
>> >> > >> >> > > >> release.
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
>> >> > >> >> > > >> > > >> >> > > > > > > >
>> >> > >> >> > > https://issues.apache.org/jira/browse/DRILL-4243
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > > >
>> >> > >> >> > > >> > > >> >> > > > > >
>> >> > >> >> > > >> > > >> >> > > > >
>> >> > >> >> > > >> > > >> >> > > >
>> >> > >> >> > > >> > > >> >> > >
>> >> > >> >> > > >> > > >> >> >
>> >> > >> >> > > >> > > >> >>
>> >> > >> >> > > >> > > >>
>> >> > >> >> > > >> > >
>> >> > >> >> > > >> >
>> >> > >> >> > > >>
>> >> > >> >> > >
>> >> > >> >> >
>> >> > >> >>
>> >> > >>
>> >> > >>
>> >> >
>> >> >
>> >>
>>
>
>

Re: Time for a 1.5 release?

Posted by Jason Altekruse <al...@gmail.com>.

The only solution to this issue I have found is disabling the new plugin to
automatically enforce the size of the jdbc-all plugin. I don't know why it
is behaving differently when performing a release than it did during a 'mvn
install', but I have verified the correct size of the JAR manually and am
planning on pushing a candidate soon with the small change to disable the
automatic jar size enforcement.

I will open a JIRA for getting this resolved after the release.

On Sun, Jan 31, 2016 at 10:46 PM, Jinfeng Ni <ji...@gmail.com> wrote:

> Good to hear this, Jacques & Jason. That makes feel relieved. :-)
>
>
>
> On Sun, Jan 31, 2016 at 10:36 PM, Jason Altekruse
> <al...@gmail.com> wrote:
> > I don't think there was any issue with the merge either Jinfeng, I don't
> > think we need to close master. I had to step out for a few hours, but I
> am
> > working on finishing up the candidate right now.
> >
> > Having a weird RAT issue running the actual maven release command, it's
> > complaining about a dependency-reduced-pom.xml in the target directory of
> > jdbc-all lacking a license header (although target directories should be
> > ignored). Checking out the commit before the recent changes to the
> jdbc-all
> > exclusions seems to get rid of the issue, but the changes in the pom file
> > don't seem like they should be causing any problems.
> >
> > Still trying to debug and hoping to have a vote up tonight.
> >
> > On Sun, Jan 31, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com>
> wrote:
> >
> >> Given how easy it is to create a branch off a random commit, I generally
> >> don't see a point in closing the master branch. I figure the release
> >> manager can propose where they'd like to fork if this type of thing
> comes
> >> up.
> >>
> >> Thanks for the feedback Jinfeng. Of course you're right about Venki's
> >> commit. My bad.
> >>
> >> --
> >> Jacques Nadeau
> >> CTO and Co-Founder, Dremio
> >>
> >> On Sun, Jan 31, 2016 at 9:21 PM, Jinfeng Ni <ji...@gmail.com>
> wrote:
> >>
> >> > Venki's backwards compatibility fix (commit id:
> >> >    03197d0) is before the partition pruning commit. So, it seems to be
> >> > fine to create a release candidate from 03197d0; no need to
> >> > cherry-pick.
> >> >
> >> > As for whether the partition pruning changes are risky, the pre-commit
> >> > run did not show any serious problems in the first run; all the plan
> >> > changes are expected, or expose some existing problem (drill-4279).
> >> > However, I agree that it impacts the main code path of query planning,
> >> > and I'm fine that we exclude the partition change from 1.5.0 release,
> >> > since it's merged right before we start a voting.
> >> >
> >> > I did not see any one mentioned that the master branch is closed, and
> >> > therefore assume it's still open for commits.  I probably should have
> >> > asked whether the master was open for new patches before I merged the
> >> > commits.
> >> >
> >> >
> >> >
> >> > On Sun, Jan 31, 2016 at 8:43 PM, Jacques Nadeau <ja...@dremio.com>
> >> > wrote:
> >> > > Jinfeng, do you agree that the changes are risky? I was going
> mostly by
> >> > > commit messages.
> >> > >
> >> > > Also just realized that the backwards compatibility fix to the
> storage
> >> > > plugin interface is after these. Could create release branch from
> >> > 03197d...
> >> > > and then cherry pick the Venki change. Or could keep them all. Or
> >> > > go without the compatibility fix....
> >> > >
> >> > > --
> >> > > Jacques Nadeau
> >> > > CTO and Co-Founder, Dremio
> >> > >
> >> > > On Sun, Jan 31, 2016 at 7:50 PM, Jinfeng Ni <ji...@gmail.com>
> >> > wrote:
> >> > >
> >> > >> Hi Jason,
> >> > >>
> >> > >> I'm fine that release 1.5.0 starts from 03197d0f2.  One thing I
> need
> >> > >> point out is that the pre-commit test suite has some testcase's
> plan
> >> > >> output modified because of the partition pruning change.  If we run
> >> > >> pre-commit against the candidate release for 1.5.0, we have to use
> a
> >> > >> prior version of pre-commit suite.
> >> > >>
> >> > >>
> >> > >>
> >> > >> On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse
> >> > >> <al...@gmail.com> wrote:
> >> > >> > That sounds reasonable to me. I'll start preparing a release
> >> > candidate.
> >> > >> >
> >> > >> > Jinfeng, are you okay waiting for the next release to include
> this
> >> > >> change?
> >> > >> >
> >> > >> > On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <
> jacques@dremio.com
> >> >
> >> > >> wrote:
> >> > >> >
> >> > >> >> Hey Jason,
> >> > >> >>
> >> > >> >> Can you start the release vote? We've let this drag on too long.
> >> > >> >>
> >> > >> >> I suggest we start from 03197d0f2. It seems like the partition
> >> > pruning
> >> > >> >> changes are pretty complicated and on a pretty main path to add
> >> right
> >> > >> >> before the vote. Since Hsuan's fix (pr 349) hasn't been
> reviewed, I
> >> > >> think
> >> > >> >> we should start the vote without it.
> >> > >> >>
> >> > >> >> Jacques
> >> > >> >>
> >> > >> >>
> >> > >> >> --
> >> > >> >> Jacques Nadeau
> >> > >> >> CTO and Co-Founder, Dremio
> >> > >> >>
> >> > >> >> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <
> hyichu@maprtech.com
> >> >
> >> > >> wrote:
> >> > >> >>
> >> > >> >> > Yes, just send a pull request.
> >> > >> >> > https://github.com/apache/drill/pull/349
> >> > >> >> >
> >> > >> >> > Can Jinfeng review it?
> >> > >> >> >
> >> > >> >> > Thanks,
> >> > >> >> >
> >> > >> >> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <
> >> jinfengni99@gmail.com
> >> > >
> >> > >> >> wrote:
> >> > >> >> >
> >> > >> >> > > Sean probably is quite close to find a fix for this issue.
> >> Let's
> >> > see
> >> > >> >> > > whether he can post a patch soon.
> >> > >> >> > >
> >> > >> >> > >
> >> > >> >> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
> >> > >> >> > > <ch...@gmail.com> wrote:
> >> > >> >> > > > DRILL-4323 only happens when assertions are turned on (my
> >> > install
> >> > >> >> > script
> >> > >> >> > > > automatically enables assertions).
> >> > >> >> > > >
> >> > >> >> > > > - Rahul
> >> > >> >> > > >
> >> > >> >> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <
> >> > >> zfong@maprtech.com>
> >> > >> >> > > wrote:
> >> > >> >> > > >
> >> > >> >> > > >> Rahul,
> >> > >> >> > > >>
> >> > >> >> > > >> Let us know if you can reproduce the issue with
> assertions
> >> > turned
> >> > >> >> OFF.
> >> > >> >> > > If
> >> > >> >> > > >> you cannot, then I think this should not be a blocker for
> >> 1.5,
> >> > >> and
> >> > >> >> > Hsuan
> >> > >> >> > > >> will continue to look into fixing this.
> >> > >> >> > > >>
> >> > >> >> > > >> -- Zelaine
> >> > >> >> > > >>
> >> > >> >> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
> >> > >> >> > knguyen@maprtech.com>
> >> > >> >> > > >> wrote:
> >> > >> >> > > >>
> >> > >> >> > > >> > I do see the same error with assertion turned on.
> >> > >> >> > > >> >
> >> > >> >> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <
> >> > >> >> jinfengni99@gmail.com>
> >> > >> >> > > >> wrote:
> >> > >> >> > > >> >
> >> > >> >> > > >> > > Krystal, by any chance, did you turn on assertion
> when
> >> you
> >> > >> start
> >> > >> >> > > >> > drillbit?
> >> > >> >> > > >> > >
> >> > >> >> > > >> > > I think the error in drill-4323 will be raised when
> >> > >> assertion =
> >> > >> >> > on.
> >> > >> >> > > >> > >
> >> > >> >> > > >> > >
> >> > >> >> > > >> > >
> >> > >> >> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
> >> > >> >> > > knguyen@maprtech.com>
> >> > >> >> > > >> > > wrote:
> >> > >> >> > > >> > > > I am running drill-1.4 and is able to run count(*)
> >> > >> >> successfully.
> >> > >> >> > > >> > > >
> >> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100>
> alter
> >> > >> session
> >> > >> >> > set
> >> > >> >> > > >> > > > `store.hive.optimize_scan_with_native_readers` =
> true;
> >> > >> >> > > >> > > >
> >> > >> >> >
> >> +-------+--------------------------------------------------------+
> >> > >> >> > > >> > > > |  ok   |                        summary
> >> > >> >> >  |
> >> > >> >> > > >> > > >
> >> > >> >> >
> >> +-------+--------------------------------------------------------+
> >> > >> >> > > >> > > > | true  |
> store.hive.optimize_scan_with_native_readers
> >> > >> >> > updated.  |
> >> > >> >> > > >> > > >
> >> > >> >> >
> >> +-------+--------------------------------------------------------+
> >> > >> >> > > >> > > > 1 row selected (0.374 seconds)
> >> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100>
> select
> >> > >> >> count(*)
> >> > >> >> > > from
> >> > >> >> > > >> > > > lineitem_parquet_partitioned_hive_string;
> >> > >> >> > > >> > > > +---------+
> >> > >> >> > > >> > > > | EXPR$0  |
> >> > >> >> > > >> > > > +---------+
> >> > >> >> > > >> > > > | 60175   |
> >> > >> >> > > >> > > > +---------+
> >> > >> >> > > >> > > > 1 row selected (2.986 seconds)
> >> > >> >> > > >> > > >
> >> > >> >> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
> >> > >> >> > > jinfengni99@gmail.com>
> >> > >> >> > > >> > > wrote:
> >> > >> >> > > >> > > >
> >> > >> >> > > >> > > >> Venki and I did some investigation for DRILL-4323.
> >> The
> >> > >> issue
> >> > >> >> > > >> reported
> >> > >> >> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as
> >> well.
> >> > >> Seems
> >> > >> >> > to
> >> > >> >> > > us
> >> > >> >> > > >> > > >> this is not a regression from 1.4.0; it's a
> >> regression
> >> > >> from
> >> > >> >> > 1.3.0
> >> > >> >> > > >> > > >> probably.
> >> > >> >> > > >> > > >>
> >> > >> >> > > >> > > >> DRILL-4083 makes the planner to use
> >> > DrillHiveNativeReader
> >> > >> in
> >> > >> >> > > stead
> >> > >> >> > > >> of
> >> > >> >> > > >> > > >> HiveReader for "select count(*) from hive_table"
> >> query.
> >> > >> >> > However,
> >> > >> >> > > >> the
> >> > >> >> > > >> > > >> Project after the scan produces empty schema.
> Before
> >> > >> >> > DRILL-4083,
> >> > >> >> > > >> > > >> Drill uses HiveScan, which works fine.
> >> > >> >> > > >> > > >>
> >> > >> >> > > >> > > >>
> >> > >> >> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
> >> > >> >> > > >> > > >> <al...@gmail.com> wrote:
> >> > >> >> > > >> > > >> > We could revert the change to the StoragePlugin
> >> > >> interface
> >> > >> >> to
> >> > >> >> > > add
> >> > >> >> > > >> the
> >> > >> >> > > >> > > >> > getLogicalOptimizerRules
> >> > >> >> > > >> > > >> > and getPhysicalOptimizerRules, and only put
> those
> >> in
> >> > >> >> > > >> > > >> AbstractStoragePlugin
> >> > >> >> > > >> > > >> > with default implementations.
> >> > >> >> > > >> > > >> >
> >> > >> >> > > >> > > >> > We could then update the docs to tell users to
> not
> >> > >> >> implement
> >> > >> >> > > the
> >> > >> >> > > >> > > >> interface
> >> > >> >> > > >> > > >> > directly, and instead extend the abstract class,
> >> > which I
> >> > >> >> > > believe
> >> > >> >> > > >> > would
> >> > >> >> > > >> > > >> > allow us to make changes like this in the future
> >> > without
> >> > >> >> > > breaking
> >> > >> >> > > >> > > >> plugins.
> >> > >> >> > > >> > > >> > At a major release version we could then just
> pull
> >> up
> >> > >> all
> >> > >> >> of
> >> > >> >> > > the
> >> > >> >> > > >> new
> >> > >> >> > > >> > > >> > methods we added to the abstract class into the
> >> > >> interface,
> >> > >> >> or
> >> > >> >> > > just
> >> > >> >> > > >> > get
> >> > >> >> > > >> > > >> rid
> >> > >> >> > > >> > > >> > of it all-together.
> >> > >> >> > > >> > > >> >
> >> > >> >> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki
> Korukanti <
> >> > >> >> > > >> > > >> venki.korukanti@gmail.com>
> >> > >> >> > > >> > > >> > wrote:
> >> > >> >> > > >> > > >> >
> >> > >> >> > > >> > > >> >> Here is the partial fix:
> >> > >> >> > > >> > > >> >>
> >> > >> >> > > >> > > >> >>
> >> > >> >> > > >> > > >>
> >> > >> >> > > >> > >
> >> > >> >> > > >> >
> >> > >> >> > > >>
> >> > >> >> > >
> >> > >> >> >
> >> > >> >>
> >> > >>
> >> >
> >>
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> >> > >> >> > > >> > > >> >>
> >> > >> >> > > >> > > >> >> If the existing StragePlugins are implementing
> >> > >> >> > > >> > AbstractStoragePlugin
> >> > >> >> > > >> > > >> class
> >> > >> >> > > >> > > >> >> this fix should resolve the backward compat
> issue.
> >> > If
> >> > >> they
> >> > >> >> > are
> >> > >> >> > > >> > > >> implementing
> >> > >> >> > > >> > > >> >> StoragePlugin interface directly then it won't
> >> > resolve
> >> > >> the
> >> > >> >> > > issue.
> >> > >> >> > > >> > Not
> >> > >> >> > > >> > > >> sure
> >> > >> >> > > >> > > >> >> how to handle it in that case. Let me know if
> >> there
> >> > any
> >> > >> >> ways
> >> > >> >> > > to
> >> > >> >> > > >> > > resolve
> >> > >> >> > > >> > > >> >> this issue completely.
> >> > >> >> > > >> > > >> >>
> >> > >> >> > > >> > > >> >> Thanks
> >> > >> >> > > >> > > >> >> Venki
> >> > >> >> > > >> > > >> >>
> >> > >> >> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <
> >> > >> >> > > asinha@maprtech.com
> >> > >> >> > > >> >
> >> > >> >> > > >> > > >> wrote:
> >> > >> >> > > >> > > >> >>
> >> > >> >> > > >> > > >> >> > I was able to do the build with maven 3.3.9
> >> (after
> >> > >> doing
> >> > >> >> > > 'brew
> >> > >> >> > > >> > > install
> >> > >> >> > > >> > > >> >> > maven').  Here are my env variables related
> to
> >> > maven:
> >> > >> >> > > >> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> >> > >> >> > > >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> >> > >> >> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> >> > >> >> > > >> > > >> >> >
> >> > >> >> > > >> > > >> >> >
> >> > >> >> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques
> Nadeau
> >> <
> >> > >> >> > > >> > > jacques@dremio.com>
> >> > >> >> > > >> > > >> >> > wrote:
> >> > >> >> > > >> > > >> >> >
> >> > >> >> > > >> > > >> >> > > Aman, for reference on the Maven version
> >> issue.
> >> > You
> >> > >> >> can
> >> > >> >> > > see
> >> > >> >> > > >> the
> >> > >> >> > > >> > > >> output
> >> > >> >> > > >> > > >> >> of
> >> > >> >> > > >> > > >> >> > > the shade plugin between the two Apache
> >> Builds.
> >> > If
> >> > >> you
> >> > >> >> > > search
> >> > >> >> > > >> > for
> >> > >> >> > > >> > > >> >> > > maven-shade-plugin you'll see that build
> 647
> >> > (maven
> >> > >> >> > 3.0.5)
> >> > >> >> > > >> > fails
> >> > >> >> > > >> > > >> >> because
> >> > >> >> > > >> > > >> >> > it
> >> > >> >> > > >> > > >> >> > > includes a bunch of hadoop (and other
> >> libraries
> >> > >> that
> >> > >> >> are
> >> > >> >> > > >> listed
> >> > >> >> > > >> > > as
> >> > >> >> > > >> > > >> >> being
> >> > >> >> > > >> > > >> >> > > excluded). If you look at build 648 (maven
> >> > 3.3.3),
> >> > >> >> those
> >> > >> >> > > >> items
> >> > >> >> > > >> > > are
> >> > >> >> > > >> > > >> >> > included
> >> > >> >> > > >> > > >> >> > > so that the build completes successfully.
> >> > >> >> > > >> > > >> >> > >
> >> > >> >> > > >> > > >> >> > >
> >> > >> >> https://builds.apache.org/job/drill-scm/647/consoleText
> >> > >> >> > > >> > > >> >> > >
> >> > >> >> https://builds.apache.org/job/drill-scm/648/consoleText
> >> > >> >> > > >> > > >> >> > >
> >> > >> >> > > >> > > >> >> > > --
> >> > >> >> > > >> > > >> >> > > Jacques Nadeau
> >> > >> >> > > >> > > >> >> > > CTO and Co-Founder, Dremio
> >> > >> >> > > >> > > >> >> > >
> >> > >> >> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul
> >> > challapalli
> >> > >> <
> >> > >> >> > > >> > > >> >> > > challapallirahul@gmail.com> wrote:
> >> > >> >> > > >> > > >> >> > >
> >> > >> >> > > >> > > >> >> > > > Jason,
> >> > >> >> > > >> > > >> >> > > >
> >> > >> >> > > >> > > >> >> > > > As of now I haven't narrowed down the
> exact
> >> > >> commit
> >> > >> >> > which
> >> > >> >> > > >> > caused
> >> > >> >> > > >> > > >> >> > > DRILL-4323
> >> > >> >> > > >> > > >> >> > > > and I do not know the scope of the fix.
> >> > >> >> > > >> > > >> >> > > >
> >> > >> >> > > >> > > >> >> > > > - Rahul
> >> > >> >> > > >> > > >> >> > > >
> >> > >> >> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason
> >> > Altekruse
> >> > >> <
> >> > >> >> > > >> > > >> >> > > altekrusejason@gmail.com
> >> > >> >> > > >> > > >> >> > > > >
> >> > >> >> > > >> > > >> >> > > > wrote:
> >> > >> >> > > >> > > >> >> > > >
> >> > >> >> > > >> > > >> >> > > > > I think it makes sense to wait for
> Venki's
> >> > fix
> >> > >> of
> >> > >> >> > the
> >> > >> >> > > >> API,
> >> > >> >> > > >> > as
> >> > >> >> > > >> > > >> >> > > releasing a
> >> > >> >> > > >> > > >> >> > > > > version with the change would make
> >> > >> >> > > reverting/refactoring
> >> > >> >> > > >> > the
> >> > >> >> > > >> > > API
> >> > >> >> > > >> > > >> >> > > breaking
> >> > >> >> > > >> > > >> >> > > > > change less useful.
> >> > >> >> > > >> > > >> >> > > > >
> >> > >> >> > > >> > > >> >> > > > > Do you have any idea about the scope of
> >> > fixing
> >> > >> the
> >> > >> >> > > hive
> >> > >> >> > > >> > > native
> >> > >> >> > > >> > > >> >> > reader,
> >> > >> >> > > >> > > >> >> > > or
> >> > >> >> > > >> > > >> >> > > > > which change since the 1.4 release
> broke
> >> > it? Is
> >> > >> >> > > someone
> >> > >> >> > > >> > > >> available
> >> > >> >> > > >> > > >> >> to
> >> > >> >> > > >> > > >> >> > > look
> >> > >> >> > > >> > > >> >> > > > > at it today?
> >> > >> >> > > >> > > >> >> > > > >
> >> > >> >> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul
> >> > >> >> challapalli <
> >> > >> >> > > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
> >> > >> >> > > >> > > >> >> > > > >
> >> > >> >> > > >> > > >> >> > > > > > Just found an issue with hive native
> >> > parquet
> >> > >> >> > reader
> >> > >> >> > > >> > > >> (DRILL-4323).
> >> > >> >> > > >> > > >> >> > > This
> >> > >> >> > > >> > > >> >> > > > > is a
> >> > >> >> > > >> > > >> >> > > > > > regression from 1.4 and in my opinion
> >> > should
> >> > >> be
> >> > >> >> > > treated
> >> > >> >> > > >> > as
> >> > >> >> > > >> > > a
> >> > >> >> > > >> > > >> >> > blocker.
> >> > >> >> > > >> > > >> >> > > > > >
> >> > >> >> > > >> > > >> >> > > > > > - Rahul
> >> > >> >> > > >> > > >> >> > > > > >
> >> > >> >> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM,
> Venki
> >> > >> >> Korukanti <
> >> > >> >> > > >> > > >> >> > > > > > venki.korukanti@gmail.com>
> >> > >> >> > > >> > > >> >> > > > > > wrote:
> >> > >> >> > > >> > > >> >> > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > Sorry I didn't realize I was
> breaking
> >> > the
> >> > >> >> public
> >> > >> >> > > >> > > interface
> >> > >> >> > > >> > > >> API.
> >> > >> >> > > >> > > >> >> > Let
> >> > >> >> > > >> > > >> >> > > > me
> >> > >> >> > > >> > > >> >> > > > > > work
> >> > >> >> > > >> > > >> >> > > > > > > on a patch to make it backward
> >> > compatible.
> >> > >> >> > > >> > > >> >> > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > Thanks
> >> > >> >> > > >> > > >> >> > > > > > > Venki
> >> > >> >> > > >> > > >> >> > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM,
> >> Steven
> >> > >> >> > Phillips
> >> > >> >> > > <
> >> > >> >> > > >> > > >> >> > > steven@dremio.com
> >> > >> >> > > >> > > >> >> > > > >
> >> > >> >> > > >> > > >> >> > > > > > > wrote:
> >> > >> >> > > >> > > >> >> > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > I just wanted to bring up an
> issue
> >> > that I
> >> > >> >> just
> >> > >> >> > > now
> >> > >> >> > > >> > > >> >> discovered,
> >> > >> >> > > >> > > >> >> > > that
> >> > >> >> > > >> > > >> >> > > > > has
> >> > >> >> > > >> > > >> >> > > > > > > > caused me a fair amount of grief.
> >> > >> >> > > >> > > >> >> > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > >
> >> > >> >> > > https://github.com/apache/drill/pull/300/commits
> >> > >> >> > > >> > > >> >> > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing
> >> API,
> >> > and
> >> > >> >> > causes
> >> > >> >> > > >> > > >> >> StoragePlugins
> >> > >> >> > > >> > > >> >> > > > that
> >> > >> >> > > >> > > >> >> > > > > > were
> >> > >> >> > > >> > > >> >> > > > > > > > compiled against currently
> released
> >> > >> versions
> >> > >> >> > of
> >> > >> >> > > >> Drill
> >> > >> >> > > >> > > to
> >> > >> >> > > >> > > >> no
> >> > >> >> > > >> > > >> >> > > longer
> >> > >> >> > > >> > > >> >> > > > > > > > functional properly. I would
> prefer
> >> > that
> >> > >> >> this
> >> > >> >> > > >> > breaking
> >> > >> >> > > >> > > >> change
> >> > >> >> > > >> > > >> >> > be
> >> > >> >> > > >> > > >> >> > > > > > modified
> >> > >> >> > > >> > > >> >> > > > > > > > to be backward compatible if
> >> possible.
> >> > >> >> > > >> > > >> >> > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM,
> >> > Jason
> >> > >> >> > > Altekruse <
> >> > >> >> > > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
> >> > >> >> > > >> > > >> >> > > > > > > > wrote:
> >> > >> >> > > >> > > >> >> > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > Hi Aman,
> >> > >> >> > > >> > > >> >> > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > This is the failure that he was
> >> > >> seeing. He
> >> > >> >> > > >> figured
> >> > >> >> > > >> > > out
> >> > >> >> > > >> > > >> that
> >> > >> >> > > >> > > >> >> > the
> >> > >> >> > > >> > > >> >> > > > new
> >> > >> >> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were not
> >> > being
> >> > >> >> > > respected
> >> > >> >> > > >> > when
> >> > >> >> > > >> > > the
> >> > >> >> > > >> > > >> >> > build
> >> > >> >> > > >> > > >> >> > > > was
> >> > >> >> > > >> > > >> >> > > > > > run
> >> > >> >> > > >> > > >> >> > > > > > > > with
> >> > >> >> > > >> > > >> >> > > > > > > > > an older Maven version, causing
> >> the
> >> > jar
> >> > >> >> size
> >> > >> >> > > to
> >> > >> >> > > >> > > increase
> >> > >> >> > > >> > > >> >> > > > > > significantly.
> >> > >> >> > > >> > > >> >> > > > > > > > He
> >> > >> >> > > >> > > >> >> > > > > > > > > added an enforcer to make sure
> the
> >> > JAR
> >> > >> >> > didn't
> >> > >> >> > > >> grow
> >> > >> >> > > >> > > >> >> > > unexpectedly.
> >> > >> >> > > >> > > >> >> > > > > Can
> >> > >> >> > > >> > > >> >> > > > > > > you
> >> > >> >> > > >> > > >> >> > > > > > > > > try to update your maven
> version
> >> and
> >> > >> >> re-run
> >> > >> >> > > the
> >> > >> >> > > >> > > build?
> >> > >> >> > > >> > > >> >> > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > - Jason
> >> > >> >> > > >> > > >> >> > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18
> AM,
> >> > Aman
> >> > >> >> > Sinha <
> >> > >> >> > > >> > > >> >> > > > asinha@maprtech.com>
> >> > >> >> > > >> > > >> >> > > > > > > > wrote:
> >> > >> >> > > >> > > >> >> > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the
> >> > following
> >> > >> >> build
> >> > >> >> > > >> failure
> >> > >> >> > > >> > > on
> >> > >> >> > > >> > > >> the
> >> > >> >> > > >> > > >> >> > > latest
> >> > >> >> > > >> > > >> >> > > > > > > master
> >> > >> >> > > >> > > >> >> > > > > > > > > > branch...is this what you saw
> >> for
> >> > the
> >> > >> >> > Apache
> >> > >> >> > > >> > build
> >> > >> >> > > >> > > ?
> >> > >> >> > > >> > > >> My
> >> > >> >> > > >> > > >> >> > mvn
> >> > >> >> > > >> > > >> >> > > > > > version
> >> > >> >> > > >> > > >> >> > > > > > > > > output
> >> > >> >> > > >> > > >> >> > > > > > > > > > is shown below.  Should we
> all
> >> be
> >> > >> >> > upgrading
> >> > >> >> > > to
> >> > >> >> > > >> a
> >> > >> >> > > >> > > newer
> >> > >> >> > > >> > > >> >> mvn
> >> > >> >> > > >> > > >> >> > ?
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > [INFO] ---
> >> > >> >> > > maven-enforcer-plugin:1.3.1:enforce
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> (enforce-jdbc-jar-compactness) @
> >> > >> >> > > drill-jdbc-all
> >> > >> >> > > >> > ---
> >> > >> >> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
> >> > >> >> > > >> > > >> >> > > > > >
> >> > >> >> org.apache.maven.plugins.enforcer.RequireFilesSize
> >> > >> >> > > >> > > >> >> > > > > > > > > failed
> >> > >> >> > > >> > > >> >> > > > > > > > > > with message:
> >> > >> >> > > >> > > >> >> > > > > > > > > > The file
> >> > >> >> drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> > >> >> > > is
> >> > >> >> > > >> > > outside
> >> > >> >> > > >> > > >> the
> >> > >> >> > > >> > > >> >> > > > > expected
> >> > >> >> > > >> > > >> >> > > > > > > size
> >> > >> >> > > >> > > >> >> > > > > > > > > > range.
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > >                   This is
> likely
> >> > due
> >> > >> to
> >> > >> >> > you
> >> > >> >> > > >> > adding
> >> > >> >> > > >> > > new
> >> > >> >> > > >> > > >> >> > > > > dependencies
> >> > >> >> > > >> > > >> >> > > > > > > to
> >> > >> >> > > >> > > >> >> > > > > > > > a
> >> > >> >> > > >> > > >> >> > > > > > > > > > java-exec and not updating
> the
> >> > >> excludes
> >> > >> >> in
> >> > >> >> > > this
> >> > >> >> > > >> > > >> module.
> >> > >> >> > > >> > > >> >> > This
> >> > >> >> > > >> > > >> >> > > is
> >> > >> >> > > >> > > >> >> > > > > > > > important
> >> > >> >> > > >> > > >> >> > > > > > > > > > as it minimizes the size of
> the
> >> > >> >> dependency
> >> > >> >> > > of
> >> > >> >> > > >> > Drill
> >> > >> >> > > >> > > >> >> > > application
> >> > >> >> > > >> > > >> >> > > > > > > users.
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > >
> >> > >> >> > > >> > > >> >> > > > > >
> >> > >> >> > > >> > > >> >> > > > >
> >> > >> >> > > >> > > >> >> > > >
> >> > >> >> > > >> > > >> >> > >
> >> > >> >> > > >> > > >> >> >
> >> > >> >> > > >> > > >> >>
> >> > >> >> > > >> > > >>
> >> > >> >> > > >> > >
> >> > >> >> > > >> >
> >> > >> >> > > >>
> >> > >> >> > >
> >> > >> >> >
> >> > >> >>
> >> > >>
> >> >
> >>
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> > >> >> > > >> > > >> >> > > > > > > > > > size (44664121) too large.
> Max.
> >> is
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > >
> >> > >> >> > > >> > > >> >> > > > > >
> >> > >> >> > > >> > > >> >> > > > >
> >> > >> >> > > >> > > >> >> > > >
> >> > >> >> > > >> > > >> >> > >
> >> > >> >> > > >> > > >> >> >
> >> > >> >> > > >> > > >> >>
> >> > >> >> > > >> > > >>
> >> > >> >> > > >> > >
> >> > >> >> > > >> >
> >> > >> >> > > >>
> >> > >> >> > >
> >> > >> >> >
> >> > >> >>
> >> > >>
> >> >
> >>
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > Administrators-MacBook-Pro-144:incubator-drill
> >> > >> >> > > >> > > asinha$
> >> > >> >> > > >> > > >> >> mvn
> >> > >> >> > > >> > > >> >> > > > > > --version
> >> > >> >> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
> >> > >> >> > > >> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> >> > >> >> > > >> > > >> >> > > > > > > > 2013-02-19
> >> > >> >> > > >> > > >> >> > > > > > > > > > 05:51:28-0800)
> >> > >> >> > > >> > > >> >> > > > > > > > > > Maven home:
> >> > >> /opt/local/share/java/maven3
> >> > >> >> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45,
> vendor:
> >> > >> Oracle
> >> > >> >> > > >> > Corporation
> >> > >> >> > > >> > > >> >> > > > > > > > > > Java home:
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > >
> >> > >> >> > > >> > > >>
> >> > >> >> > >
> >> > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> >> > >> >> > > >> > > >> >> > > > > > > > > > Default locale: en_US,
> platform
> >> > >> >> encoding:
> >> > >> >> > > UTF-8
> >> > >> >> > > >> > > >> >> > > > > > > > > > OS name: "mac os x", version:
> >> > >> "10.9.5",
> >> > >> >> > > arch:
> >> > >> >> > > >> > > >> "x86_64",
> >> > >> >> > > >> > > >> >> > > family:
> >> > >> >> > > >> > > >> >> > > > > > "mac"
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20
> AM,
> >> > >> Jacques
> >> > >> >> > > >> Nadeau <
> >> > >> >> > > >> > > >> >> > > > > > jacques@dremio.com>
> >> > >> >> > > >> > > >> >> > > > > > > > > > wrote:
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > Build back to normal. It
> looks
> >> > like
> >> > >> >> the
> >> > >> >> > > >> Apache
> >> > >> >> > > >> > > >> server
> >> > >> >> > > >> > > >> >> was
> >> > >> >> > > >> > > >> >> > > > using
> >> > >> >> > > >> > > >> >> > > > > > an
> >> > >> >> > > >> > > >> >> > > > > > > > old
> >> > >> >> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I
> >> > switched
> >> > >> to
> >> > >> >> > > >> something
> >> > >> >> > > >> > > more
> >> > >> >> > > >> > > >> >> > recent,
> >> > >> >> > > >> > > >> >> > > > the
> >> > >> >> > > >> > > >> >> > > > > > > build
> >> > >> >> > > >> > > >> >> > > > > > > > > > > passed.
> >> > >> >> > > >> > > >> >> > > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > --
> >> > >> >> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau
> >> > >> >> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
> >> > >> >> > > >> > > >> >> > > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at
> 7:02
> >> AM,
> >> > >> >> Jacques
> >> > >> >> > > >> > Nadeau <
> >> > >> >> > > >> > > >> >> > > > > > > jacques@dremio.com>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > wrote:
> >> > >> >> > > >> > > >> >> > > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge caused
> the
> >> > >> Apache
> >> > >> >> > > build
> >> > >> >> > > >> to
> >> > >> >> > > >> > > fail.
> >> > >> >> > > >> > > >> >> > > > > > > Investigating...
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > > --
> >> > >> >> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
> >> > >> >> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder,
> Dremio
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at
> 6:31
> >> > AM,
> >> > >> >> > Jacques
> >> > >> >> > > >> > > Nadeau <
> >> > >> >> > > >> > > >> >> > > > > > > > jacques@dremio.com>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > > wrote:
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> I got clean regression
> runs
> >> > as
> >> > >> >> well.
> >> > >> >> > > I've
> >> > >> >> > > >> > > merged
> >> > >> >> > > >> > > >> the
> >> > >> >> > > >> > > >> >> > > > patch.
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to start
> >> the
> >> > >> >> release
> >> > >> >> > > >> > process?
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> --
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder,
> Dremio
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at
> >> 10:42
> >> > >> PM,
> >> > >> >> > > Abhishek
> >> > >> >> > > >> > > >> Girish <
> >> > >> >> > > >> > > >> >> > > > > > > > agirish@mapr.com
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >> wrote:
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> Had two clean
> Functional
> >> > runs.
> >> > >> >> TPC-H
> >> > >> >> > > >> SF100
> >> > >> >> > > >> > > was
> >> > >> >> > > >> > > >> also
> >> > >> >> > > >> > > >> >> > > > > > successful.
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at
> >> > 10:07
> >> > >> PM,
> >> > >> >> > > rahul
> >> > >> >> > > >> > > >> >> challapalli <
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> challapallirahul@gmail.com>
> >> > >> >> wrote:
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a
> functional
> >> > run
> >> > >> with
> >> > >> >> > > your
> >> > >> >> > > >> > > branch.
> >> > >> >> > > >> > > >> >> Will
> >> > >> >> > > >> > > >> >> > > let
> >> > >> >> > > >> > > >> >> > > > > you
> >> > >> >> > > >> > > >> >> > > > > > > know
> >> > >> >> > > >> > > >> >> > > > > > > > > > once
> >> > >> >> > > >> > > >> >> > > > > > > > > > > it
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > finishes
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016
> at
> >> > 9:56
> >> > >> PM,
> >> > >> >> > > >> Jacques
> >> > >> >> > > >> > > >> Nadeau <
> >> > >> >> > > >> > > >> >> > > > > > > > > > jacques@dremio.com>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > wrote:
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged
> >> today. I
> >> > >> have
> >> > >> >> an
> >> > >> >> > > >> > updated
> >> > >> >> > > >> > > >> patch
> >> > >> >> > > >> > > >> >> > for
> >> > >> >> > > >> > > >> >> > > > 4291
> >> > >> >> > > >> > > >> >> > > > > > > that
> >> > >> >> > > >> > > >> >> > > > > > > > is
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> ready.
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it
> >> seems
> >> > >> that
> >> > >> >> > > >> something
> >> > >> >> > > >> > > isn't
> >> > >> >> > > >> > > >> >> > > working
> >> > >> >> > > >> > > >> >> > > > > with
> >> > >> >> > > >> > > >> >> > > > > > > our
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> extended
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't
> >> been
> >> > >> able
> >> > >> >> to
> >> > >> >> > > run
> >> > >> >> > > >> an
> >> > >> >> > > >> > > >> extended
> >> > >> >> > > >> > > >> >> > > > > > regression.
> >> > >> >> > > >> > > >> >> > > > > > > > > Unit
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> tests
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone
> else
> >> > >> possibly
> >> > >> >> > > able
> >> > >> >> > > >> to
> >> > >> >> > > >> > > run a
> >> > >> >> > > >> > > >> >> > > > regression
> >> > >> >> > > >> > > >> >> > > > > > > suite
> >> > >> >> > > >> > > >> >> > > > > > > > > > > against
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > this
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we
> can
> >> > >> confirm
> >> > >> >> > > things
> >> > >> >> > > >> > look
> >> > >> >> > > >> > > >> good
> >> > >> >> > > >> > > >> >> and
> >> > >> >> > > >> > > >> >> > > > start
> >> > >> >> > > >> > > >> >> > > > > > the
> >> > >> >> > > >> > > >> >> > > > > > > > > > release
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > process?
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > thanks,
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
> >> > >> >> > > >> > > >> >> > > > >
> >> > >> >> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > --
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder,
> >> > Dremio
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25,
> 2016
> >> at
> >> > >> 11:20
> >> > >> >> > AM,
> >> > >> >> > > >> > Jacques
> >> > >> >> > > >> > > >> >> Nadeau
> >> > >> >> > > >> > > >> >> > <
> >> > >> >> > > >> > > >> >> > > > > > > > > > > jacques@dremio.com
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the main
> >> > things
> >> > >> are
> >> > >> >> > 4196
> >> > >> >> > > >> and
> >> > >> >> > > >> > > 4291
> >> > >> >> > > >> > > >> >> > should
> >> > >> >> > > >> > > >> >> > > be
> >> > >> >> > > >> > > >> >> > > > > > > > > completed.
> >> > >> >> > > >> > > >> >> > > > > > > > > > I
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> know
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Amit
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to
> >> reproduce
> >> > >> 4196
> >> > >> >> > > locally
> >> > >> >> > > >> > this
> >> > >> >> > > >> > > >> >> weekend
> >> > >> >> > > >> > > >> >> > > so
> >> > >> >> > > >> > > >> >> > > > I
> >> > >> >> > > >> > > >> >> > > > > > > think
> >> > >> >> > > >> > > >> >> > > > > > > > > > we're
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> close
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > on
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > that.
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > --
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and
> Co-Founder,
> >> > >> Dremio
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25,
> 2016
> >> > at
> >> > >> >> 10:14
> >> > >> >> > > AM,
> >> > >> >> > > >> > > Zelaine
> >> > >> >> > > >> > > >> >> Fong
> >> > >> >> > > >> > > >> >> > <
> >> > >> >> > > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on
> >> this?
> >> > >> >> What's
> >> > >> >> > > >> > blocking
> >> > >> >> > > >> > > us
> >> > >> >> > > >> > > >> >> from
> >> > >> >> > > >> > > >> >> > > > taking
> >> > >> >> > > >> > > >> >> > > > > > > this
> >> > >> >> > > >> > > >> >> > > > > > > > > to a
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> vote --
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > the
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join
> >> > issues?
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19,
> >> 2016
> >> > at
> >> > >> >> 4:18
> >> > >> >> > > PM,
> >> > >> >> > > >> > > Jacques
> >> > >> >> > > >> > > >> >> > Nadeau
> >> > >> >> > > >> > > >> >> > > <
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this
> >> > thread...
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the
> >> issues
> >> > >> that
> >> > >> >> > were
> >> > >> >> > > >> > > >> mentioned in
> >> > >> >> > > >> > > >> >> > > this
> >> > >> >> > > >> > > >> >> > > > > > thread
> >> > >> >> > > >> > > >> >> > > > > > > > > along
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> with a
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed
> >> > >> categorization:
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release
> Blockers
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
> Amit
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4190
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
> Amit
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4196
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge
> >> > Jacques
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > >> >> > > >> > > >> >>
> https://issues.apache.org/jira/browse/DRILL-4246
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review
> Jinfeng
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4256
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
> >> Jacques
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4278
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge
> >> > Laurent
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > >> >> > > >> > > >> >>
> https://issues.apache.org/jira/browse/DRILL-4285
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open
> Jason/Hakim
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4247
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
> Jason
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4203
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> https://issues.apache.org/jira/browse/DRILL-4266
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge
> >> > Jacques
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > >> >> > > >> > > >> >>
> https://issues.apache.org/jira/browse/DRILL-4131
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do others
> >> > think?
> >> > >> >> Let's
> >> > >> >> > > try
> >> > >> >> > > >> to
> >> > >> >> > > >> > > get
> >> > >> >> > > >> > > >> the
> >> > >> >> > > >> > > >> >> > > > > blockers
> >> > >> >> > > >> > > >> >> > > > > > > > > wrapped
> >> > >> >> > > >> > > >> >> > > > > > > > > > up
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> in
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > the
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> next
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two and
> >> > start a
> >> > >> >> > > release
> >> > >> >> > > >> > > vote...
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > --
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and
> >> Co-Founder,
> >> > >> >> Dremio
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4,
> >> > 2016 at
> >> > >> >> 1:48
> >> > >> >> > > PM,
> >> > >> >> > > >> > > Jason
> >> > >> >> > > >> > > >> >> > > Altekruse
> >> > >> >> > > >> > > >> >> > > > <
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > >> altekrusejason@gmail.com>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the
> >> > allocator
> >> > >> >> > changes
> >> > >> >> > > >> > merged
> >> > >> >> > > >> > > and
> >> > >> >> > > >> > > >> >> > about
> >> > >> >> > > >> > > >> >> > > a
> >> > >> >> > > >> > > >> >> > > > > > month
> >> > >> >> > > >> > > >> >> > > > > > > > > since
> >> > >> >> > > >> > > >> >> > > > > > > > > > > the
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> last
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it
> >> would
> >> > be
> >> > >> >> good
> >> > >> >> > to
> >> > >> >> > > >> > start
> >> > >> >> > > >> > > a
> >> > >> >> > > >> > > >> vote
> >> > >> >> > > >> > > >> >> > > > soon. I
> >> > >> >> > > >> > > >> >> > > > > > > would
> >> > >> >> > > >> > > >> >> > > > > > > > > > like
> >> > >> >> > > >> > > >> >> > > > > > > > > > > to
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release
> >> > manager.
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that
> >> there
> >> > >> were
> >> > >> >> > some
> >> > >> >> > > >> > issues
> >> > >> >> > > >> > > >> that
> >> > >> >> > > >> > > >> >> > were
> >> > >> >> > > >> > > >> >> > > > > > > > identified
> >> > >> >> > > >> > > >> >> > > > > > > > > > > after
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> the
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was
> >> > merged. I
> >> > >> >> think
> >> > >> >> > > that
> >> > >> >> > > >> > > these
> >> > >> >> > > >> > > >> >> > issues
> >> > >> >> > > >> > > >> >> > > > > should
> >> > >> >> > > >> > > >> >> > > > > > > be
> >> > >> >> > > >> > > >> >> > > > > > > > > > fixed
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> before
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > we
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release
> >> > candidate.
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From
> looking at
> >> > the
> >> > >> >> > > associated
> >> > >> >> > > >> > > JIRAs
> >> > >> >> > > >> > > >> it
> >> > >> >> > > >> > > >> >> > > looked
> >> > >> >> > > >> > > >> >> > > > > > like
> >> > >> >> > > >> > > >> >> > > > > > > > > there
> >> > >> >> > > >> > > >> >> > > > > > > > > > > was
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> a
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short term
> fix
> >> > just
> >> > >> >> > > adjusting
> >> > >> >> > > >> > the
> >> > >> >> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > option,
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> and
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more
> >> > involved
> >> > >> work
> >> > >> >> > to
> >> > >> >> > > >> > change
> >> > >> >> > > >> > > >> how we
> >> > >> >> > > >> > > >> >> > > > > determine
> >> > >> >> > > >> > > >> >> > > > > > > the
> >> > >> >> > > >> > > >> >> > > > > > > > > > > correct
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > time
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > to
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill during
> >> > >> external
> >> > >> >> > > sort. I
> >> > >> >> > > >> > > >> believe it
> >> > >> >> > > >> > > >> >> > > makes
> >> > >> >> > > >> > > >> >> > > > > > sense
> >> > >> >> > > >> > > >> >> > > > > > > > to
> >> > >> >> > > >> > > >> >> > > > > > > > > > make
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > external
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well
> with
> >> > the
> >> > >> >> newly
> >> > >> >> > > >> > improved
> >> > >> >> > > >> > > >> memory
> >> > >> >> > > >> > > >> >> > > > > > accounting
> >> > >> >> > > >> > > >> >> > > > > > > > > before
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > cutting a
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release, but
> >> I'm
> >> > not
> >> > >> >> sure
> >> > >> >> > > how
> >> > >> >> > > >> > much
> >> > >> >> > > >> > > >> work
> >> > >> >> > > >> > > >> >> is
> >> > >> >> > > >> > > >> >> > > > left
> >> > >> >> > > >> > > >> >> > > > > to
> >> > >> >> > > >> > > >> >> > > > > > > be
> >> > >> >> > > >> > > >> >> > > > > > > > > done
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> there.
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please
> respond
> >> > with
> >> > >> >> your
> >> > >> >> > > >> > thoughts
> >> > >> >> > > >> > > on
> >> > >> >> > > >> > > >> a
> >> > >> >> > > >> > > >> >> > > release
> >> > >> >> > > >> > > >> >> > > > > > soon
> >> > >> >> > > >> > > >> >> > > > > > > > and
> >> > >> >> > > >> > > >> >> > > > > > > > > > any
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> JIRAs
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > you
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to
> include
> >> > in
> >> > >> the
> >> > >> >> > > >> release.
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
> >> > >> >> > > >> > > >> >> > > > > > > >
> >> > >> >> > > https://issues.apache.org/jira/browse/DRILL-4243
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >>
> >> > >> >> > > >> > > >> >> > > > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > > >
> >> > >> >> > > >> > > >> >> > > > > > >
> >> > >> >> > > >> > > >> >> > > > > >
> >> > >> >> > > >> > > >> >> > > > >
> >> > >> >> > > >> > > >> >> > > >
> >> > >> >> > > >> > > >> >> > >
> >> > >> >> > > >> > > >> >> >
> >> > >> >> > > >> > > >> >>
> >> > >> >> > > >> > > >>
> >> > >> >> > > >> > >
> >> > >> >> > > >> >
> >> > >> >> > > >>
> >> > >> >> > >
> >> > >> >> >
> >> > >> >>
> >> > >>
> >> > >>
> >> >
> >> >
> >>
>

Re: Time for a 1.5 release?

Posted by Jinfeng Ni <ji...@gmail.com>.

Good to hear this, Jacques & Jason. That makes feel relieved. :-)



On Sun, Jan 31, 2016 at 10:36 PM, Jason Altekruse
<al...@gmail.com> wrote:
> I don't think there was any issue with the merge either Jinfeng, I don't
> think we need to close master. I had to step out for a few hours, but I am
> working on finishing up the candidate right now.
>
> Having a weird RAT issue running the actual maven release command, it's
> complaining about a dependency-reduced-pom.xml in the target directory of
> jdbc-all lacking a license header (although target directories should be
> ignored). Checking out the commit before the recent changes to the jdbc-all
> exclusions seems to get rid of the issue, but the changes in the pom file
> don't seem like they should be causing any problems.
>
> Still trying to debug and hoping to have a vote up tonight.
>
> On Sun, Jan 31, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com> wrote:
>
>> Given how easy it is to create a branch off a random commit, I generally
>> don't see a point in closing the master branch. I figure the release
>> manager can propose where they'd like to fork if this type of thing comes
>> up.
>>
>> Thanks for the feedback Jinfeng. Of course you're right about Venki's
>> commit. My bad.
>>
>> --
>> Jacques Nadeau
>> CTO and Co-Founder, Dremio
>>
>> On Sun, Jan 31, 2016 at 9:21 PM, Jinfeng Ni <ji...@gmail.com> wrote:
>>
>> > Venki's backwards compatibility fix (commit id:
>> >    03197d0) is before the partition pruning commit. So, it seems to be
>> > fine to create a release candidate from 03197d0; no need to
>> > cherry-pick.
>> >
>> > As for whether the partition pruning changes are risky, the pre-commit
>> > run did not show any serious problems in the first run; all the plan
>> > changes are expected, or expose some existing problem (drill-4279).
>> > However, I agree that it impacts the main code path of query planning,
>> > and I'm fine that we exclude the partition change from 1.5.0 release,
>> > since it's merged right before we start a voting.
>> >
>> > I did not see any one mentioned that the master branch is closed, and
>> > therefore assume it's still open for commits.  I probably should have
>> > asked whether the master was open for new patches before I merged the
>> > commits.
>> >
>> >
>> >
>> > On Sun, Jan 31, 2016 at 8:43 PM, Jacques Nadeau <ja...@dremio.com>
>> > wrote:
>> > > Jinfeng, do you agree that the changes are risky? I was going mostly by
>> > > commit messages.
>> > >
>> > > Also just realized that the backwards compatibility fix to the storage
>> > > plugin interface is after these. Could create release branch from
>> > 03197d...
>> > > and then cherry pick the Venki change. Or could keep them all. Or
>> > > go without the compatibility fix....
>> > >
>> > > --
>> > > Jacques Nadeau
>> > > CTO and Co-Founder, Dremio
>> > >
>> > > On Sun, Jan 31, 2016 at 7:50 PM, Jinfeng Ni <ji...@gmail.com>
>> > wrote:
>> > >
>> > >> Hi Jason,
>> > >>
>> > >> I'm fine that release 1.5.0 starts from 03197d0f2.  One thing I need
>> > >> point out is that the pre-commit test suite has some testcase's plan
>> > >> output modified because of the partition pruning change.  If we run
>> > >> pre-commit against the candidate release for 1.5.0, we have to use a
>> > >> prior version of pre-commit suite.
>> > >>
>> > >>
>> > >>
>> > >> On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse
>> > >> <al...@gmail.com> wrote:
>> > >> > That sounds reasonable to me. I'll start preparing a release
>> > candidate.
>> > >> >
>> > >> > Jinfeng, are you okay waiting for the next release to include this
>> > >> change?
>> > >> >
>> > >> > On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <jacques@dremio.com
>> >
>> > >> wrote:
>> > >> >
>> > >> >> Hey Jason,
>> > >> >>
>> > >> >> Can you start the release vote? We've let this drag on too long.
>> > >> >>
>> > >> >> I suggest we start from 03197d0f2. It seems like the partition
>> > pruning
>> > >> >> changes are pretty complicated and on a pretty main path to add
>> right
>> > >> >> before the vote. Since Hsuan's fix (pr 349) hasn't been reviewed, I
>> > >> think
>> > >> >> we should start the vote without it.
>> > >> >>
>> > >> >> Jacques
>> > >> >>
>> > >> >>
>> > >> >> --
>> > >> >> Jacques Nadeau
>> > >> >> CTO and Co-Founder, Dremio
>> > >> >>
>> > >> >> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <hyichu@maprtech.com
>> >
>> > >> wrote:
>> > >> >>
>> > >> >> > Yes, just send a pull request.
>> > >> >> > https://github.com/apache/drill/pull/349
>> > >> >> >
>> > >> >> > Can Jinfeng review it?
>> > >> >> >
>> > >> >> > Thanks,
>> > >> >> >
>> > >> >> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <
>> jinfengni99@gmail.com
>> > >
>> > >> >> wrote:
>> > >> >> >
>> > >> >> > > Sean probably is quite close to find a fix for this issue.
>> Let's
>> > see
>> > >> >> > > whether he can post a patch soon.
>> > >> >> > >
>> > >> >> > >
>> > >> >> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
>> > >> >> > > <ch...@gmail.com> wrote:
>> > >> >> > > > DRILL-4323 only happens when assertions are turned on (my
>> > install
>> > >> >> > script
>> > >> >> > > > automatically enables assertions).
>> > >> >> > > >
>> > >> >> > > > - Rahul
>> > >> >> > > >
>> > >> >> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <
>> > >> zfong@maprtech.com>
>> > >> >> > > wrote:
>> > >> >> > > >
>> > >> >> > > >> Rahul,
>> > >> >> > > >>
>> > >> >> > > >> Let us know if you can reproduce the issue with assertions
>> > turned
>> > >> >> OFF.
>> > >> >> > > If
>> > >> >> > > >> you cannot, then I think this should not be a blocker for
>> 1.5,
>> > >> and
>> > >> >> > Hsuan
>> > >> >> > > >> will continue to look into fixing this.
>> > >> >> > > >>
>> > >> >> > > >> -- Zelaine
>> > >> >> > > >>
>> > >> >> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
>> > >> >> > knguyen@maprtech.com>
>> > >> >> > > >> wrote:
>> > >> >> > > >>
>> > >> >> > > >> > I do see the same error with assertion turned on.
>> > >> >> > > >> >
>> > >> >> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <
>> > >> >> jinfengni99@gmail.com>
>> > >> >> > > >> wrote:
>> > >> >> > > >> >
>> > >> >> > > >> > > Krystal, by any chance, did you turn on assertion when
>> you
>> > >> start
>> > >> >> > > >> > drillbit?
>> > >> >> > > >> > >
>> > >> >> > > >> > > I think the error in drill-4323 will be raised when
>> > >> assertion =
>> > >> >> > on.
>> > >> >> > > >> > >
>> > >> >> > > >> > >
>> > >> >> > > >> > >
>> > >> >> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
>> > >> >> > > knguyen@maprtech.com>
>> > >> >> > > >> > > wrote:
>> > >> >> > > >> > > > I am running drill-1.4 and is able to run count(*)
>> > >> >> successfully.
>> > >> >> > > >> > > >
>> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter
>> > >> session
>> > >> >> > set
>> > >> >> > > >> > > > `store.hive.optimize_scan_with_native_readers` = true;
>> > >> >> > > >> > > >
>> > >> >> >
>> +-------+--------------------------------------------------------+
>> > >> >> > > >> > > > |  ok   |                        summary
>> > >> >> >  |
>> > >> >> > > >> > > >
>> > >> >> >
>> +-------+--------------------------------------------------------+
>> > >> >> > > >> > > > | true  | store.hive.optimize_scan_with_native_readers
>> > >> >> > updated.  |
>> > >> >> > > >> > > >
>> > >> >> >
>> +-------+--------------------------------------------------------+
>> > >> >> > > >> > > > 1 row selected (0.374 seconds)
>> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select
>> > >> >> count(*)
>> > >> >> > > from
>> > >> >> > > >> > > > lineitem_parquet_partitioned_hive_string;
>> > >> >> > > >> > > > +---------+
>> > >> >> > > >> > > > | EXPR$0  |
>> > >> >> > > >> > > > +---------+
>> > >> >> > > >> > > > | 60175   |
>> > >> >> > > >> > > > +---------+
>> > >> >> > > >> > > > 1 row selected (2.986 seconds)
>> > >> >> > > >> > > >
>> > >> >> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
>> > >> >> > > jinfengni99@gmail.com>
>> > >> >> > > >> > > wrote:
>> > >> >> > > >> > > >
>> > >> >> > > >> > > >> Venki and I did some investigation for DRILL-4323.
>> The
>> > >> issue
>> > >> >> > > >> reported
>> > >> >> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as
>> well.
>> > >> Seems
>> > >> >> > to
>> > >> >> > > us
>> > >> >> > > >> > > >> this is not a regression from 1.4.0; it's a
>> regression
>> > >> from
>> > >> >> > 1.3.0
>> > >> >> > > >> > > >> probably.
>> > >> >> > > >> > > >>
>> > >> >> > > >> > > >> DRILL-4083 makes the planner to use
>> > DrillHiveNativeReader
>> > >> in
>> > >> >> > > stead
>> > >> >> > > >> of
>> > >> >> > > >> > > >> HiveReader for "select count(*) from hive_table"
>> query.
>> > >> >> > However,
>> > >> >> > > >> the
>> > >> >> > > >> > > >> Project after the scan produces empty schema.  Before
>> > >> >> > DRILL-4083,
>> > >> >> > > >> > > >> Drill uses HiveScan, which works fine.
>> > >> >> > > >> > > >>
>> > >> >> > > >> > > >>
>> > >> >> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
>> > >> >> > > >> > > >> <al...@gmail.com> wrote:
>> > >> >> > > >> > > >> > We could revert the change to the StoragePlugin
>> > >> interface
>> > >> >> to
>> > >> >> > > add
>> > >> >> > > >> the
>> > >> >> > > >> > > >> > getLogicalOptimizerRules
>> > >> >> > > >> > > >> > and getPhysicalOptimizerRules, and only put those
>> in
>> > >> >> > > >> > > >> AbstractStoragePlugin
>> > >> >> > > >> > > >> > with default implementations.
>> > >> >> > > >> > > >> >
>> > >> >> > > >> > > >> > We could then update the docs to tell users to not
>> > >> >> implement
>> > >> >> > > the
>> > >> >> > > >> > > >> interface
>> > >> >> > > >> > > >> > directly, and instead extend the abstract class,
>> > which I
>> > >> >> > > believe
>> > >> >> > > >> > would
>> > >> >> > > >> > > >> > allow us to make changes like this in the future
>> > without
>> > >> >> > > breaking
>> > >> >> > > >> > > >> plugins.
>> > >> >> > > >> > > >> > At a major release version we could then just pull
>> up
>> > >> all
>> > >> >> of
>> > >> >> > > the
>> > >> >> > > >> new
>> > >> >> > > >> > > >> > methods we added to the abstract class into the
>> > >> interface,
>> > >> >> or
>> > >> >> > > just
>> > >> >> > > >> > get
>> > >> >> > > >> > > >> rid
>> > >> >> > > >> > > >> > of it all-together.
>> > >> >> > > >> > > >> >
>> > >> >> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
>> > >> >> > > >> > > >> venki.korukanti@gmail.com>
>> > >> >> > > >> > > >> > wrote:
>> > >> >> > > >> > > >> >
>> > >> >> > > >> > > >> >> Here is the partial fix:
>> > >> >> > > >> > > >> >>
>> > >> >> > > >> > > >> >>
>> > >> >> > > >> > > >>
>> > >> >> > > >> > >
>> > >> >> > > >> >
>> > >> >> > > >>
>> > >> >> > >
>> > >> >> >
>> > >> >>
>> > >>
>> >
>> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
>> > >> >> > > >> > > >> >>
>> > >> >> > > >> > > >> >> If the existing StragePlugins are implementing
>> > >> >> > > >> > AbstractStoragePlugin
>> > >> >> > > >> > > >> class
>> > >> >> > > >> > > >> >> this fix should resolve the backward compat issue.
>> > If
>> > >> they
>> > >> >> > are
>> > >> >> > > >> > > >> implementing
>> > >> >> > > >> > > >> >> StoragePlugin interface directly then it won't
>> > resolve
>> > >> the
>> > >> >> > > issue.
>> > >> >> > > >> > Not
>> > >> >> > > >> > > >> sure
>> > >> >> > > >> > > >> >> how to handle it in that case. Let me know if
>> there
>> > any
>> > >> >> ways
>> > >> >> > > to
>> > >> >> > > >> > > resolve
>> > >> >> > > >> > > >> >> this issue completely.
>> > >> >> > > >> > > >> >>
>> > >> >> > > >> > > >> >> Thanks
>> > >> >> > > >> > > >> >> Venki
>> > >> >> > > >> > > >> >>
>> > >> >> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <
>> > >> >> > > asinha@maprtech.com
>> > >> >> > > >> >
>> > >> >> > > >> > > >> wrote:
>> > >> >> > > >> > > >> >>
>> > >> >> > > >> > > >> >> > I was able to do the build with maven 3.3.9
>> (after
>> > >> doing
>> > >> >> > > 'brew
>> > >> >> > > >> > > install
>> > >> >> > > >> > > >> >> > maven').  Here are my env variables related to
>> > maven:
>> > >> >> > > >> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
>> > >> >> > > >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
>> > >> >> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
>> > >> >> > > >> > > >> >> >
>> > >> >> > > >> > > >> >> >
>> > >> >> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau
>> <
>> > >> >> > > >> > > jacques@dremio.com>
>> > >> >> > > >> > > >> >> > wrote:
>> > >> >> > > >> > > >> >> >
>> > >> >> > > >> > > >> >> > > Aman, for reference on the Maven version
>> issue.
>> > You
>> > >> >> can
>> > >> >> > > see
>> > >> >> > > >> the
>> > >> >> > > >> > > >> output
>> > >> >> > > >> > > >> >> of
>> > >> >> > > >> > > >> >> > > the shade plugin between the two Apache
>> Builds.
>> > If
>> > >> you
>> > >> >> > > search
>> > >> >> > > >> > for
>> > >> >> > > >> > > >> >> > > maven-shade-plugin you'll see that build 647
>> > (maven
>> > >> >> > 3.0.5)
>> > >> >> > > >> > fails
>> > >> >> > > >> > > >> >> because
>> > >> >> > > >> > > >> >> > it
>> > >> >> > > >> > > >> >> > > includes a bunch of hadoop (and other
>> libraries
>> > >> that
>> > >> >> are
>> > >> >> > > >> listed
>> > >> >> > > >> > > as
>> > >> >> > > >> > > >> >> being
>> > >> >> > > >> > > >> >> > > excluded). If you look at build 648 (maven
>> > 3.3.3),
>> > >> >> those
>> > >> >> > > >> items
>> > >> >> > > >> > > are
>> > >> >> > > >> > > >> >> > included
>> > >> >> > > >> > > >> >> > > so that the build completes successfully.
>> > >> >> > > >> > > >> >> > >
>> > >> >> > > >> > > >> >> > >
>> > >> >> https://builds.apache.org/job/drill-scm/647/consoleText
>> > >> >> > > >> > > >> >> > >
>> > >> >> https://builds.apache.org/job/drill-scm/648/consoleText
>> > >> >> > > >> > > >> >> > >
>> > >> >> > > >> > > >> >> > > --
>> > >> >> > > >> > > >> >> > > Jacques Nadeau
>> > >> >> > > >> > > >> >> > > CTO and Co-Founder, Dremio
>> > >> >> > > >> > > >> >> > >
>> > >> >> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul
>> > challapalli
>> > >> <
>> > >> >> > > >> > > >> >> > > challapallirahul@gmail.com> wrote:
>> > >> >> > > >> > > >> >> > >
>> > >> >> > > >> > > >> >> > > > Jason,
>> > >> >> > > >> > > >> >> > > >
>> > >> >> > > >> > > >> >> > > > As of now I haven't narrowed down the exact
>> > >> commit
>> > >> >> > which
>> > >> >> > > >> > caused
>> > >> >> > > >> > > >> >> > > DRILL-4323
>> > >> >> > > >> > > >> >> > > > and I do not know the scope of the fix.
>> > >> >> > > >> > > >> >> > > >
>> > >> >> > > >> > > >> >> > > > - Rahul
>> > >> >> > > >> > > >> >> > > >
>> > >> >> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason
>> > Altekruse
>> > >> <
>> > >> >> > > >> > > >> >> > > altekrusejason@gmail.com
>> > >> >> > > >> > > >> >> > > > >
>> > >> >> > > >> > > >> >> > > > wrote:
>> > >> >> > > >> > > >> >> > > >
>> > >> >> > > >> > > >> >> > > > > I think it makes sense to wait for Venki's
>> > fix
>> > >> of
>> > >> >> > the
>> > >> >> > > >> API,
>> > >> >> > > >> > as
>> > >> >> > > >> > > >> >> > > releasing a
>> > >> >> > > >> > > >> >> > > > > version with the change would make
>> > >> >> > > reverting/refactoring
>> > >> >> > > >> > the
>> > >> >> > > >> > > API
>> > >> >> > > >> > > >> >> > > breaking
>> > >> >> > > >> > > >> >> > > > > change less useful.
>> > >> >> > > >> > > >> >> > > > >
>> > >> >> > > >> > > >> >> > > > > Do you have any idea about the scope of
>> > fixing
>> > >> the
>> > >> >> > > hive
>> > >> >> > > >> > > native
>> > >> >> > > >> > > >> >> > reader,
>> > >> >> > > >> > > >> >> > > or
>> > >> >> > > >> > > >> >> > > > > which change since the 1.4 release broke
>> > it? Is
>> > >> >> > > someone
>> > >> >> > > >> > > >> available
>> > >> >> > > >> > > >> >> to
>> > >> >> > > >> > > >> >> > > look
>> > >> >> > > >> > > >> >> > > > > at it today?
>> > >> >> > > >> > > >> >> > > > >
>> > >> >> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul
>> > >> >> challapalli <
>> > >> >> > > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
>> > >> >> > > >> > > >> >> > > > >
>> > >> >> > > >> > > >> >> > > > > > Just found an issue with hive native
>> > parquet
>> > >> >> > reader
>> > >> >> > > >> > > >> (DRILL-4323).
>> > >> >> > > >> > > >> >> > > This
>> > >> >> > > >> > > >> >> > > > > is a
>> > >> >> > > >> > > >> >> > > > > > regression from 1.4 and in my opinion
>> > should
>> > >> be
>> > >> >> > > treated
>> > >> >> > > >> > as
>> > >> >> > > >> > > a
>> > >> >> > > >> > > >> >> > blocker.
>> > >> >> > > >> > > >> >> > > > > >
>> > >> >> > > >> > > >> >> > > > > > - Rahul
>> > >> >> > > >> > > >> >> > > > > >
>> > >> >> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki
>> > >> >> Korukanti <
>> > >> >> > > >> > > >> >> > > > > > venki.korukanti@gmail.com>
>> > >> >> > > >> > > >> >> > > > > > wrote:
>> > >> >> > > >> > > >> >> > > > > >
>> > >> >> > > >> > > >> >> > > > > > > Sorry I didn't realize I was breaking
>> > the
>> > >> >> public
>> > >> >> > > >> > > interface
>> > >> >> > > >> > > >> API.
>> > >> >> > > >> > > >> >> > Let
>> > >> >> > > >> > > >> >> > > > me
>> > >> >> > > >> > > >> >> > > > > > work
>> > >> >> > > >> > > >> >> > > > > > > on a patch to make it backward
>> > compatible.
>> > >> >> > > >> > > >> >> > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > Thanks
>> > >> >> > > >> > > >> >> > > > > > > Venki
>> > >> >> > > >> > > >> >> > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM,
>> Steven
>> > >> >> > Phillips
>> > >> >> > > <
>> > >> >> > > >> > > >> >> > > steven@dremio.com
>> > >> >> > > >> > > >> >> > > > >
>> > >> >> > > >> > > >> >> > > > > > > wrote:
>> > >> >> > > >> > > >> >> > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > I just wanted to bring up an issue
>> > that I
>> > >> >> just
>> > >> >> > > now
>> > >> >> > > >> > > >> >> discovered,
>> > >> >> > > >> > > >> >> > > that
>> > >> >> > > >> > > >> >> > > > > has
>> > >> >> > > >> > > >> >> > > > > > > > caused me a fair amount of grief.
>> > >> >> > > >> > > >> >> > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > >
>> > >> >> > > https://github.com/apache/drill/pull/300/commits
>> > >> >> > > >> > > >> >> > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing
>> API,
>> > and
>> > >> >> > causes
>> > >> >> > > >> > > >> >> StoragePlugins
>> > >> >> > > >> > > >> >> > > > that
>> > >> >> > > >> > > >> >> > > > > > were
>> > >> >> > > >> > > >> >> > > > > > > > compiled against currently released
>> > >> versions
>> > >> >> > of
>> > >> >> > > >> Drill
>> > >> >> > > >> > > to
>> > >> >> > > >> > > >> no
>> > >> >> > > >> > > >> >> > > longer
>> > >> >> > > >> > > >> >> > > > > > > > functional properly. I would prefer
>> > that
>> > >> >> this
>> > >> >> > > >> > breaking
>> > >> >> > > >> > > >> change
>> > >> >> > > >> > > >> >> > be
>> > >> >> > > >> > > >> >> > > > > > modified
>> > >> >> > > >> > > >> >> > > > > > > > to be backward compatible if
>> possible.
>> > >> >> > > >> > > >> >> > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM,
>> > Jason
>> > >> >> > > Altekruse <
>> > >> >> > > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
>> > >> >> > > >> > > >> >> > > > > > > > wrote:
>> > >> >> > > >> > > >> >> > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > Hi Aman,
>> > >> >> > > >> > > >> >> > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > This is the failure that he was
>> > >> seeing. He
>> > >> >> > > >> figured
>> > >> >> > > >> > > out
>> > >> >> > > >> > > >> that
>> > >> >> > > >> > > >> >> > the
>> > >> >> > > >> > > >> >> > > > new
>> > >> >> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were not
>> > being
>> > >> >> > > respected
>> > >> >> > > >> > when
>> > >> >> > > >> > > the
>> > >> >> > > >> > > >> >> > build
>> > >> >> > > >> > > >> >> > > > was
>> > >> >> > > >> > > >> >> > > > > > run
>> > >> >> > > >> > > >> >> > > > > > > > with
>> > >> >> > > >> > > >> >> > > > > > > > > an older Maven version, causing
>> the
>> > jar
>> > >> >> size
>> > >> >> > > to
>> > >> >> > > >> > > increase
>> > >> >> > > >> > > >> >> > > > > > significantly.
>> > >> >> > > >> > > >> >> > > > > > > > He
>> > >> >> > > >> > > >> >> > > > > > > > > added an enforcer to make sure the
>> > JAR
>> > >> >> > didn't
>> > >> >> > > >> grow
>> > >> >> > > >> > > >> >> > > unexpectedly.
>> > >> >> > > >> > > >> >> > > > > Can
>> > >> >> > > >> > > >> >> > > > > > > you
>> > >> >> > > >> > > >> >> > > > > > > > > try to update your maven version
>> and
>> > >> >> re-run
>> > >> >> > > the
>> > >> >> > > >> > > build?
>> > >> >> > > >> > > >> >> > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > - Jason
>> > >> >> > > >> > > >> >> > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM,
>> > Aman
>> > >> >> > Sinha <
>> > >> >> > > >> > > >> >> > > > asinha@maprtech.com>
>> > >> >> > > >> > > >> >> > > > > > > > wrote:
>> > >> >> > > >> > > >> >> > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the
>> > following
>> > >> >> build
>> > >> >> > > >> failure
>> > >> >> > > >> > > on
>> > >> >> > > >> > > >> the
>> > >> >> > > >> > > >> >> > > latest
>> > >> >> > > >> > > >> >> > > > > > > master
>> > >> >> > > >> > > >> >> > > > > > > > > > branch...is this what you saw
>> for
>> > the
>> > >> >> > Apache
>> > >> >> > > >> > build
>> > >> >> > > >> > > ?
>> > >> >> > > >> > > >> My
>> > >> >> > > >> > > >> >> > mvn
>> > >> >> > > >> > > >> >> > > > > > version
>> > >> >> > > >> > > >> >> > > > > > > > > output
>> > >> >> > > >> > > >> >> > > > > > > > > > is shown below.  Should we all
>> be
>> > >> >> > upgrading
>> > >> >> > > to
>> > >> >> > > >> a
>> > >> >> > > >> > > newer
>> > >> >> > > >> > > >> >> mvn
>> > >> >> > > >> > > >> >> > ?
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > [INFO] ---
>> > >> >> > > maven-enforcer-plugin:1.3.1:enforce
>> > >> >> > > >> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @
>> > >> >> > > drill-jdbc-all
>> > >> >> > > >> > ---
>> > >> >> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
>> > >> >> > > >> > > >> >> > > > > >
>> > >> >> org.apache.maven.plugins.enforcer.RequireFilesSize
>> > >> >> > > >> > > >> >> > > > > > > > > failed
>> > >> >> > > >> > > >> >> > > > > > > > > > with message:
>> > >> >> > > >> > > >> >> > > > > > > > > > The file
>> > >> >> drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> > >> >> > > is
>> > >> >> > > >> > > outside
>> > >> >> > > >> > > >> the
>> > >> >> > > >> > > >> >> > > > > expected
>> > >> >> > > >> > > >> >> > > > > > > size
>> > >> >> > > >> > > >> >> > > > > > > > > > range.
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > >                   This is likely
>> > due
>> > >> to
>> > >> >> > you
>> > >> >> > > >> > adding
>> > >> >> > > >> > > new
>> > >> >> > > >> > > >> >> > > > > dependencies
>> > >> >> > > >> > > >> >> > > > > > > to
>> > >> >> > > >> > > >> >> > > > > > > > a
>> > >> >> > > >> > > >> >> > > > > > > > > > java-exec and not updating the
>> > >> excludes
>> > >> >> in
>> > >> >> > > this
>> > >> >> > > >> > > >> module.
>> > >> >> > > >> > > >> >> > This
>> > >> >> > > >> > > >> >> > > is
>> > >> >> > > >> > > >> >> > > > > > > > important
>> > >> >> > > >> > > >> >> > > > > > > > > > as it minimizes the size of the
>> > >> >> dependency
>> > >> >> > > of
>> > >> >> > > >> > Drill
>> > >> >> > > >> > > >> >> > > application
>> > >> >> > > >> > > >> >> > > > > > > users.
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > >
>> > >> >> > > >> > > >> >> > > > > >
>> > >> >> > > >> > > >> >> > > > >
>> > >> >> > > >> > > >> >> > > >
>> > >> >> > > >> > > >> >> > >
>> > >> >> > > >> > > >> >> >
>> > >> >> > > >> > > >> >>
>> > >> >> > > >> > > >>
>> > >> >> > > >> > >
>> > >> >> > > >> >
>> > >> >> > > >>
>> > >> >> > >
>> > >> >> >
>> > >> >>
>> > >>
>> >
>> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> > >> >> > > >> > > >> >> > > > > > > > > > size (44664121) too large. Max.
>> is
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > >
>> > >> >> > > >> > > >> >> > > > > >
>> > >> >> > > >> > > >> >> > > > >
>> > >> >> > > >> > > >> >> > > >
>> > >> >> > > >> > > >> >> > >
>> > >> >> > > >> > > >> >> >
>> > >> >> > > >> > > >> >>
>> > >> >> > > >> > > >>
>> > >> >> > > >> > >
>> > >> >> > > >> >
>> > >> >> > > >>
>> > >> >> > >
>> > >> >> >
>> > >> >>
>> > >>
>> >
>> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > Administrators-MacBook-Pro-144:incubator-drill
>> > >> >> > > >> > > asinha$
>> > >> >> > > >> > > >> >> mvn
>> > >> >> > > >> > > >> >> > > > > > --version
>> > >> >> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
>> > >> >> > > >> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
>> > >> >> > > >> > > >> >> > > > > > > > 2013-02-19
>> > >> >> > > >> > > >> >> > > > > > > > > > 05:51:28-0800)
>> > >> >> > > >> > > >> >> > > > > > > > > > Maven home:
>> > >> /opt/local/share/java/maven3
>> > >> >> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor:
>> > >> Oracle
>> > >> >> > > >> > Corporation
>> > >> >> > > >> > > >> >> > > > > > > > > > Java home:
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > >
>> > >> >> > > >> > > >>
>> > >> >> > >
>> > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
>> > >> >> > > >> > > >> >> > > > > > > > > > Default locale: en_US, platform
>> > >> >> encoding:
>> > >> >> > > UTF-8
>> > >> >> > > >> > > >> >> > > > > > > > > > OS name: "mac os x", version:
>> > >> "10.9.5",
>> > >> >> > > arch:
>> > >> >> > > >> > > >> "x86_64",
>> > >> >> > > >> > > >> >> > > family:
>> > >> >> > > >> > > >> >> > > > > > "mac"
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM,
>> > >> Jacques
>> > >> >> > > >> Nadeau <
>> > >> >> > > >> > > >> >> > > > > > jacques@dremio.com>
>> > >> >> > > >> > > >> >> > > > > > > > > > wrote:
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > Build back to normal. It looks
>> > like
>> > >> >> the
>> > >> >> > > >> Apache
>> > >> >> > > >> > > >> server
>> > >> >> > > >> > > >> >> was
>> > >> >> > > >> > > >> >> > > > using
>> > >> >> > > >> > > >> >> > > > > > an
>> > >> >> > > >> > > >> >> > > > > > > > old
>> > >> >> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I
>> > switched
>> > >> to
>> > >> >> > > >> something
>> > >> >> > > >> > > more
>> > >> >> > > >> > > >> >> > recent,
>> > >> >> > > >> > > >> >> > > > the
>> > >> >> > > >> > > >> >> > > > > > > build
>> > >> >> > > >> > > >> >> > > > > > > > > > > passed.
>> > >> >> > > >> > > >> >> > > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > --
>> > >> >> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau
>> > >> >> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
>> > >> >> > > >> > > >> >> > > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02
>> AM,
>> > >> >> Jacques
>> > >> >> > > >> > Nadeau <
>> > >> >> > > >> > > >> >> > > > > > > jacques@dremio.com>
>> > >> >> > > >> > > >> >> > > > > > > > > > > wrote:
>> > >> >> > > >> > > >> >> > > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge caused the
>> > >> Apache
>> > >> >> > > build
>> > >> >> > > >> to
>> > >> >> > > >> > > fail.
>> > >> >> > > >> > > >> >> > > > > > > Investigating...
>> > >> >> > > >> > > >> >> > > > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > > --
>> > >> >> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
>> > >> >> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
>> > >> >> > > >> > > >> >> > > > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31
>> > AM,
>> > >> >> > Jacques
>> > >> >> > > >> > > Nadeau <
>> > >> >> > > >> > > >> >> > > > > > > > jacques@dremio.com>
>> > >> >> > > >> > > >> >> > > > > > > > > > > > wrote:
>> > >> >> > > >> > > >> >> > > > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >> I got clean regression runs
>> > as
>> > >> >> well.
>> > >> >> > > I've
>> > >> >> > > >> > > merged
>> > >> >> > > >> > > >> the
>> > >> >> > > >> > > >> >> > > > patch.
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to start
>> the
>> > >> >> release
>> > >> >> > > >> > process?
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >> --
>> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
>> > >> >> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at
>> 10:42
>> > >> PM,
>> > >> >> > > Abhishek
>> > >> >> > > >> > > >> Girish <
>> > >> >> > > >> > > >> >> > > > > > > > agirish@mapr.com
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >> wrote:
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> Had two clean Functional
>> > runs.
>> > >> >> TPC-H
>> > >> >> > > >> SF100
>> > >> >> > > >> > > was
>> > >> >> > > >> > > >> also
>> > >> >> > > >> > > >> >> > > > > > successful.
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at
>> > 10:07
>> > >> PM,
>> > >> >> > > rahul
>> > >> >> > > >> > > >> >> challapalli <
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> challapallirahul@gmail.com>
>> > >> >> wrote:
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a functional
>> > run
>> > >> with
>> > >> >> > > your
>> > >> >> > > >> > > branch.
>> > >> >> > > >> > > >> >> Will
>> > >> >> > > >> > > >> >> > > let
>> > >> >> > > >> > > >> >> > > > > you
>> > >> >> > > >> > > >> >> > > > > > > know
>> > >> >> > > >> > > >> >> > > > > > > > > > once
>> > >> >> > > >> > > >> >> > > > > > > > > > > it
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > finishes
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at
>> > 9:56
>> > >> PM,
>> > >> >> > > >> Jacques
>> > >> >> > > >> > > >> Nadeau <
>> > >> >> > > >> > > >> >> > > > > > > > > > jacques@dremio.com>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > wrote:
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged
>> today. I
>> > >> have
>> > >> >> an
>> > >> >> > > >> > updated
>> > >> >> > > >> > > >> patch
>> > >> >> > > >> > > >> >> > for
>> > >> >> > > >> > > >> >> > > > 4291
>> > >> >> > > >> > > >> >> > > > > > > that
>> > >> >> > > >> > > >> >> > > > > > > > is
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> ready.
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it
>> seems
>> > >> that
>> > >> >> > > >> something
>> > >> >> > > >> > > isn't
>> > >> >> > > >> > > >> >> > > working
>> > >> >> > > >> > > >> >> > > > > with
>> > >> >> > > >> > > >> >> > > > > > > our
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> extended
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't
>> been
>> > >> able
>> > >> >> to
>> > >> >> > > run
>> > >> >> > > >> an
>> > >> >> > > >> > > >> extended
>> > >> >> > > >> > > >> >> > > > > > regression.
>> > >> >> > > >> > > >> >> > > > > > > > > Unit
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> tests
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else
>> > >> possibly
>> > >> >> > > able
>> > >> >> > > >> to
>> > >> >> > > >> > > run a
>> > >> >> > > >> > > >> >> > > > regression
>> > >> >> > > >> > > >> >> > > > > > > suite
>> > >> >> > > >> > > >> >> > > > > > > > > > > against
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > this
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can
>> > >> confirm
>> > >> >> > > things
>> > >> >> > > >> > look
>> > >> >> > > >> > > >> good
>> > >> >> > > >> > > >> >> and
>> > >> >> > > >> > > >> >> > > > start
>> > >> >> > > >> > > >> >> > > > > > the
>> > >> >> > > >> > > >> >> > > > > > > > > > release
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > process?
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > thanks,
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>> > >> >> > > >> > > >> >> > > > >
>> > >> >> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > --
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder,
>> > Dremio
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016
>> at
>> > >> 11:20
>> > >> >> > AM,
>> > >> >> > > >> > Jacques
>> > >> >> > > >> > > >> >> Nadeau
>> > >> >> > > >> > > >> >> > <
>> > >> >> > > >> > > >> >> > > > > > > > > > > jacques@dremio.com
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the main
>> > things
>> > >> are
>> > >> >> > 4196
>> > >> >> > > >> and
>> > >> >> > > >> > > 4291
>> > >> >> > > >> > > >> >> > should
>> > >> >> > > >> > > >> >> > > be
>> > >> >> > > >> > > >> >> > > > > > > > > completed.
>> > >> >> > > >> > > >> >> > > > > > > > > > I
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> know
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Amit
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to
>> reproduce
>> > >> 4196
>> > >> >> > > locally
>> > >> >> > > >> > this
>> > >> >> > > >> > > >> >> weekend
>> > >> >> > > >> > > >> >> > > so
>> > >> >> > > >> > > >> >> > > > I
>> > >> >> > > >> > > >> >> > > > > > > think
>> > >> >> > > >> > > >> >> > > > > > > > > > we're
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> close
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > on
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > that.
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > --
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder,
>> > >> Dremio
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016
>> > at
>> > >> >> 10:14
>> > >> >> > > AM,
>> > >> >> > > >> > > Zelaine
>> > >> >> > > >> > > >> >> Fong
>> > >> >> > > >> > > >> >> > <
>> > >> >> > > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on
>> this?
>> > >> >> What's
>> > >> >> > > >> > blocking
>> > >> >> > > >> > > us
>> > >> >> > > >> > > >> >> from
>> > >> >> > > >> > > >> >> > > > taking
>> > >> >> > > >> > > >> >> > > > > > > this
>> > >> >> > > >> > > >> >> > > > > > > > > to a
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> vote --
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > the
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join
>> > issues?
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19,
>> 2016
>> > at
>> > >> >> 4:18
>> > >> >> > > PM,
>> > >> >> > > >> > > Jacques
>> > >> >> > > >> > > >> >> > Nadeau
>> > >> >> > > >> > > >> >> > > <
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this
>> > thread...
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the
>> issues
>> > >> that
>> > >> >> > were
>> > >> >> > > >> > > >> mentioned in
>> > >> >> > > >> > > >> >> > > this
>> > >> >> > > >> > > >> >> > > > > > thread
>> > >> >> > > >> > > >> >> > > > > > > > > along
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> with a
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed
>> > >> categorization:
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4190
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4196
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge
>> > Jacques
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > >> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4256
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
>> Jacques
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4278
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge
>> > Laurent
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > >> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4247
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4203
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> https://issues.apache.org/jira/browse/DRILL-4266
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge
>> > Jacques
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > >> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do others
>> > think?
>> > >> >> Let's
>> > >> >> > > try
>> > >> >> > > >> to
>> > >> >> > > >> > > get
>> > >> >> > > >> > > >> the
>> > >> >> > > >> > > >> >> > > > > blockers
>> > >> >> > > >> > > >> >> > > > > > > > > wrapped
>> > >> >> > > >> > > >> >> > > > > > > > > > up
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> in
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > the
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> next
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two and
>> > start a
>> > >> >> > > release
>> > >> >> > > >> > > vote...
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > --
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and
>> Co-Founder,
>> > >> >> Dremio
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4,
>> > 2016 at
>> > >> >> 1:48
>> > >> >> > > PM,
>> > >> >> > > >> > > Jason
>> > >> >> > > >> > > >> >> > > Altekruse
>> > >> >> > > >> > > >> >> > > > <
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> > >> altekrusejason@gmail.com>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the
>> > allocator
>> > >> >> > changes
>> > >> >> > > >> > merged
>> > >> >> > > >> > > and
>> > >> >> > > >> > > >> >> > about
>> > >> >> > > >> > > >> >> > > a
>> > >> >> > > >> > > >> >> > > > > > month
>> > >> >> > > >> > > >> >> > > > > > > > > since
>> > >> >> > > >> > > >> >> > > > > > > > > > > the
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> last
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it
>> would
>> > be
>> > >> >> good
>> > >> >> > to
>> > >> >> > > >> > start
>> > >> >> > > >> > > a
>> > >> >> > > >> > > >> vote
>> > >> >> > > >> > > >> >> > > > soon. I
>> > >> >> > > >> > > >> >> > > > > > > would
>> > >> >> > > >> > > >> >> > > > > > > > > > like
>> > >> >> > > >> > > >> >> > > > > > > > > > > to
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release
>> > manager.
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that
>> there
>> > >> were
>> > >> >> > some
>> > >> >> > > >> > issues
>> > >> >> > > >> > > >> that
>> > >> >> > > >> > > >> >> > were
>> > >> >> > > >> > > >> >> > > > > > > > identified
>> > >> >> > > >> > > >> >> > > > > > > > > > > after
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> the
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was
>> > merged. I
>> > >> >> think
>> > >> >> > > that
>> > >> >> > > >> > > these
>> > >> >> > > >> > > >> >> > issues
>> > >> >> > > >> > > >> >> > > > > should
>> > >> >> > > >> > > >> >> > > > > > > be
>> > >> >> > > >> > > >> >> > > > > > > > > > fixed
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> before
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > we
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release
>> > candidate.
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at
>> > the
>> > >> >> > > associated
>> > >> >> > > >> > > JIRAs
>> > >> >> > > >> > > >> it
>> > >> >> > > >> > > >> >> > > looked
>> > >> >> > > >> > > >> >> > > > > > like
>> > >> >> > > >> > > >> >> > > > > > > > > there
>> > >> >> > > >> > > >> >> > > > > > > > > > > was
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> a
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix
>> > just
>> > >> >> > > adjusting
>> > >> >> > > >> > the
>> > >> >> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > option,
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> and
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more
>> > involved
>> > >> work
>> > >> >> > to
>> > >> >> > > >> > change
>> > >> >> > > >> > > >> how we
>> > >> >> > > >> > > >> >> > > > > determine
>> > >> >> > > >> > > >> >> > > > > > > the
>> > >> >> > > >> > > >> >> > > > > > > > > > > correct
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > time
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > to
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill during
>> > >> external
>> > >> >> > > sort. I
>> > >> >> > > >> > > >> believe it
>> > >> >> > > >> > > >> >> > > makes
>> > >> >> > > >> > > >> >> > > > > > sense
>> > >> >> > > >> > > >> >> > > > > > > > to
>> > >> >> > > >> > > >> >> > > > > > > > > > make
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > external
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well with
>> > the
>> > >> >> newly
>> > >> >> > > >> > improved
>> > >> >> > > >> > > >> memory
>> > >> >> > > >> > > >> >> > > > > > accounting
>> > >> >> > > >> > > >> >> > > > > > > > > before
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > cutting a
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release, but
>> I'm
>> > not
>> > >> >> sure
>> > >> >> > > how
>> > >> >> > > >> > much
>> > >> >> > > >> > > >> work
>> > >> >> > > >> > > >> >> is
>> > >> >> > > >> > > >> >> > > > left
>> > >> >> > > >> > > >> >> > > > > to
>> > >> >> > > >> > > >> >> > > > > > > be
>> > >> >> > > >> > > >> >> > > > > > > > > done
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> there.
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond
>> > with
>> > >> >> your
>> > >> >> > > >> > thoughts
>> > >> >> > > >> > > on
>> > >> >> > > >> > > >> a
>> > >> >> > > >> > > >> >> > > release
>> > >> >> > > >> > > >> >> > > > > > soon
>> > >> >> > > >> > > >> >> > > > > > > > and
>> > >> >> > > >> > > >> >> > > > > > > > > > any
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> JIRAs
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > you
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to include
>> > in
>> > >> the
>> > >> >> > > >> release.
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
>> > >> >> > > >> > > >> >> > > > > > > >
>> > >> >> > > https://issues.apache.org/jira/browse/DRILL-4243
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >>
>> > >> >> > > >> > > >> >> > > > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > > >
>> > >> >> > > >> > > >> >> > > > > > >
>> > >> >> > > >> > > >> >> > > > > >
>> > >> >> > > >> > > >> >> > > > >
>> > >> >> > > >> > > >> >> > > >
>> > >> >> > > >> > > >> >> > >
>> > >> >> > > >> > > >> >> >
>> > >> >> > > >> > > >> >>
>> > >> >> > > >> > > >>
>> > >> >> > > >> > >
>> > >> >> > > >> >
>> > >> >> > > >>
>> > >> >> > >
>> > >> >> >
>> > >> >>
>> > >>
>> > >>
>> >
>> >
>>

Re: Time for a 1.5 release?

Posted by Jason Altekruse <al...@gmail.com>.

I don't think there was any issue with the merge either Jinfeng, I don't
think we need to close master. I had to step out for a few hours, but I am
working on finishing up the candidate right now.

Having a weird RAT issue running the actual maven release command, it's
complaining about a dependency-reduced-pom.xml in the target directory of
jdbc-all lacking a license header (although target directories should be
ignored). Checking out the commit before the recent changes to the jdbc-all
exclusions seems to get rid of the issue, but the changes in the pom file
don't seem like they should be causing any problems.

Still trying to debug and hoping to have a vote up tonight.

On Sun, Jan 31, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com> wrote:

> Given how easy it is to create a branch off a random commit, I generally
> don't see a point in closing the master branch. I figure the release
> manager can propose where they'd like to fork if this type of thing comes
> up.
>
> Thanks for the feedback Jinfeng. Of course you're right about Venki's
> commit. My bad.
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Sun, Jan 31, 2016 at 9:21 PM, Jinfeng Ni <ji...@gmail.com> wrote:
>
> > Venki's backwards compatibility fix (commit id:
> >    03197d0) is before the partition pruning commit. So, it seems to be
> > fine to create a release candidate from 03197d0; no need to
> > cherry-pick.
> >
> > As for whether the partition pruning changes are risky, the pre-commit
> > run did not show any serious problems in the first run; all the plan
> > changes are expected, or expose some existing problem (drill-4279).
> > However, I agree that it impacts the main code path of query planning,
> > and I'm fine that we exclude the partition change from 1.5.0 release,
> > since it's merged right before we start a voting.
> >
> > I did not see any one mentioned that the master branch is closed, and
> > therefore assume it's still open for commits.  I probably should have
> > asked whether the master was open for new patches before I merged the
> > commits.
> >
> >
> >
> > On Sun, Jan 31, 2016 at 8:43 PM, Jacques Nadeau <ja...@dremio.com>
> > wrote:
> > > Jinfeng, do you agree that the changes are risky? I was going mostly by
> > > commit messages.
> > >
> > > Also just realized that the backwards compatibility fix to the storage
> > > plugin interface is after these. Could create release branch from
> > 03197d...
> > > and then cherry pick the Venki change. Or could keep them all. Or
> > > go without the compatibility fix....
> > >
> > > --
> > > Jacques Nadeau
> > > CTO and Co-Founder, Dremio
> > >
> > > On Sun, Jan 31, 2016 at 7:50 PM, Jinfeng Ni <ji...@gmail.com>
> > wrote:
> > >
> > >> Hi Jason,
> > >>
> > >> I'm fine that release 1.5.0 starts from 03197d0f2.  One thing I need
> > >> point out is that the pre-commit test suite has some testcase's plan
> > >> output modified because of the partition pruning change.  If we run
> > >> pre-commit against the candidate release for 1.5.0, we have to use a
> > >> prior version of pre-commit suite.
> > >>
> > >>
> > >>
> > >> On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse
> > >> <al...@gmail.com> wrote:
> > >> > That sounds reasonable to me. I'll start preparing a release
> > candidate.
> > >> >
> > >> > Jinfeng, are you okay waiting for the next release to include this
> > >> change?
> > >> >
> > >> > On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <jacques@dremio.com
> >
> > >> wrote:
> > >> >
> > >> >> Hey Jason,
> > >> >>
> > >> >> Can you start the release vote? We've let this drag on too long.
> > >> >>
> > >> >> I suggest we start from 03197d0f2. It seems like the partition
> > pruning
> > >> >> changes are pretty complicated and on a pretty main path to add
> right
> > >> >> before the vote. Since Hsuan's fix (pr 349) hasn't been reviewed, I
> > >> think
> > >> >> we should start the vote without it.
> > >> >>
> > >> >> Jacques
> > >> >>
> > >> >>
> > >> >> --
> > >> >> Jacques Nadeau
> > >> >> CTO and Co-Founder, Dremio
> > >> >>
> > >> >> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <hyichu@maprtech.com
> >
> > >> wrote:
> > >> >>
> > >> >> > Yes, just send a pull request.
> > >> >> > https://github.com/apache/drill/pull/349
> > >> >> >
> > >> >> > Can Jinfeng review it?
> > >> >> >
> > >> >> > Thanks,
> > >> >> >
> > >> >> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <
> jinfengni99@gmail.com
> > >
> > >> >> wrote:
> > >> >> >
> > >> >> > > Sean probably is quite close to find a fix for this issue.
> Let's
> > see
> > >> >> > > whether he can post a patch soon.
> > >> >> > >
> > >> >> > >
> > >> >> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
> > >> >> > > <ch...@gmail.com> wrote:
> > >> >> > > > DRILL-4323 only happens when assertions are turned on (my
> > install
> > >> >> > script
> > >> >> > > > automatically enables assertions).
> > >> >> > > >
> > >> >> > > > - Rahul
> > >> >> > > >
> > >> >> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <
> > >> zfong@maprtech.com>
> > >> >> > > wrote:
> > >> >> > > >
> > >> >> > > >> Rahul,
> > >> >> > > >>
> > >> >> > > >> Let us know if you can reproduce the issue with assertions
> > turned
> > >> >> OFF.
> > >> >> > > If
> > >> >> > > >> you cannot, then I think this should not be a blocker for
> 1.5,
> > >> and
> > >> >> > Hsuan
> > >> >> > > >> will continue to look into fixing this.
> > >> >> > > >>
> > >> >> > > >> -- Zelaine
> > >> >> > > >>
> > >> >> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
> > >> >> > knguyen@maprtech.com>
> > >> >> > > >> wrote:
> > >> >> > > >>
> > >> >> > > >> > I do see the same error with assertion turned on.
> > >> >> > > >> >
> > >> >> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <
> > >> >> jinfengni99@gmail.com>
> > >> >> > > >> wrote:
> > >> >> > > >> >
> > >> >> > > >> > > Krystal, by any chance, did you turn on assertion when
> you
> > >> start
> > >> >> > > >> > drillbit?
> > >> >> > > >> > >
> > >> >> > > >> > > I think the error in drill-4323 will be raised when
> > >> assertion =
> > >> >> > on.
> > >> >> > > >> > >
> > >> >> > > >> > >
> > >> >> > > >> > >
> > >> >> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
> > >> >> > > knguyen@maprtech.com>
> > >> >> > > >> > > wrote:
> > >> >> > > >> > > > I am running drill-1.4 and is able to run count(*)
> > >> >> successfully.
> > >> >> > > >> > > >
> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter
> > >> session
> > >> >> > set
> > >> >> > > >> > > > `store.hive.optimize_scan_with_native_readers` = true;
> > >> >> > > >> > > >
> > >> >> >
> +-------+--------------------------------------------------------+
> > >> >> > > >> > > > |  ok   |                        summary
> > >> >> >  |
> > >> >> > > >> > > >
> > >> >> >
> +-------+--------------------------------------------------------+
> > >> >> > > >> > > > | true  | store.hive.optimize_scan_with_native_readers
> > >> >> > updated.  |
> > >> >> > > >> > > >
> > >> >> >
> +-------+--------------------------------------------------------+
> > >> >> > > >> > > > 1 row selected (0.374 seconds)
> > >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select
> > >> >> count(*)
> > >> >> > > from
> > >> >> > > >> > > > lineitem_parquet_partitioned_hive_string;
> > >> >> > > >> > > > +---------+
> > >> >> > > >> > > > | EXPR$0  |
> > >> >> > > >> > > > +---------+
> > >> >> > > >> > > > | 60175   |
> > >> >> > > >> > > > +---------+
> > >> >> > > >> > > > 1 row selected (2.986 seconds)
> > >> >> > > >> > > >
> > >> >> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
> > >> >> > > jinfengni99@gmail.com>
> > >> >> > > >> > > wrote:
> > >> >> > > >> > > >
> > >> >> > > >> > > >> Venki and I did some investigation for DRILL-4323.
> The
> > >> issue
> > >> >> > > >> reported
> > >> >> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as
> well.
> > >> Seems
> > >> >> > to
> > >> >> > > us
> > >> >> > > >> > > >> this is not a regression from 1.4.0; it's a
> regression
> > >> from
> > >> >> > 1.3.0
> > >> >> > > >> > > >> probably.
> > >> >> > > >> > > >>
> > >> >> > > >> > > >> DRILL-4083 makes the planner to use
> > DrillHiveNativeReader
> > >> in
> > >> >> > > stead
> > >> >> > > >> of
> > >> >> > > >> > > >> HiveReader for "select count(*) from hive_table"
> query.
> > >> >> > However,
> > >> >> > > >> the
> > >> >> > > >> > > >> Project after the scan produces empty schema.  Before
> > >> >> > DRILL-4083,
> > >> >> > > >> > > >> Drill uses HiveScan, which works fine.
> > >> >> > > >> > > >>
> > >> >> > > >> > > >>
> > >> >> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
> > >> >> > > >> > > >> <al...@gmail.com> wrote:
> > >> >> > > >> > > >> > We could revert the change to the StoragePlugin
> > >> interface
> > >> >> to
> > >> >> > > add
> > >> >> > > >> the
> > >> >> > > >> > > >> > getLogicalOptimizerRules
> > >> >> > > >> > > >> > and getPhysicalOptimizerRules, and only put those
> in
> > >> >> > > >> > > >> AbstractStoragePlugin
> > >> >> > > >> > > >> > with default implementations.
> > >> >> > > >> > > >> >
> > >> >> > > >> > > >> > We could then update the docs to tell users to not
> > >> >> implement
> > >> >> > > the
> > >> >> > > >> > > >> interface
> > >> >> > > >> > > >> > directly, and instead extend the abstract class,
> > which I
> > >> >> > > believe
> > >> >> > > >> > would
> > >> >> > > >> > > >> > allow us to make changes like this in the future
> > without
> > >> >> > > breaking
> > >> >> > > >> > > >> plugins.
> > >> >> > > >> > > >> > At a major release version we could then just pull
> up
> > >> all
> > >> >> of
> > >> >> > > the
> > >> >> > > >> new
> > >> >> > > >> > > >> > methods we added to the abstract class into the
> > >> interface,
> > >> >> or
> > >> >> > > just
> > >> >> > > >> > get
> > >> >> > > >> > > >> rid
> > >> >> > > >> > > >> > of it all-together.
> > >> >> > > >> > > >> >
> > >> >> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
> > >> >> > > >> > > >> venki.korukanti@gmail.com>
> > >> >> > > >> > > >> > wrote:
> > >> >> > > >> > > >> >
> > >> >> > > >> > > >> >> Here is the partial fix:
> > >> >> > > >> > > >> >>
> > >> >> > > >> > > >> >>
> > >> >> > > >> > > >>
> > >> >> > > >> > >
> > >> >> > > >> >
> > >> >> > > >>
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> > >> >> > > >> > > >> >>
> > >> >> > > >> > > >> >> If the existing StragePlugins are implementing
> > >> >> > > >> > AbstractStoragePlugin
> > >> >> > > >> > > >> class
> > >> >> > > >> > > >> >> this fix should resolve the backward compat issue.
> > If
> > >> they
> > >> >> > are
> > >> >> > > >> > > >> implementing
> > >> >> > > >> > > >> >> StoragePlugin interface directly then it won't
> > resolve
> > >> the
> > >> >> > > issue.
> > >> >> > > >> > Not
> > >> >> > > >> > > >> sure
> > >> >> > > >> > > >> >> how to handle it in that case. Let me know if
> there
> > any
> > >> >> ways
> > >> >> > > to
> > >> >> > > >> > > resolve
> > >> >> > > >> > > >> >> this issue completely.
> > >> >> > > >> > > >> >>
> > >> >> > > >> > > >> >> Thanks
> > >> >> > > >> > > >> >> Venki
> > >> >> > > >> > > >> >>
> > >> >> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <
> > >> >> > > asinha@maprtech.com
> > >> >> > > >> >
> > >> >> > > >> > > >> wrote:
> > >> >> > > >> > > >> >>
> > >> >> > > >> > > >> >> > I was able to do the build with maven 3.3.9
> (after
> > >> doing
> > >> >> > > 'brew
> > >> >> > > >> > > install
> > >> >> > > >> > > >> >> > maven').  Here are my env variables related to
> > maven:
> > >> >> > > >> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> > >> >> > > >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> > >> >> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> > >> >> > > >> > > >> >> >
> > >> >> > > >> > > >> >> >
> > >> >> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau
> <
> > >> >> > > >> > > jacques@dremio.com>
> > >> >> > > >> > > >> >> > wrote:
> > >> >> > > >> > > >> >> >
> > >> >> > > >> > > >> >> > > Aman, for reference on the Maven version
> issue.
> > You
> > >> >> can
> > >> >> > > see
> > >> >> > > >> the
> > >> >> > > >> > > >> output
> > >> >> > > >> > > >> >> of
> > >> >> > > >> > > >> >> > > the shade plugin between the two Apache
> Builds.
> > If
> > >> you
> > >> >> > > search
> > >> >> > > >> > for
> > >> >> > > >> > > >> >> > > maven-shade-plugin you'll see that build 647
> > (maven
> > >> >> > 3.0.5)
> > >> >> > > >> > fails
> > >> >> > > >> > > >> >> because
> > >> >> > > >> > > >> >> > it
> > >> >> > > >> > > >> >> > > includes a bunch of hadoop (and other
> libraries
> > >> that
> > >> >> are
> > >> >> > > >> listed
> > >> >> > > >> > > as
> > >> >> > > >> > > >> >> being
> > >> >> > > >> > > >> >> > > excluded). If you look at build 648 (maven
> > 3.3.3),
> > >> >> those
> > >> >> > > >> items
> > >> >> > > >> > > are
> > >> >> > > >> > > >> >> > included
> > >> >> > > >> > > >> >> > > so that the build completes successfully.
> > >> >> > > >> > > >> >> > >
> > >> >> > > >> > > >> >> > >
> > >> >> https://builds.apache.org/job/drill-scm/647/consoleText
> > >> >> > > >> > > >> >> > >
> > >> >> https://builds.apache.org/job/drill-scm/648/consoleText
> > >> >> > > >> > > >> >> > >
> > >> >> > > >> > > >> >> > > --
> > >> >> > > >> > > >> >> > > Jacques Nadeau
> > >> >> > > >> > > >> >> > > CTO and Co-Founder, Dremio
> > >> >> > > >> > > >> >> > >
> > >> >> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul
> > challapalli
> > >> <
> > >> >> > > >> > > >> >> > > challapallirahul@gmail.com> wrote:
> > >> >> > > >> > > >> >> > >
> > >> >> > > >> > > >> >> > > > Jason,
> > >> >> > > >> > > >> >> > > >
> > >> >> > > >> > > >> >> > > > As of now I haven't narrowed down the exact
> > >> commit
> > >> >> > which
> > >> >> > > >> > caused
> > >> >> > > >> > > >> >> > > DRILL-4323
> > >> >> > > >> > > >> >> > > > and I do not know the scope of the fix.
> > >> >> > > >> > > >> >> > > >
> > >> >> > > >> > > >> >> > > > - Rahul
> > >> >> > > >> > > >> >> > > >
> > >> >> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason
> > Altekruse
> > >> <
> > >> >> > > >> > > >> >> > > altekrusejason@gmail.com
> > >> >> > > >> > > >> >> > > > >
> > >> >> > > >> > > >> >> > > > wrote:
> > >> >> > > >> > > >> >> > > >
> > >> >> > > >> > > >> >> > > > > I think it makes sense to wait for Venki's
> > fix
> > >> of
> > >> >> > the
> > >> >> > > >> API,
> > >> >> > > >> > as
> > >> >> > > >> > > >> >> > > releasing a
> > >> >> > > >> > > >> >> > > > > version with the change would make
> > >> >> > > reverting/refactoring
> > >> >> > > >> > the
> > >> >> > > >> > > API
> > >> >> > > >> > > >> >> > > breaking
> > >> >> > > >> > > >> >> > > > > change less useful.
> > >> >> > > >> > > >> >> > > > >
> > >> >> > > >> > > >> >> > > > > Do you have any idea about the scope of
> > fixing
> > >> the
> > >> >> > > hive
> > >> >> > > >> > > native
> > >> >> > > >> > > >> >> > reader,
> > >> >> > > >> > > >> >> > > or
> > >> >> > > >> > > >> >> > > > > which change since the 1.4 release broke
> > it? Is
> > >> >> > > someone
> > >> >> > > >> > > >> available
> > >> >> > > >> > > >> >> to
> > >> >> > > >> > > >> >> > > look
> > >> >> > > >> > > >> >> > > > > at it today?
> > >> >> > > >> > > >> >> > > > >
> > >> >> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul
> > >> >> challapalli <
> > >> >> > > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
> > >> >> > > >> > > >> >> > > > >
> > >> >> > > >> > > >> >> > > > > > Just found an issue with hive native
> > parquet
> > >> >> > reader
> > >> >> > > >> > > >> (DRILL-4323).
> > >> >> > > >> > > >> >> > > This
> > >> >> > > >> > > >> >> > > > > is a
> > >> >> > > >> > > >> >> > > > > > regression from 1.4 and in my opinion
> > should
> > >> be
> > >> >> > > treated
> > >> >> > > >> > as
> > >> >> > > >> > > a
> > >> >> > > >> > > >> >> > blocker.
> > >> >> > > >> > > >> >> > > > > >
> > >> >> > > >> > > >> >> > > > > > - Rahul
> > >> >> > > >> > > >> >> > > > > >
> > >> >> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki
> > >> >> Korukanti <
> > >> >> > > >> > > >> >> > > > > > venki.korukanti@gmail.com>
> > >> >> > > >> > > >> >> > > > > > wrote:
> > >> >> > > >> > > >> >> > > > > >
> > >> >> > > >> > > >> >> > > > > > > Sorry I didn't realize I was breaking
> > the
> > >> >> public
> > >> >> > > >> > > interface
> > >> >> > > >> > > >> API.
> > >> >> > > >> > > >> >> > Let
> > >> >> > > >> > > >> >> > > > me
> > >> >> > > >> > > >> >> > > > > > work
> > >> >> > > >> > > >> >> > > > > > > on a patch to make it backward
> > compatible.
> > >> >> > > >> > > >> >> > > > > > >
> > >> >> > > >> > > >> >> > > > > > > Thanks
> > >> >> > > >> > > >> >> > > > > > > Venki
> > >> >> > > >> > > >> >> > > > > > >
> > >> >> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM,
> Steven
> > >> >> > Phillips
> > >> >> > > <
> > >> >> > > >> > > >> >> > > steven@dremio.com
> > >> >> > > >> > > >> >> > > > >
> > >> >> > > >> > > >> >> > > > > > > wrote:
> > >> >> > > >> > > >> >> > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > I just wanted to bring up an issue
> > that I
> > >> >> just
> > >> >> > > now
> > >> >> > > >> > > >> >> discovered,
> > >> >> > > >> > > >> >> > > that
> > >> >> > > >> > > >> >> > > > > has
> > >> >> > > >> > > >> >> > > > > > > > caused me a fair amount of grief.
> > >> >> > > >> > > >> >> > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > >
> > >> >> > > https://github.com/apache/drill/pull/300/commits
> > >> >> > > >> > > >> >> > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing
> API,
> > and
> > >> >> > causes
> > >> >> > > >> > > >> >> StoragePlugins
> > >> >> > > >> > > >> >> > > > that
> > >> >> > > >> > > >> >> > > > > > were
> > >> >> > > >> > > >> >> > > > > > > > compiled against currently released
> > >> versions
> > >> >> > of
> > >> >> > > >> Drill
> > >> >> > > >> > > to
> > >> >> > > >> > > >> no
> > >> >> > > >> > > >> >> > > longer
> > >> >> > > >> > > >> >> > > > > > > > functional properly. I would prefer
> > that
> > >> >> this
> > >> >> > > >> > breaking
> > >> >> > > >> > > >> change
> > >> >> > > >> > > >> >> > be
> > >> >> > > >> > > >> >> > > > > > modified
> > >> >> > > >> > > >> >> > > > > > > > to be backward compatible if
> possible.
> > >> >> > > >> > > >> >> > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM,
> > Jason
> > >> >> > > Altekruse <
> > >> >> > > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
> > >> >> > > >> > > >> >> > > > > > > > wrote:
> > >> >> > > >> > > >> >> > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > Hi Aman,
> > >> >> > > >> > > >> >> > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > This is the failure that he was
> > >> seeing. He
> > >> >> > > >> figured
> > >> >> > > >> > > out
> > >> >> > > >> > > >> that
> > >> >> > > >> > > >> >> > the
> > >> >> > > >> > > >> >> > > > new
> > >> >> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were not
> > being
> > >> >> > > respected
> > >> >> > > >> > when
> > >> >> > > >> > > the
> > >> >> > > >> > > >> >> > build
> > >> >> > > >> > > >> >> > > > was
> > >> >> > > >> > > >> >> > > > > > run
> > >> >> > > >> > > >> >> > > > > > > > with
> > >> >> > > >> > > >> >> > > > > > > > > an older Maven version, causing
> the
> > jar
> > >> >> size
> > >> >> > > to
> > >> >> > > >> > > increase
> > >> >> > > >> > > >> >> > > > > > significantly.
> > >> >> > > >> > > >> >> > > > > > > > He
> > >> >> > > >> > > >> >> > > > > > > > > added an enforcer to make sure the
> > JAR
> > >> >> > didn't
> > >> >> > > >> grow
> > >> >> > > >> > > >> >> > > unexpectedly.
> > >> >> > > >> > > >> >> > > > > Can
> > >> >> > > >> > > >> >> > > > > > > you
> > >> >> > > >> > > >> >> > > > > > > > > try to update your maven version
> and
> > >> >> re-run
> > >> >> > > the
> > >> >> > > >> > > build?
> > >> >> > > >> > > >> >> > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > - Jason
> > >> >> > > >> > > >> >> > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM,
> > Aman
> > >> >> > Sinha <
> > >> >> > > >> > > >> >> > > > asinha@maprtech.com>
> > >> >> > > >> > > >> >> > > > > > > > wrote:
> > >> >> > > >> > > >> >> > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the
> > following
> > >> >> build
> > >> >> > > >> failure
> > >> >> > > >> > > on
> > >> >> > > >> > > >> the
> > >> >> > > >> > > >> >> > > latest
> > >> >> > > >> > > >> >> > > > > > > master
> > >> >> > > >> > > >> >> > > > > > > > > > branch...is this what you saw
> for
> > the
> > >> >> > Apache
> > >> >> > > >> > build
> > >> >> > > >> > > ?
> > >> >> > > >> > > >> My
> > >> >> > > >> > > >> >> > mvn
> > >> >> > > >> > > >> >> > > > > > version
> > >> >> > > >> > > >> >> > > > > > > > > output
> > >> >> > > >> > > >> >> > > > > > > > > > is shown below.  Should we all
> be
> > >> >> > upgrading
> > >> >> > > to
> > >> >> > > >> a
> > >> >> > > >> > > newer
> > >> >> > > >> > > >> >> mvn
> > >> >> > > >> > > >> >> > ?
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > > [INFO] ---
> > >> >> > > maven-enforcer-plugin:1.3.1:enforce
> > >> >> > > >> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @
> > >> >> > > drill-jdbc-all
> > >> >> > > >> > ---
> > >> >> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
> > >> >> > > >> > > >> >> > > > > >
> > >> >> org.apache.maven.plugins.enforcer.RequireFilesSize
> > >> >> > > >> > > >> >> > > > > > > > > failed
> > >> >> > > >> > > >> >> > > > > > > > > > with message:
> > >> >> > > >> > > >> >> > > > > > > > > > The file
> > >> >> drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > >> >> > > is
> > >> >> > > >> > > outside
> > >> >> > > >> > > >> the
> > >> >> > > >> > > >> >> > > > > expected
> > >> >> > > >> > > >> >> > > > > > > size
> > >> >> > > >> > > >> >> > > > > > > > > > range.
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > >                   This is likely
> > due
> > >> to
> > >> >> > you
> > >> >> > > >> > adding
> > >> >> > > >> > > new
> > >> >> > > >> > > >> >> > > > > dependencies
> > >> >> > > >> > > >> >> > > > > > > to
> > >> >> > > >> > > >> >> > > > > > > > a
> > >> >> > > >> > > >> >> > > > > > > > > > java-exec and not updating the
> > >> excludes
> > >> >> in
> > >> >> > > this
> > >> >> > > >> > > >> module.
> > >> >> > > >> > > >> >> > This
> > >> >> > > >> > > >> >> > > is
> > >> >> > > >> > > >> >> > > > > > > > important
> > >> >> > > >> > > >> >> > > > > > > > > > as it minimizes the size of the
> > >> >> dependency
> > >> >> > > of
> > >> >> > > >> > Drill
> > >> >> > > >> > > >> >> > > application
> > >> >> > > >> > > >> >> > > > > > > users.
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > >
> > >> >> > > >> > > >> >> > > > > > >
> > >> >> > > >> > > >> >> > > > > >
> > >> >> > > >> > > >> >> > > > >
> > >> >> > > >> > > >> >> > > >
> > >> >> > > >> > > >> >> > >
> > >> >> > > >> > > >> >> >
> > >> >> > > >> > > >> >>
> > >> >> > > >> > > >>
> > >> >> > > >> > >
> > >> >> > > >> >
> > >> >> > > >>
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > >> >> > > >> > > >> >> > > > > > > > > > size (44664121) too large. Max.
> is
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > >
> > >> >> > > >> > > >> >> > > > > > >
> > >> >> > > >> > > >> >> > > > > >
> > >> >> > > >> > > >> >> > > > >
> > >> >> > > >> > > >> >> > > >
> > >> >> > > >> > > >> >> > >
> > >> >> > > >> > > >> >> >
> > >> >> > > >> > > >> >>
> > >> >> > > >> > > >>
> > >> >> > > >> > >
> > >> >> > > >> >
> > >> >> > > >>
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > Administrators-MacBook-Pro-144:incubator-drill
> > >> >> > > >> > > asinha$
> > >> >> > > >> > > >> >> mvn
> > >> >> > > >> > > >> >> > > > > > --version
> > >> >> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
> > >> >> > > >> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > >> >> > > >> > > >> >> > > > > > > > 2013-02-19
> > >> >> > > >> > > >> >> > > > > > > > > > 05:51:28-0800)
> > >> >> > > >> > > >> >> > > > > > > > > > Maven home:
> > >> /opt/local/share/java/maven3
> > >> >> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor:
> > >> Oracle
> > >> >> > > >> > Corporation
> > >> >> > > >> > > >> >> > > > > > > > > > Java home:
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > >
> > >> >> > > >> > > >>
> > >> >> > >
> > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > >> >> > > >> > > >> >> > > > > > > > > > Default locale: en_US, platform
> > >> >> encoding:
> > >> >> > > UTF-8
> > >> >> > > >> > > >> >> > > > > > > > > > OS name: "mac os x", version:
> > >> "10.9.5",
> > >> >> > > arch:
> > >> >> > > >> > > >> "x86_64",
> > >> >> > > >> > > >> >> > > family:
> > >> >> > > >> > > >> >> > > > > > "mac"
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM,
> > >> Jacques
> > >> >> > > >> Nadeau <
> > >> >> > > >> > > >> >> > > > > > jacques@dremio.com>
> > >> >> > > >> > > >> >> > > > > > > > > > wrote:
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > Build back to normal. It looks
> > like
> > >> >> the
> > >> >> > > >> Apache
> > >> >> > > >> > > >> server
> > >> >> > > >> > > >> >> was
> > >> >> > > >> > > >> >> > > > using
> > >> >> > > >> > > >> >> > > > > > an
> > >> >> > > >> > > >> >> > > > > > > > old
> > >> >> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I
> > switched
> > >> to
> > >> >> > > >> something
> > >> >> > > >> > > more
> > >> >> > > >> > > >> >> > recent,
> > >> >> > > >> > > >> >> > > > the
> > >> >> > > >> > > >> >> > > > > > > build
> > >> >> > > >> > > >> >> > > > > > > > > > > passed.
> > >> >> > > >> > > >> >> > > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > --
> > >> >> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau
> > >> >> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
> > >> >> > > >> > > >> >> > > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02
> AM,
> > >> >> Jacques
> > >> >> > > >> > Nadeau <
> > >> >> > > >> > > >> >> > > > > > > jacques@dremio.com>
> > >> >> > > >> > > >> >> > > > > > > > > > > wrote:
> > >> >> > > >> > > >> >> > > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge caused the
> > >> Apache
> > >> >> > > build
> > >> >> > > >> to
> > >> >> > > >> > > fail.
> > >> >> > > >> > > >> >> > > > > > > Investigating...
> > >> >> > > >> > > >> >> > > > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > > --
> > >> >> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
> > >> >> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
> > >> >> > > >> > > >> >> > > > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31
> > AM,
> > >> >> > Jacques
> > >> >> > > >> > > Nadeau <
> > >> >> > > >> > > >> >> > > > > > > > jacques@dremio.com>
> > >> >> > > >> > > >> >> > > > > > > > > > > > wrote:
> > >> >> > > >> > > >> >> > > > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >> I got clean regression runs
> > as
> > >> >> well.
> > >> >> > > I've
> > >> >> > > >> > > merged
> > >> >> > > >> > > >> the
> > >> >> > > >> > > >> >> > > > patch.
> > >> >> > > >> > > >> >> > > > > > > > > > > >>
> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to start
> the
> > >> >> release
> > >> >> > > >> > process?
> > >> >> > > >> > > >> >> > > > > > > > > > > >>
> > >> >> > > >> > > >> >> > > > > > > > > > > >> --
> > >> >> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
> > >> >> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
> > >> >> > > >> > > >> >> > > > > > > > > > > >>
> > >> >> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at
> 10:42
> > >> PM,
> > >> >> > > Abhishek
> > >> >> > > >> > > >> Girish <
> > >> >> > > >> > > >> >> > > > > > > > agirish@mapr.com
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >> wrote:
> > >> >> > > >> > > >> >> > > > > > > > > > > >>
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> Had two clean Functional
> > runs.
> > >> >> TPC-H
> > >> >> > > >> SF100
> > >> >> > > >> > > was
> > >> >> > > >> > > >> also
> > >> >> > > >> > > >> >> > > > > > successful.
> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at
> > 10:07
> > >> PM,
> > >> >> > > rahul
> > >> >> > > >> > > >> >> challapalli <
> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> challapallirahul@gmail.com>
> > >> >> wrote:
> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a functional
> > run
> > >> with
> > >> >> > > your
> > >> >> > > >> > > branch.
> > >> >> > > >> > > >> >> Will
> > >> >> > > >> > > >> >> > > let
> > >> >> > > >> > > >> >> > > > > you
> > >> >> > > >> > > >> >> > > > > > > know
> > >> >> > > >> > > >> >> > > > > > > > > > once
> > >> >> > > >> > > >> >> > > > > > > > > > > it
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > finishes
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at
> > 9:56
> > >> PM,
> > >> >> > > >> Jacques
> > >> >> > > >> > > >> Nadeau <
> > >> >> > > >> > > >> >> > > > > > > > > > jacques@dremio.com>
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > wrote:
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged
> today. I
> > >> have
> > >> >> an
> > >> >> > > >> > updated
> > >> >> > > >> > > >> patch
> > >> >> > > >> > > >> >> > for
> > >> >> > > >> > > >> >> > > > 4291
> > >> >> > > >> > > >> >> > > > > > > that
> > >> >> > > >> > > >> >> > > > > > > > is
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> ready.
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it
> seems
> > >> that
> > >> >> > > >> something
> > >> >> > > >> > > isn't
> > >> >> > > >> > > >> >> > > working
> > >> >> > > >> > > >> >> > > > > with
> > >> >> > > >> > > >> >> > > > > > > our
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> extended
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't
> been
> > >> able
> > >> >> to
> > >> >> > > run
> > >> >> > > >> an
> > >> >> > > >> > > >> extended
> > >> >> > > >> > > >> >> > > > > > regression.
> > >> >> > > >> > > >> >> > > > > > > > > Unit
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> tests
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else
> > >> possibly
> > >> >> > > able
> > >> >> > > >> to
> > >> >> > > >> > > run a
> > >> >> > > >> > > >> >> > > > regression
> > >> >> > > >> > > >> >> > > > > > > suite
> > >> >> > > >> > > >> >> > > > > > > > > > > against
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > this
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can
> > >> confirm
> > >> >> > > things
> > >> >> > > >> > look
> > >> >> > > >> > > >> good
> > >> >> > > >> > > >> >> and
> > >> >> > > >> > > >> >> > > > start
> > >> >> > > >> > > >> >> > > > > > the
> > >> >> > > >> > > >> >> > > > > > > > > > release
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > process?
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > thanks,
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
> > >> >> > > >> > > >> >> > > > >
> > >> >> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > --
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder,
> > Dremio
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016
> at
> > >> 11:20
> > >> >> > AM,
> > >> >> > > >> > Jacques
> > >> >> > > >> > > >> >> Nadeau
> > >> >> > > >> > > >> >> > <
> > >> >> > > >> > > >> >> > > > > > > > > > > jacques@dremio.com
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the main
> > things
> > >> are
> > >> >> > 4196
> > >> >> > > >> and
> > >> >> > > >> > > 4291
> > >> >> > > >> > > >> >> > should
> > >> >> > > >> > > >> >> > > be
> > >> >> > > >> > > >> >> > > > > > > > > completed.
> > >> >> > > >> > > >> >> > > > > > > > > > I
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> know
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Amit
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to
> reproduce
> > >> 4196
> > >> >> > > locally
> > >> >> > > >> > this
> > >> >> > > >> > > >> >> weekend
> > >> >> > > >> > > >> >> > > so
> > >> >> > > >> > > >> >> > > > I
> > >> >> > > >> > > >> >> > > > > > > think
> > >> >> > > >> > > >> >> > > > > > > > > > we're
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> close
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > on
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > that.
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > --
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder,
> > >> Dremio
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016
> > at
> > >> >> 10:14
> > >> >> > > AM,
> > >> >> > > >> > > Zelaine
> > >> >> > > >> > > >> >> Fong
> > >> >> > > >> > > >> >> > <
> > >> >> > > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on
> this?
> > >> >> What's
> > >> >> > > >> > blocking
> > >> >> > > >> > > us
> > >> >> > > >> > > >> >> from
> > >> >> > > >> > > >> >> > > > taking
> > >> >> > > >> > > >> >> > > > > > > this
> > >> >> > > >> > > >> >> > > > > > > > > to a
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> vote --
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > the
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join
> > issues?
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19,
> 2016
> > at
> > >> >> 4:18
> > >> >> > > PM,
> > >> >> > > >> > > Jacques
> > >> >> > > >> > > >> >> > Nadeau
> > >> >> > > >> > > >> >> > > <
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this
> > thread...
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the
> issues
> > >> that
> > >> >> > were
> > >> >> > > >> > > >> mentioned in
> > >> >> > > >> > > >> >> > > this
> > >> >> > > >> > > >> >> > > > > > thread
> > >> >> > > >> > > >> >> > > > > > > > > along
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> with a
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed
> > >> categorization:
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4190
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4196
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge
> > Jacques
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> > >> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4256
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress
> Jacques
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4278
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge
> > Laurent
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> > >> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4247
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> > >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4203
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> https://issues.apache.org/jira/browse/DRILL-4266
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge
> > Jacques
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> > >> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do others
> > think?
> > >> >> Let's
> > >> >> > > try
> > >> >> > > >> to
> > >> >> > > >> > > get
> > >> >> > > >> > > >> the
> > >> >> > > >> > > >> >> > > > > blockers
> > >> >> > > >> > > >> >> > > > > > > > > wrapped
> > >> >> > > >> > > >> >> > > > > > > > > > up
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> in
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > the
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> next
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two and
> > start a
> > >> >> > > release
> > >> >> > > >> > > vote...
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > --
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and
> Co-Founder,
> > >> >> Dremio
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4,
> > 2016 at
> > >> >> 1:48
> > >> >> > > PM,
> > >> >> > > >> > > Jason
> > >> >> > > >> > > >> >> > > Altekruse
> > >> >> > > >> > > >> >> > > > <
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> > >> altekrusejason@gmail.com>
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the
> > allocator
> > >> >> > changes
> > >> >> > > >> > merged
> > >> >> > > >> > > and
> > >> >> > > >> > > >> >> > about
> > >> >> > > >> > > >> >> > > a
> > >> >> > > >> > > >> >> > > > > > month
> > >> >> > > >> > > >> >> > > > > > > > > since
> > >> >> > > >> > > >> >> > > > > > > > > > > the
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> last
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it
> would
> > be
> > >> >> good
> > >> >> > to
> > >> >> > > >> > start
> > >> >> > > >> > > a
> > >> >> > > >> > > >> vote
> > >> >> > > >> > > >> >> > > > soon. I
> > >> >> > > >> > > >> >> > > > > > > would
> > >> >> > > >> > > >> >> > > > > > > > > > like
> > >> >> > > >> > > >> >> > > > > > > > > > > to
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release
> > manager.
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that
> there
> > >> were
> > >> >> > some
> > >> >> > > >> > issues
> > >> >> > > >> > > >> that
> > >> >> > > >> > > >> >> > were
> > >> >> > > >> > > >> >> > > > > > > > identified
> > >> >> > > >> > > >> >> > > > > > > > > > > after
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> the
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was
> > merged. I
> > >> >> think
> > >> >> > > that
> > >> >> > > >> > > these
> > >> >> > > >> > > >> >> > issues
> > >> >> > > >> > > >> >> > > > > should
> > >> >> > > >> > > >> >> > > > > > > be
> > >> >> > > >> > > >> >> > > > > > > > > > fixed
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> before
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > we
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release
> > candidate.
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at
> > the
> > >> >> > > associated
> > >> >> > > >> > > JIRAs
> > >> >> > > >> > > >> it
> > >> >> > > >> > > >> >> > > looked
> > >> >> > > >> > > >> >> > > > > > like
> > >> >> > > >> > > >> >> > > > > > > > > there
> > >> >> > > >> > > >> >> > > > > > > > > > > was
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> a
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix
> > just
> > >> >> > > adjusting
> > >> >> > > >> > the
> > >> >> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > option,
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> and
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more
> > involved
> > >> work
> > >> >> > to
> > >> >> > > >> > change
> > >> >> > > >> > > >> how we
> > >> >> > > >> > > >> >> > > > > determine
> > >> >> > > >> > > >> >> > > > > > > the
> > >> >> > > >> > > >> >> > > > > > > > > > > correct
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > time
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > to
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill during
> > >> external
> > >> >> > > sort. I
> > >> >> > > >> > > >> believe it
> > >> >> > > >> > > >> >> > > makes
> > >> >> > > >> > > >> >> > > > > > sense
> > >> >> > > >> > > >> >> > > > > > > > to
> > >> >> > > >> > > >> >> > > > > > > > > > make
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > external
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well with
> > the
> > >> >> newly
> > >> >> > > >> > improved
> > >> >> > > >> > > >> memory
> > >> >> > > >> > > >> >> > > > > > accounting
> > >> >> > > >> > > >> >> > > > > > > > > before
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > cutting a
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release, but
> I'm
> > not
> > >> >> sure
> > >> >> > > how
> > >> >> > > >> > much
> > >> >> > > >> > > >> work
> > >> >> > > >> > > >> >> is
> > >> >> > > >> > > >> >> > > > left
> > >> >> > > >> > > >> >> > > > > to
> > >> >> > > >> > > >> >> > > > > > > be
> > >> >> > > >> > > >> >> > > > > > > > > done
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> there.
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond
> > with
> > >> >> your
> > >> >> > > >> > thoughts
> > >> >> > > >> > > on
> > >> >> > > >> > > >> a
> > >> >> > > >> > > >> >> > > release
> > >> >> > > >> > > >> >> > > > > > soon
> > >> >> > > >> > > >> >> > > > > > > > and
> > >> >> > > >> > > >> >> > > > > > > > > > any
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> JIRAs
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > you
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to include
> > in
> > >> the
> > >> >> > > >> release.
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
> > >> >> > > >> > > >> >> > > > > > > >
> > >> >> > > https://issues.apache.org/jira/browse/DRILL-4243
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> > >> >> > > >> > > >> >> > > > > > > > > > > >>>
> > >> >> > > >> > > >> >> > > > > > > > > > > >>
> > >> >> > > >> > > >> >> > > > > > > > > > > >>
> > >> >> > > >> > > >> >> > > > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > > >
> > >> >> > > >> > > >> >> > > > > > > >
> > >> >> > > >> > > >> >> > > > > > >
> > >> >> > > >> > > >> >> > > > > >
> > >> >> > > >> > > >> >> > > > >
> > >> >> > > >> > > >> >> > > >
> > >> >> > > >> > > >> >> > >
> > >> >> > > >> > > >> >> >
> > >> >> > > >> > > >> >>
> > >> >> > > >> > > >>
> > >> >> > > >> > >
> > >> >> > > >> >
> > >> >> > > >>
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> > >>
> >
> >
>

Re: Time for a 1.5 release?

Posted by Jacques Nadeau <ja...@dremio.com>.

Given how easy it is to create a branch off a random commit, I generally
don't see a point in closing the master branch. I figure the release
manager can propose where they'd like to fork if this type of thing comes
up.

Thanks for the feedback Jinfeng. Of course you're right about Venki's
commit. My bad.

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Sun, Jan 31, 2016 at 9:21 PM, Jinfeng Ni <ji...@gmail.com> wrote:

> Venki's backwards compatibility fix (commit id:
>    03197d0) is before the partition pruning commit. So, it seems to be
> fine to create a release candidate from 03197d0; no need to
> cherry-pick.
>
> As for whether the partition pruning changes are risky, the pre-commit
> run did not show any serious problems in the first run; all the plan
> changes are expected, or expose some existing problem (drill-4279).
> However, I agree that it impacts the main code path of query planning,
> and I'm fine that we exclude the partition change from 1.5.0 release,
> since it's merged right before we start a voting.
>
> I did not see any one mentioned that the master branch is closed, and
> therefore assume it's still open for commits.  I probably should have
> asked whether the master was open for new patches before I merged the
> commits.
>
>
>
> On Sun, Jan 31, 2016 at 8:43 PM, Jacques Nadeau <ja...@dremio.com>
> wrote:
> > Jinfeng, do you agree that the changes are risky? I was going mostly by
> > commit messages.
> >
> > Also just realized that the backwards compatibility fix to the storage
> > plugin interface is after these. Could create release branch from
> 03197d...
> > and then cherry pick the Venki change. Or could keep them all. Or
> > go without the compatibility fix....
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Sun, Jan 31, 2016 at 7:50 PM, Jinfeng Ni <ji...@gmail.com>
> wrote:
> >
> >> Hi Jason,
> >>
> >> I'm fine that release 1.5.0 starts from 03197d0f2.  One thing I need
> >> point out is that the pre-commit test suite has some testcase's plan
> >> output modified because of the partition pruning change.  If we run
> >> pre-commit against the candidate release for 1.5.0, we have to use a
> >> prior version of pre-commit suite.
> >>
> >>
> >>
> >> On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse
> >> <al...@gmail.com> wrote:
> >> > That sounds reasonable to me. I'll start preparing a release
> candidate.
> >> >
> >> > Jinfeng, are you okay waiting for the next release to include this
> >> change?
> >> >
> >> > On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <ja...@dremio.com>
> >> wrote:
> >> >
> >> >> Hey Jason,
> >> >>
> >> >> Can you start the release vote? We've let this drag on too long.
> >> >>
> >> >> I suggest we start from 03197d0f2. It seems like the partition
> pruning
> >> >> changes are pretty complicated and on a pretty main path to add right
> >> >> before the vote. Since Hsuan's fix (pr 349) hasn't been reviewed, I
> >> think
> >> >> we should start the vote without it.
> >> >>
> >> >> Jacques
> >> >>
> >> >>
> >> >> --
> >> >> Jacques Nadeau
> >> >> CTO and Co-Founder, Dremio
> >> >>
> >> >> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <hy...@maprtech.com>
> >> wrote:
> >> >>
> >> >> > Yes, just send a pull request.
> >> >> > https://github.com/apache/drill/pull/349
> >> >> >
> >> >> > Can Jinfeng review it?
> >> >> >
> >> >> > Thanks,
> >> >> >
> >> >> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <jinfengni99@gmail.com
> >
> >> >> wrote:
> >> >> >
> >> >> > > Sean probably is quite close to find a fix for this issue. Let's
> see
> >> >> > > whether he can post a patch soon.
> >> >> > >
> >> >> > >
> >> >> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
> >> >> > > <ch...@gmail.com> wrote:
> >> >> > > > DRILL-4323 only happens when assertions are turned on (my
> install
> >> >> > script
> >> >> > > > automatically enables assertions).
> >> >> > > >
> >> >> > > > - Rahul
> >> >> > > >
> >> >> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <
> >> zfong@maprtech.com>
> >> >> > > wrote:
> >> >> > > >
> >> >> > > >> Rahul,
> >> >> > > >>
> >> >> > > >> Let us know if you can reproduce the issue with assertions
> turned
> >> >> OFF.
> >> >> > > If
> >> >> > > >> you cannot, then I think this should not be a blocker for 1.5,
> >> and
> >> >> > Hsuan
> >> >> > > >> will continue to look into fixing this.
> >> >> > > >>
> >> >> > > >> -- Zelaine
> >> >> > > >>
> >> >> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
> >> >> > knguyen@maprtech.com>
> >> >> > > >> wrote:
> >> >> > > >>
> >> >> > > >> > I do see the same error with assertion turned on.
> >> >> > > >> >
> >> >> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <
> >> >> jinfengni99@gmail.com>
> >> >> > > >> wrote:
> >> >> > > >> >
> >> >> > > >> > > Krystal, by any chance, did you turn on assertion when you
> >> start
> >> >> > > >> > drillbit?
> >> >> > > >> > >
> >> >> > > >> > > I think the error in drill-4323 will be raised when
> >> assertion =
> >> >> > on.
> >> >> > > >> > >
> >> >> > > >> > >
> >> >> > > >> > >
> >> >> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
> >> >> > > knguyen@maprtech.com>
> >> >> > > >> > > wrote:
> >> >> > > >> > > > I am running drill-1.4 and is able to run count(*)
> >> >> successfully.
> >> >> > > >> > > >
> >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter
> >> session
> >> >> > set
> >> >> > > >> > > > `store.hive.optimize_scan_with_native_readers` = true;
> >> >> > > >> > > >
> >> >> > +-------+--------------------------------------------------------+
> >> >> > > >> > > > |  ok   |                        summary
> >> >> >  |
> >> >> > > >> > > >
> >> >> > +-------+--------------------------------------------------------+
> >> >> > > >> > > > | true  | store.hive.optimize_scan_with_native_readers
> >> >> > updated.  |
> >> >> > > >> > > >
> >> >> > +-------+--------------------------------------------------------+
> >> >> > > >> > > > 1 row selected (0.374 seconds)
> >> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select
> >> >> count(*)
> >> >> > > from
> >> >> > > >> > > > lineitem_parquet_partitioned_hive_string;
> >> >> > > >> > > > +---------+
> >> >> > > >> > > > | EXPR$0  |
> >> >> > > >> > > > +---------+
> >> >> > > >> > > > | 60175   |
> >> >> > > >> > > > +---------+
> >> >> > > >> > > > 1 row selected (2.986 seconds)
> >> >> > > >> > > >
> >> >> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
> >> >> > > jinfengni99@gmail.com>
> >> >> > > >> > > wrote:
> >> >> > > >> > > >
> >> >> > > >> > > >> Venki and I did some investigation for DRILL-4323. The
> >> issue
> >> >> > > >> reported
> >> >> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as well.
> >> Seems
> >> >> > to
> >> >> > > us
> >> >> > > >> > > >> this is not a regression from 1.4.0; it's a regression
> >> from
> >> >> > 1.3.0
> >> >> > > >> > > >> probably.
> >> >> > > >> > > >>
> >> >> > > >> > > >> DRILL-4083 makes the planner to use
> DrillHiveNativeReader
> >> in
> >> >> > > stead
> >> >> > > >> of
> >> >> > > >> > > >> HiveReader for "select count(*) from hive_table" query.
> >> >> > However,
> >> >> > > >> the
> >> >> > > >> > > >> Project after the scan produces empty schema.  Before
> >> >> > DRILL-4083,
> >> >> > > >> > > >> Drill uses HiveScan, which works fine.
> >> >> > > >> > > >>
> >> >> > > >> > > >>
> >> >> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
> >> >> > > >> > > >> <al...@gmail.com> wrote:
> >> >> > > >> > > >> > We could revert the change to the StoragePlugin
> >> interface
> >> >> to
> >> >> > > add
> >> >> > > >> the
> >> >> > > >> > > >> > getLogicalOptimizerRules
> >> >> > > >> > > >> > and getPhysicalOptimizerRules, and only put those in
> >> >> > > >> > > >> AbstractStoragePlugin
> >> >> > > >> > > >> > with default implementations.
> >> >> > > >> > > >> >
> >> >> > > >> > > >> > We could then update the docs to tell users to not
> >> >> implement
> >> >> > > the
> >> >> > > >> > > >> interface
> >> >> > > >> > > >> > directly, and instead extend the abstract class,
> which I
> >> >> > > believe
> >> >> > > >> > would
> >> >> > > >> > > >> > allow us to make changes like this in the future
> without
> >> >> > > breaking
> >> >> > > >> > > >> plugins.
> >> >> > > >> > > >> > At a major release version we could then just pull up
> >> all
> >> >> of
> >> >> > > the
> >> >> > > >> new
> >> >> > > >> > > >> > methods we added to the abstract class into the
> >> interface,
> >> >> or
> >> >> > > just
> >> >> > > >> > get
> >> >> > > >> > > >> rid
> >> >> > > >> > > >> > of it all-together.
> >> >> > > >> > > >> >
> >> >> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
> >> >> > > >> > > >> venki.korukanti@gmail.com>
> >> >> > > >> > > >> > wrote:
> >> >> > > >> > > >> >
> >> >> > > >> > > >> >> Here is the partial fix:
> >> >> > > >> > > >> >>
> >> >> > > >> > > >> >>
> >> >> > > >> > > >>
> >> >> > > >> > >
> >> >> > > >> >
> >> >> > > >>
> >> >> > >
> >> >> >
> >> >>
> >>
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> >> >> > > >> > > >> >>
> >> >> > > >> > > >> >> If the existing StragePlugins are implementing
> >> >> > > >> > AbstractStoragePlugin
> >> >> > > >> > > >> class
> >> >> > > >> > > >> >> this fix should resolve the backward compat issue.
> If
> >> they
> >> >> > are
> >> >> > > >> > > >> implementing
> >> >> > > >> > > >> >> StoragePlugin interface directly then it won't
> resolve
> >> the
> >> >> > > issue.
> >> >> > > >> > Not
> >> >> > > >> > > >> sure
> >> >> > > >> > > >> >> how to handle it in that case. Let me know if there
> any
> >> >> ways
> >> >> > > to
> >> >> > > >> > > resolve
> >> >> > > >> > > >> >> this issue completely.
> >> >> > > >> > > >> >>
> >> >> > > >> > > >> >> Thanks
> >> >> > > >> > > >> >> Venki
> >> >> > > >> > > >> >>
> >> >> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <
> >> >> > > asinha@maprtech.com
> >> >> > > >> >
> >> >> > > >> > > >> wrote:
> >> >> > > >> > > >> >>
> >> >> > > >> > > >> >> > I was able to do the build with maven 3.3.9 (after
> >> doing
> >> >> > > 'brew
> >> >> > > >> > > install
> >> >> > > >> > > >> >> > maven').  Here are my env variables related to
> maven:
> >> >> > > >> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> >> >> > > >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> >> >> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> >> >> > > >> > > >> >> >
> >> >> > > >> > > >> >> >
> >> >> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <
> >> >> > > >> > > jacques@dremio.com>
> >> >> > > >> > > >> >> > wrote:
> >> >> > > >> > > >> >> >
> >> >> > > >> > > >> >> > > Aman, for reference on the Maven version issue.
> You
> >> >> can
> >> >> > > see
> >> >> > > >> the
> >> >> > > >> > > >> output
> >> >> > > >> > > >> >> of
> >> >> > > >> > > >> >> > > the shade plugin between the two Apache Builds.
> If
> >> you
> >> >> > > search
> >> >> > > >> > for
> >> >> > > >> > > >> >> > > maven-shade-plugin you'll see that build 647
> (maven
> >> >> > 3.0.5)
> >> >> > > >> > fails
> >> >> > > >> > > >> >> because
> >> >> > > >> > > >> >> > it
> >> >> > > >> > > >> >> > > includes a bunch of hadoop (and other libraries
> >> that
> >> >> are
> >> >> > > >> listed
> >> >> > > >> > > as
> >> >> > > >> > > >> >> being
> >> >> > > >> > > >> >> > > excluded). If you look at build 648 (maven
> 3.3.3),
> >> >> those
> >> >> > > >> items
> >> >> > > >> > > are
> >> >> > > >> > > >> >> > included
> >> >> > > >> > > >> >> > > so that the build completes successfully.
> >> >> > > >> > > >> >> > >
> >> >> > > >> > > >> >> > >
> >> >> https://builds.apache.org/job/drill-scm/647/consoleText
> >> >> > > >> > > >> >> > >
> >> >> https://builds.apache.org/job/drill-scm/648/consoleText
> >> >> > > >> > > >> >> > >
> >> >> > > >> > > >> >> > > --
> >> >> > > >> > > >> >> > > Jacques Nadeau
> >> >> > > >> > > >> >> > > CTO and Co-Founder, Dremio
> >> >> > > >> > > >> >> > >
> >> >> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul
> challapalli
> >> <
> >> >> > > >> > > >> >> > > challapallirahul@gmail.com> wrote:
> >> >> > > >> > > >> >> > >
> >> >> > > >> > > >> >> > > > Jason,
> >> >> > > >> > > >> >> > > >
> >> >> > > >> > > >> >> > > > As of now I haven't narrowed down the exact
> >> commit
> >> >> > which
> >> >> > > >> > caused
> >> >> > > >> > > >> >> > > DRILL-4323
> >> >> > > >> > > >> >> > > > and I do not know the scope of the fix.
> >> >> > > >> > > >> >> > > >
> >> >> > > >> > > >> >> > > > - Rahul
> >> >> > > >> > > >> >> > > >
> >> >> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason
> Altekruse
> >> <
> >> >> > > >> > > >> >> > > altekrusejason@gmail.com
> >> >> > > >> > > >> >> > > > >
> >> >> > > >> > > >> >> > > > wrote:
> >> >> > > >> > > >> >> > > >
> >> >> > > >> > > >> >> > > > > I think it makes sense to wait for Venki's
> fix
> >> of
> >> >> > the
> >> >> > > >> API,
> >> >> > > >> > as
> >> >> > > >> > > >> >> > > releasing a
> >> >> > > >> > > >> >> > > > > version with the change would make
> >> >> > > reverting/refactoring
> >> >> > > >> > the
> >> >> > > >> > > API
> >> >> > > >> > > >> >> > > breaking
> >> >> > > >> > > >> >> > > > > change less useful.
> >> >> > > >> > > >> >> > > > >
> >> >> > > >> > > >> >> > > > > Do you have any idea about the scope of
> fixing
> >> the
> >> >> > > hive
> >> >> > > >> > > native
> >> >> > > >> > > >> >> > reader,
> >> >> > > >> > > >> >> > > or
> >> >> > > >> > > >> >> > > > > which change since the 1.4 release broke
> it? Is
> >> >> > > someone
> >> >> > > >> > > >> available
> >> >> > > >> > > >> >> to
> >> >> > > >> > > >> >> > > look
> >> >> > > >> > > >> >> > > > > at it today?
> >> >> > > >> > > >> >> > > > >
> >> >> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul
> >> >> challapalli <
> >> >> > > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
> >> >> > > >> > > >> >> > > > >
> >> >> > > >> > > >> >> > > > > > Just found an issue with hive native
> parquet
> >> >> > reader
> >> >> > > >> > > >> (DRILL-4323).
> >> >> > > >> > > >> >> > > This
> >> >> > > >> > > >> >> > > > > is a
> >> >> > > >> > > >> >> > > > > > regression from 1.4 and in my opinion
> should
> >> be
> >> >> > > treated
> >> >> > > >> > as
> >> >> > > >> > > a
> >> >> > > >> > > >> >> > blocker.
> >> >> > > >> > > >> >> > > > > >
> >> >> > > >> > > >> >> > > > > > - Rahul
> >> >> > > >> > > >> >> > > > > >
> >> >> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki
> >> >> Korukanti <
> >> >> > > >> > > >> >> > > > > > venki.korukanti@gmail.com>
> >> >> > > >> > > >> >> > > > > > wrote:
> >> >> > > >> > > >> >> > > > > >
> >> >> > > >> > > >> >> > > > > > > Sorry I didn't realize I was breaking
> the
> >> >> public
> >> >> > > >> > > interface
> >> >> > > >> > > >> API.
> >> >> > > >> > > >> >> > Let
> >> >> > > >> > > >> >> > > > me
> >> >> > > >> > > >> >> > > > > > work
> >> >> > > >> > > >> >> > > > > > > on a patch to make it backward
> compatible.
> >> >> > > >> > > >> >> > > > > > >
> >> >> > > >> > > >> >> > > > > > > Thanks
> >> >> > > >> > > >> >> > > > > > > Venki
> >> >> > > >> > > >> >> > > > > > >
> >> >> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven
> >> >> > Phillips
> >> >> > > <
> >> >> > > >> > > >> >> > > steven@dremio.com
> >> >> > > >> > > >> >> > > > >
> >> >> > > >> > > >> >> > > > > > > wrote:
> >> >> > > >> > > >> >> > > > > > >
> >> >> > > >> > > >> >> > > > > > > > I just wanted to bring up an issue
> that I
> >> >> just
> >> >> > > now
> >> >> > > >> > > >> >> discovered,
> >> >> > > >> > > >> >> > > that
> >> >> > > >> > > >> >> > > > > has
> >> >> > > >> > > >> >> > > > > > > > caused me a fair amount of grief.
> >> >> > > >> > > >> >> > > > > > > >
> >> >> > > >> > > >> >> > > > > > > >
> >> >> > > https://github.com/apache/drill/pull/300/commits
> >> >> > > >> > > >> >> > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing API,
> and
> >> >> > causes
> >> >> > > >> > > >> >> StoragePlugins
> >> >> > > >> > > >> >> > > > that
> >> >> > > >> > > >> >> > > > > > were
> >> >> > > >> > > >> >> > > > > > > > compiled against currently released
> >> versions
> >> >> > of
> >> >> > > >> Drill
> >> >> > > >> > > to
> >> >> > > >> > > >> no
> >> >> > > >> > > >> >> > > longer
> >> >> > > >> > > >> >> > > > > > > > functional properly. I would prefer
> that
> >> >> this
> >> >> > > >> > breaking
> >> >> > > >> > > >> change
> >> >> > > >> > > >> >> > be
> >> >> > > >> > > >> >> > > > > > modified
> >> >> > > >> > > >> >> > > > > > > > to be backward compatible if possible.
> >> >> > > >> > > >> >> > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM,
> Jason
> >> >> > > Altekruse <
> >> >> > > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
> >> >> > > >> > > >> >> > > > > > > > wrote:
> >> >> > > >> > > >> >> > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > Hi Aman,
> >> >> > > >> > > >> >> > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > This is the failure that he was
> >> seeing. He
> >> >> > > >> figured
> >> >> > > >> > > out
> >> >> > > >> > > >> that
> >> >> > > >> > > >> >> > the
> >> >> > > >> > > >> >> > > > new
> >> >> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were not
> being
> >> >> > > respected
> >> >> > > >> > when
> >> >> > > >> > > the
> >> >> > > >> > > >> >> > build
> >> >> > > >> > > >> >> > > > was
> >> >> > > >> > > >> >> > > > > > run
> >> >> > > >> > > >> >> > > > > > > > with
> >> >> > > >> > > >> >> > > > > > > > > an older Maven version, causing the
> jar
> >> >> size
> >> >> > > to
> >> >> > > >> > > increase
> >> >> > > >> > > >> >> > > > > > significantly.
> >> >> > > >> > > >> >> > > > > > > > He
> >> >> > > >> > > >> >> > > > > > > > > added an enforcer to make sure the
> JAR
> >> >> > didn't
> >> >> > > >> grow
> >> >> > > >> > > >> >> > > unexpectedly.
> >> >> > > >> > > >> >> > > > > Can
> >> >> > > >> > > >> >> > > > > > > you
> >> >> > > >> > > >> >> > > > > > > > > try to update your maven version and
> >> >> re-run
> >> >> > > the
> >> >> > > >> > > build?
> >> >> > > >> > > >> >> > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > - Jason
> >> >> > > >> > > >> >> > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM,
> Aman
> >> >> > Sinha <
> >> >> > > >> > > >> >> > > > asinha@maprtech.com>
> >> >> > > >> > > >> >> > > > > > > > wrote:
> >> >> > > >> > > >> >> > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the
> following
> >> >> build
> >> >> > > >> failure
> >> >> > > >> > > on
> >> >> > > >> > > >> the
> >> >> > > >> > > >> >> > > latest
> >> >> > > >> > > >> >> > > > > > > master
> >> >> > > >> > > >> >> > > > > > > > > > branch...is this what you saw for
> the
> >> >> > Apache
> >> >> > > >> > build
> >> >> > > >> > > ?
> >> >> > > >> > > >> My
> >> >> > > >> > > >> >> > mvn
> >> >> > > >> > > >> >> > > > > > version
> >> >> > > >> > > >> >> > > > > > > > > output
> >> >> > > >> > > >> >> > > > > > > > > > is shown below.  Should we all be
> >> >> > upgrading
> >> >> > > to
> >> >> > > >> a
> >> >> > > >> > > newer
> >> >> > > >> > > >> >> mvn
> >> >> > > >> > > >> >> > ?
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > > [INFO] ---
> >> >> > > maven-enforcer-plugin:1.3.1:enforce
> >> >> > > >> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @
> >> >> > > drill-jdbc-all
> >> >> > > >> > ---
> >> >> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
> >> >> > > >> > > >> >> > > > > >
> >> >> org.apache.maven.plugins.enforcer.RequireFilesSize
> >> >> > > >> > > >> >> > > > > > > > > failed
> >> >> > > >> > > >> >> > > > > > > > > > with message:
> >> >> > > >> > > >> >> > > > > > > > > > The file
> >> >> drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> >> > > is
> >> >> > > >> > > outside
> >> >> > > >> > > >> the
> >> >> > > >> > > >> >> > > > > expected
> >> >> > > >> > > >> >> > > > > > > size
> >> >> > > >> > > >> >> > > > > > > > > > range.
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > >                   This is likely
> due
> >> to
> >> >> > you
> >> >> > > >> > adding
> >> >> > > >> > > new
> >> >> > > >> > > >> >> > > > > dependencies
> >> >> > > >> > > >> >> > > > > > > to
> >> >> > > >> > > >> >> > > > > > > > a
> >> >> > > >> > > >> >> > > > > > > > > > java-exec and not updating the
> >> excludes
> >> >> in
> >> >> > > this
> >> >> > > >> > > >> module.
> >> >> > > >> > > >> >> > This
> >> >> > > >> > > >> >> > > is
> >> >> > > >> > > >> >> > > > > > > > important
> >> >> > > >> > > >> >> > > > > > > > > > as it minimizes the size of the
> >> >> dependency
> >> >> > > of
> >> >> > > >> > Drill
> >> >> > > >> > > >> >> > > application
> >> >> > > >> > > >> >> > > > > > > users.
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > >
> >> >> > > >> > > >> >> > > > > > >
> >> >> > > >> > > >> >> > > > > >
> >> >> > > >> > > >> >> > > > >
> >> >> > > >> > > >> >> > > >
> >> >> > > >> > > >> >> > >
> >> >> > > >> > > >> >> >
> >> >> > > >> > > >> >>
> >> >> > > >> > > >>
> >> >> > > >> > >
> >> >> > > >> >
> >> >> > > >>
> >> >> > >
> >> >> >
> >> >>
> >>
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> >> > > >> > > >> >> > > > > > > > > > size (44664121) too large. Max. is
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > >
> >> >> > > >> > > >> >> > > > > > >
> >> >> > > >> > > >> >> > > > > >
> >> >> > > >> > > >> >> > > > >
> >> >> > > >> > > >> >> > > >
> >> >> > > >> > > >> >> > >
> >> >> > > >> > > >> >> >
> >> >> > > >> > > >> >>
> >> >> > > >> > > >>
> >> >> > > >> > >
> >> >> > > >> >
> >> >> > > >>
> >> >> > >
> >> >> >
> >> >>
> >>
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > Administrators-MacBook-Pro-144:incubator-drill
> >> >> > > >> > > asinha$
> >> >> > > >> > > >> >> mvn
> >> >> > > >> > > >> >> > > > > > --version
> >> >> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
> >> >> > > >> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> >> >> > > >> > > >> >> > > > > > > > 2013-02-19
> >> >> > > >> > > >> >> > > > > > > > > > 05:51:28-0800)
> >> >> > > >> > > >> >> > > > > > > > > > Maven home:
> >> /opt/local/share/java/maven3
> >> >> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor:
> >> Oracle
> >> >> > > >> > Corporation
> >> >> > > >> > > >> >> > > > > > > > > > Java home:
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > >
> >> >> > > >> > > >>
> >> >> > >
> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> >> >> > > >> > > >> >> > > > > > > > > > Default locale: en_US, platform
> >> >> encoding:
> >> >> > > UTF-8
> >> >> > > >> > > >> >> > > > > > > > > > OS name: "mac os x", version:
> >> "10.9.5",
> >> >> > > arch:
> >> >> > > >> > > >> "x86_64",
> >> >> > > >> > > >> >> > > family:
> >> >> > > >> > > >> >> > > > > > "mac"
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM,
> >> Jacques
> >> >> > > >> Nadeau <
> >> >> > > >> > > >> >> > > > > > jacques@dremio.com>
> >> >> > > >> > > >> >> > > > > > > > > > wrote:
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > > > Build back to normal. It looks
> like
> >> >> the
> >> >> > > >> Apache
> >> >> > > >> > > >> server
> >> >> > > >> > > >> >> was
> >> >> > > >> > > >> >> > > > using
> >> >> > > >> > > >> >> > > > > > an
> >> >> > > >> > > >> >> > > > > > > > old
> >> >> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I
> switched
> >> to
> >> >> > > >> something
> >> >> > > >> > > more
> >> >> > > >> > > >> >> > recent,
> >> >> > > >> > > >> >> > > > the
> >> >> > > >> > > >> >> > > > > > > build
> >> >> > > >> > > >> >> > > > > > > > > > > passed.
> >> >> > > >> > > >> >> > > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > > > --
> >> >> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau
> >> >> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
> >> >> > > >> > > >> >> > > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM,
> >> >> Jacques
> >> >> > > >> > Nadeau <
> >> >> > > >> > > >> >> > > > > > > jacques@dremio.com>
> >> >> > > >> > > >> >> > > > > > > > > > > wrote:
> >> >> > > >> > > >> >> > > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge caused the
> >> Apache
> >> >> > > build
> >> >> > > >> to
> >> >> > > >> > > fail.
> >> >> > > >> > > >> >> > > > > > > Investigating...
> >> >> > > >> > > >> >> > > > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > > > > --
> >> >> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
> >> >> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
> >> >> > > >> > > >> >> > > > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31
> AM,
> >> >> > Jacques
> >> >> > > >> > > Nadeau <
> >> >> > > >> > > >> >> > > > > > > > jacques@dremio.com>
> >> >> > > >> > > >> >> > > > > > > > > > > > wrote:
> >> >> > > >> > > >> >> > > > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > > > >> I got clean regression runs
> as
> >> >> well.
> >> >> > > I've
> >> >> > > >> > > merged
> >> >> > > >> > > >> the
> >> >> > > >> > > >> >> > > > patch.
> >> >> > > >> > > >> >> > > > > > > > > > > >>
> >> >> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to start the
> >> >> release
> >> >> > > >> > process?
> >> >> > > >> > > >> >> > > > > > > > > > > >>
> >> >> > > >> > > >> >> > > > > > > > > > > >> --
> >> >> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
> >> >> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
> >> >> > > >> > > >> >> > > > > > > > > > > >>
> >> >> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42
> >> PM,
> >> >> > > Abhishek
> >> >> > > >> > > >> Girish <
> >> >> > > >> > > >> >> > > > > > > > agirish@mapr.com
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > > > >> wrote:
> >> >> > > >> > > >> >> > > > > > > > > > > >>
> >> >> > > >> > > >> >> > > > > > > > > > > >>> Had two clean Functional
> runs.
> >> >> TPC-H
> >> >> > > >> SF100
> >> >> > > >> > > was
> >> >> > > >> > > >> also
> >> >> > > >> > > >> >> > > > > > successful.
> >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> >> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at
> 10:07
> >> PM,
> >> >> > > rahul
> >> >> > > >> > > >> >> challapalli <
> >> >> > > >> > > >> >> > > > > > > > > > > >>> challapallirahul@gmail.com>
> >> >> wrote:
> >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a functional
> run
> >> with
> >> >> > > your
> >> >> > > >> > > branch.
> >> >> > > >> > > >> >> Will
> >> >> > > >> > > >> >> > > let
> >> >> > > >> > > >> >> > > > > you
> >> >> > > >> > > >> >> > > > > > > know
> >> >> > > >> > > >> >> > > > > > > > > > once
> >> >> > > >> > > >> >> > > > > > > > > > > it
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > finishes
> >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul
> >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at
> 9:56
> >> PM,
> >> >> > > >> Jacques
> >> >> > > >> > > >> Nadeau <
> >> >> > > >> > > >> >> > > > > > > > > > jacques@dremio.com>
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > wrote:
> >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I
> >> have
> >> >> an
> >> >> > > >> > updated
> >> >> > > >> > > >> patch
> >> >> > > >> > > >> >> > for
> >> >> > > >> > > >> >> > > > 4291
> >> >> > > >> > > >> >> > > > > > > that
> >> >> > > >> > > >> >> > > > > > > > is
> >> >> > > >> > > >> >> > > > > > > > > > > >>> ready.
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems
> >> that
> >> >> > > >> something
> >> >> > > >> > > isn't
> >> >> > > >> > > >> >> > > working
> >> >> > > >> > > >> >> > > > > with
> >> >> > > >> > > >> >> > > > > > > our
> >> >> > > >> > > >> >> > > > > > > > > > > >>> extended
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't been
> >> able
> >> >> to
> >> >> > > run
> >> >> > > >> an
> >> >> > > >> > > >> extended
> >> >> > > >> > > >> >> > > > > > regression.
> >> >> > > >> > > >> >> > > > > > > > > Unit
> >> >> > > >> > > >> >> > > > > > > > > > > >>> tests
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else
> >> possibly
> >> >> > > able
> >> >> > > >> to
> >> >> > > >> > > run a
> >> >> > > >> > > >> >> > > > regression
> >> >> > > >> > > >> >> > > > > > > suite
> >> >> > > >> > > >> >> > > > > > > > > > > against
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > this
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can
> >> confirm
> >> >> > > things
> >> >> > > >> > look
> >> >> > > >> > > >> good
> >> >> > > >> > > >> >> and
> >> >> > > >> > > >> >> > > > start
> >> >> > > >> > > >> >> > > > > > the
> >> >> > > >> > > >> >> > > > > > > > > > release
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > process?
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > thanks,
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
> >> >> > > >> > > >> >> > > > >
> >> >> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > --
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder,
> Dremio
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at
> >> 11:20
> >> >> > AM,
> >> >> > > >> > Jacques
> >> >> > > >> > > >> >> Nadeau
> >> >> > > >> > > >> >> > <
> >> >> > > >> > > >> >> > > > > > > > > > > jacques@dremio.com
> >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the main
> things
> >> are
> >> >> > 4196
> >> >> > > >> and
> >> >> > > >> > > 4291
> >> >> > > >> > > >> >> > should
> >> >> > > >> > > >> >> > > be
> >> >> > > >> > > >> >> > > > > > > > > completed.
> >> >> > > >> > > >> >> > > > > > > > > > I
> >> >> > > >> > > >> >> > > > > > > > > > > >>> know
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > Amit
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to reproduce
> >> 4196
> >> >> > > locally
> >> >> > > >> > this
> >> >> > > >> > > >> >> weekend
> >> >> > > >> > > >> >> > > so
> >> >> > > >> > > >> >> > > > I
> >> >> > > >> > > >> >> > > > > > > think
> >> >> > > >> > > >> >> > > > > > > > > > we're
> >> >> > > >> > > >> >> > > > > > > > > > > >>> close
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > on
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > that.
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > --
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder,
> >> Dremio
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016
> at
> >> >> 10:14
> >> >> > > AM,
> >> >> > > >> > > Zelaine
> >> >> > > >> > > >> >> Fong
> >> >> > > >> > > >> >> > <
> >> >> > > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
> >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on this?
> >> >> What's
> >> >> > > >> > blocking
> >> >> > > >> > > us
> >> >> > > >> > > >> >> from
> >> >> > > >> > > >> >> > > > taking
> >> >> > > >> > > >> >> > > > > > > this
> >> >> > > >> > > >> >> > > > > > > > > to a
> >> >> > > >> > > >> >> > > > > > > > > > > >>> vote --
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > the
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join
> issues?
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016
> at
> >> >> 4:18
> >> >> > > PM,
> >> >> > > >> > > Jacques
> >> >> > > >> > > >> >> > Nadeau
> >> >> > > >> > > >> >> > > <
> >> >> > > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this
> thread...
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the issues
> >> that
> >> >> > were
> >> >> > > >> > > >> mentioned in
> >> >> > > >> > > >> >> > > this
> >> >> > > >> > > >> >> > > > > > thread
> >> >> > > >> > > >> >> > > > > > > > > along
> >> >> > > >> > > >> >> > > > > > > > > > > >>> with a
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed
> >> categorization:
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4190
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4196
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge
> Jacques
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
> >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4256
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
> >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4278
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge
> Laurent
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
> >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4247
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
> >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4203
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> https://issues.apache.org/jira/browse/DRILL-4266
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge
> Jacques
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do others
> think?
> >> >> Let's
> >> >> > > try
> >> >> > > >> to
> >> >> > > >> > > get
> >> >> > > >> > > >> the
> >> >> > > >> > > >> >> > > > > blockers
> >> >> > > >> > > >> >> > > > > > > > > wrapped
> >> >> > > >> > > >> >> > > > > > > > > > up
> >> >> > > >> > > >> >> > > > > > > > > > > >>> in
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > the
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> next
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two and
> start a
> >> >> > > release
> >> >> > > >> > > vote...
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > --
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder,
> >> >> Dremio
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4,
> 2016 at
> >> >> 1:48
> >> >> > > PM,
> >> >> > > >> > > Jason
> >> >> > > >> > > >> >> > > Altekruse
> >> >> > > >> > > >> >> > > > <
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> altekrusejason@gmail.com>
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the
> allocator
> >> >> > changes
> >> >> > > >> > merged
> >> >> > > >> > > and
> >> >> > > >> > > >> >> > about
> >> >> > > >> > > >> >> > > a
> >> >> > > >> > > >> >> > > > > > month
> >> >> > > >> > > >> >> > > > > > > > > since
> >> >> > > >> > > >> >> > > > > > > > > > > the
> >> >> > > >> > > >> >> > > > > > > > > > > >>> last
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it would
> be
> >> >> good
> >> >> > to
> >> >> > > >> > start
> >> >> > > >> > > a
> >> >> > > >> > > >> vote
> >> >> > > >> > > >> >> > > > soon. I
> >> >> > > >> > > >> >> > > > > > > would
> >> >> > > >> > > >> >> > > > > > > > > > like
> >> >> > > >> > > >> >> > > > > > > > > > > to
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release
> manager.
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that there
> >> were
> >> >> > some
> >> >> > > >> > issues
> >> >> > > >> > > >> that
> >> >> > > >> > > >> >> > were
> >> >> > > >> > > >> >> > > > > > > > identified
> >> >> > > >> > > >> >> > > > > > > > > > > after
> >> >> > > >> > > >> >> > > > > > > > > > > >>> the
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was
> merged. I
> >> >> think
> >> >> > > that
> >> >> > > >> > > these
> >> >> > > >> > > >> >> > issues
> >> >> > > >> > > >> >> > > > > should
> >> >> > > >> > > >> >> > > > > > > be
> >> >> > > >> > > >> >> > > > > > > > > > fixed
> >> >> > > >> > > >> >> > > > > > > > > > > >>> before
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > we
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release
> candidate.
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at
> the
> >> >> > > associated
> >> >> > > >> > > JIRAs
> >> >> > > >> > > >> it
> >> >> > > >> > > >> >> > > looked
> >> >> > > >> > > >> >> > > > > > like
> >> >> > > >> > > >> >> > > > > > > > > there
> >> >> > > >> > > >> >> > > > > > > > > > > was
> >> >> > > >> > > >> >> > > > > > > > > > > >>> a
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix
> just
> >> >> > > adjusting
> >> >> > > >> > the
> >> >> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > option,
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> and
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more
> involved
> >> work
> >> >> > to
> >> >> > > >> > change
> >> >> > > >> > > >> how we
> >> >> > > >> > > >> >> > > > > determine
> >> >> > > >> > > >> >> > > > > > > the
> >> >> > > >> > > >> >> > > > > > > > > > > correct
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > time
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > to
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill during
> >> external
> >> >> > > sort. I
> >> >> > > >> > > >> believe it
> >> >> > > >> > > >> >> > > makes
> >> >> > > >> > > >> >> > > > > > sense
> >> >> > > >> > > >> >> > > > > > > > to
> >> >> > > >> > > >> >> > > > > > > > > > make
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > external
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well with
> the
> >> >> newly
> >> >> > > >> > improved
> >> >> > > >> > > >> memory
> >> >> > > >> > > >> >> > > > > > accounting
> >> >> > > >> > > >> >> > > > > > > > > before
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > cutting a
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm
> not
> >> >> sure
> >> >> > > how
> >> >> > > >> > much
> >> >> > > >> > > >> work
> >> >> > > >> > > >> >> is
> >> >> > > >> > > >> >> > > > left
> >> >> > > >> > > >> >> > > > > to
> >> >> > > >> > > >> >> > > > > > > be
> >> >> > > >> > > >> >> > > > > > > > > done
> >> >> > > >> > > >> >> > > > > > > > > > > >>> there.
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond
> with
> >> >> your
> >> >> > > >> > thoughts
> >> >> > > >> > > on
> >> >> > > >> > > >> a
> >> >> > > >> > > >> >> > > release
> >> >> > > >> > > >> >> > > > > > soon
> >> >> > > >> > > >> >> > > > > > > > and
> >> >> > > >> > > >> >> > > > > > > > > > any
> >> >> > > >> > > >> >> > > > > > > > > > > >>> JIRAs
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > you
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to include
> in
> >> the
> >> >> > > >> release.
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
> >> >> > > >> > > >> >> > > > > > > >
> >> >> > > https://issues.apache.org/jira/browse/DRILL-4243
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> >> > > >> > > >> >> > > > > > > > > > > >>
> >> >> > > >> > > >> >> > > > > > > > > > > >>
> >> >> > > >> > > >> >> > > > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > > >
> >> >> > > >> > > >> >> > > > > > > >
> >> >> > > >> > > >> >> > > > > > >
> >> >> > > >> > > >> >> > > > > >
> >> >> > > >> > > >> >> > > > >
> >> >> > > >> > > >> >> > > >
> >> >> > > >> > > >> >> > >
> >> >> > > >> > > >> >> >
> >> >> > > >> > > >> >>
> >> >> > > >> > > >>
> >> >> > > >> > >
> >> >> > > >> >
> >> >> > > >>
> >> >> > >
> >> >> >
> >> >>
> >>
> >>
>
>

Re: Time for a 1.5 release?

Posted by Jinfeng Ni <ji...@gmail.com>.

Venki's backwards compatibility fix (commit id:
   03197d0) is before the partition pruning commit. So, it seems to be
fine to create a release candidate from 03197d0; no need to
cherry-pick.

As for whether the partition pruning changes are risky, the pre-commit
run did not show any serious problems in the first run; all the plan
changes are expected, or expose some existing problem (drill-4279).
However, I agree that it impacts the main code path of query planning,
and I'm fine that we exclude the partition change from 1.5.0 release,
since it's merged right before we start a voting.

I did not see any one mentioned that the master branch is closed, and
therefore assume it's still open for commits.  I probably should have
asked whether the master was open for new patches before I merged the
commits.



On Sun, Jan 31, 2016 at 8:43 PM, Jacques Nadeau <ja...@dremio.com> wrote:
> Jinfeng, do you agree that the changes are risky? I was going mostly by
> commit messages.
>
> Also just realized that the backwards compatibility fix to the storage
> plugin interface is after these. Could create release branch from 03197d...
> and then cherry pick the Venki change. Or could keep them all. Or
> go without the compatibility fix....
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Sun, Jan 31, 2016 at 7:50 PM, Jinfeng Ni <ji...@gmail.com> wrote:
>
>> Hi Jason,
>>
>> I'm fine that release 1.5.0 starts from 03197d0f2.  One thing I need
>> point out is that the pre-commit test suite has some testcase's plan
>> output modified because of the partition pruning change.  If we run
>> pre-commit against the candidate release for 1.5.0, we have to use a
>> prior version of pre-commit suite.
>>
>>
>>
>> On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse
>> <al...@gmail.com> wrote:
>> > That sounds reasonable to me. I'll start preparing a release candidate.
>> >
>> > Jinfeng, are you okay waiting for the next release to include this
>> change?
>> >
>> > On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <ja...@dremio.com>
>> wrote:
>> >
>> >> Hey Jason,
>> >>
>> >> Can you start the release vote? We've let this drag on too long.
>> >>
>> >> I suggest we start from 03197d0f2. It seems like the partition pruning
>> >> changes are pretty complicated and on a pretty main path to add right
>> >> before the vote. Since Hsuan's fix (pr 349) hasn't been reviewed, I
>> think
>> >> we should start the vote without it.
>> >>
>> >> Jacques
>> >>
>> >>
>> >> --
>> >> Jacques Nadeau
>> >> CTO and Co-Founder, Dremio
>> >>
>> >> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <hy...@maprtech.com>
>> wrote:
>> >>
>> >> > Yes, just send a pull request.
>> >> > https://github.com/apache/drill/pull/349
>> >> >
>> >> > Can Jinfeng review it?
>> >> >
>> >> > Thanks,
>> >> >
>> >> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <ji...@gmail.com>
>> >> wrote:
>> >> >
>> >> > > Sean probably is quite close to find a fix for this issue. Let's see
>> >> > > whether he can post a patch soon.
>> >> > >
>> >> > >
>> >> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
>> >> > > <ch...@gmail.com> wrote:
>> >> > > > DRILL-4323 only happens when assertions are turned on (my install
>> >> > script
>> >> > > > automatically enables assertions).
>> >> > > >
>> >> > > > - Rahul
>> >> > > >
>> >> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <
>> zfong@maprtech.com>
>> >> > > wrote:
>> >> > > >
>> >> > > >> Rahul,
>> >> > > >>
>> >> > > >> Let us know if you can reproduce the issue with assertions turned
>> >> OFF.
>> >> > > If
>> >> > > >> you cannot, then I think this should not be a blocker for 1.5,
>> and
>> >> > Hsuan
>> >> > > >> will continue to look into fixing this.
>> >> > > >>
>> >> > > >> -- Zelaine
>> >> > > >>
>> >> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
>> >> > knguyen@maprtech.com>
>> >> > > >> wrote:
>> >> > > >>
>> >> > > >> > I do see the same error with assertion turned on.
>> >> > > >> >
>> >> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <
>> >> jinfengni99@gmail.com>
>> >> > > >> wrote:
>> >> > > >> >
>> >> > > >> > > Krystal, by any chance, did you turn on assertion when you
>> start
>> >> > > >> > drillbit?
>> >> > > >> > >
>> >> > > >> > > I think the error in drill-4323 will be raised when
>> assertion =
>> >> > on.
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
>> >> > > knguyen@maprtech.com>
>> >> > > >> > > wrote:
>> >> > > >> > > > I am running drill-1.4 and is able to run count(*)
>> >> successfully.
>> >> > > >> > > >
>> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter
>> session
>> >> > set
>> >> > > >> > > > `store.hive.optimize_scan_with_native_readers` = true;
>> >> > > >> > > >
>> >> > +-------+--------------------------------------------------------+
>> >> > > >> > > > |  ok   |                        summary
>> >> >  |
>> >> > > >> > > >
>> >> > +-------+--------------------------------------------------------+
>> >> > > >> > > > | true  | store.hive.optimize_scan_with_native_readers
>> >> > updated.  |
>> >> > > >> > > >
>> >> > +-------+--------------------------------------------------------+
>> >> > > >> > > > 1 row selected (0.374 seconds)
>> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select
>> >> count(*)
>> >> > > from
>> >> > > >> > > > lineitem_parquet_partitioned_hive_string;
>> >> > > >> > > > +---------+
>> >> > > >> > > > | EXPR$0  |
>> >> > > >> > > > +---------+
>> >> > > >> > > > | 60175   |
>> >> > > >> > > > +---------+
>> >> > > >> > > > 1 row selected (2.986 seconds)
>> >> > > >> > > >
>> >> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
>> >> > > jinfengni99@gmail.com>
>> >> > > >> > > wrote:
>> >> > > >> > > >
>> >> > > >> > > >> Venki and I did some investigation for DRILL-4323. The
>> issue
>> >> > > >> reported
>> >> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as well.
>> Seems
>> >> > to
>> >> > > us
>> >> > > >> > > >> this is not a regression from 1.4.0; it's a regression
>> from
>> >> > 1.3.0
>> >> > > >> > > >> probably.
>> >> > > >> > > >>
>> >> > > >> > > >> DRILL-4083 makes the planner to use DrillHiveNativeReader
>> in
>> >> > > stead
>> >> > > >> of
>> >> > > >> > > >> HiveReader for "select count(*) from hive_table" query.
>> >> > However,
>> >> > > >> the
>> >> > > >> > > >> Project after the scan produces empty schema.  Before
>> >> > DRILL-4083,
>> >> > > >> > > >> Drill uses HiveScan, which works fine.
>> >> > > >> > > >>
>> >> > > >> > > >>
>> >> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
>> >> > > >> > > >> <al...@gmail.com> wrote:
>> >> > > >> > > >> > We could revert the change to the StoragePlugin
>> interface
>> >> to
>> >> > > add
>> >> > > >> the
>> >> > > >> > > >> > getLogicalOptimizerRules
>> >> > > >> > > >> > and getPhysicalOptimizerRules, and only put those in
>> >> > > >> > > >> AbstractStoragePlugin
>> >> > > >> > > >> > with default implementations.
>> >> > > >> > > >> >
>> >> > > >> > > >> > We could then update the docs to tell users to not
>> >> implement
>> >> > > the
>> >> > > >> > > >> interface
>> >> > > >> > > >> > directly, and instead extend the abstract class, which I
>> >> > > believe
>> >> > > >> > would
>> >> > > >> > > >> > allow us to make changes like this in the future without
>> >> > > breaking
>> >> > > >> > > >> plugins.
>> >> > > >> > > >> > At a major release version we could then just pull up
>> all
>> >> of
>> >> > > the
>> >> > > >> new
>> >> > > >> > > >> > methods we added to the abstract class into the
>> interface,
>> >> or
>> >> > > just
>> >> > > >> > get
>> >> > > >> > > >> rid
>> >> > > >> > > >> > of it all-together.
>> >> > > >> > > >> >
>> >> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
>> >> > > >> > > >> venki.korukanti@gmail.com>
>> >> > > >> > > >> > wrote:
>> >> > > >> > > >> >
>> >> > > >> > > >> >> Here is the partial fix:
>> >> > > >> > > >> >>
>> >> > > >> > > >> >>
>> >> > > >> > > >>
>> >> > > >> > >
>> >> > > >> >
>> >> > > >>
>> >> > >
>> >> >
>> >>
>> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
>> >> > > >> > > >> >>
>> >> > > >> > > >> >> If the existing StragePlugins are implementing
>> >> > > >> > AbstractStoragePlugin
>> >> > > >> > > >> class
>> >> > > >> > > >> >> this fix should resolve the backward compat issue. If
>> they
>> >> > are
>> >> > > >> > > >> implementing
>> >> > > >> > > >> >> StoragePlugin interface directly then it won't resolve
>> the
>> >> > > issue.
>> >> > > >> > Not
>> >> > > >> > > >> sure
>> >> > > >> > > >> >> how to handle it in that case. Let me know if there any
>> >> ways
>> >> > > to
>> >> > > >> > > resolve
>> >> > > >> > > >> >> this issue completely.
>> >> > > >> > > >> >>
>> >> > > >> > > >> >> Thanks
>> >> > > >> > > >> >> Venki
>> >> > > >> > > >> >>
>> >> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <
>> >> > > asinha@maprtech.com
>> >> > > >> >
>> >> > > >> > > >> wrote:
>> >> > > >> > > >> >>
>> >> > > >> > > >> >> > I was able to do the build with maven 3.3.9 (after
>> doing
>> >> > > 'brew
>> >> > > >> > > install
>> >> > > >> > > >> >> > maven').  Here are my env variables related to maven:
>> >> > > >> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
>> >> > > >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
>> >> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
>> >> > > >> > > >> >> >
>> >> > > >> > > >> >> >
>> >> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <
>> >> > > >> > > jacques@dremio.com>
>> >> > > >> > > >> >> > wrote:
>> >> > > >> > > >> >> >
>> >> > > >> > > >> >> > > Aman, for reference on the Maven version issue. You
>> >> can
>> >> > > see
>> >> > > >> the
>> >> > > >> > > >> output
>> >> > > >> > > >> >> of
>> >> > > >> > > >> >> > > the shade plugin between the two Apache Builds. If
>> you
>> >> > > search
>> >> > > >> > for
>> >> > > >> > > >> >> > > maven-shade-plugin you'll see that build 647 (maven
>> >> > 3.0.5)
>> >> > > >> > fails
>> >> > > >> > > >> >> because
>> >> > > >> > > >> >> > it
>> >> > > >> > > >> >> > > includes a bunch of hadoop (and other libraries
>> that
>> >> are
>> >> > > >> listed
>> >> > > >> > > as
>> >> > > >> > > >> >> being
>> >> > > >> > > >> >> > > excluded). If you look at build 648 (maven 3.3.3),
>> >> those
>> >> > > >> items
>> >> > > >> > > are
>> >> > > >> > > >> >> > included
>> >> > > >> > > >> >> > > so that the build completes successfully.
>> >> > > >> > > >> >> > >
>> >> > > >> > > >> >> > >
>> >> https://builds.apache.org/job/drill-scm/647/consoleText
>> >> > > >> > > >> >> > >
>> >> https://builds.apache.org/job/drill-scm/648/consoleText
>> >> > > >> > > >> >> > >
>> >> > > >> > > >> >> > > --
>> >> > > >> > > >> >> > > Jacques Nadeau
>> >> > > >> > > >> >> > > CTO and Co-Founder, Dremio
>> >> > > >> > > >> >> > >
>> >> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli
>> <
>> >> > > >> > > >> >> > > challapallirahul@gmail.com> wrote:
>> >> > > >> > > >> >> > >
>> >> > > >> > > >> >> > > > Jason,
>> >> > > >> > > >> >> > > >
>> >> > > >> > > >> >> > > > As of now I haven't narrowed down the exact
>> commit
>> >> > which
>> >> > > >> > caused
>> >> > > >> > > >> >> > > DRILL-4323
>> >> > > >> > > >> >> > > > and I do not know the scope of the fix.
>> >> > > >> > > >> >> > > >
>> >> > > >> > > >> >> > > > - Rahul
>> >> > > >> > > >> >> > > >
>> >> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse
>> <
>> >> > > >> > > >> >> > > altekrusejason@gmail.com
>> >> > > >> > > >> >> > > > >
>> >> > > >> > > >> >> > > > wrote:
>> >> > > >> > > >> >> > > >
>> >> > > >> > > >> >> > > > > I think it makes sense to wait for Venki's fix
>> of
>> >> > the
>> >> > > >> API,
>> >> > > >> > as
>> >> > > >> > > >> >> > > releasing a
>> >> > > >> > > >> >> > > > > version with the change would make
>> >> > > reverting/refactoring
>> >> > > >> > the
>> >> > > >> > > API
>> >> > > >> > > >> >> > > breaking
>> >> > > >> > > >> >> > > > > change less useful.
>> >> > > >> > > >> >> > > > >
>> >> > > >> > > >> >> > > > > Do you have any idea about the scope of fixing
>> the
>> >> > > hive
>> >> > > >> > > native
>> >> > > >> > > >> >> > reader,
>> >> > > >> > > >> >> > > or
>> >> > > >> > > >> >> > > > > which change since the 1.4 release broke it? Is
>> >> > > someone
>> >> > > >> > > >> available
>> >> > > >> > > >> >> to
>> >> > > >> > > >> >> > > look
>> >> > > >> > > >> >> > > > > at it today?
>> >> > > >> > > >> >> > > > >
>> >> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul
>> >> challapalli <
>> >> > > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
>> >> > > >> > > >> >> > > > >
>> >> > > >> > > >> >> > > > > > Just found an issue with hive native parquet
>> >> > reader
>> >> > > >> > > >> (DRILL-4323).
>> >> > > >> > > >> >> > > This
>> >> > > >> > > >> >> > > > > is a
>> >> > > >> > > >> >> > > > > > regression from 1.4 and in my opinion should
>> be
>> >> > > treated
>> >> > > >> > as
>> >> > > >> > > a
>> >> > > >> > > >> >> > blocker.
>> >> > > >> > > >> >> > > > > >
>> >> > > >> > > >> >> > > > > > - Rahul
>> >> > > >> > > >> >> > > > > >
>> >> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki
>> >> Korukanti <
>> >> > > >> > > >> >> > > > > > venki.korukanti@gmail.com>
>> >> > > >> > > >> >> > > > > > wrote:
>> >> > > >> > > >> >> > > > > >
>> >> > > >> > > >> >> > > > > > > Sorry I didn't realize I was breaking the
>> >> public
>> >> > > >> > > interface
>> >> > > >> > > >> API.
>> >> > > >> > > >> >> > Let
>> >> > > >> > > >> >> > > > me
>> >> > > >> > > >> >> > > > > > work
>> >> > > >> > > >> >> > > > > > > on a patch to make it backward compatible.
>> >> > > >> > > >> >> > > > > > >
>> >> > > >> > > >> >> > > > > > > Thanks
>> >> > > >> > > >> >> > > > > > > Venki
>> >> > > >> > > >> >> > > > > > >
>> >> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven
>> >> > Phillips
>> >> > > <
>> >> > > >> > > >> >> > > steven@dremio.com
>> >> > > >> > > >> >> > > > >
>> >> > > >> > > >> >> > > > > > > wrote:
>> >> > > >> > > >> >> > > > > > >
>> >> > > >> > > >> >> > > > > > > > I just wanted to bring up an issue that I
>> >> just
>> >> > > now
>> >> > > >> > > >> >> discovered,
>> >> > > >> > > >> >> > > that
>> >> > > >> > > >> >> > > > > has
>> >> > > >> > > >> >> > > > > > > > caused me a fair amount of grief.
>> >> > > >> > > >> >> > > > > > > >
>> >> > > >> > > >> >> > > > > > > >
>> >> > > https://github.com/apache/drill/pull/300/commits
>> >> > > >> > > >> >> > > > > > > >
>> >> > > >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing API, and
>> >> > causes
>> >> > > >> > > >> >> StoragePlugins
>> >> > > >> > > >> >> > > > that
>> >> > > >> > > >> >> > > > > > were
>> >> > > >> > > >> >> > > > > > > > compiled against currently released
>> versions
>> >> > of
>> >> > > >> Drill
>> >> > > >> > > to
>> >> > > >> > > >> no
>> >> > > >> > > >> >> > > longer
>> >> > > >> > > >> >> > > > > > > > functional properly. I would prefer that
>> >> this
>> >> > > >> > breaking
>> >> > > >> > > >> change
>> >> > > >> > > >> >> > be
>> >> > > >> > > >> >> > > > > > modified
>> >> > > >> > > >> >> > > > > > > > to be backward compatible if possible.
>> >> > > >> > > >> >> > > > > > > >
>> >> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason
>> >> > > Altekruse <
>> >> > > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
>> >> > > >> > > >> >> > > > > > > > wrote:
>> >> > > >> > > >> >> > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > Hi Aman,
>> >> > > >> > > >> >> > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > This is the failure that he was
>> seeing. He
>> >> > > >> figured
>> >> > > >> > > out
>> >> > > >> > > >> that
>> >> > > >> > > >> >> > the
>> >> > > >> > > >> >> > > > new
>> >> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were not being
>> >> > > respected
>> >> > > >> > when
>> >> > > >> > > the
>> >> > > >> > > >> >> > build
>> >> > > >> > > >> >> > > > was
>> >> > > >> > > >> >> > > > > > run
>> >> > > >> > > >> >> > > > > > > > with
>> >> > > >> > > >> >> > > > > > > > > an older Maven version, causing the jar
>> >> size
>> >> > > to
>> >> > > >> > > increase
>> >> > > >> > > >> >> > > > > > significantly.
>> >> > > >> > > >> >> > > > > > > > He
>> >> > > >> > > >> >> > > > > > > > > added an enforcer to make sure the JAR
>> >> > didn't
>> >> > > >> grow
>> >> > > >> > > >> >> > > unexpectedly.
>> >> > > >> > > >> >> > > > > Can
>> >> > > >> > > >> >> > > > > > > you
>> >> > > >> > > >> >> > > > > > > > > try to update your maven version and
>> >> re-run
>> >> > > the
>> >> > > >> > > build?
>> >> > > >> > > >> >> > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > - Jason
>> >> > > >> > > >> >> > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman
>> >> > Sinha <
>> >> > > >> > > >> >> > > > asinha@maprtech.com>
>> >> > > >> > > >> >> > > > > > > > wrote:
>> >> > > >> > > >> >> > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the following
>> >> build
>> >> > > >> failure
>> >> > > >> > > on
>> >> > > >> > > >> the
>> >> > > >> > > >> >> > > latest
>> >> > > >> > > >> >> > > > > > > master
>> >> > > >> > > >> >> > > > > > > > > > branch...is this what you saw for the
>> >> > Apache
>> >> > > >> > build
>> >> > > >> > > ?
>> >> > > >> > > >> My
>> >> > > >> > > >> >> > mvn
>> >> > > >> > > >> >> > > > > > version
>> >> > > >> > > >> >> > > > > > > > > output
>> >> > > >> > > >> >> > > > > > > > > > is shown below.  Should we all be
>> >> > upgrading
>> >> > > to
>> >> > > >> a
>> >> > > >> > > newer
>> >> > > >> > > >> >> mvn
>> >> > > >> > > >> >> > ?
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > > [INFO] ---
>> >> > > maven-enforcer-plugin:1.3.1:enforce
>> >> > > >> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @
>> >> > > drill-jdbc-all
>> >> > > >> > ---
>> >> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
>> >> > > >> > > >> >> > > > > >
>> >> org.apache.maven.plugins.enforcer.RequireFilesSize
>> >> > > >> > > >> >> > > > > > > > > failed
>> >> > > >> > > >> >> > > > > > > > > > with message:
>> >> > > >> > > >> >> > > > > > > > > > The file
>> >> drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> >> > > is
>> >> > > >> > > outside
>> >> > > >> > > >> the
>> >> > > >> > > >> >> > > > > expected
>> >> > > >> > > >> >> > > > > > > size
>> >> > > >> > > >> >> > > > > > > > > > range.
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > >                   This is likely due
>> to
>> >> > you
>> >> > > >> > adding
>> >> > > >> > > new
>> >> > > >> > > >> >> > > > > dependencies
>> >> > > >> > > >> >> > > > > > > to
>> >> > > >> > > >> >> > > > > > > > a
>> >> > > >> > > >> >> > > > > > > > > > java-exec and not updating the
>> excludes
>> >> in
>> >> > > this
>> >> > > >> > > >> module.
>> >> > > >> > > >> >> > This
>> >> > > >> > > >> >> > > is
>> >> > > >> > > >> >> > > > > > > > important
>> >> > > >> > > >> >> > > > > > > > > > as it minimizes the size of the
>> >> dependency
>> >> > > of
>> >> > > >> > Drill
>> >> > > >> > > >> >> > > application
>> >> > > >> > > >> >> > > > > > > users.
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > >
>> >> > > >> > > >> >> > > > > > > >
>> >> > > >> > > >> >> > > > > > >
>> >> > > >> > > >> >> > > > > >
>> >> > > >> > > >> >> > > > >
>> >> > > >> > > >> >> > > >
>> >> > > >> > > >> >> > >
>> >> > > >> > > >> >> >
>> >> > > >> > > >> >>
>> >> > > >> > > >>
>> >> > > >> > >
>> >> > > >> >
>> >> > > >>
>> >> > >
>> >> >
>> >>
>> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> >> > > >> > > >> >> > > > > > > > > > size (44664121) too large. Max. is
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > >
>> >> > > >> > > >> >> > > > > > > >
>> >> > > >> > > >> >> > > > > > >
>> >> > > >> > > >> >> > > > > >
>> >> > > >> > > >> >> > > > >
>> >> > > >> > > >> >> > > >
>> >> > > >> > > >> >> > >
>> >> > > >> > > >> >> >
>> >> > > >> > > >> >>
>> >> > > >> > > >>
>> >> > > >> > >
>> >> > > >> >
>> >> > > >>
>> >> > >
>> >> >
>> >>
>> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > Administrators-MacBook-Pro-144:incubator-drill
>> >> > > >> > > asinha$
>> >> > > >> > > >> >> mvn
>> >> > > >> > > >> >> > > > > > --version
>> >> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
>> >> > > >> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
>> >> > > >> > > >> >> > > > > > > > 2013-02-19
>> >> > > >> > > >> >> > > > > > > > > > 05:51:28-0800)
>> >> > > >> > > >> >> > > > > > > > > > Maven home:
>> /opt/local/share/java/maven3
>> >> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor:
>> Oracle
>> >> > > >> > Corporation
>> >> > > >> > > >> >> > > > > > > > > > Java home:
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > >
>> >> > > >> > > >>
>> >> > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
>> >> > > >> > > >> >> > > > > > > > > > Default locale: en_US, platform
>> >> encoding:
>> >> > > UTF-8
>> >> > > >> > > >> >> > > > > > > > > > OS name: "mac os x", version:
>> "10.9.5",
>> >> > > arch:
>> >> > > >> > > >> "x86_64",
>> >> > > >> > > >> >> > > family:
>> >> > > >> > > >> >> > > > > > "mac"
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM,
>> Jacques
>> >> > > >> Nadeau <
>> >> > > >> > > >> >> > > > > > jacques@dremio.com>
>> >> > > >> > > >> >> > > > > > > > > > wrote:
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > > > Build back to normal. It looks like
>> >> the
>> >> > > >> Apache
>> >> > > >> > > >> server
>> >> > > >> > > >> >> was
>> >> > > >> > > >> >> > > > using
>> >> > > >> > > >> >> > > > > > an
>> >> > > >> > > >> >> > > > > > > > old
>> >> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I switched
>> to
>> >> > > >> something
>> >> > > >> > > more
>> >> > > >> > > >> >> > recent,
>> >> > > >> > > >> >> > > > the
>> >> > > >> > > >> >> > > > > > > build
>> >> > > >> > > >> >> > > > > > > > > > > passed.
>> >> > > >> > > >> >> > > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > > > --
>> >> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau
>> >> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
>> >> > > >> > > >> >> > > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM,
>> >> Jacques
>> >> > > >> > Nadeau <
>> >> > > >> > > >> >> > > > > > > jacques@dremio.com>
>> >> > > >> > > >> >> > > > > > > > > > > wrote:
>> >> > > >> > > >> >> > > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge caused the
>> Apache
>> >> > > build
>> >> > > >> to
>> >> > > >> > > fail.
>> >> > > >> > > >> >> > > > > > > Investigating...
>> >> > > >> > > >> >> > > > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > > > > --
>> >> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
>> >> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
>> >> > > >> > > >> >> > > > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM,
>> >> > Jacques
>> >> > > >> > > Nadeau <
>> >> > > >> > > >> >> > > > > > > > jacques@dremio.com>
>> >> > > >> > > >> >> > > > > > > > > > > > wrote:
>> >> > > >> > > >> >> > > > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > > > >> I got clean regression runs as
>> >> well.
>> >> > > I've
>> >> > > >> > > merged
>> >> > > >> > > >> the
>> >> > > >> > > >> >> > > > patch.
>> >> > > >> > > >> >> > > > > > > > > > > >>
>> >> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to start the
>> >> release
>> >> > > >> > process?
>> >> > > >> > > >> >> > > > > > > > > > > >>
>> >> > > >> > > >> >> > > > > > > > > > > >> --
>> >> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
>> >> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
>> >> > > >> > > >> >> > > > > > > > > > > >>
>> >> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42
>> PM,
>> >> > > Abhishek
>> >> > > >> > > >> Girish <
>> >> > > >> > > >> >> > > > > > > > agirish@mapr.com
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > > > >> wrote:
>> >> > > >> > > >> >> > > > > > > > > > > >>
>> >> > > >> > > >> >> > > > > > > > > > > >>> Had two clean Functional runs.
>> >> TPC-H
>> >> > > >> SF100
>> >> > > >> > > was
>> >> > > >> > > >> also
>> >> > > >> > > >> >> > > > > > successful.
>> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07
>> PM,
>> >> > > rahul
>> >> > > >> > > >> >> challapalli <
>> >> > > >> > > >> >> > > > > > > > > > > >>> challapallirahul@gmail.com>
>> >> wrote:
>> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a functional run
>> with
>> >> > > your
>> >> > > >> > > branch.
>> >> > > >> > > >> >> Will
>> >> > > >> > > >> >> > > let
>> >> > > >> > > >> >> > > > > you
>> >> > > >> > > >> >> > > > > > > know
>> >> > > >> > > >> >> > > > > > > > > > once
>> >> > > >> > > >> >> > > > > > > > > > > it
>> >> > > >> > > >> >> > > > > > > > > > > >>> > finishes
>> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul
>> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56
>> PM,
>> >> > > >> Jacques
>> >> > > >> > > >> Nadeau <
>> >> > > >> > > >> >> > > > > > > > > > jacques@dremio.com>
>> >> > > >> > > >> >> > > > > > > > > > > >>> > wrote:
>> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I
>> have
>> >> an
>> >> > > >> > updated
>> >> > > >> > > >> patch
>> >> > > >> > > >> >> > for
>> >> > > >> > > >> >> > > > 4291
>> >> > > >> > > >> >> > > > > > > that
>> >> > > >> > > >> >> > > > > > > > is
>> >> > > >> > > >> >> > > > > > > > > > > >>> ready.
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems
>> that
>> >> > > >> something
>> >> > > >> > > isn't
>> >> > > >> > > >> >> > > working
>> >> > > >> > > >> >> > > > > with
>> >> > > >> > > >> >> > > > > > > our
>> >> > > >> > > >> >> > > > > > > > > > > >>> extended
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't been
>> able
>> >> to
>> >> > > run
>> >> > > >> an
>> >> > > >> > > >> extended
>> >> > > >> > > >> >> > > > > > regression.
>> >> > > >> > > >> >> > > > > > > > > Unit
>> >> > > >> > > >> >> > > > > > > > > > > >>> tests
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else
>> possibly
>> >> > > able
>> >> > > >> to
>> >> > > >> > > run a
>> >> > > >> > > >> >> > > > regression
>> >> > > >> > > >> >> > > > > > > suite
>> >> > > >> > > >> >> > > > > > > > > > > against
>> >> > > >> > > >> >> > > > > > > > > > > >>> > this
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can
>> confirm
>> >> > > things
>> >> > > >> > look
>> >> > > >> > > >> good
>> >> > > >> > > >> >> and
>> >> > > >> > > >> >> > > > start
>> >> > > >> > > >> >> > > > > > the
>> >> > > >> > > >> >> > > > > > > > > > release
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > process?
>> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > thanks,
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques
>> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>> >> > > >> > > >> >> > > > >
>> >> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
>> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > --
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
>> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at
>> 11:20
>> >> > AM,
>> >> > > >> > Jacques
>> >> > > >> > > >> >> Nadeau
>> >> > > >> > > >> >> > <
>> >> > > >> > > >> >> > > > > > > > > > > jacques@dremio.com
>> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the main things
>> are
>> >> > 4196
>> >> > > >> and
>> >> > > >> > > 4291
>> >> > > >> > > >> >> > should
>> >> > > >> > > >> >> > > be
>> >> > > >> > > >> >> > > > > > > > > completed.
>> >> > > >> > > >> >> > > > > > > > > > I
>> >> > > >> > > >> >> > > > > > > > > > > >>> know
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > Amit
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to reproduce
>> 4196
>> >> > > locally
>> >> > > >> > this
>> >> > > >> > > >> >> weekend
>> >> > > >> > > >> >> > > so
>> >> > > >> > > >> >> > > > I
>> >> > > >> > > >> >> > > > > > > think
>> >> > > >> > > >> >> > > > > > > > > > we're
>> >> > > >> > > >> >> > > > > > > > > > > >>> close
>> >> > > >> > > >> >> > > > > > > > > > > >>> > on
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > > that.
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > > --
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder,
>> Dremio
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at
>> >> 10:14
>> >> > > AM,
>> >> > > >> > > Zelaine
>> >> > > >> > > >> >> Fong
>> >> > > >> > > >> >> > <
>> >> > > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
>> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on this?
>> >> What's
>> >> > > >> > blocking
>> >> > > >> > > us
>> >> > > >> > > >> >> from
>> >> > > >> > > >> >> > > > taking
>> >> > > >> > > >> >> > > > > > > this
>> >> > > >> > > >> >> > > > > > > > > to a
>> >> > > >> > > >> >> > > > > > > > > > > >>> vote --
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > the
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join issues?
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at
>> >> 4:18
>> >> > > PM,
>> >> > > >> > > Jacques
>> >> > > >> > > >> >> > Nadeau
>> >> > > >> > > >> >> > > <
>> >> > > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the issues
>> that
>> >> > were
>> >> > > >> > > >> mentioned in
>> >> > > >> > > >> >> > > this
>> >> > > >> > > >> >> > > > > > thread
>> >> > > >> > > >> >> > > > > > > > > along
>> >> > > >> > > >> >> > > > > > > > > > > >>> with a
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed
>> categorization:
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
>> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4190
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
>> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4196
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
>> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
>> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4256
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
>> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4278
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
>> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
>> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4247
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
>> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4203
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> https://issues.apache.org/jira/browse/DRILL-4266
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
>> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do others think?
>> >> Let's
>> >> > > try
>> >> > > >> to
>> >> > > >> > > get
>> >> > > >> > > >> the
>> >> > > >> > > >> >> > > > > blockers
>> >> > > >> > > >> >> > > > > > > > > wrapped
>> >> > > >> > > >> >> > > > > > > > > > up
>> >> > > >> > > >> >> > > > > > > > > > > >>> in
>> >> > > >> > > >> >> > > > > > > > > > > >>> > the
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> next
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two and start a
>> >> > > release
>> >> > > >> > > vote...
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > --
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder,
>> >> Dremio
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at
>> >> 1:48
>> >> > > PM,
>> >> > > >> > > Jason
>> >> > > >> > > >> >> > > Altekruse
>> >> > > >> > > >> >> > > > <
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> altekrusejason@gmail.com>
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the allocator
>> >> > changes
>> >> > > >> > merged
>> >> > > >> > > and
>> >> > > >> > > >> >> > about
>> >> > > >> > > >> >> > > a
>> >> > > >> > > >> >> > > > > > month
>> >> > > >> > > >> >> > > > > > > > > since
>> >> > > >> > > >> >> > > > > > > > > > > the
>> >> > > >> > > >> >> > > > > > > > > > > >>> last
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it would be
>> >> good
>> >> > to
>> >> > > >> > start
>> >> > > >> > > a
>> >> > > >> > > >> vote
>> >> > > >> > > >> >> > > > soon. I
>> >> > > >> > > >> >> > > > > > > would
>> >> > > >> > > >> >> > > > > > > > > > like
>> >> > > >> > > >> >> > > > > > > > > > > to
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release manager.
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that there
>> were
>> >> > some
>> >> > > >> > issues
>> >> > > >> > > >> that
>> >> > > >> > > >> >> > were
>> >> > > >> > > >> >> > > > > > > > identified
>> >> > > >> > > >> >> > > > > > > > > > > after
>> >> > > >> > > >> >> > > > > > > > > > > >>> the
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I
>> >> think
>> >> > > that
>> >> > > >> > > these
>> >> > > >> > > >> >> > issues
>> >> > > >> > > >> >> > > > > should
>> >> > > >> > > >> >> > > > > > > be
>> >> > > >> > > >> >> > > > > > > > > > fixed
>> >> > > >> > > >> >> > > > > > > > > > > >>> before
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > we
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release candidate.
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at the
>> >> > > associated
>> >> > > >> > > JIRAs
>> >> > > >> > > >> it
>> >> > > >> > > >> >> > > looked
>> >> > > >> > > >> >> > > > > > like
>> >> > > >> > > >> >> > > > > > > > > there
>> >> > > >> > > >> >> > > > > > > > > > > was
>> >> > > >> > > >> >> > > > > > > > > > > >>> a
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix just
>> >> > > adjusting
>> >> > > >> > the
>> >> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node
>> >> > > >> > > >> >> > > > > > > > > > > >>> > option,
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> and
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more involved
>> work
>> >> > to
>> >> > > >> > change
>> >> > > >> > > >> how we
>> >> > > >> > > >> >> > > > > determine
>> >> > > >> > > >> >> > > > > > > the
>> >> > > >> > > >> >> > > > > > > > > > > correct
>> >> > > >> > > >> >> > > > > > > > > > > >>> > time
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > to
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill during
>> external
>> >> > > sort. I
>> >> > > >> > > >> believe it
>> >> > > >> > > >> >> > > makes
>> >> > > >> > > >> >> > > > > > sense
>> >> > > >> > > >> >> > > > > > > > to
>> >> > > >> > > >> >> > > > > > > > > > make
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > external
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well with the
>> >> newly
>> >> > > >> > improved
>> >> > > >> > > >> memory
>> >> > > >> > > >> >> > > > > > accounting
>> >> > > >> > > >> >> > > > > > > > > before
>> >> > > >> > > >> >> > > > > > > > > > > >>> > cutting a
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not
>> >> sure
>> >> > > how
>> >> > > >> > much
>> >> > > >> > > >> work
>> >> > > >> > > >> >> is
>> >> > > >> > > >> >> > > > left
>> >> > > >> > > >> >> > > > > to
>> >> > > >> > > >> >> > > > > > > be
>> >> > > >> > > >> >> > > > > > > > > done
>> >> > > >> > > >> >> > > > > > > > > > > >>> there.
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond with
>> >> your
>> >> > > >> > thoughts
>> >> > > >> > > on
>> >> > > >> > > >> a
>> >> > > >> > > >> >> > > release
>> >> > > >> > > >> >> > > > > > soon
>> >> > > >> > > >> >> > > > > > > > and
>> >> > > >> > > >> >> > > > > > > > > > any
>> >> > > >> > > >> >> > > > > > > > > > > >>> JIRAs
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > you
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to include in
>> the
>> >> > > >> release.
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
>> >> > > >> > > >> >> > > > > > > >
>> >> > > https://issues.apache.org/jira/browse/DRILL-4243
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> > >
>> >> > > >> > > >> >> > > > > > > > > > > >>> >
>> >> > > >> > > >> >> > > > > > > > > > > >>>
>> >> > > >> > > >> >> > > > > > > > > > > >>
>> >> > > >> > > >> >> > > > > > > > > > > >>
>> >> > > >> > > >> >> > > > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > > >
>> >> > > >> > > >> >> > > > > > > > >
>> >> > > >> > > >> >> > > > > > > >
>> >> > > >> > > >> >> > > > > > >
>> >> > > >> > > >> >> > > > > >
>> >> > > >> > > >> >> > > > >
>> >> > > >> > > >> >> > > >
>> >> > > >> > > >> >> > >
>> >> > > >> > > >> >> >
>> >> > > >> > > >> >>
>> >> > > >> > > >>
>> >> > > >> > >
>> >> > > >> >
>> >> > > >>
>> >> > >
>> >> >
>> >>
>>
>>

Re: Time for a 1.5 release?

Posted by Jacques Nadeau <ja...@dremio.com>.

Jinfeng, do you agree that the changes are risky? I was going mostly by
commit messages.

Also just realized that the backwards compatibility fix to the storage
plugin interface is after these. Could create release branch from 03197d...
and then cherry pick the Venki change. Or could keep them all. Or
go without the compatibility fix....

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Sun, Jan 31, 2016 at 7:50 PM, Jinfeng Ni <ji...@gmail.com> wrote:

> Hi Jason,
>
> I'm fine that release 1.5.0 starts from 03197d0f2.  One thing I need
> point out is that the pre-commit test suite has some testcase's plan
> output modified because of the partition pruning change.  If we run
> pre-commit against the candidate release for 1.5.0, we have to use a
> prior version of pre-commit suite.
>
>
>
> On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse
> <al...@gmail.com> wrote:
> > That sounds reasonable to me. I'll start preparing a release candidate.
> >
> > Jinfeng, are you okay waiting for the next release to include this
> change?
> >
> > On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <ja...@dremio.com>
> wrote:
> >
> >> Hey Jason,
> >>
> >> Can you start the release vote? We've let this drag on too long.
> >>
> >> I suggest we start from 03197d0f2. It seems like the partition pruning
> >> changes are pretty complicated and on a pretty main path to add right
> >> before the vote. Since Hsuan's fix (pr 349) hasn't been reviewed, I
> think
> >> we should start the vote without it.
> >>
> >> Jacques
> >>
> >>
> >> --
> >> Jacques Nadeau
> >> CTO and Co-Founder, Dremio
> >>
> >> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <hy...@maprtech.com>
> wrote:
> >>
> >> > Yes, just send a pull request.
> >> > https://github.com/apache/drill/pull/349
> >> >
> >> > Can Jinfeng review it?
> >> >
> >> > Thanks,
> >> >
> >> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <ji...@gmail.com>
> >> wrote:
> >> >
> >> > > Sean probably is quite close to find a fix for this issue. Let's see
> >> > > whether he can post a patch soon.
> >> > >
> >> > >
> >> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
> >> > > <ch...@gmail.com> wrote:
> >> > > > DRILL-4323 only happens when assertions are turned on (my install
> >> > script
> >> > > > automatically enables assertions).
> >> > > >
> >> > > > - Rahul
> >> > > >
> >> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <
> zfong@maprtech.com>
> >> > > wrote:
> >> > > >
> >> > > >> Rahul,
> >> > > >>
> >> > > >> Let us know if you can reproduce the issue with assertions turned
> >> OFF.
> >> > > If
> >> > > >> you cannot, then I think this should not be a blocker for 1.5,
> and
> >> > Hsuan
> >> > > >> will continue to look into fixing this.
> >> > > >>
> >> > > >> -- Zelaine
> >> > > >>
> >> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
> >> > knguyen@maprtech.com>
> >> > > >> wrote:
> >> > > >>
> >> > > >> > I do see the same error with assertion turned on.
> >> > > >> >
> >> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <
> >> jinfengni99@gmail.com>
> >> > > >> wrote:
> >> > > >> >
> >> > > >> > > Krystal, by any chance, did you turn on assertion when you
> start
> >> > > >> > drillbit?
> >> > > >> > >
> >> > > >> > > I think the error in drill-4323 will be raised when
> assertion =
> >> > on.
> >> > > >> > >
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
> >> > > knguyen@maprtech.com>
> >> > > >> > > wrote:
> >> > > >> > > > I am running drill-1.4 and is able to run count(*)
> >> successfully.
> >> > > >> > > >
> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter
> session
> >> > set
> >> > > >> > > > `store.hive.optimize_scan_with_native_readers` = true;
> >> > > >> > > >
> >> > +-------+--------------------------------------------------------+
> >> > > >> > > > |  ok   |                        summary
> >> >  |
> >> > > >> > > >
> >> > +-------+--------------------------------------------------------+
> >> > > >> > > > | true  | store.hive.optimize_scan_with_native_readers
> >> > updated.  |
> >> > > >> > > >
> >> > +-------+--------------------------------------------------------+
> >> > > >> > > > 1 row selected (0.374 seconds)
> >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select
> >> count(*)
> >> > > from
> >> > > >> > > > lineitem_parquet_partitioned_hive_string;
> >> > > >> > > > +---------+
> >> > > >> > > > | EXPR$0  |
> >> > > >> > > > +---------+
> >> > > >> > > > | 60175   |
> >> > > >> > > > +---------+
> >> > > >> > > > 1 row selected (2.986 seconds)
> >> > > >> > > >
> >> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
> >> > > jinfengni99@gmail.com>
> >> > > >> > > wrote:
> >> > > >> > > >
> >> > > >> > > >> Venki and I did some investigation for DRILL-4323. The
> issue
> >> > > >> reported
> >> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as well.
> Seems
> >> > to
> >> > > us
> >> > > >> > > >> this is not a regression from 1.4.0; it's a regression
> from
> >> > 1.3.0
> >> > > >> > > >> probably.
> >> > > >> > > >>
> >> > > >> > > >> DRILL-4083 makes the planner to use DrillHiveNativeReader
> in
> >> > > stead
> >> > > >> of
> >> > > >> > > >> HiveReader for "select count(*) from hive_table" query.
> >> > However,
> >> > > >> the
> >> > > >> > > >> Project after the scan produces empty schema.  Before
> >> > DRILL-4083,
> >> > > >> > > >> Drill uses HiveScan, which works fine.
> >> > > >> > > >>
> >> > > >> > > >>
> >> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
> >> > > >> > > >> <al...@gmail.com> wrote:
> >> > > >> > > >> > We could revert the change to the StoragePlugin
> interface
> >> to
> >> > > add
> >> > > >> the
> >> > > >> > > >> > getLogicalOptimizerRules
> >> > > >> > > >> > and getPhysicalOptimizerRules, and only put those in
> >> > > >> > > >> AbstractStoragePlugin
> >> > > >> > > >> > with default implementations.
> >> > > >> > > >> >
> >> > > >> > > >> > We could then update the docs to tell users to not
> >> implement
> >> > > the
> >> > > >> > > >> interface
> >> > > >> > > >> > directly, and instead extend the abstract class, which I
> >> > > believe
> >> > > >> > would
> >> > > >> > > >> > allow us to make changes like this in the future without
> >> > > breaking
> >> > > >> > > >> plugins.
> >> > > >> > > >> > At a major release version we could then just pull up
> all
> >> of
> >> > > the
> >> > > >> new
> >> > > >> > > >> > methods we added to the abstract class into the
> interface,
> >> or
> >> > > just
> >> > > >> > get
> >> > > >> > > >> rid
> >> > > >> > > >> > of it all-together.
> >> > > >> > > >> >
> >> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
> >> > > >> > > >> venki.korukanti@gmail.com>
> >> > > >> > > >> > wrote:
> >> > > >> > > >> >
> >> > > >> > > >> >> Here is the partial fix:
> >> > > >> > > >> >>
> >> > > >> > > >> >>
> >> > > >> > > >>
> >> > > >> > >
> >> > > >> >
> >> > > >>
> >> > >
> >> >
> >>
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> >> > > >> > > >> >>
> >> > > >> > > >> >> If the existing StragePlugins are implementing
> >> > > >> > AbstractStoragePlugin
> >> > > >> > > >> class
> >> > > >> > > >> >> this fix should resolve the backward compat issue. If
> they
> >> > are
> >> > > >> > > >> implementing
> >> > > >> > > >> >> StoragePlugin interface directly then it won't resolve
> the
> >> > > issue.
> >> > > >> > Not
> >> > > >> > > >> sure
> >> > > >> > > >> >> how to handle it in that case. Let me know if there any
> >> ways
> >> > > to
> >> > > >> > > resolve
> >> > > >> > > >> >> this issue completely.
> >> > > >> > > >> >>
> >> > > >> > > >> >> Thanks
> >> > > >> > > >> >> Venki
> >> > > >> > > >> >>
> >> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <
> >> > > asinha@maprtech.com
> >> > > >> >
> >> > > >> > > >> wrote:
> >> > > >> > > >> >>
> >> > > >> > > >> >> > I was able to do the build with maven 3.3.9 (after
> doing
> >> > > 'brew
> >> > > >> > > install
> >> > > >> > > >> >> > maven').  Here are my env variables related to maven:
> >> > > >> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> >> > > >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> >> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> >> > > >> > > >> >> >
> >> > > >> > > >> >> >
> >> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <
> >> > > >> > > jacques@dremio.com>
> >> > > >> > > >> >> > wrote:
> >> > > >> > > >> >> >
> >> > > >> > > >> >> > > Aman, for reference on the Maven version issue. You
> >> can
> >> > > see
> >> > > >> the
> >> > > >> > > >> output
> >> > > >> > > >> >> of
> >> > > >> > > >> >> > > the shade plugin between the two Apache Builds. If
> you
> >> > > search
> >> > > >> > for
> >> > > >> > > >> >> > > maven-shade-plugin you'll see that build 647 (maven
> >> > 3.0.5)
> >> > > >> > fails
> >> > > >> > > >> >> because
> >> > > >> > > >> >> > it
> >> > > >> > > >> >> > > includes a bunch of hadoop (and other libraries
> that
> >> are
> >> > > >> listed
> >> > > >> > > as
> >> > > >> > > >> >> being
> >> > > >> > > >> >> > > excluded). If you look at build 648 (maven 3.3.3),
> >> those
> >> > > >> items
> >> > > >> > > are
> >> > > >> > > >> >> > included
> >> > > >> > > >> >> > > so that the build completes successfully.
> >> > > >> > > >> >> > >
> >> > > >> > > >> >> > >
> >> https://builds.apache.org/job/drill-scm/647/consoleText
> >> > > >> > > >> >> > >
> >> https://builds.apache.org/job/drill-scm/648/consoleText
> >> > > >> > > >> >> > >
> >> > > >> > > >> >> > > --
> >> > > >> > > >> >> > > Jacques Nadeau
> >> > > >> > > >> >> > > CTO and Co-Founder, Dremio
> >> > > >> > > >> >> > >
> >> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli
> <
> >> > > >> > > >> >> > > challapallirahul@gmail.com> wrote:
> >> > > >> > > >> >> > >
> >> > > >> > > >> >> > > > Jason,
> >> > > >> > > >> >> > > >
> >> > > >> > > >> >> > > > As of now I haven't narrowed down the exact
> commit
> >> > which
> >> > > >> > caused
> >> > > >> > > >> >> > > DRILL-4323
> >> > > >> > > >> >> > > > and I do not know the scope of the fix.
> >> > > >> > > >> >> > > >
> >> > > >> > > >> >> > > > - Rahul
> >> > > >> > > >> >> > > >
> >> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse
> <
> >> > > >> > > >> >> > > altekrusejason@gmail.com
> >> > > >> > > >> >> > > > >
> >> > > >> > > >> >> > > > wrote:
> >> > > >> > > >> >> > > >
> >> > > >> > > >> >> > > > > I think it makes sense to wait for Venki's fix
> of
> >> > the
> >> > > >> API,
> >> > > >> > as
> >> > > >> > > >> >> > > releasing a
> >> > > >> > > >> >> > > > > version with the change would make
> >> > > reverting/refactoring
> >> > > >> > the
> >> > > >> > > API
> >> > > >> > > >> >> > > breaking
> >> > > >> > > >> >> > > > > change less useful.
> >> > > >> > > >> >> > > > >
> >> > > >> > > >> >> > > > > Do you have any idea about the scope of fixing
> the
> >> > > hive
> >> > > >> > > native
> >> > > >> > > >> >> > reader,
> >> > > >> > > >> >> > > or
> >> > > >> > > >> >> > > > > which change since the 1.4 release broke it? Is
> >> > > someone
> >> > > >> > > >> available
> >> > > >> > > >> >> to
> >> > > >> > > >> >> > > look
> >> > > >> > > >> >> > > > > at it today?
> >> > > >> > > >> >> > > > >
> >> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul
> >> challapalli <
> >> > > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
> >> > > >> > > >> >> > > > >
> >> > > >> > > >> >> > > > > > Just found an issue with hive native parquet
> >> > reader
> >> > > >> > > >> (DRILL-4323).
> >> > > >> > > >> >> > > This
> >> > > >> > > >> >> > > > > is a
> >> > > >> > > >> >> > > > > > regression from 1.4 and in my opinion should
> be
> >> > > treated
> >> > > >> > as
> >> > > >> > > a
> >> > > >> > > >> >> > blocker.
> >> > > >> > > >> >> > > > > >
> >> > > >> > > >> >> > > > > > - Rahul
> >> > > >> > > >> >> > > > > >
> >> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki
> >> Korukanti <
> >> > > >> > > >> >> > > > > > venki.korukanti@gmail.com>
> >> > > >> > > >> >> > > > > > wrote:
> >> > > >> > > >> >> > > > > >
> >> > > >> > > >> >> > > > > > > Sorry I didn't realize I was breaking the
> >> public
> >> > > >> > > interface
> >> > > >> > > >> API.
> >> > > >> > > >> >> > Let
> >> > > >> > > >> >> > > > me
> >> > > >> > > >> >> > > > > > work
> >> > > >> > > >> >> > > > > > > on a patch to make it backward compatible.
> >> > > >> > > >> >> > > > > > >
> >> > > >> > > >> >> > > > > > > Thanks
> >> > > >> > > >> >> > > > > > > Venki
> >> > > >> > > >> >> > > > > > >
> >> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven
> >> > Phillips
> >> > > <
> >> > > >> > > >> >> > > steven@dremio.com
> >> > > >> > > >> >> > > > >
> >> > > >> > > >> >> > > > > > > wrote:
> >> > > >> > > >> >> > > > > > >
> >> > > >> > > >> >> > > > > > > > I just wanted to bring up an issue that I
> >> just
> >> > > now
> >> > > >> > > >> >> discovered,
> >> > > >> > > >> >> > > that
> >> > > >> > > >> >> > > > > has
> >> > > >> > > >> >> > > > > > > > caused me a fair amount of grief.
> >> > > >> > > >> >> > > > > > > >
> >> > > >> > > >> >> > > > > > > >
> >> > > https://github.com/apache/drill/pull/300/commits
> >> > > >> > > >> >> > > > > > > >
> >> > > >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing API, and
> >> > causes
> >> > > >> > > >> >> StoragePlugins
> >> > > >> > > >> >> > > > that
> >> > > >> > > >> >> > > > > > were
> >> > > >> > > >> >> > > > > > > > compiled against currently released
> versions
> >> > of
> >> > > >> Drill
> >> > > >> > > to
> >> > > >> > > >> no
> >> > > >> > > >> >> > > longer
> >> > > >> > > >> >> > > > > > > > functional properly. I would prefer that
> >> this
> >> > > >> > breaking
> >> > > >> > > >> change
> >> > > >> > > >> >> > be
> >> > > >> > > >> >> > > > > > modified
> >> > > >> > > >> >> > > > > > > > to be backward compatible if possible.
> >> > > >> > > >> >> > > > > > > >
> >> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason
> >> > > Altekruse <
> >> > > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
> >> > > >> > > >> >> > > > > > > > wrote:
> >> > > >> > > >> >> > > > > > > >
> >> > > >> > > >> >> > > > > > > > > Hi Aman,
> >> > > >> > > >> >> > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > This is the failure that he was
> seeing. He
> >> > > >> figured
> >> > > >> > > out
> >> > > >> > > >> that
> >> > > >> > > >> >> > the
> >> > > >> > > >> >> > > > new
> >> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were not being
> >> > > respected
> >> > > >> > when
> >> > > >> > > the
> >> > > >> > > >> >> > build
> >> > > >> > > >> >> > > > was
> >> > > >> > > >> >> > > > > > run
> >> > > >> > > >> >> > > > > > > > with
> >> > > >> > > >> >> > > > > > > > > an older Maven version, causing the jar
> >> size
> >> > > to
> >> > > >> > > increase
> >> > > >> > > >> >> > > > > > significantly.
> >> > > >> > > >> >> > > > > > > > He
> >> > > >> > > >> >> > > > > > > > > added an enforcer to make sure the JAR
> >> > didn't
> >> > > >> grow
> >> > > >> > > >> >> > > unexpectedly.
> >> > > >> > > >> >> > > > > Can
> >> > > >> > > >> >> > > > > > > you
> >> > > >> > > >> >> > > > > > > > > try to update your maven version and
> >> re-run
> >> > > the
> >> > > >> > > build?
> >> > > >> > > >> >> > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > - Jason
> >> > > >> > > >> >> > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman
> >> > Sinha <
> >> > > >> > > >> >> > > > asinha@maprtech.com>
> >> > > >> > > >> >> > > > > > > > wrote:
> >> > > >> > > >> >> > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the following
> >> build
> >> > > >> failure
> >> > > >> > > on
> >> > > >> > > >> the
> >> > > >> > > >> >> > > latest
> >> > > >> > > >> >> > > > > > > master
> >> > > >> > > >> >> > > > > > > > > > branch...is this what you saw for the
> >> > Apache
> >> > > >> > build
> >> > > >> > > ?
> >> > > >> > > >> My
> >> > > >> > > >> >> > mvn
> >> > > >> > > >> >> > > > > > version
> >> > > >> > > >> >> > > > > > > > > output
> >> > > >> > > >> >> > > > > > > > > > is shown below.  Should we all be
> >> > upgrading
> >> > > to
> >> > > >> a
> >> > > >> > > newer
> >> > > >> > > >> >> mvn
> >> > > >> > > >> >> > ?
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > > [INFO] ---
> >> > > maven-enforcer-plugin:1.3.1:enforce
> >> > > >> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @
> >> > > drill-jdbc-all
> >> > > >> > ---
> >> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
> >> > > >> > > >> >> > > > > >
> >> org.apache.maven.plugins.enforcer.RequireFilesSize
> >> > > >> > > >> >> > > > > > > > > failed
> >> > > >> > > >> >> > > > > > > > > > with message:
> >> > > >> > > >> >> > > > > > > > > > The file
> >> drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> > > is
> >> > > >> > > outside
> >> > > >> > > >> the
> >> > > >> > > >> >> > > > > expected
> >> > > >> > > >> >> > > > > > > size
> >> > > >> > > >> >> > > > > > > > > > range.
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > >                   This is likely due
> to
> >> > you
> >> > > >> > adding
> >> > > >> > > new
> >> > > >> > > >> >> > > > > dependencies
> >> > > >> > > >> >> > > > > > > to
> >> > > >> > > >> >> > > > > > > > a
> >> > > >> > > >> >> > > > > > > > > > java-exec and not updating the
> excludes
> >> in
> >> > > this
> >> > > >> > > >> module.
> >> > > >> > > >> >> > This
> >> > > >> > > >> >> > > is
> >> > > >> > > >> >> > > > > > > > important
> >> > > >> > > >> >> > > > > > > > > > as it minimizes the size of the
> >> dependency
> >> > > of
> >> > > >> > Drill
> >> > > >> > > >> >> > > application
> >> > > >> > > >> >> > > > > > > users.
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > >
> >> > > >> > > >> >> > > > > > > >
> >> > > >> > > >> >> > > > > > >
> >> > > >> > > >> >> > > > > >
> >> > > >> > > >> >> > > > >
> >> > > >> > > >> >> > > >
> >> > > >> > > >> >> > >
> >> > > >> > > >> >> >
> >> > > >> > > >> >>
> >> > > >> > > >>
> >> > > >> > >
> >> > > >> >
> >> > > >>
> >> > >
> >> >
> >>
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> > > >> > > >> >> > > > > > > > > > size (44664121) too large. Max. is
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > >
> >> > > >> > > >> >> > > > > > > >
> >> > > >> > > >> >> > > > > > >
> >> > > >> > > >> >> > > > > >
> >> > > >> > > >> >> > > > >
> >> > > >> > > >> >> > > >
> >> > > >> > > >> >> > >
> >> > > >> > > >> >> >
> >> > > >> > > >> >>
> >> > > >> > > >>
> >> > > >> > >
> >> > > >> >
> >> > > >>
> >> > >
> >> >
> >>
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > >
> >> > > Administrators-MacBook-Pro-144:incubator-drill
> >> > > >> > > asinha$
> >> > > >> > > >> >> mvn
> >> > > >> > > >> >> > > > > > --version
> >> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
> >> > > >> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> >> > > >> > > >> >> > > > > > > > 2013-02-19
> >> > > >> > > >> >> > > > > > > > > > 05:51:28-0800)
> >> > > >> > > >> >> > > > > > > > > > Maven home:
> /opt/local/share/java/maven3
> >> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor:
> Oracle
> >> > > >> > Corporation
> >> > > >> > > >> >> > > > > > > > > > Java home:
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > >
> >> > > >> > > >>
> >> > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> >> > > >> > > >> >> > > > > > > > > > Default locale: en_US, platform
> >> encoding:
> >> > > UTF-8
> >> > > >> > > >> >> > > > > > > > > > OS name: "mac os x", version:
> "10.9.5",
> >> > > arch:
> >> > > >> > > >> "x86_64",
> >> > > >> > > >> >> > > family:
> >> > > >> > > >> >> > > > > > "mac"
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM,
> Jacques
> >> > > >> Nadeau <
> >> > > >> > > >> >> > > > > > jacques@dremio.com>
> >> > > >> > > >> >> > > > > > > > > > wrote:
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > > > Build back to normal. It looks like
> >> the
> >> > > >> Apache
> >> > > >> > > >> server
> >> > > >> > > >> >> was
> >> > > >> > > >> >> > > > using
> >> > > >> > > >> >> > > > > > an
> >> > > >> > > >> >> > > > > > > > old
> >> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I switched
> to
> >> > > >> something
> >> > > >> > > more
> >> > > >> > > >> >> > recent,
> >> > > >> > > >> >> > > > the
> >> > > >> > > >> >> > > > > > > build
> >> > > >> > > >> >> > > > > > > > > > > passed.
> >> > > >> > > >> >> > > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > > > --
> >> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau
> >> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
> >> > > >> > > >> >> > > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM,
> >> Jacques
> >> > > >> > Nadeau <
> >> > > >> > > >> >> > > > > > > jacques@dremio.com>
> >> > > >> > > >> >> > > > > > > > > > > wrote:
> >> > > >> > > >> >> > > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge caused the
> Apache
> >> > > build
> >> > > >> to
> >> > > >> > > fail.
> >> > > >> > > >> >> > > > > > > Investigating...
> >> > > >> > > >> >> > > > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > > > > --
> >> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
> >> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
> >> > > >> > > >> >> > > > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM,
> >> > Jacques
> >> > > >> > > Nadeau <
> >> > > >> > > >> >> > > > > > > > jacques@dremio.com>
> >> > > >> > > >> >> > > > > > > > > > > > wrote:
> >> > > >> > > >> >> > > > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > > > >> I got clean regression runs as
> >> well.
> >> > > I've
> >> > > >> > > merged
> >> > > >> > > >> the
> >> > > >> > > >> >> > > > patch.
> >> > > >> > > >> >> > > > > > > > > > > >>
> >> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to start the
> >> release
> >> > > >> > process?
> >> > > >> > > >> >> > > > > > > > > > > >>
> >> > > >> > > >> >> > > > > > > > > > > >> --
> >> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
> >> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
> >> > > >> > > >> >> > > > > > > > > > > >>
> >> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42
> PM,
> >> > > Abhishek
> >> > > >> > > >> Girish <
> >> > > >> > > >> >> > > > > > > > agirish@mapr.com
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > > > >> wrote:
> >> > > >> > > >> >> > > > > > > > > > > >>
> >> > > >> > > >> >> > > > > > > > > > > >>> Had two clean Functional runs.
> >> TPC-H
> >> > > >> SF100
> >> > > >> > > was
> >> > > >> > > >> also
> >> > > >> > > >> >> > > > > > successful.
> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07
> PM,
> >> > > rahul
> >> > > >> > > >> >> challapalli <
> >> > > >> > > >> >> > > > > > > > > > > >>> challapallirahul@gmail.com>
> >> wrote:
> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a functional run
> with
> >> > > your
> >> > > >> > > branch.
> >> > > >> > > >> >> Will
> >> > > >> > > >> >> > > let
> >> > > >> > > >> >> > > > > you
> >> > > >> > > >> >> > > > > > > know
> >> > > >> > > >> >> > > > > > > > > > once
> >> > > >> > > >> >> > > > > > > > > > > it
> >> > > >> > > >> >> > > > > > > > > > > >>> > finishes
> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul
> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56
> PM,
> >> > > >> Jacques
> >> > > >> > > >> Nadeau <
> >> > > >> > > >> >> > > > > > > > > > jacques@dremio.com>
> >> > > >> > > >> >> > > > > > > > > > > >>> > wrote:
> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I
> have
> >> an
> >> > > >> > updated
> >> > > >> > > >> patch
> >> > > >> > > >> >> > for
> >> > > >> > > >> >> > > > 4291
> >> > > >> > > >> >> > > > > > > that
> >> > > >> > > >> >> > > > > > > > is
> >> > > >> > > >> >> > > > > > > > > > > >>> ready.
> >> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems
> that
> >> > > >> something
> >> > > >> > > isn't
> >> > > >> > > >> >> > > working
> >> > > >> > > >> >> > > > > with
> >> > > >> > > >> >> > > > > > > our
> >> > > >> > > >> >> > > > > > > > > > > >>> extended
> >> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't been
> able
> >> to
> >> > > run
> >> > > >> an
> >> > > >> > > >> extended
> >> > > >> > > >> >> > > > > > regression.
> >> > > >> > > >> >> > > > > > > > > Unit
> >> > > >> > > >> >> > > > > > > > > > > >>> tests
> >> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else
> possibly
> >> > > able
> >> > > >> to
> >> > > >> > > run a
> >> > > >> > > >> >> > > > regression
> >> > > >> > > >> >> > > > > > > suite
> >> > > >> > > >> >> > > > > > > > > > > against
> >> > > >> > > >> >> > > > > > > > > > > >>> > this
> >> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can
> confirm
> >> > > things
> >> > > >> > look
> >> > > >> > > >> good
> >> > > >> > > >> >> and
> >> > > >> > > >> >> > > > start
> >> > > >> > > >> >> > > > > > the
> >> > > >> > > >> >> > > > > > > > > > release
> >> > > >> > > >> >> > > > > > > > > > > >>> > > process?
> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > thanks,
> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques
> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
> >> > > >> > > >> >> > > > >
> >> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > --
> >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
> >> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at
> 11:20
> >> > AM,
> >> > > >> > Jacques
> >> > > >> > > >> >> Nadeau
> >> > > >> > > >> >> > <
> >> > > >> > > >> >> > > > > > > > > > > jacques@dremio.com
> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the main things
> are
> >> > 4196
> >> > > >> and
> >> > > >> > > 4291
> >> > > >> > > >> >> > should
> >> > > >> > > >> >> > > be
> >> > > >> > > >> >> > > > > > > > > completed.
> >> > > >> > > >> >> > > > > > > > > > I
> >> > > >> > > >> >> > > > > > > > > > > >>> know
> >> > > >> > > >> >> > > > > > > > > > > >>> > > Amit
> >> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to reproduce
> 4196
> >> > > locally
> >> > > >> > this
> >> > > >> > > >> >> weekend
> >> > > >> > > >> >> > > so
> >> > > >> > > >> >> > > > I
> >> > > >> > > >> >> > > > > > > think
> >> > > >> > > >> >> > > > > > > > > > we're
> >> > > >> > > >> >> > > > > > > > > > > >>> close
> >> > > >> > > >> >> > > > > > > > > > > >>> > on
> >> > > >> > > >> >> > > > > > > > > > > >>> > > > that.
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > > --
> >> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
> >> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder,
> Dremio
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at
> >> 10:14
> >> > > AM,
> >> > > >> > > Zelaine
> >> > > >> > > >> >> Fong
> >> > > >> > > >> >> > <
> >> > > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on this?
> >> What's
> >> > > >> > blocking
> >> > > >> > > us
> >> > > >> > > >> >> from
> >> > > >> > > >> >> > > > taking
> >> > > >> > > >> >> > > > > > > this
> >> > > >> > > >> >> > > > > > > > > to a
> >> > > >> > > >> >> > > > > > > > > > > >>> vote --
> >> > > >> > > >> >> > > > > > > > > > > >>> > > the
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join issues?
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at
> >> 4:18
> >> > > PM,
> >> > > >> > > Jacques
> >> > > >> > > >> >> > Nadeau
> >> > > >> > > >> >> > > <
> >> > > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the issues
> that
> >> > were
> >> > > >> > > >> mentioned in
> >> > > >> > > >> >> > > this
> >> > > >> > > >> >> > > > > > thread
> >> > > >> > > >> >> > > > > > > > > along
> >> > > >> > > >> >> > > > > > > > > > > >>> with a
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed
> categorization:
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4190
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4196
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4256
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4278
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4247
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4203
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> https://issues.apache.org/jira/browse/DRILL-4266
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do others think?
> >> Let's
> >> > > try
> >> > > >> to
> >> > > >> > > get
> >> > > >> > > >> the
> >> > > >> > > >> >> > > > > blockers
> >> > > >> > > >> >> > > > > > > > > wrapped
> >> > > >> > > >> >> > > > > > > > > > up
> >> > > >> > > >> >> > > > > > > > > > > >>> in
> >> > > >> > > >> >> > > > > > > > > > > >>> > the
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> next
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two and start a
> >> > > release
> >> > > >> > > vote...
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > --
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder,
> >> Dremio
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at
> >> 1:48
> >> > > PM,
> >> > > >> > > Jason
> >> > > >> > > >> >> > > Altekruse
> >> > > >> > > >> >> > > > <
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> altekrusejason@gmail.com>
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the allocator
> >> > changes
> >> > > >> > merged
> >> > > >> > > and
> >> > > >> > > >> >> > about
> >> > > >> > > >> >> > > a
> >> > > >> > > >> >> > > > > > month
> >> > > >> > > >> >> > > > > > > > > since
> >> > > >> > > >> >> > > > > > > > > > > the
> >> > > >> > > >> >> > > > > > > > > > > >>> last
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it would be
> >> good
> >> > to
> >> > > >> > start
> >> > > >> > > a
> >> > > >> > > >> vote
> >> > > >> > > >> >> > > > soon. I
> >> > > >> > > >> >> > > > > > > would
> >> > > >> > > >> >> > > > > > > > > > like
> >> > > >> > > >> >> > > > > > > > > > > to
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release manager.
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that there
> were
> >> > some
> >> > > >> > issues
> >> > > >> > > >> that
> >> > > >> > > >> >> > were
> >> > > >> > > >> >> > > > > > > > identified
> >> > > >> > > >> >> > > > > > > > > > > after
> >> > > >> > > >> >> > > > > > > > > > > >>> the
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I
> >> think
> >> > > that
> >> > > >> > > these
> >> > > >> > > >> >> > issues
> >> > > >> > > >> >> > > > > should
> >> > > >> > > >> >> > > > > > > be
> >> > > >> > > >> >> > > > > > > > > > fixed
> >> > > >> > > >> >> > > > > > > > > > > >>> before
> >> > > >> > > >> >> > > > > > > > > > > >>> > > we
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release candidate.
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at the
> >> > > associated
> >> > > >> > > JIRAs
> >> > > >> > > >> it
> >> > > >> > > >> >> > > looked
> >> > > >> > > >> >> > > > > > like
> >> > > >> > > >> >> > > > > > > > > there
> >> > > >> > > >> >> > > > > > > > > > > was
> >> > > >> > > >> >> > > > > > > > > > > >>> a
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix just
> >> > > adjusting
> >> > > >> > the
> >> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node
> >> > > >> > > >> >> > > > > > > > > > > >>> > option,
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> and
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more involved
> work
> >> > to
> >> > > >> > change
> >> > > >> > > >> how we
> >> > > >> > > >> >> > > > > determine
> >> > > >> > > >> >> > > > > > > the
> >> > > >> > > >> >> > > > > > > > > > > correct
> >> > > >> > > >> >> > > > > > > > > > > >>> > time
> >> > > >> > > >> >> > > > > > > > > > > >>> > > to
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill during
> external
> >> > > sort. I
> >> > > >> > > >> believe it
> >> > > >> > > >> >> > > makes
> >> > > >> > > >> >> > > > > > sense
> >> > > >> > > >> >> > > > > > > > to
> >> > > >> > > >> >> > > > > > > > > > make
> >> > > >> > > >> >> > > > > > > > > > > >>> > > external
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well with the
> >> newly
> >> > > >> > improved
> >> > > >> > > >> memory
> >> > > >> > > >> >> > > > > > accounting
> >> > > >> > > >> >> > > > > > > > > before
> >> > > >> > > >> >> > > > > > > > > > > >>> > cutting a
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not
> >> sure
> >> > > how
> >> > > >> > much
> >> > > >> > > >> work
> >> > > >> > > >> >> is
> >> > > >> > > >> >> > > > left
> >> > > >> > > >> >> > > > > to
> >> > > >> > > >> >> > > > > > > be
> >> > > >> > > >> >> > > > > > > > > done
> >> > > >> > > >> >> > > > > > > > > > > >>> there.
> >> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond with
> >> your
> >> > > >> > thoughts
> >> > > >> > > on
> >> > > >> > > >> a
> >> > > >> > > >> >> > > release
> >> > > >> > > >> >> > > > > > soon
> >> > > >> > > >> >> > > > > > > > and
> >> > > >> > > >> >> > > > > > > > > > any
> >> > > >> > > >> >> > > > > > > > > > > >>> JIRAs
> >> > > >> > > >> >> > > > > > > > > > > >>> > > you
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to include in
> the
> >> > > >> release.
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
> >> > > >> > > >> >> > > > > > > >
> >> > > https://issues.apache.org/jira/browse/DRILL-4243
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > > >
> >> > > >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> > > >> >> > > > > > > > > > > >>>
> >> > > >> > > >> >> > > > > > > > > > > >>
> >> > > >> > > >> >> > > > > > > > > > > >>
> >> > > >> > > >> >> > > > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > > >
> >> > > >> > > >> >> > > > > > > > >
> >> > > >> > > >> >> > > > > > > >
> >> > > >> > > >> >> > > > > > >
> >> > > >> > > >> >> > > > > >
> >> > > >> > > >> >> > > > >
> >> > > >> > > >> >> > > >
> >> > > >> > > >> >> > >
> >> > > >> > > >> >> >
> >> > > >> > > >> >>
> >> > > >> > > >>
> >> > > >> > >
> >> > > >> >
> >> > > >>
> >> > >
> >> >
> >>
>
>

Re: Time for a 1.5 release?

Posted by Jinfeng Ni <ji...@gmail.com>.

Hi Jason,

I'm fine that release 1.5.0 starts from 03197d0f2.  One thing I need
point out is that the pre-commit test suite has some testcase's plan
output modified because of the partition pruning change.  If we run
pre-commit against the candidate release for 1.5.0, we have to use a
prior version of pre-commit suite.



On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse
<al...@gmail.com> wrote:
> That sounds reasonable to me. I'll start preparing a release candidate.
>
> Jinfeng, are you okay waiting for the next release to include this change?
>
> On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <ja...@dremio.com> wrote:
>
>> Hey Jason,
>>
>> Can you start the release vote? We've let this drag on too long.
>>
>> I suggest we start from 03197d0f2. It seems like the partition pruning
>> changes are pretty complicated and on a pretty main path to add right
>> before the vote. Since Hsuan's fix (pr 349) hasn't been reviewed, I think
>> we should start the vote without it.
>>
>> Jacques
>>
>>
>> --
>> Jacques Nadeau
>> CTO and Co-Founder, Dremio
>>
>> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <hy...@maprtech.com> wrote:
>>
>> > Yes, just send a pull request.
>> > https://github.com/apache/drill/pull/349
>> >
>> > Can Jinfeng review it?
>> >
>> > Thanks,
>> >
>> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <ji...@gmail.com>
>> wrote:
>> >
>> > > Sean probably is quite close to find a fix for this issue. Let's see
>> > > whether he can post a patch soon.
>> > >
>> > >
>> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
>> > > <ch...@gmail.com> wrote:
>> > > > DRILL-4323 only happens when assertions are turned on (my install
>> > script
>> > > > automatically enables assertions).
>> > > >
>> > > > - Rahul
>> > > >
>> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <zf...@maprtech.com>
>> > > wrote:
>> > > >
>> > > >> Rahul,
>> > > >>
>> > > >> Let us know if you can reproduce the issue with assertions turned
>> OFF.
>> > > If
>> > > >> you cannot, then I think this should not be a blocker for 1.5, and
>> > Hsuan
>> > > >> will continue to look into fixing this.
>> > > >>
>> > > >> -- Zelaine
>> > > >>
>> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
>> > knguyen@maprtech.com>
>> > > >> wrote:
>> > > >>
>> > > >> > I do see the same error with assertion turned on.
>> > > >> >
>> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <
>> jinfengni99@gmail.com>
>> > > >> wrote:
>> > > >> >
>> > > >> > > Krystal, by any chance, did you turn on assertion when you start
>> > > >> > drillbit?
>> > > >> > >
>> > > >> > > I think the error in drill-4323 will be raised when assertion =
>> > on.
>> > > >> > >
>> > > >> > >
>> > > >> > >
>> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
>> > > knguyen@maprtech.com>
>> > > >> > > wrote:
>> > > >> > > > I am running drill-1.4 and is able to run count(*)
>> successfully.
>> > > >> > > >
>> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session
>> > set
>> > > >> > > > `store.hive.optimize_scan_with_native_readers` = true;
>> > > >> > > >
>> > +-------+--------------------------------------------------------+
>> > > >> > > > |  ok   |                        summary
>> >  |
>> > > >> > > >
>> > +-------+--------------------------------------------------------+
>> > > >> > > > | true  | store.hive.optimize_scan_with_native_readers
>> > updated.  |
>> > > >> > > >
>> > +-------+--------------------------------------------------------+
>> > > >> > > > 1 row selected (0.374 seconds)
>> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select
>> count(*)
>> > > from
>> > > >> > > > lineitem_parquet_partitioned_hive_string;
>> > > >> > > > +---------+
>> > > >> > > > | EXPR$0  |
>> > > >> > > > +---------+
>> > > >> > > > | 60175   |
>> > > >> > > > +---------+
>> > > >> > > > 1 row selected (2.986 seconds)
>> > > >> > > >
>> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
>> > > jinfengni99@gmail.com>
>> > > >> > > wrote:
>> > > >> > > >
>> > > >> > > >> Venki and I did some investigation for DRILL-4323. The issue
>> > > >> reported
>> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems
>> > to
>> > > us
>> > > >> > > >> this is not a regression from 1.4.0; it's a regression from
>> > 1.3.0
>> > > >> > > >> probably.
>> > > >> > > >>
>> > > >> > > >> DRILL-4083 makes the planner to use DrillHiveNativeReader in
>> > > stead
>> > > >> of
>> > > >> > > >> HiveReader for "select count(*) from hive_table" query.
>> > However,
>> > > >> the
>> > > >> > > >> Project after the scan produces empty schema.  Before
>> > DRILL-4083,
>> > > >> > > >> Drill uses HiveScan, which works fine.
>> > > >> > > >>
>> > > >> > > >>
>> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
>> > > >> > > >> <al...@gmail.com> wrote:
>> > > >> > > >> > We could revert the change to the StoragePlugin interface
>> to
>> > > add
>> > > >> the
>> > > >> > > >> > getLogicalOptimizerRules
>> > > >> > > >> > and getPhysicalOptimizerRules, and only put those in
>> > > >> > > >> AbstractStoragePlugin
>> > > >> > > >> > with default implementations.
>> > > >> > > >> >
>> > > >> > > >> > We could then update the docs to tell users to not
>> implement
>> > > the
>> > > >> > > >> interface
>> > > >> > > >> > directly, and instead extend the abstract class, which I
>> > > believe
>> > > >> > would
>> > > >> > > >> > allow us to make changes like this in the future without
>> > > breaking
>> > > >> > > >> plugins.
>> > > >> > > >> > At a major release version we could then just pull up all
>> of
>> > > the
>> > > >> new
>> > > >> > > >> > methods we added to the abstract class into the interface,
>> or
>> > > just
>> > > >> > get
>> > > >> > > >> rid
>> > > >> > > >> > of it all-together.
>> > > >> > > >> >
>> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
>> > > >> > > >> venki.korukanti@gmail.com>
>> > > >> > > >> > wrote:
>> > > >> > > >> >
>> > > >> > > >> >> Here is the partial fix:
>> > > >> > > >> >>
>> > > >> > > >> >>
>> > > >> > > >>
>> > > >> > >
>> > > >> >
>> > > >>
>> > >
>> >
>> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
>> > > >> > > >> >>
>> > > >> > > >> >> If the existing StragePlugins are implementing
>> > > >> > AbstractStoragePlugin
>> > > >> > > >> class
>> > > >> > > >> >> this fix should resolve the backward compat issue. If they
>> > are
>> > > >> > > >> implementing
>> > > >> > > >> >> StoragePlugin interface directly then it won't resolve the
>> > > issue.
>> > > >> > Not
>> > > >> > > >> sure
>> > > >> > > >> >> how to handle it in that case. Let me know if there any
>> ways
>> > > to
>> > > >> > > resolve
>> > > >> > > >> >> this issue completely.
>> > > >> > > >> >>
>> > > >> > > >> >> Thanks
>> > > >> > > >> >> Venki
>> > > >> > > >> >>
>> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <
>> > > asinha@maprtech.com
>> > > >> >
>> > > >> > > >> wrote:
>> > > >> > > >> >>
>> > > >> > > >> >> > I was able to do the build with maven 3.3.9 (after doing
>> > > 'brew
>> > > >> > > install
>> > > >> > > >> >> > maven').  Here are my env variables related to maven:
>> > > >> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
>> > > >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
>> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
>> > > >> > > >> >> >
>> > > >> > > >> >> >
>> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <
>> > > >> > > jacques@dremio.com>
>> > > >> > > >> >> > wrote:
>> > > >> > > >> >> >
>> > > >> > > >> >> > > Aman, for reference on the Maven version issue. You
>> can
>> > > see
>> > > >> the
>> > > >> > > >> output
>> > > >> > > >> >> of
>> > > >> > > >> >> > > the shade plugin between the two Apache Builds. If you
>> > > search
>> > > >> > for
>> > > >> > > >> >> > > maven-shade-plugin you'll see that build 647 (maven
>> > 3.0.5)
>> > > >> > fails
>> > > >> > > >> >> because
>> > > >> > > >> >> > it
>> > > >> > > >> >> > > includes a bunch of hadoop (and other libraries that
>> are
>> > > >> listed
>> > > >> > > as
>> > > >> > > >> >> being
>> > > >> > > >> >> > > excluded). If you look at build 648 (maven 3.3.3),
>> those
>> > > >> items
>> > > >> > > are
>> > > >> > > >> >> > included
>> > > >> > > >> >> > > so that the build completes successfully.
>> > > >> > > >> >> > >
>> > > >> > > >> >> > >
>> https://builds.apache.org/job/drill-scm/647/consoleText
>> > > >> > > >> >> > >
>> https://builds.apache.org/job/drill-scm/648/consoleText
>> > > >> > > >> >> > >
>> > > >> > > >> >> > > --
>> > > >> > > >> >> > > Jacques Nadeau
>> > > >> > > >> >> > > CTO and Co-Founder, Dremio
>> > > >> > > >> >> > >
>> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
>> > > >> > > >> >> > > challapallirahul@gmail.com> wrote:
>> > > >> > > >> >> > >
>> > > >> > > >> >> > > > Jason,
>> > > >> > > >> >> > > >
>> > > >> > > >> >> > > > As of now I haven't narrowed down the exact commit
>> > which
>> > > >> > caused
>> > > >> > > >> >> > > DRILL-4323
>> > > >> > > >> >> > > > and I do not know the scope of the fix.
>> > > >> > > >> >> > > >
>> > > >> > > >> >> > > > - Rahul
>> > > >> > > >> >> > > >
>> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
>> > > >> > > >> >> > > altekrusejason@gmail.com
>> > > >> > > >> >> > > > >
>> > > >> > > >> >> > > > wrote:
>> > > >> > > >> >> > > >
>> > > >> > > >> >> > > > > I think it makes sense to wait for Venki's fix of
>> > the
>> > > >> API,
>> > > >> > as
>> > > >> > > >> >> > > releasing a
>> > > >> > > >> >> > > > > version with the change would make
>> > > reverting/refactoring
>> > > >> > the
>> > > >> > > API
>> > > >> > > >> >> > > breaking
>> > > >> > > >> >> > > > > change less useful.
>> > > >> > > >> >> > > > >
>> > > >> > > >> >> > > > > Do you have any idea about the scope of fixing the
>> > > hive
>> > > >> > > native
>> > > >> > > >> >> > reader,
>> > > >> > > >> >> > > or
>> > > >> > > >> >> > > > > which change since the 1.4 release broke it? Is
>> > > someone
>> > > >> > > >> available
>> > > >> > > >> >> to
>> > > >> > > >> >> > > look
>> > > >> > > >> >> > > > > at it today?
>> > > >> > > >> >> > > > >
>> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul
>> challapalli <
>> > > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
>> > > >> > > >> >> > > > >
>> > > >> > > >> >> > > > > > Just found an issue with hive native parquet
>> > reader
>> > > >> > > >> (DRILL-4323).
>> > > >> > > >> >> > > This
>> > > >> > > >> >> > > > > is a
>> > > >> > > >> >> > > > > > regression from 1.4 and in my opinion should be
>> > > treated
>> > > >> > as
>> > > >> > > a
>> > > >> > > >> >> > blocker.
>> > > >> > > >> >> > > > > >
>> > > >> > > >> >> > > > > > - Rahul
>> > > >> > > >> >> > > > > >
>> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki
>> Korukanti <
>> > > >> > > >> >> > > > > > venki.korukanti@gmail.com>
>> > > >> > > >> >> > > > > > wrote:
>> > > >> > > >> >> > > > > >
>> > > >> > > >> >> > > > > > > Sorry I didn't realize I was breaking the
>> public
>> > > >> > > interface
>> > > >> > > >> API.
>> > > >> > > >> >> > Let
>> > > >> > > >> >> > > > me
>> > > >> > > >> >> > > > > > work
>> > > >> > > >> >> > > > > > > on a patch to make it backward compatible.
>> > > >> > > >> >> > > > > > >
>> > > >> > > >> >> > > > > > > Thanks
>> > > >> > > >> >> > > > > > > Venki
>> > > >> > > >> >> > > > > > >
>> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven
>> > Phillips
>> > > <
>> > > >> > > >> >> > > steven@dremio.com
>> > > >> > > >> >> > > > >
>> > > >> > > >> >> > > > > > > wrote:
>> > > >> > > >> >> > > > > > >
>> > > >> > > >> >> > > > > > > > I just wanted to bring up an issue that I
>> just
>> > > now
>> > > >> > > >> >> discovered,
>> > > >> > > >> >> > > that
>> > > >> > > >> >> > > > > has
>> > > >> > > >> >> > > > > > > > caused me a fair amount of grief.
>> > > >> > > >> >> > > > > > > >
>> > > >> > > >> >> > > > > > > >
>> > > https://github.com/apache/drill/pull/300/commits
>> > > >> > > >> >> > > > > > > >
>> > > >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing API, and
>> > causes
>> > > >> > > >> >> StoragePlugins
>> > > >> > > >> >> > > > that
>> > > >> > > >> >> > > > > > were
>> > > >> > > >> >> > > > > > > > compiled against currently released versions
>> > of
>> > > >> Drill
>> > > >> > > to
>> > > >> > > >> no
>> > > >> > > >> >> > > longer
>> > > >> > > >> >> > > > > > > > functional properly. I would prefer that
>> this
>> > > >> > breaking
>> > > >> > > >> change
>> > > >> > > >> >> > be
>> > > >> > > >> >> > > > > > modified
>> > > >> > > >> >> > > > > > > > to be backward compatible if possible.
>> > > >> > > >> >> > > > > > > >
>> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason
>> > > Altekruse <
>> > > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
>> > > >> > > >> >> > > > > > > > wrote:
>> > > >> > > >> >> > > > > > > >
>> > > >> > > >> >> > > > > > > > > Hi Aman,
>> > > >> > > >> >> > > > > > > > >
>> > > >> > > >> >> > > > > > > > > This is the failure that he was seeing. He
>> > > >> figured
>> > > >> > > out
>> > > >> > > >> that
>> > > >> > > >> >> > the
>> > > >> > > >> >> > > > new
>> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were not being
>> > > respected
>> > > >> > when
>> > > >> > > the
>> > > >> > > >> >> > build
>> > > >> > > >> >> > > > was
>> > > >> > > >> >> > > > > > run
>> > > >> > > >> >> > > > > > > > with
>> > > >> > > >> >> > > > > > > > > an older Maven version, causing the jar
>> size
>> > > to
>> > > >> > > increase
>> > > >> > > >> >> > > > > > significantly.
>> > > >> > > >> >> > > > > > > > He
>> > > >> > > >> >> > > > > > > > > added an enforcer to make sure the JAR
>> > didn't
>> > > >> grow
>> > > >> > > >> >> > > unexpectedly.
>> > > >> > > >> >> > > > > Can
>> > > >> > > >> >> > > > > > > you
>> > > >> > > >> >> > > > > > > > > try to update your maven version and
>> re-run
>> > > the
>> > > >> > > build?
>> > > >> > > >> >> > > > > > > > >
>> > > >> > > >> >> > > > > > > > > - Jason
>> > > >> > > >> >> > > > > > > > >
>> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman
>> > Sinha <
>> > > >> > > >> >> > > > asinha@maprtech.com>
>> > > >> > > >> >> > > > > > > > wrote:
>> > > >> > > >> >> > > > > > > > >
>> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the following
>> build
>> > > >> failure
>> > > >> > > on
>> > > >> > > >> the
>> > > >> > > >> >> > > latest
>> > > >> > > >> >> > > > > > > master
>> > > >> > > >> >> > > > > > > > > > branch...is this what you saw for the
>> > Apache
>> > > >> > build
>> > > >> > > ?
>> > > >> > > >> My
>> > > >> > > >> >> > mvn
>> > > >> > > >> >> > > > > > version
>> > > >> > > >> >> > > > > > > > > output
>> > > >> > > >> >> > > > > > > > > > is shown below.  Should we all be
>> > upgrading
>> > > to
>> > > >> a
>> > > >> > > newer
>> > > >> > > >> >> mvn
>> > > >> > > >> >> > ?
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > > [INFO] ---
>> > > maven-enforcer-plugin:1.3.1:enforce
>> > > >> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @
>> > > drill-jdbc-all
>> > > >> > ---
>> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
>> > > >> > > >> >> > > > > >
>> org.apache.maven.plugins.enforcer.RequireFilesSize
>> > > >> > > >> >> > > > > > > > > failed
>> > > >> > > >> >> > > > > > > > > > with message:
>> > > >> > > >> >> > > > > > > > > > The file
>> drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> > > is
>> > > >> > > outside
>> > > >> > > >> the
>> > > >> > > >> >> > > > > expected
>> > > >> > > >> >> > > > > > > size
>> > > >> > > >> >> > > > > > > > > > range.
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > >                   This is likely due to
>> > you
>> > > >> > adding
>> > > >> > > new
>> > > >> > > >> >> > > > > dependencies
>> > > >> > > >> >> > > > > > > to
>> > > >> > > >> >> > > > > > > > a
>> > > >> > > >> >> > > > > > > > > > java-exec and not updating the excludes
>> in
>> > > this
>> > > >> > > >> module.
>> > > >> > > >> >> > This
>> > > >> > > >> >> > > is
>> > > >> > > >> >> > > > > > > > important
>> > > >> > > >> >> > > > > > > > > > as it minimizes the size of the
>> dependency
>> > > of
>> > > >> > Drill
>> > > >> > > >> >> > > application
>> > > >> > > >> >> > > > > > > users.
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > >
>> > > >> > > >> >> > > > > > > >
>> > > >> > > >> >> > > > > > >
>> > > >> > > >> >> > > > > >
>> > > >> > > >> >> > > > >
>> > > >> > > >> >> > > >
>> > > >> > > >> >> > >
>> > > >> > > >> >> >
>> > > >> > > >> >>
>> > > >> > > >>
>> > > >> > >
>> > > >> >
>> > > >>
>> > >
>> >
>> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> > > >> > > >> >> > > > > > > > > > size (44664121) too large. Max. is
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > >
>> > > >> > > >> >> > > > > > > >
>> > > >> > > >> >> > > > > > >
>> > > >> > > >> >> > > > > >
>> > > >> > > >> >> > > > >
>> > > >> > > >> >> > > >
>> > > >> > > >> >> > >
>> > > >> > > >> >> >
>> > > >> > > >> >>
>> > > >> > > >>
>> > > >> > >
>> > > >> >
>> > > >>
>> > >
>> >
>> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > >
>> > > Administrators-MacBook-Pro-144:incubator-drill
>> > > >> > > asinha$
>> > > >> > > >> >> mvn
>> > > >> > > >> >> > > > > > --version
>> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
>> > > >> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
>> > > >> > > >> >> > > > > > > > 2013-02-19
>> > > >> > > >> >> > > > > > > > > > 05:51:28-0800)
>> > > >> > > >> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3
>> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle
>> > > >> > Corporation
>> > > >> > > >> >> > > > > > > > > > Java home:
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > >
>> > > >> > > >>
>> > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
>> > > >> > > >> >> > > > > > > > > > Default locale: en_US, platform
>> encoding:
>> > > UTF-8
>> > > >> > > >> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5",
>> > > arch:
>> > > >> > > >> "x86_64",
>> > > >> > > >> >> > > family:
>> > > >> > > >> >> > > > > > "mac"
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques
>> > > >> Nadeau <
>> > > >> > > >> >> > > > > > jacques@dremio.com>
>> > > >> > > >> >> > > > > > > > > > wrote:
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > > > Build back to normal. It looks like
>> the
>> > > >> Apache
>> > > >> > > >> server
>> > > >> > > >> >> was
>> > > >> > > >> >> > > > using
>> > > >> > > >> >> > > > > > an
>> > > >> > > >> >> > > > > > > > old
>> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I switched to
>> > > >> something
>> > > >> > > more
>> > > >> > > >> >> > recent,
>> > > >> > > >> >> > > > the
>> > > >> > > >> >> > > > > > > build
>> > > >> > > >> >> > > > > > > > > > > passed.
>> > > >> > > >> >> > > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > > > --
>> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau
>> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
>> > > >> > > >> >> > > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM,
>> Jacques
>> > > >> > Nadeau <
>> > > >> > > >> >> > > > > > > jacques@dremio.com>
>> > > >> > > >> >> > > > > > > > > > > wrote:
>> > > >> > > >> >> > > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge caused the Apache
>> > > build
>> > > >> to
>> > > >> > > fail.
>> > > >> > > >> >> > > > > > > Investigating...
>> > > >> > > >> >> > > > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > > > > --
>> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
>> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
>> > > >> > > >> >> > > > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM,
>> > Jacques
>> > > >> > > Nadeau <
>> > > >> > > >> >> > > > > > > > jacques@dremio.com>
>> > > >> > > >> >> > > > > > > > > > > > wrote:
>> > > >> > > >> >> > > > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > > > >> I got clean regression runs as
>> well.
>> > > I've
>> > > >> > > merged
>> > > >> > > >> the
>> > > >> > > >> >> > > > patch.
>> > > >> > > >> >> > > > > > > > > > > >>
>> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to start the
>> release
>> > > >> > process?
>> > > >> > > >> >> > > > > > > > > > > >>
>> > > >> > > >> >> > > > > > > > > > > >> --
>> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
>> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
>> > > >> > > >> >> > > > > > > > > > > >>
>> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM,
>> > > Abhishek
>> > > >> > > >> Girish <
>> > > >> > > >> >> > > > > > > > agirish@mapr.com
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > > > >> wrote:
>> > > >> > > >> >> > > > > > > > > > > >>
>> > > >> > > >> >> > > > > > > > > > > >>> Had two clean Functional runs.
>> TPC-H
>> > > >> SF100
>> > > >> > > was
>> > > >> > > >> also
>> > > >> > > >> >> > > > > > successful.
>> > > >> > > >> >> > > > > > > > > > > >>>
>> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM,
>> > > rahul
>> > > >> > > >> >> challapalli <
>> > > >> > > >> >> > > > > > > > > > > >>> challapallirahul@gmail.com>
>> wrote:
>> > > >> > > >> >> > > > > > > > > > > >>>
>> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a functional run with
>> > > your
>> > > >> > > branch.
>> > > >> > > >> >> Will
>> > > >> > > >> >> > > let
>> > > >> > > >> >> > > > > you
>> > > >> > > >> >> > > > > > > know
>> > > >> > > >> >> > > > > > > > > > once
>> > > >> > > >> >> > > > > > > > > > > it
>> > > >> > > >> >> > > > > > > > > > > >>> > finishes
>> > > >> > > >> >> > > > > > > > > > > >>> >
>> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul
>> > > >> > > >> >> > > > > > > > > > > >>> >
>> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM,
>> > > >> Jacques
>> > > >> > > >> Nadeau <
>> > > >> > > >> >> > > > > > > > > > jacques@dremio.com>
>> > > >> > > >> >> > > > > > > > > > > >>> > wrote:
>> > > >> > > >> >> > > > > > > > > > > >>> >
>> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I have
>> an
>> > > >> > updated
>> > > >> > > >> patch
>> > > >> > > >> >> > for
>> > > >> > > >> >> > > > 4291
>> > > >> > > >> >> > > > > > > that
>> > > >> > > >> >> > > > > > > > is
>> > > >> > > >> >> > > > > > > > > > > >>> ready.
>> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems that
>> > > >> something
>> > > >> > > isn't
>> > > >> > > >> >> > > working
>> > > >> > > >> >> > > > > with
>> > > >> > > >> >> > > > > > > our
>> > > >> > > >> >> > > > > > > > > > > >>> extended
>> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't been able
>> to
>> > > run
>> > > >> an
>> > > >> > > >> extended
>> > > >> > > >> >> > > > > > regression.
>> > > >> > > >> >> > > > > > > > > Unit
>> > > >> > > >> >> > > > > > > > > > > >>> tests
>> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else possibly
>> > > able
>> > > >> to
>> > > >> > > run a
>> > > >> > > >> >> > > > regression
>> > > >> > > >> >> > > > > > > suite
>> > > >> > > >> >> > > > > > > > > > > against
>> > > >> > > >> >> > > > > > > > > > > >>> > this
>> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can confirm
>> > > things
>> > > >> > look
>> > > >> > > >> good
>> > > >> > > >> >> and
>> > > >> > > >> >> > > > start
>> > > >> > > >> >> > > > > > the
>> > > >> > > >> >> > > > > > > > > > release
>> > > >> > > >> >> > > > > > > > > > > >>> > > process?
>> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > thanks,
>> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques
>> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>> > > >> > > >> >> > > > >
>> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
>> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > --
>> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
>> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
>> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20
>> > AM,
>> > > >> > Jacques
>> > > >> > > >> >> Nadeau
>> > > >> > > >> >> > <
>> > > >> > > >> >> > > > > > > > > > > jacques@dremio.com
>> > > >> > > >> >> > > > > > > > > > > >>> >
>> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the main things are
>> > 4196
>> > > >> and
>> > > >> > > 4291
>> > > >> > > >> >> > should
>> > > >> > > >> >> > > be
>> > > >> > > >> >> > > > > > > > > completed.
>> > > >> > > >> >> > > > > > > > > > I
>> > > >> > > >> >> > > > > > > > > > > >>> know
>> > > >> > > >> >> > > > > > > > > > > >>> > > Amit
>> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to reproduce 4196
>> > > locally
>> > > >> > this
>> > > >> > > >> >> weekend
>> > > >> > > >> >> > > so
>> > > >> > > >> >> > > > I
>> > > >> > > >> >> > > > > > > think
>> > > >> > > >> >> > > > > > > > > > we're
>> > > >> > > >> >> > > > > > > > > > > >>> close
>> > > >> > > >> >> > > > > > > > > > > >>> > on
>> > > >> > > >> >> > > > > > > > > > > >>> > > > that.
>> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > > --
>> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
>> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
>> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at
>> 10:14
>> > > AM,
>> > > >> > > Zelaine
>> > > >> > > >> >> Fong
>> > > >> > > >> >> > <
>> > > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
>> > > >> > > >> >> > > > > > > > > > > >>> >
>> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
>> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on this?
>> What's
>> > > >> > blocking
>> > > >> > > us
>> > > >> > > >> >> from
>> > > >> > > >> >> > > > taking
>> > > >> > > >> >> > > > > > > this
>> > > >> > > >> >> > > > > > > > > to a
>> > > >> > > >> >> > > > > > > > > > > >>> vote --
>> > > >> > > >> >> > > > > > > > > > > >>> > > the
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join issues?
>> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
>> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at
>> 4:18
>> > > PM,
>> > > >> > > Jacques
>> > > >> > > >> >> > Nadeau
>> > > >> > > >> >> > > <
>> > > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
>> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the issues that
>> > were
>> > > >> > > >> mentioned in
>> > > >> > > >> >> > > this
>> > > >> > > >> >> > > > > > thread
>> > > >> > > >> >> > > > > > > > > along
>> > > >> > > >> >> > > > > > > > > > > >>> with a
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed categorization:
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
>> > > >> > > >> >> > > > > > > > > > > >>>
>> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4190
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
>> > > >> > > >> >> > > > > > > > > > > >>>
>> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4196
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
>> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
>> > > >> > > >> >> > > > > > > > > > > >>>
>> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4256
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
>> > > >> > > >> >> > > > > > > > > > > >>> >
>> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4278
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
>> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
>> > > >> > > >> >> > > > > > > > > > > >>>
>> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4247
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
>> > > >> > > >> >> > > > > > > > > > > >>>
>> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4203
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
>> > > >> > > >> >> > > > > > > > > >
>> > > >> https://issues.apache.org/jira/browse/DRILL-4266
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
>> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do others think?
>> Let's
>> > > try
>> > > >> to
>> > > >> > > get
>> > > >> > > >> the
>> > > >> > > >> >> > > > > blockers
>> > > >> > > >> >> > > > > > > > > wrapped
>> > > >> > > >> >> > > > > > > > > > up
>> > > >> > > >> >> > > > > > > > > > > >>> in
>> > > >> > > >> >> > > > > > > > > > > >>> > the
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> next
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two and start a
>> > > release
>> > > >> > > vote...
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > --
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder,
>> Dremio
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at
>> 1:48
>> > > PM,
>> > > >> > > Jason
>> > > >> > > >> >> > > Altekruse
>> > > >> > > >> >> > > > <
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the allocator
>> > changes
>> > > >> > merged
>> > > >> > > and
>> > > >> > > >> >> > about
>> > > >> > > >> >> > > a
>> > > >> > > >> >> > > > > > month
>> > > >> > > >> >> > > > > > > > > since
>> > > >> > > >> >> > > > > > > > > > > the
>> > > >> > > >> >> > > > > > > > > > > >>> last
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it would be
>> good
>> > to
>> > > >> > start
>> > > >> > > a
>> > > >> > > >> vote
>> > > >> > > >> >> > > > soon. I
>> > > >> > > >> >> > > > > > > would
>> > > >> > > >> >> > > > > > > > > > like
>> > > >> > > >> >> > > > > > > > > > > to
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release manager.
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that there were
>> > some
>> > > >> > issues
>> > > >> > > >> that
>> > > >> > > >> >> > were
>> > > >> > > >> >> > > > > > > > identified
>> > > >> > > >> >> > > > > > > > > > > after
>> > > >> > > >> >> > > > > > > > > > > >>> the
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I
>> think
>> > > that
>> > > >> > > these
>> > > >> > > >> >> > issues
>> > > >> > > >> >> > > > > should
>> > > >> > > >> >> > > > > > > be
>> > > >> > > >> >> > > > > > > > > > fixed
>> > > >> > > >> >> > > > > > > > > > > >>> before
>> > > >> > > >> >> > > > > > > > > > > >>> > > we
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release candidate.
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at the
>> > > associated
>> > > >> > > JIRAs
>> > > >> > > >> it
>> > > >> > > >> >> > > looked
>> > > >> > > >> >> > > > > > like
>> > > >> > > >> >> > > > > > > > > there
>> > > >> > > >> >> > > > > > > > > > > was
>> > > >> > > >> >> > > > > > > > > > > >>> a
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix just
>> > > adjusting
>> > > >> > the
>> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node
>> > > >> > > >> >> > > > > > > > > > > >>> > option,
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> and
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more involved work
>> > to
>> > > >> > change
>> > > >> > > >> how we
>> > > >> > > >> >> > > > > determine
>> > > >> > > >> >> > > > > > > the
>> > > >> > > >> >> > > > > > > > > > > correct
>> > > >> > > >> >> > > > > > > > > > > >>> > time
>> > > >> > > >> >> > > > > > > > > > > >>> > > to
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill during external
>> > > sort. I
>> > > >> > > >> believe it
>> > > >> > > >> >> > > makes
>> > > >> > > >> >> > > > > > sense
>> > > >> > > >> >> > > > > > > > to
>> > > >> > > >> >> > > > > > > > > > make
>> > > >> > > >> >> > > > > > > > > > > >>> > > external
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well with the
>> newly
>> > > >> > improved
>> > > >> > > >> memory
>> > > >> > > >> >> > > > > > accounting
>> > > >> > > >> >> > > > > > > > > before
>> > > >> > > >> >> > > > > > > > > > > >>> > cutting a
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not
>> sure
>> > > how
>> > > >> > much
>> > > >> > > >> work
>> > > >> > > >> >> is
>> > > >> > > >> >> > > > left
>> > > >> > > >> >> > > > > to
>> > > >> > > >> >> > > > > > > be
>> > > >> > > >> >> > > > > > > > > done
>> > > >> > > >> >> > > > > > > > > > > >>> there.
>> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond with
>> your
>> > > >> > thoughts
>> > > >> > > on
>> > > >> > > >> a
>> > > >> > > >> >> > > release
>> > > >> > > >> >> > > > > > soon
>> > > >> > > >> >> > > > > > > > and
>> > > >> > > >> >> > > > > > > > > > any
>> > > >> > > >> >> > > > > > > > > > > >>> JIRAs
>> > > >> > > >> >> > > > > > > > > > > >>> > > you
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to include in the
>> > > >> release.
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
>> > > >> > > >> >> > > > > > > >
>> > > https://issues.apache.org/jira/browse/DRILL-4243
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >>
>> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> > > >> >> > > > > > > > > > > >>> > >
>> > > >> > > >> >> > > > > > > > > > > >>> >
>> > > >> > > >> >> > > > > > > > > > > >>>
>> > > >> > > >> >> > > > > > > > > > > >>
>> > > >> > > >> >> > > > > > > > > > > >>
>> > > >> > > >> >> > > > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > > >
>> > > >> > > >> >> > > > > > > > > >
>> > > >> > > >> >> > > > > > > > >
>> > > >> > > >> >> > > > > > > >
>> > > >> > > >> >> > > > > > >
>> > > >> > > >> >> > > > > >
>> > > >> > > >> >> > > > >
>> > > >> > > >> >> > > >
>> > > >> > > >> >> > >
>> > > >> > > >> >> >
>> > > >> > > >> >>
>> > > >> > > >>
>> > > >> > >
>> > > >> >
>> > > >>
>> > >
>> >
>>

Re: Time for a 1.5 release?

Posted by Jason Altekruse <al...@gmail.com>.

That sounds reasonable to me. I'll start preparing a release candidate.

Jinfeng, are you okay waiting for the next release to include this change?

On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <ja...@dremio.com> wrote:

> Hey Jason,
>
> Can you start the release vote? We've let this drag on too long.
>
> I suggest we start from 03197d0f2. It seems like the partition pruning
> changes are pretty complicated and on a pretty main path to add right
> before the vote. Since Hsuan's fix (pr 349) hasn't been reviewed, I think
> we should start the vote without it.
>
> Jacques
>
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <hy...@maprtech.com> wrote:
>
> > Yes, just send a pull request.
> > https://github.com/apache/drill/pull/349
> >
> > Can Jinfeng review it?
> >
> > Thanks,
> >
> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <ji...@gmail.com>
> wrote:
> >
> > > Sean probably is quite close to find a fix for this issue. Let's see
> > > whether he can post a patch soon.
> > >
> > >
> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
> > > <ch...@gmail.com> wrote:
> > > > DRILL-4323 only happens when assertions are turned on (my install
> > script
> > > > automatically enables assertions).
> > > >
> > > > - Rahul
> > > >
> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <zf...@maprtech.com>
> > > wrote:
> > > >
> > > >> Rahul,
> > > >>
> > > >> Let us know if you can reproduce the issue with assertions turned
> OFF.
> > > If
> > > >> you cannot, then I think this should not be a blocker for 1.5, and
> > Hsuan
> > > >> will continue to look into fixing this.
> > > >>
> > > >> -- Zelaine
> > > >>
> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
> > knguyen@maprtech.com>
> > > >> wrote:
> > > >>
> > > >> > I do see the same error with assertion turned on.
> > > >> >
> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <
> jinfengni99@gmail.com>
> > > >> wrote:
> > > >> >
> > > >> > > Krystal, by any chance, did you turn on assertion when you start
> > > >> > drillbit?
> > > >> > >
> > > >> > > I think the error in drill-4323 will be raised when assertion =
> > on.
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
> > > knguyen@maprtech.com>
> > > >> > > wrote:
> > > >> > > > I am running drill-1.4 and is able to run count(*)
> successfully.
> > > >> > > >
> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session
> > set
> > > >> > > > `store.hive.optimize_scan_with_native_readers` = true;
> > > >> > > >
> > +-------+--------------------------------------------------------+
> > > >> > > > |  ok   |                        summary
> >  |
> > > >> > > >
> > +-------+--------------------------------------------------------+
> > > >> > > > | true  | store.hive.optimize_scan_with_native_readers
> > updated.  |
> > > >> > > >
> > +-------+--------------------------------------------------------+
> > > >> > > > 1 row selected (0.374 seconds)
> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select
> count(*)
> > > from
> > > >> > > > lineitem_parquet_partitioned_hive_string;
> > > >> > > > +---------+
> > > >> > > > | EXPR$0  |
> > > >> > > > +---------+
> > > >> > > > | 60175   |
> > > >> > > > +---------+
> > > >> > > > 1 row selected (2.986 seconds)
> > > >> > > >
> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
> > > jinfengni99@gmail.com>
> > > >> > > wrote:
> > > >> > > >
> > > >> > > >> Venki and I did some investigation for DRILL-4323. The issue
> > > >> reported
> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems
> > to
> > > us
> > > >> > > >> this is not a regression from 1.4.0; it's a regression from
> > 1.3.0
> > > >> > > >> probably.
> > > >> > > >>
> > > >> > > >> DRILL-4083 makes the planner to use DrillHiveNativeReader in
> > > stead
> > > >> of
> > > >> > > >> HiveReader for "select count(*) from hive_table" query.
> > However,
> > > >> the
> > > >> > > >> Project after the scan produces empty schema.  Before
> > DRILL-4083,
> > > >> > > >> Drill uses HiveScan, which works fine.
> > > >> > > >>
> > > >> > > >>
> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
> > > >> > > >> <al...@gmail.com> wrote:
> > > >> > > >> > We could revert the change to the StoragePlugin interface
> to
> > > add
> > > >> the
> > > >> > > >> > getLogicalOptimizerRules
> > > >> > > >> > and getPhysicalOptimizerRules, and only put those in
> > > >> > > >> AbstractStoragePlugin
> > > >> > > >> > with default implementations.
> > > >> > > >> >
> > > >> > > >> > We could then update the docs to tell users to not
> implement
> > > the
> > > >> > > >> interface
> > > >> > > >> > directly, and instead extend the abstract class, which I
> > > believe
> > > >> > would
> > > >> > > >> > allow us to make changes like this in the future without
> > > breaking
> > > >> > > >> plugins.
> > > >> > > >> > At a major release version we could then just pull up all
> of
> > > the
> > > >> new
> > > >> > > >> > methods we added to the abstract class into the interface,
> or
> > > just
> > > >> > get
> > > >> > > >> rid
> > > >> > > >> > of it all-together.
> > > >> > > >> >
> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
> > > >> > > >> venki.korukanti@gmail.com>
> > > >> > > >> > wrote:
> > > >> > > >> >
> > > >> > > >> >> Here is the partial fix:
> > > >> > > >> >>
> > > >> > > >> >>
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> > > >> > > >> >>
> > > >> > > >> >> If the existing StragePlugins are implementing
> > > >> > AbstractStoragePlugin
> > > >> > > >> class
> > > >> > > >> >> this fix should resolve the backward compat issue. If they
> > are
> > > >> > > >> implementing
> > > >> > > >> >> StoragePlugin interface directly then it won't resolve the
> > > issue.
> > > >> > Not
> > > >> > > >> sure
> > > >> > > >> >> how to handle it in that case. Let me know if there any
> ways
> > > to
> > > >> > > resolve
> > > >> > > >> >> this issue completely.
> > > >> > > >> >>
> > > >> > > >> >> Thanks
> > > >> > > >> >> Venki
> > > >> > > >> >>
> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <
> > > asinha@maprtech.com
> > > >> >
> > > >> > > >> wrote:
> > > >> > > >> >>
> > > >> > > >> >> > I was able to do the build with maven 3.3.9 (after doing
> > > 'brew
> > > >> > > install
> > > >> > > >> >> > maven').  Here are my env variables related to maven:
> > > >> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> > > >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <
> > > >> > > jacques@dremio.com>
> > > >> > > >> >> > wrote:
> > > >> > > >> >> >
> > > >> > > >> >> > > Aman, for reference on the Maven version issue. You
> can
> > > see
> > > >> the
> > > >> > > >> output
> > > >> > > >> >> of
> > > >> > > >> >> > > the shade plugin between the two Apache Builds. If you
> > > search
> > > >> > for
> > > >> > > >> >> > > maven-shade-plugin you'll see that build 647 (maven
> > 3.0.5)
> > > >> > fails
> > > >> > > >> >> because
> > > >> > > >> >> > it
> > > >> > > >> >> > > includes a bunch of hadoop (and other libraries that
> are
> > > >> listed
> > > >> > > as
> > > >> > > >> >> being
> > > >> > > >> >> > > excluded). If you look at build 648 (maven 3.3.3),
> those
> > > >> items
> > > >> > > are
> > > >> > > >> >> > included
> > > >> > > >> >> > > so that the build completes successfully.
> > > >> > > >> >> > >
> > > >> > > >> >> > >
> https://builds.apache.org/job/drill-scm/647/consoleText
> > > >> > > >> >> > >
> https://builds.apache.org/job/drill-scm/648/consoleText
> > > >> > > >> >> > >
> > > >> > > >> >> > > --
> > > >> > > >> >> > > Jacques Nadeau
> > > >> > > >> >> > > CTO and Co-Founder, Dremio
> > > >> > > >> >> > >
> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
> > > >> > > >> >> > > challapallirahul@gmail.com> wrote:
> > > >> > > >> >> > >
> > > >> > > >> >> > > > Jason,
> > > >> > > >> >> > > >
> > > >> > > >> >> > > > As of now I haven't narrowed down the exact commit
> > which
> > > >> > caused
> > > >> > > >> >> > > DRILL-4323
> > > >> > > >> >> > > > and I do not know the scope of the fix.
> > > >> > > >> >> > > >
> > > >> > > >> >> > > > - Rahul
> > > >> > > >> >> > > >
> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
> > > >> > > >> >> > > altekrusejason@gmail.com
> > > >> > > >> >> > > > >
> > > >> > > >> >> > > > wrote:
> > > >> > > >> >> > > >
> > > >> > > >> >> > > > > I think it makes sense to wait for Venki's fix of
> > the
> > > >> API,
> > > >> > as
> > > >> > > >> >> > > releasing a
> > > >> > > >> >> > > > > version with the change would make
> > > reverting/refactoring
> > > >> > the
> > > >> > > API
> > > >> > > >> >> > > breaking
> > > >> > > >> >> > > > > change less useful.
> > > >> > > >> >> > > > >
> > > >> > > >> >> > > > > Do you have any idea about the scope of fixing the
> > > hive
> > > >> > > native
> > > >> > > >> >> > reader,
> > > >> > > >> >> > > or
> > > >> > > >> >> > > > > which change since the 1.4 release broke it? Is
> > > someone
> > > >> > > >> available
> > > >> > > >> >> to
> > > >> > > >> >> > > look
> > > >> > > >> >> > > > > at it today?
> > > >> > > >> >> > > > >
> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul
> challapalli <
> > > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
> > > >> > > >> >> > > > >
> > > >> > > >> >> > > > > > Just found an issue with hive native parquet
> > reader
> > > >> > > >> (DRILL-4323).
> > > >> > > >> >> > > This
> > > >> > > >> >> > > > > is a
> > > >> > > >> >> > > > > > regression from 1.4 and in my opinion should be
> > > treated
> > > >> > as
> > > >> > > a
> > > >> > > >> >> > blocker.
> > > >> > > >> >> > > > > >
> > > >> > > >> >> > > > > > - Rahul
> > > >> > > >> >> > > > > >
> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki
> Korukanti <
> > > >> > > >> >> > > > > > venki.korukanti@gmail.com>
> > > >> > > >> >> > > > > > wrote:
> > > >> > > >> >> > > > > >
> > > >> > > >> >> > > > > > > Sorry I didn't realize I was breaking the
> public
> > > >> > > interface
> > > >> > > >> API.
> > > >> > > >> >> > Let
> > > >> > > >> >> > > > me
> > > >> > > >> >> > > > > > work
> > > >> > > >> >> > > > > > > on a patch to make it backward compatible.
> > > >> > > >> >> > > > > > >
> > > >> > > >> >> > > > > > > Thanks
> > > >> > > >> >> > > > > > > Venki
> > > >> > > >> >> > > > > > >
> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven
> > Phillips
> > > <
> > > >> > > >> >> > > steven@dremio.com
> > > >> > > >> >> > > > >
> > > >> > > >> >> > > > > > > wrote:
> > > >> > > >> >> > > > > > >
> > > >> > > >> >> > > > > > > > I just wanted to bring up an issue that I
> just
> > > now
> > > >> > > >> >> discovered,
> > > >> > > >> >> > > that
> > > >> > > >> >> > > > > has
> > > >> > > >> >> > > > > > > > caused me a fair amount of grief.
> > > >> > > >> >> > > > > > > >
> > > >> > > >> >> > > > > > > >
> > > https://github.com/apache/drill/pull/300/commits
> > > >> > > >> >> > > > > > > >
> > > >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing API, and
> > causes
> > > >> > > >> >> StoragePlugins
> > > >> > > >> >> > > > that
> > > >> > > >> >> > > > > > were
> > > >> > > >> >> > > > > > > > compiled against currently released versions
> > of
> > > >> Drill
> > > >> > > to
> > > >> > > >> no
> > > >> > > >> >> > > longer
> > > >> > > >> >> > > > > > > > functional properly. I would prefer that
> this
> > > >> > breaking
> > > >> > > >> change
> > > >> > > >> >> > be
> > > >> > > >> >> > > > > > modified
> > > >> > > >> >> > > > > > > > to be backward compatible if possible.
> > > >> > > >> >> > > > > > > >
> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason
> > > Altekruse <
> > > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
> > > >> > > >> >> > > > > > > > wrote:
> > > >> > > >> >> > > > > > > >
> > > >> > > >> >> > > > > > > > > Hi Aman,
> > > >> > > >> >> > > > > > > > >
> > > >> > > >> >> > > > > > > > > This is the failure that he was seeing. He
> > > >> figured
> > > >> > > out
> > > >> > > >> that
> > > >> > > >> >> > the
> > > >> > > >> >> > > > new
> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were not being
> > > respected
> > > >> > when
> > > >> > > the
> > > >> > > >> >> > build
> > > >> > > >> >> > > > was
> > > >> > > >> >> > > > > > run
> > > >> > > >> >> > > > > > > > with
> > > >> > > >> >> > > > > > > > > an older Maven version, causing the jar
> size
> > > to
> > > >> > > increase
> > > >> > > >> >> > > > > > significantly.
> > > >> > > >> >> > > > > > > > He
> > > >> > > >> >> > > > > > > > > added an enforcer to make sure the JAR
> > didn't
> > > >> grow
> > > >> > > >> >> > > unexpectedly.
> > > >> > > >> >> > > > > Can
> > > >> > > >> >> > > > > > > you
> > > >> > > >> >> > > > > > > > > try to update your maven version and
> re-run
> > > the
> > > >> > > build?
> > > >> > > >> >> > > > > > > > >
> > > >> > > >> >> > > > > > > > > - Jason
> > > >> > > >> >> > > > > > > > >
> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman
> > Sinha <
> > > >> > > >> >> > > > asinha@maprtech.com>
> > > >> > > >> >> > > > > > > > wrote:
> > > >> > > >> >> > > > > > > > >
> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the following
> build
> > > >> failure
> > > >> > > on
> > > >> > > >> the
> > > >> > > >> >> > > latest
> > > >> > > >> >> > > > > > > master
> > > >> > > >> >> > > > > > > > > > branch...is this what you saw for the
> > Apache
> > > >> > build
> > > >> > > ?
> > > >> > > >> My
> > > >> > > >> >> > mvn
> > > >> > > >> >> > > > > > version
> > > >> > > >> >> > > > > > > > > output
> > > >> > > >> >> > > > > > > > > > is shown below.  Should we all be
> > upgrading
> > > to
> > > >> a
> > > >> > > newer
> > > >> > > >> >> mvn
> > > >> > > >> >> > ?
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > > > [INFO] ---
> > > maven-enforcer-plugin:1.3.1:enforce
> > > >> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @
> > > drill-jdbc-all
> > > >> > ---
> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
> > > >> > > >> >> > > > > >
> org.apache.maven.plugins.enforcer.RequireFilesSize
> > > >> > > >> >> > > > > > > > > failed
> > > >> > > >> >> > > > > > > > > > with message:
> > > >> > > >> >> > > > > > > > > > The file
> drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > is
> > > >> > > outside
> > > >> > > >> the
> > > >> > > >> >> > > > > expected
> > > >> > > >> >> > > > > > > size
> > > >> > > >> >> > > > > > > > > > range.
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > > >                   This is likely due to
> > you
> > > >> > adding
> > > >> > > new
> > > >> > > >> >> > > > > dependencies
> > > >> > > >> >> > > > > > > to
> > > >> > > >> >> > > > > > > > a
> > > >> > > >> >> > > > > > > > > > java-exec and not updating the excludes
> in
> > > this
> > > >> > > >> module.
> > > >> > > >> >> > This
> > > >> > > >> >> > > is
> > > >> > > >> >> > > > > > > > important
> > > >> > > >> >> > > > > > > > > > as it minimizes the size of the
> dependency
> > > of
> > > >> > Drill
> > > >> > > >> >> > > application
> > > >> > > >> >> > > > > > > users.
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > >
> > > >> > > >> >> > > > > > > >
> > > >> > > >> >> > > > > > >
> > > >> > > >> >> > > > > >
> > > >> > > >> >> > > > >
> > > >> > > >> >> > > >
> > > >> > > >> >> > >
> > > >> > > >> >> >
> > > >> > > >> >>
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > >> > > >> >> > > > > > > > > > size (44664121) too large. Max. is
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > >
> > > >> > > >> >> > > > > > > >
> > > >> > > >> >> > > > > > >
> > > >> > > >> >> > > > > >
> > > >> > > >> >> > > > >
> > > >> > > >> >> > > >
> > > >> > > >> >> > >
> > > >> > > >> >> >
> > > >> > > >> >>
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > > >
> > > Administrators-MacBook-Pro-144:incubator-drill
> > > >> > > asinha$
> > > >> > > >> >> mvn
> > > >> > > >> >> > > > > > --version
> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
> > > >> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > > >> > > >> >> > > > > > > > 2013-02-19
> > > >> > > >> >> > > > > > > > > > 05:51:28-0800)
> > > >> > > >> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3
> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle
> > > >> > Corporation
> > > >> > > >> >> > > > > > > > > > Java home:
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > >
> > > >> > > >>
> > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > > >> > > >> >> > > > > > > > > > Default locale: en_US, platform
> encoding:
> > > UTF-8
> > > >> > > >> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5",
> > > arch:
> > > >> > > >> "x86_64",
> > > >> > > >> >> > > family:
> > > >> > > >> >> > > > > > "mac"
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques
> > > >> Nadeau <
> > > >> > > >> >> > > > > > jacques@dremio.com>
> > > >> > > >> >> > > > > > > > > > wrote:
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > > > > Build back to normal. It looks like
> the
> > > >> Apache
> > > >> > > >> server
> > > >> > > >> >> was
> > > >> > > >> >> > > > using
> > > >> > > >> >> > > > > > an
> > > >> > > >> >> > > > > > > > old
> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I switched to
> > > >> something
> > > >> > > more
> > > >> > > >> >> > recent,
> > > >> > > >> >> > > > the
> > > >> > > >> >> > > > > > > build
> > > >> > > >> >> > > > > > > > > > > passed.
> > > >> > > >> >> > > > > > > > > > >
> > > >> > > >> >> > > > > > > > > > > --
> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau
> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
> > > >> > > >> >> > > > > > > > > > >
> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM,
> Jacques
> > > >> > Nadeau <
> > > >> > > >> >> > > > > > > jacques@dremio.com>
> > > >> > > >> >> > > > > > > > > > > wrote:
> > > >> > > >> >> > > > > > > > > > >
> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge caused the Apache
> > > build
> > > >> to
> > > >> > > fail.
> > > >> > > >> >> > > > > > > Investigating...
> > > >> > > >> >> > > > > > > > > > > >
> > > >> > > >> >> > > > > > > > > > > > --
> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
> > > >> > > >> >> > > > > > > > > > > >
> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM,
> > Jacques
> > > >> > > Nadeau <
> > > >> > > >> >> > > > > > > > jacques@dremio.com>
> > > >> > > >> >> > > > > > > > > > > > wrote:
> > > >> > > >> >> > > > > > > > > > > >
> > > >> > > >> >> > > > > > > > > > > >> I got clean regression runs as
> well.
> > > I've
> > > >> > > merged
> > > >> > > >> the
> > > >> > > >> >> > > > patch.
> > > >> > > >> >> > > > > > > > > > > >>
> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to start the
> release
> > > >> > process?
> > > >> > > >> >> > > > > > > > > > > >>
> > > >> > > >> >> > > > > > > > > > > >> --
> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
> > > >> > > >> >> > > > > > > > > > > >>
> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM,
> > > Abhishek
> > > >> > > >> Girish <
> > > >> > > >> >> > > > > > > > agirish@mapr.com
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > > > > >> wrote:
> > > >> > > >> >> > > > > > > > > > > >>
> > > >> > > >> >> > > > > > > > > > > >>> Had two clean Functional runs.
> TPC-H
> > > >> SF100
> > > >> > > was
> > > >> > > >> also
> > > >> > > >> >> > > > > > successful.
> > > >> > > >> >> > > > > > > > > > > >>>
> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM,
> > > rahul
> > > >> > > >> >> challapalli <
> > > >> > > >> >> > > > > > > > > > > >>> challapallirahul@gmail.com>
> wrote:
> > > >> > > >> >> > > > > > > > > > > >>>
> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a functional run with
> > > your
> > > >> > > branch.
> > > >> > > >> >> Will
> > > >> > > >> >> > > let
> > > >> > > >> >> > > > > you
> > > >> > > >> >> > > > > > > know
> > > >> > > >> >> > > > > > > > > > once
> > > >> > > >> >> > > > > > > > > > > it
> > > >> > > >> >> > > > > > > > > > > >>> > finishes
> > > >> > > >> >> > > > > > > > > > > >>> >
> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul
> > > >> > > >> >> > > > > > > > > > > >>> >
> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM,
> > > >> Jacques
> > > >> > > >> Nadeau <
> > > >> > > >> >> > > > > > > > > > jacques@dremio.com>
> > > >> > > >> >> > > > > > > > > > > >>> > wrote:
> > > >> > > >> >> > > > > > > > > > > >>> >
> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I have
> an
> > > >> > updated
> > > >> > > >> patch
> > > >> > > >> >> > for
> > > >> > > >> >> > > > 4291
> > > >> > > >> >> > > > > > > that
> > > >> > > >> >> > > > > > > > is
> > > >> > > >> >> > > > > > > > > > > >>> ready.
> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems that
> > > >> something
> > > >> > > isn't
> > > >> > > >> >> > > working
> > > >> > > >> >> > > > > with
> > > >> > > >> >> > > > > > > our
> > > >> > > >> >> > > > > > > > > > > >>> extended
> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't been able
> to
> > > run
> > > >> an
> > > >> > > >> extended
> > > >> > > >> >> > > > > > regression.
> > > >> > > >> >> > > > > > > > > Unit
> > > >> > > >> >> > > > > > > > > > > >>> tests
> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else possibly
> > > able
> > > >> to
> > > >> > > run a
> > > >> > > >> >> > > > regression
> > > >> > > >> >> > > > > > > suite
> > > >> > > >> >> > > > > > > > > > > against
> > > >> > > >> >> > > > > > > > > > > >>> > this
> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can confirm
> > > things
> > > >> > look
> > > >> > > >> good
> > > >> > > >> >> and
> > > >> > > >> >> > > > start
> > > >> > > >> >> > > > > > the
> > > >> > > >> >> > > > > > > > > > release
> > > >> > > >> >> > > > > > > > > > > >>> > > process?
> > > >> > > >> >> > > > > > > > > > > >>> > >
> > > >> > > >> >> > > > > > > > > > > >>> > > thanks,
> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques
> > > >> > > >> >> > > > > > > > > > > >>> > >
> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
> > > >> > > >> >> > > > >
> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > >> > > >> >> > > > > > > > > > > >>> > >
> > > >> > > >> >> > > > > > > > > > > >>> > > --
> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
> > > >> > > >> >> > > > > > > > > > > >>> > >
> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20
> > AM,
> > > >> > Jacques
> > > >> > > >> >> Nadeau
> > > >> > > >> >> > <
> > > >> > > >> >> > > > > > > > > > > jacques@dremio.com
> > > >> > > >> >> > > > > > > > > > > >>> >
> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
> > > >> > > >> >> > > > > > > > > > > >>> > >
> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the main things are
> > 4196
> > > >> and
> > > >> > > 4291
> > > >> > > >> >> > should
> > > >> > > >> >> > > be
> > > >> > > >> >> > > > > > > > > completed.
> > > >> > > >> >> > > > > > > > > > I
> > > >> > > >> >> > > > > > > > > > > >>> know
> > > >> > > >> >> > > > > > > > > > > >>> > > Amit
> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to reproduce 4196
> > > locally
> > > >> > this
> > > >> > > >> >> weekend
> > > >> > > >> >> > > so
> > > >> > > >> >> > > > I
> > > >> > > >> >> > > > > > > think
> > > >> > > >> >> > > > > > > > > > we're
> > > >> > > >> >> > > > > > > > > > > >>> close
> > > >> > > >> >> > > > > > > > > > > >>> > on
> > > >> > > >> >> > > > > > > > > > > >>> > > > that.
> > > >> > > >> >> > > > > > > > > > > >>> > > >
> > > >> > > >> >> > > > > > > > > > > >>> > > > --
> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
> > > >> > > >> >> > > > > > > > > > > >>> > > >
> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at
> 10:14
> > > AM,
> > > >> > > Zelaine
> > > >> > > >> >> Fong
> > > >> > > >> >> > <
> > > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
> > > >> > > >> >> > > > > > > > > > > >>> >
> > > >> > > >> >> > > > > > > > > > > >>> > > wrote:
> > > >> > > >> >> > > > > > > > > > > >>> > > >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on this?
> What's
> > > >> > blocking
> > > >> > > us
> > > >> > > >> >> from
> > > >> > > >> >> > > > taking
> > > >> > > >> >> > > > > > > this
> > > >> > > >> >> > > > > > > > > to a
> > > >> > > >> >> > > > > > > > > > > >>> vote --
> > > >> > > >> >> > > > > > > > > > > >>> > > the
> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join issues?
> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at
> 4:18
> > > PM,
> > > >> > > Jacques
> > > >> > > >> >> > Nadeau
> > > >> > > >> >> > > <
> > > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the issues that
> > were
> > > >> > > >> mentioned in
> > > >> > > >> >> > > this
> > > >> > > >> >> > > > > > thread
> > > >> > > >> >> > > > > > > > > along
> > > >> > > >> >> > > > > > > > > > > >>> with a
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed categorization:
> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> > > >> > > >> >> > > > > > > > > > > >>>
> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4190
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> > > >> > > >> >> > > > > > > > > > > >>>
> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4196
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > > >> > > >> >> > > > > > > > > > > >>> > >
> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
> > > >> > > >> >> > > > > > > > > > > >>>
> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4256
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
> > > >> > > >> >> > > > > > > > > > > >>> >
> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4278
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
> > > >> > > >> >> > > > > > > > > > > >>> > >
> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
> > > >> > > >> >> > > > > > > > > > > >>>
> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4247
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
> > > >> > > >> >> > > > > > > > > > > >>>
> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4203
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
> > > >> > > >> >> > > > > > > > > >
> > > >> https://issues.apache.org/jira/browse/DRILL-4266
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > > >> > > >> >> > > > > > > > > > > >>> > >
> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131
> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do others think?
> Let's
> > > try
> > > >> to
> > > >> > > get
> > > >> > > >> the
> > > >> > > >> >> > > > > blockers
> > > >> > > >> >> > > > > > > > > wrapped
> > > >> > > >> >> > > > > > > > > > up
> > > >> > > >> >> > > > > > > > > > > >>> in
> > > >> > > >> >> > > > > > > > > > > >>> > the
> > > >> > > >> >> > > > > > > > > > > >>> > > >> next
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two and start a
> > > release
> > > >> > > vote...
> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > --
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder,
> Dremio
> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at
> 1:48
> > > PM,
> > > >> > > Jason
> > > >> > > >> >> > > Altekruse
> > > >> > > >> >> > > > <
> > > >> > > >> >> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the allocator
> > changes
> > > >> > merged
> > > >> > > and
> > > >> > > >> >> > about
> > > >> > > >> >> > > a
> > > >> > > >> >> > > > > > month
> > > >> > > >> >> > > > > > > > > since
> > > >> > > >> >> > > > > > > > > > > the
> > > >> > > >> >> > > > > > > > > > > >>> last
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it would be
> good
> > to
> > > >> > start
> > > >> > > a
> > > >> > > >> vote
> > > >> > > >> >> > > > soon. I
> > > >> > > >> >> > > > > > > would
> > > >> > > >> >> > > > > > > > > > like
> > > >> > > >> >> > > > > > > > > > > to
> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release manager.
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that there were
> > some
> > > >> > issues
> > > >> > > >> that
> > > >> > > >> >> > were
> > > >> > > >> >> > > > > > > > identified
> > > >> > > >> >> > > > > > > > > > > after
> > > >> > > >> >> > > > > > > > > > > >>> the
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I
> think
> > > that
> > > >> > > these
> > > >> > > >> >> > issues
> > > >> > > >> >> > > > > should
> > > >> > > >> >> > > > > > > be
> > > >> > > >> >> > > > > > > > > > fixed
> > > >> > > >> >> > > > > > > > > > > >>> before
> > > >> > > >> >> > > > > > > > > > > >>> > > we
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release candidate.
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at the
> > > associated
> > > >> > > JIRAs
> > > >> > > >> it
> > > >> > > >> >> > > looked
> > > >> > > >> >> > > > > > like
> > > >> > > >> >> > > > > > > > > there
> > > >> > > >> >> > > > > > > > > > > was
> > > >> > > >> >> > > > > > > > > > > >>> a
> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix just
> > > adjusting
> > > >> > the
> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node
> > > >> > > >> >> > > > > > > > > > > >>> > option,
> > > >> > > >> >> > > > > > > > > > > >>> > > >> and
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more involved work
> > to
> > > >> > change
> > > >> > > >> how we
> > > >> > > >> >> > > > > determine
> > > >> > > >> >> > > > > > > the
> > > >> > > >> >> > > > > > > > > > > correct
> > > >> > > >> >> > > > > > > > > > > >>> > time
> > > >> > > >> >> > > > > > > > > > > >>> > > to
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill during external
> > > sort. I
> > > >> > > >> believe it
> > > >> > > >> >> > > makes
> > > >> > > >> >> > > > > > sense
> > > >> > > >> >> > > > > > > > to
> > > >> > > >> >> > > > > > > > > > make
> > > >> > > >> >> > > > > > > > > > > >>> > > external
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well with the
> newly
> > > >> > improved
> > > >> > > >> memory
> > > >> > > >> >> > > > > > accounting
> > > >> > > >> >> > > > > > > > > before
> > > >> > > >> >> > > > > > > > > > > >>> > cutting a
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not
> sure
> > > how
> > > >> > much
> > > >> > > >> work
> > > >> > > >> >> is
> > > >> > > >> >> > > > left
> > > >> > > >> >> > > > > to
> > > >> > > >> >> > > > > > > be
> > > >> > > >> >> > > > > > > > > done
> > > >> > > >> >> > > > > > > > > > > >>> there.
> > > >> > > >> >> > > > > > > > > > > >>> > > [1]
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond with
> your
> > > >> > thoughts
> > > >> > > on
> > > >> > > >> a
> > > >> > > >> >> > > release
> > > >> > > >> >> > > > > > soon
> > > >> > > >> >> > > > > > > > and
> > > >> > > >> >> > > > > > > > > > any
> > > >> > > >> >> > > > > > > > > > > >>> JIRAs
> > > >> > > >> >> > > > > > > > > > > >>> > > you
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to include in the
> > > >> release.
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
> > > >> > > >> >> > > > > > > >
> > > https://issues.apache.org/jira/browse/DRILL-4243
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> > > >> >> > > > > > > > > > > >>> > > >>
> > > >> > > >> >> > > > > > > > > > > >>> > > >
> > > >> > > >> >> > > > > > > > > > > >>> > > >
> > > >> > > >> >> > > > > > > > > > > >>> > >
> > > >> > > >> >> > > > > > > > > > > >>> >
> > > >> > > >> >> > > > > > > > > > > >>>
> > > >> > > >> >> > > > > > > > > > > >>
> > > >> > > >> >> > > > > > > > > > > >>
> > > >> > > >> >> > > > > > > > > > > >
> > > >> > > >> >> > > > > > > > > > >
> > > >> > > >> >> > > > > > > > > >
> > > >> > > >> >> > > > > > > > >
> > > >> > > >> >> > > > > > > >
> > > >> > > >> >> > > > > > >
> > > >> > > >> >> > > > > >
> > > >> > > >> >> > > > >
> > > >> > > >> >> > > >
> > > >> > > >> >> > >
> > > >> > > >> >> >
> > > >> > > >> >>
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Jacques Nadeau <ja...@dremio.com>.

Hey Jason,

Can you start the release vote? We've let this drag on too long.

I suggest we start from 03197d0f2. It seems like the partition pruning
changes are pretty complicated and on a pretty main path to add right
before the vote. Since Hsuan's fix (pr 349) hasn't been reviewed, I think
we should start the vote without it.

Jacques


--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <hy...@maprtech.com> wrote:

> Yes, just send a pull request.
> https://github.com/apache/drill/pull/349
>
> Can Jinfeng review it?
>
> Thanks,
>
> On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <ji...@gmail.com> wrote:
>
> > Sean probably is quite close to find a fix for this issue. Let's see
> > whether he can post a patch soon.
> >
> >
> > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
> > <ch...@gmail.com> wrote:
> > > DRILL-4323 only happens when assertions are turned on (my install
> script
> > > automatically enables assertions).
> > >
> > > - Rahul
> > >
> > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <zf...@maprtech.com>
> > wrote:
> > >
> > >> Rahul,
> > >>
> > >> Let us know if you can reproduce the issue with assertions turned OFF.
> > If
> > >> you cannot, then I think this should not be a blocker for 1.5, and
> Hsuan
> > >> will continue to look into fixing this.
> > >>
> > >> -- Zelaine
> > >>
> > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <
> knguyen@maprtech.com>
> > >> wrote:
> > >>
> > >> > I do see the same error with assertion turned on.
> > >> >
> > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <ji...@gmail.com>
> > >> wrote:
> > >> >
> > >> > > Krystal, by any chance, did you turn on assertion when you start
> > >> > drillbit?
> > >> > >
> > >> > > I think the error in drill-4323 will be raised when assertion =
> on.
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
> > knguyen@maprtech.com>
> > >> > > wrote:
> > >> > > > I am running drill-1.4 and is able to run count(*) successfully.
> > >> > > >
> > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session
> set
> > >> > > > `store.hive.optimize_scan_with_native_readers` = true;
> > >> > > >
> +-------+--------------------------------------------------------+
> > >> > > > |  ok   |                        summary
>  |
> > >> > > >
> +-------+--------------------------------------------------------+
> > >> > > > | true  | store.hive.optimize_scan_with_native_readers
> updated.  |
> > >> > > >
> +-------+--------------------------------------------------------+
> > >> > > > 1 row selected (0.374 seconds)
> > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select count(*)
> > from
> > >> > > > lineitem_parquet_partitioned_hive_string;
> > >> > > > +---------+
> > >> > > > | EXPR$0  |
> > >> > > > +---------+
> > >> > > > | 60175   |
> > >> > > > +---------+
> > >> > > > 1 row selected (2.986 seconds)
> > >> > > >
> > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
> > jinfengni99@gmail.com>
> > >> > > wrote:
> > >> > > >
> > >> > > >> Venki and I did some investigation for DRILL-4323. The issue
> > >> reported
> > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems
> to
> > us
> > >> > > >> this is not a regression from 1.4.0; it's a regression from
> 1.3.0
> > >> > > >> probably.
> > >> > > >>
> > >> > > >> DRILL-4083 makes the planner to use DrillHiveNativeReader in
> > stead
> > >> of
> > >> > > >> HiveReader for "select count(*) from hive_table" query.
> However,
> > >> the
> > >> > > >> Project after the scan produces empty schema.  Before
> DRILL-4083,
> > >> > > >> Drill uses HiveScan, which works fine.
> > >> > > >>
> > >> > > >>
> > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
> > >> > > >> <al...@gmail.com> wrote:
> > >> > > >> > We could revert the change to the StoragePlugin interface to
> > add
> > >> the
> > >> > > >> > getLogicalOptimizerRules
> > >> > > >> > and getPhysicalOptimizerRules, and only put those in
> > >> > > >> AbstractStoragePlugin
> > >> > > >> > with default implementations.
> > >> > > >> >
> > >> > > >> > We could then update the docs to tell users to not implement
> > the
> > >> > > >> interface
> > >> > > >> > directly, and instead extend the abstract class, which I
> > believe
> > >> > would
> > >> > > >> > allow us to make changes like this in the future without
> > breaking
> > >> > > >> plugins.
> > >> > > >> > At a major release version we could then just pull up all of
> > the
> > >> new
> > >> > > >> > methods we added to the abstract class into the interface, or
> > just
> > >> > get
> > >> > > >> rid
> > >> > > >> > of it all-together.
> > >> > > >> >
> > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
> > >> > > >> venki.korukanti@gmail.com>
> > >> > > >> > wrote:
> > >> > > >> >
> > >> > > >> >> Here is the partial fix:
> > >> > > >> >>
> > >> > > >> >>
> > >> > > >>
> > >> > >
> > >> >
> > >>
> >
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> > >> > > >> >>
> > >> > > >> >> If the existing StragePlugins are implementing
> > >> > AbstractStoragePlugin
> > >> > > >> class
> > >> > > >> >> this fix should resolve the backward compat issue. If they
> are
> > >> > > >> implementing
> > >> > > >> >> StoragePlugin interface directly then it won't resolve the
> > issue.
> > >> > Not
> > >> > > >> sure
> > >> > > >> >> how to handle it in that case. Let me know if there any ways
> > to
> > >> > > resolve
> > >> > > >> >> this issue completely.
> > >> > > >> >>
> > >> > > >> >> Thanks
> > >> > > >> >> Venki
> > >> > > >> >>
> > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <
> > asinha@maprtech.com
> > >> >
> > >> > > >> wrote:
> > >> > > >> >>
> > >> > > >> >> > I was able to do the build with maven 3.3.9 (after doing
> > 'brew
> > >> > > install
> > >> > > >> >> > maven').  Here are my env variables related to maven:
> > >> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> > >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> > >> > > >> >> >
> > >> > > >> >> >
> > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <
> > >> > > jacques@dremio.com>
> > >> > > >> >> > wrote:
> > >> > > >> >> >
> > >> > > >> >> > > Aman, for reference on the Maven version issue. You can
> > see
> > >> the
> > >> > > >> output
> > >> > > >> >> of
> > >> > > >> >> > > the shade plugin between the two Apache Builds. If you
> > search
> > >> > for
> > >> > > >> >> > > maven-shade-plugin you'll see that build 647 (maven
> 3.0.5)
> > >> > fails
> > >> > > >> >> because
> > >> > > >> >> > it
> > >> > > >> >> > > includes a bunch of hadoop (and other libraries that are
> > >> listed
> > >> > > as
> > >> > > >> >> being
> > >> > > >> >> > > excluded). If you look at build 648 (maven 3.3.3), those
> > >> items
> > >> > > are
> > >> > > >> >> > included
> > >> > > >> >> > > so that the build completes successfully.
> > >> > > >> >> > >
> > >> > > >> >> > > https://builds.apache.org/job/drill-scm/647/consoleText
> > >> > > >> >> > > https://builds.apache.org/job/drill-scm/648/consoleText
> > >> > > >> >> > >
> > >> > > >> >> > > --
> > >> > > >> >> > > Jacques Nadeau
> > >> > > >> >> > > CTO and Co-Founder, Dremio
> > >> > > >> >> > >
> > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
> > >> > > >> >> > > challapallirahul@gmail.com> wrote:
> > >> > > >> >> > >
> > >> > > >> >> > > > Jason,
> > >> > > >> >> > > >
> > >> > > >> >> > > > As of now I haven't narrowed down the exact commit
> which
> > >> > caused
> > >> > > >> >> > > DRILL-4323
> > >> > > >> >> > > > and I do not know the scope of the fix.
> > >> > > >> >> > > >
> > >> > > >> >> > > > - Rahul
> > >> > > >> >> > > >
> > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
> > >> > > >> >> > > altekrusejason@gmail.com
> > >> > > >> >> > > > >
> > >> > > >> >> > > > wrote:
> > >> > > >> >> > > >
> > >> > > >> >> > > > > I think it makes sense to wait for Venki's fix of
> the
> > >> API,
> > >> > as
> > >> > > >> >> > > releasing a
> > >> > > >> >> > > > > version with the change would make
> > reverting/refactoring
> > >> > the
> > >> > > API
> > >> > > >> >> > > breaking
> > >> > > >> >> > > > > change less useful.
> > >> > > >> >> > > > >
> > >> > > >> >> > > > > Do you have any idea about the scope of fixing the
> > hive
> > >> > > native
> > >> > > >> >> > reader,
> > >> > > >> >> > > or
> > >> > > >> >> > > > > which change since the 1.4 release broke it? Is
> > someone
> > >> > > >> available
> > >> > > >> >> to
> > >> > > >> >> > > look
> > >> > > >> >> > > > > at it today?
> > >> > > >> >> > > > >
> > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> > >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
> > >> > > >> >> > > > >
> > >> > > >> >> > > > > > Just found an issue with hive native parquet
> reader
> > >> > > >> (DRILL-4323).
> > >> > > >> >> > > This
> > >> > > >> >> > > > > is a
> > >> > > >> >> > > > > > regression from 1.4 and in my opinion should be
> > treated
> > >> > as
> > >> > > a
> > >> > > >> >> > blocker.
> > >> > > >> >> > > > > >
> > >> > > >> >> > > > > > - Rahul
> > >> > > >> >> > > > > >
> > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> > >> > > >> >> > > > > > venki.korukanti@gmail.com>
> > >> > > >> >> > > > > > wrote:
> > >> > > >> >> > > > > >
> > >> > > >> >> > > > > > > Sorry I didn't realize I was breaking the public
> > >> > > interface
> > >> > > >> API.
> > >> > > >> >> > Let
> > >> > > >> >> > > > me
> > >> > > >> >> > > > > > work
> > >> > > >> >> > > > > > > on a patch to make it backward compatible.
> > >> > > >> >> > > > > > >
> > >> > > >> >> > > > > > > Thanks
> > >> > > >> >> > > > > > > Venki
> > >> > > >> >> > > > > > >
> > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven
> Phillips
> > <
> > >> > > >> >> > > steven@dremio.com
> > >> > > >> >> > > > >
> > >> > > >> >> > > > > > > wrote:
> > >> > > >> >> > > > > > >
> > >> > > >> >> > > > > > > > I just wanted to bring up an issue that I just
> > now
> > >> > > >> >> discovered,
> > >> > > >> >> > > that
> > >> > > >> >> > > > > has
> > >> > > >> >> > > > > > > > caused me a fair amount of grief.
> > >> > > >> >> > > > > > > >
> > >> > > >> >> > > > > > > >
> > https://github.com/apache/drill/pull/300/commits
> > >> > > >> >> > > > > > > >
> > >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing API, and
> causes
> > >> > > >> >> StoragePlugins
> > >> > > >> >> > > > that
> > >> > > >> >> > > > > > were
> > >> > > >> >> > > > > > > > compiled against currently released versions
> of
> > >> Drill
> > >> > > to
> > >> > > >> no
> > >> > > >> >> > > longer
> > >> > > >> >> > > > > > > > functional properly. I would prefer that this
> > >> > breaking
> > >> > > >> change
> > >> > > >> >> > be
> > >> > > >> >> > > > > > modified
> > >> > > >> >> > > > > > > > to be backward compatible if possible.
> > >> > > >> >> > > > > > > >
> > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason
> > Altekruse <
> > >> > > >> >> > > > > > > > altekrusejason@gmail.com>
> > >> > > >> >> > > > > > > > wrote:
> > >> > > >> >> > > > > > > >
> > >> > > >> >> > > > > > > > > Hi Aman,
> > >> > > >> >> > > > > > > > >
> > >> > > >> >> > > > > > > > > This is the failure that he was seeing. He
> > >> figured
> > >> > > out
> > >> > > >> that
> > >> > > >> >> > the
> > >> > > >> >> > > > new
> > >> > > >> >> > > > > > > > > exclusions in jdbc-all were not being
> > respected
> > >> > when
> > >> > > the
> > >> > > >> >> > build
> > >> > > >> >> > > > was
> > >> > > >> >> > > > > > run
> > >> > > >> >> > > > > > > > with
> > >> > > >> >> > > > > > > > > an older Maven version, causing the jar size
> > to
> > >> > > increase
> > >> > > >> >> > > > > > significantly.
> > >> > > >> >> > > > > > > > He
> > >> > > >> >> > > > > > > > > added an enforcer to make sure the JAR
> didn't
> > >> grow
> > >> > > >> >> > > unexpectedly.
> > >> > > >> >> > > > > Can
> > >> > > >> >> > > > > > > you
> > >> > > >> >> > > > > > > > > try to update your maven version and re-run
> > the
> > >> > > build?
> > >> > > >> >> > > > > > > > >
> > >> > > >> >> > > > > > > > > - Jason
> > >> > > >> >> > > > > > > > >
> > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman
> Sinha <
> > >> > > >> >> > > > asinha@maprtech.com>
> > >> > > >> >> > > > > > > > wrote:
> > >> > > >> >> > > > > > > > >
> > >> > > >> >> > > > > > > > > > Jacques, I am getting the following build
> > >> failure
> > >> > > on
> > >> > > >> the
> > >> > > >> >> > > latest
> > >> > > >> >> > > > > > > master
> > >> > > >> >> > > > > > > > > > branch...is this what you saw for the
> Apache
> > >> > build
> > >> > > ?
> > >> > > >> My
> > >> > > >> >> > mvn
> > >> > > >> >> > > > > > version
> > >> > > >> >> > > > > > > > > output
> > >> > > >> >> > > > > > > > > > is shown below.  Should we all be
> upgrading
> > to
> > >> a
> > >> > > newer
> > >> > > >> >> mvn
> > >> > > >> >> > ?
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > > > [INFO] ---
> > maven-enforcer-plugin:1.3.1:enforce
> > >> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @
> > drill-jdbc-all
> > >> > ---
> > >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
> > >> > > >> >> > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
> > >> > > >> >> > > > > > > > > failed
> > >> > > >> >> > > > > > > > > > with message:
> > >> > > >> >> > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > is
> > >> > > outside
> > >> > > >> the
> > >> > > >> >> > > > > expected
> > >> > > >> >> > > > > > > size
> > >> > > >> >> > > > > > > > > > range.
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > > >                   This is likely due to
> you
> > >> > adding
> > >> > > new
> > >> > > >> >> > > > > dependencies
> > >> > > >> >> > > > > > > to
> > >> > > >> >> > > > > > > > a
> > >> > > >> >> > > > > > > > > > java-exec and not updating the excludes in
> > this
> > >> > > >> module.
> > >> > > >> >> > This
> > >> > > >> >> > > is
> > >> > > >> >> > > > > > > > important
> > >> > > >> >> > > > > > > > > > as it minimizes the size of the dependency
> > of
> > >> > Drill
> > >> > > >> >> > > application
> > >> > > >> >> > > > > > > users.
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > >
> > >> > > >> >> > > > > > > >
> > >> > > >> >> > > > > > >
> > >> > > >> >> > > > > >
> > >> > > >> >> > > > >
> > >> > > >> >> > > >
> > >> > > >> >> > >
> > >> > > >> >> >
> > >> > > >> >>
> > >> > > >>
> > >> > >
> > >> >
> > >>
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > >> > > >> >> > > > > > > > > > size (44664121) too large. Max. is
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > >
> > >> > > >> >> > > > > > > >
> > >> > > >> >> > > > > > >
> > >> > > >> >> > > > > >
> > >> > > >> >> > > > >
> > >> > > >> >> > > >
> > >> > > >> >> > >
> > >> > > >> >> >
> > >> > > >> >>
> > >> > > >>
> > >> > >
> > >> >
> > >>
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > > >
> > Administrators-MacBook-Pro-144:incubator-drill
> > >> > > asinha$
> > >> > > >> >> mvn
> > >> > > >> >> > > > > > --version
> > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
> > >> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > >> > > >> >> > > > > > > > 2013-02-19
> > >> > > >> >> > > > > > > > > > 05:51:28-0800)
> > >> > > >> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3
> > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle
> > >> > Corporation
> > >> > > >> >> > > > > > > > > > Java home:
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > >
> > >> > > >>
> > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > >> > > >> >> > > > > > > > > > Default locale: en_US, platform encoding:
> > UTF-8
> > >> > > >> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5",
> > arch:
> > >> > > >> "x86_64",
> > >> > > >> >> > > family:
> > >> > > >> >> > > > > > "mac"
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques
> > >> Nadeau <
> > >> > > >> >> > > > > > jacques@dremio.com>
> > >> > > >> >> > > > > > > > > > wrote:
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > > > > Build back to normal. It looks like the
> > >> Apache
> > >> > > >> server
> > >> > > >> >> was
> > >> > > >> >> > > > using
> > >> > > >> >> > > > > > an
> > >> > > >> >> > > > > > > > old
> > >> > > >> >> > > > > > > > > > > version of Maven. Once I switched to
> > >> something
> > >> > > more
> > >> > > >> >> > recent,
> > >> > > >> >> > > > the
> > >> > > >> >> > > > > > > build
> > >> > > >> >> > > > > > > > > > > passed.
> > >> > > >> >> > > > > > > > > > >
> > >> > > >> >> > > > > > > > > > > --
> > >> > > >> >> > > > > > > > > > > Jacques Nadeau
> > >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
> > >> > > >> >> > > > > > > > > > >
> > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques
> > >> > Nadeau <
> > >> > > >> >> > > > > > > jacques@dremio.com>
> > >> > > >> >> > > > > > > > > > > wrote:
> > >> > > >> >> > > > > > > > > > >
> > >> > > >> >> > > > > > > > > > > > Hmm... this merge caused the Apache
> > build
> > >> to
> > >> > > fail.
> > >> > > >> >> > > > > > > Investigating...
> > >> > > >> >> > > > > > > > > > > >
> > >> > > >> >> > > > > > > > > > > > --
> > >> > > >> >> > > > > > > > > > > > Jacques Nadeau
> > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
> > >> > > >> >> > > > > > > > > > > >
> > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM,
> Jacques
> > >> > > Nadeau <
> > >> > > >> >> > > > > > > > jacques@dremio.com>
> > >> > > >> >> > > > > > > > > > > > wrote:
> > >> > > >> >> > > > > > > > > > > >
> > >> > > >> >> > > > > > > > > > > >> I got clean regression runs as well.
> > I've
> > >> > > merged
> > >> > > >> the
> > >> > > >> >> > > > patch.
> > >> > > >> >> > > > > > > > > > > >>
> > >> > > >> >> > > > > > > > > > > >> Jason, you want to start the release
> > >> > process?
> > >> > > >> >> > > > > > > > > > > >>
> > >> > > >> >> > > > > > > > > > > >> --
> > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
> > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
> > >> > > >> >> > > > > > > > > > > >>
> > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM,
> > Abhishek
> > >> > > >> Girish <
> > >> > > >> >> > > > > > > > agirish@mapr.com
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > > > > >> wrote:
> > >> > > >> >> > > > > > > > > > > >>
> > >> > > >> >> > > > > > > > > > > >>> Had two clean Functional runs. TPC-H
> > >> SF100
> > >> > > was
> > >> > > >> also
> > >> > > >> >> > > > > > successful.
> > >> > > >> >> > > > > > > > > > > >>>
> > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM,
> > rahul
> > >> > > >> >> challapalli <
> > >> > > >> >> > > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
> > >> > > >> >> > > > > > > > > > > >>>
> > >> > > >> >> > > > > > > > > > > >>> > Kicked off a functional run with
> > your
> > >> > > branch.
> > >> > > >> >> Will
> > >> > > >> >> > > let
> > >> > > >> >> > > > > you
> > >> > > >> >> > > > > > > know
> > >> > > >> >> > > > > > > > > > once
> > >> > > >> >> > > > > > > > > > > it
> > >> > > >> >> > > > > > > > > > > >>> > finishes
> > >> > > >> >> > > > > > > > > > > >>> >
> > >> > > >> >> > > > > > > > > > > >>> > - Rahul
> > >> > > >> >> > > > > > > > > > > >>> >
> > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM,
> > >> Jacques
> > >> > > >> Nadeau <
> > >> > > >> >> > > > > > > > > > jacques@dremio.com>
> > >> > > >> >> > > > > > > > > > > >>> > wrote:
> > >> > > >> >> > > > > > > > > > > >>> >
> > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I have an
> > >> > updated
> > >> > > >> patch
> > >> > > >> >> > for
> > >> > > >> >> > > > 4291
> > >> > > >> >> > > > > > > that
> > >> > > >> >> > > > > > > > is
> > >> > > >> >> > > > > > > > > > > >>> ready.
> > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems that
> > >> something
> > >> > > isn't
> > >> > > >> >> > > working
> > >> > > >> >> > > > > with
> > >> > > >> >> > > > > > > our
> > >> > > >> >> > > > > > > > > > > >>> extended
> > >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't been able to
> > run
> > >> an
> > >> > > >> extended
> > >> > > >> >> > > > > > regression.
> > >> > > >> >> > > > > > > > > Unit
> > >> > > >> >> > > > > > > > > > > >>> tests
> > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else possibly
> > able
> > >> to
> > >> > > run a
> > >> > > >> >> > > > regression
> > >> > > >> >> > > > > > > suite
> > >> > > >> >> > > > > > > > > > > against
> > >> > > >> >> > > > > > > > > > > >>> > this
> > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can confirm
> > things
> > >> > look
> > >> > > >> good
> > >> > > >> >> and
> > >> > > >> >> > > > start
> > >> > > >> >> > > > > > the
> > >> > > >> >> > > > > > > > > > release
> > >> > > >> >> > > > > > > > > > > >>> > > process?
> > >> > > >> >> > > > > > > > > > > >>> > >
> > >> > > >> >> > > > > > > > > > > >>> > > thanks,
> > >> > > >> >> > > > > > > > > > > >>> > > Jacques
> > >> > > >> >> > > > > > > > > > > >>> > >
> > >> > > >> >> > > > > > > > > > > >>> > > [1]
> > >> > > >> >> > > > >
> https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > >> > > >> >> > > > > > > > > > > >>> > >
> > >> > > >> >> > > > > > > > > > > >>> > > --
> > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
> > >> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
> > >> > > >> >> > > > > > > > > > > >>> > >
> > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20
> AM,
> > >> > Jacques
> > >> > > >> >> Nadeau
> > >> > > >> >> > <
> > >> > > >> >> > > > > > > > > > > jacques@dremio.com
> > >> > > >> >> > > > > > > > > > > >>> >
> > >> > > >> >> > > > > > > > > > > >>> > > wrote:
> > >> > > >> >> > > > > > > > > > > >>> > >
> > >> > > >> >> > > > > > > > > > > >>> > > > I think the main things are
> 4196
> > >> and
> > >> > > 4291
> > >> > > >> >> > should
> > >> > > >> >> > > be
> > >> > > >> >> > > > > > > > > completed.
> > >> > > >> >> > > > > > > > > > I
> > >> > > >> >> > > > > > > > > > > >>> know
> > >> > > >> >> > > > > > > > > > > >>> > > Amit
> > >> > > >> >> > > > > > > > > > > >>> > > > was able to reproduce 4196
> > locally
> > >> > this
> > >> > > >> >> weekend
> > >> > > >> >> > > so
> > >> > > >> >> > > > I
> > >> > > >> >> > > > > > > think
> > >> > > >> >> > > > > > > > > > we're
> > >> > > >> >> > > > > > > > > > > >>> close
> > >> > > >> >> > > > > > > > > > > >>> > on
> > >> > > >> >> > > > > > > > > > > >>> > > > that.
> > >> > > >> >> > > > > > > > > > > >>> > > >
> > >> > > >> >> > > > > > > > > > > >>> > > > --
> > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
> > >> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
> > >> > > >> >> > > > > > > > > > > >>> > > >
> > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14
> > AM,
> > >> > > Zelaine
> > >> > > >> >> Fong
> > >> > > >> >> > <
> > >> > > >> >> > > > > > > > > > > zfong@maprtech.com
> > >> > > >> >> > > > > > > > > > > >>> >
> > >> > > >> >> > > > > > > > > > > >>> > > wrote:
> > >> > > >> >> > > > > > > > > > > >>> > > >
> > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on this?  What's
> > >> > blocking
> > >> > > us
> > >> > > >> >> from
> > >> > > >> >> > > > taking
> > >> > > >> >> > > > > > > this
> > >> > > >> >> > > > > > > > > to a
> > >> > > >> >> > > > > > > > > > > >>> vote --
> > >> > > >> >> > > > > > > > > > > >>> > > the
> > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join issues?
> > >> > > >> >> > > > > > > > > > > >>> > > >>
> > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
> > >> > > >> >> > > > > > > > > > > >>> > > >>
> > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18
> > PM,
> > >> > > Jacques
> > >> > > >> >> > Nadeau
> > >> > > >> >> > > <
> > >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
> > >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
> > >> > > >> >> > > > > > > > > > > >>> > > >>
> > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
> > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the issues that
> were
> > >> > > >> mentioned in
> > >> > > >> >> > > this
> > >> > > >> >> > > > > > thread
> > >> > > >> >> > > > > > > > > along
> > >> > > >> >> > > > > > > > > > > >>> with a
> > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed categorization:
> > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
> > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> > >> > > >> >> > > > > > > > > > > >>>
> > >> > > >> https://issues.apache.org/jira/browse/DRILL-4190
> > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> > >> > > >> >> > > > > > > > > > > >>>
> > >> > > >> https://issues.apache.org/jira/browse/DRILL-4196
> > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > >> > > >> >> > > > > > > > > > > >>> > >
> > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246
> > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
> > >> > > >> >> > > > > > > > > > > >>>
> > >> > > >> https://issues.apache.org/jira/browse/DRILL-4256
> > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
> > >> > > >> >> > > > > > > > > > > >>> >
> > >> > > >> https://issues.apache.org/jira/browse/DRILL-4278
> > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
> > >> > > >> >> > > > > > > > > > > >>> > >
> > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285
> > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
> > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
> > >> > > >> >> > > > > > > > > > > >>>
> > >> > > >> https://issues.apache.org/jira/browse/DRILL-4247
> > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
> > >> > > >> >> > > > > > > > > > > >>>
> > >> > > >> https://issues.apache.org/jira/browse/DRILL-4203
> > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
> > >> > > >> >> > > > > > > > > >
> > >> https://issues.apache.org/jira/browse/DRILL-4266
> > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > >> > > >> >> > > > > > > > > > > >>> > >
> > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131
> > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> > > >> >> > > > > > > > > > > >>> > > >> > What do others think? Let's
> > try
> > >> to
> > >> > > get
> > >> > > >> the
> > >> > > >> >> > > > > blockers
> > >> > > >> >> > > > > > > > > wrapped
> > >> > > >> >> > > > > > > > > > up
> > >> > > >> >> > > > > > > > > > > >>> in
> > >> > > >> >> > > > > > > > > > > >>> > the
> > >> > > >> >> > > > > > > > > > > >>> > > >> next
> > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two and start a
> > release
> > >> > > vote...
> > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> > > >> >> > > > > > > > > > > >>> > > >> > --
> > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
> > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48
> > PM,
> > >> > > Jason
> > >> > > >> >> > > Altekruse
> > >> > > >> >> > > > <
> > >> > > >> >> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
> > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
> > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
> > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the allocator
> changes
> > >> > merged
> > >> > > and
> > >> > > >> >> > about
> > >> > > >> >> > > a
> > >> > > >> >> > > > > > month
> > >> > > >> >> > > > > > > > > since
> > >> > > >> >> > > > > > > > > > > the
> > >> > > >> >> > > > > > > > > > > >>> last
> > >> > > >> >> > > > > > > > > > > >>> > > >> > release
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it would be good
> to
> > >> > start
> > >> > > a
> > >> > > >> vote
> > >> > > >> >> > > > soon. I
> > >> > > >> >> > > > > > > would
> > >> > > >> >> > > > > > > > > > like
> > >> > > >> >> > > > > > > > > > > to
> > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
> > >> > > >> >> > > > > > > > > > > >>> > > >> > to
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release manager.
> > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that there were
> some
> > >> > issues
> > >> > > >> that
> > >> > > >> >> > were
> > >> > > >> >> > > > > > > > identified
> > >> > > >> >> > > > > > > > > > > after
> > >> > > >> >> > > > > > > > > > > >>> the
> > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I think
> > that
> > >> > > these
> > >> > > >> >> > issues
> > >> > > >> >> > > > > should
> > >> > > >> >> > > > > > > be
> > >> > > >> >> > > > > > > > > > fixed
> > >> > > >> >> > > > > > > > > > > >>> before
> > >> > > >> >> > > > > > > > > > > >>> > > we
> > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > release candidate.
> > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at the
> > associated
> > >> > > JIRAs
> > >> > > >> it
> > >> > > >> >> > > looked
> > >> > > >> >> > > > > > like
> > >> > > >> >> > > > > > > > > there
> > >> > > >> >> > > > > > > > > > > was
> > >> > > >> >> > > > > > > > > > > >>> a
> > >> > > >> >> > > > > > > > > > > >>> > > >> possible
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix just
> > adjusting
> > >> > the
> > >> > > >> >> > > > > > > > > > max_query_memory_per_node
> > >> > > >> >> > > > > > > > > > > >>> > option,
> > >> > > >> >> > > > > > > > > > > >>> > > >> and
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more involved work
> to
> > >> > change
> > >> > > >> how we
> > >> > > >> >> > > > > determine
> > >> > > >> >> > > > > > > the
> > >> > > >> >> > > > > > > > > > > correct
> > >> > > >> >> > > > > > > > > > > >>> > time
> > >> > > >> >> > > > > > > > > > > >>> > > to
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill during external
> > sort. I
> > >> > > >> believe it
> > >> > > >> >> > > makes
> > >> > > >> >> > > > > > sense
> > >> > > >> >> > > > > > > > to
> > >> > > >> >> > > > > > > > > > make
> > >> > > >> >> > > > > > > > > > > >>> > > external
> > >> > > >> >> > > > > > > > > > > >>> > > >> > sort
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well with the newly
> > >> > improved
> > >> > > >> memory
> > >> > > >> >> > > > > > accounting
> > >> > > >> >> > > > > > > > > before
> > >> > > >> >> > > > > > > > > > > >>> > cutting a
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not sure
> > how
> > >> > much
> > >> > > >> work
> > >> > > >> >> is
> > >> > > >> >> > > > left
> > >> > > >> >> > > > > to
> > >> > > >> >> > > > > > > be
> > >> > > >> >> > > > > > > > > done
> > >> > > >> >> > > > > > > > > > > >>> there.
> > >> > > >> >> > > > > > > > > > > >>> > > [1]
> > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond with your
> > >> > thoughts
> > >> > > on
> > >> > > >> a
> > >> > > >> >> > > release
> > >> > > >> >> > > > > > soon
> > >> > > >> >> > > > > > > > and
> > >> > > >> >> > > > > > > > > > any
> > >> > > >> >> > > > > > > > > > > >>> JIRAs
> > >> > > >> >> > > > > > > > > > > >>> > > you
> > >> > > >> >> > > > > > > > > > > >>> > > >> > would
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to include in the
> > >> release.
> > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
> > >> > > >> >> > > > > > > >
> > https://issues.apache.org/jira/browse/DRILL-4243
> > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
> > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
> > >> > > >> >> > > > > > > > > > > >>> > > >> > >
> > >> > > >> >> > > > > > > > > > > >>> > > >> >
> > >> > > >> >> > > > > > > > > > > >>> > > >>
> > >> > > >> >> > > > > > > > > > > >>> > > >
> > >> > > >> >> > > > > > > > > > > >>> > > >
> > >> > > >> >> > > > > > > > > > > >>> > >
> > >> > > >> >> > > > > > > > > > > >>> >
> > >> > > >> >> > > > > > > > > > > >>>
> > >> > > >> >> > > > > > > > > > > >>
> > >> > > >> >> > > > > > > > > > > >>
> > >> > > >> >> > > > > > > > > > > >
> > >> > > >> >> > > > > > > > > > >
> > >> > > >> >> > > > > > > > > >
> > >> > > >> >> > > > > > > > >
> > >> > > >> >> > > > > > > >
> > >> > > >> >> > > > > > >
> > >> > > >> >> > > > > >
> > >> > > >> >> > > > >
> > >> > > >> >> > > >
> > >> > > >> >> > >
> > >> > > >> >> >
> > >> > > >> >>
> > >> > > >>
> > >> > >
> > >> >
> > >>
> >
>

Re: Time for a 1.5 release?

Posted by Hsuan Yi Chu <hy...@maprtech.com>.

Yes, just send a pull request.
https://github.com/apache/drill/pull/349

Can Jinfeng review it?

Thanks,

On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <ji...@gmail.com> wrote:

> Sean probably is quite close to find a fix for this issue. Let's see
> whether he can post a patch soon.
>
>
> On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
> <ch...@gmail.com> wrote:
> > DRILL-4323 only happens when assertions are turned on (my install script
> > automatically enables assertions).
> >
> > - Rahul
> >
> > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <zf...@maprtech.com>
> wrote:
> >
> >> Rahul,
> >>
> >> Let us know if you can reproduce the issue with assertions turned OFF.
> If
> >> you cannot, then I think this should not be a blocker for 1.5, and Hsuan
> >> will continue to look into fixing this.
> >>
> >> -- Zelaine
> >>
> >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <kn...@maprtech.com>
> >> wrote:
> >>
> >> > I do see the same error with assertion turned on.
> >> >
> >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <ji...@gmail.com>
> >> wrote:
> >> >
> >> > > Krystal, by any chance, did you turn on assertion when you start
> >> > drillbit?
> >> > >
> >> > > I think the error in drill-4323 will be raised when assertion = on.
> >> > >
> >> > >
> >> > >
> >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <
> knguyen@maprtech.com>
> >> > > wrote:
> >> > > > I am running drill-1.4 and is able to run count(*) successfully.
> >> > > >
> >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session set
> >> > > > `store.hive.optimize_scan_with_native_readers` = true;
> >> > > > +-------+--------------------------------------------------------+
> >> > > > |  ok   |                        summary                         |
> >> > > > +-------+--------------------------------------------------------+
> >> > > > | true  | store.hive.optimize_scan_with_native_readers updated.  |
> >> > > > +-------+--------------------------------------------------------+
> >> > > > 1 row selected (0.374 seconds)
> >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select count(*)
> from
> >> > > > lineitem_parquet_partitioned_hive_string;
> >> > > > +---------+
> >> > > > | EXPR$0  |
> >> > > > +---------+
> >> > > > | 60175   |
> >> > > > +---------+
> >> > > > 1 row selected (2.986 seconds)
> >> > > >
> >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <
> jinfengni99@gmail.com>
> >> > > wrote:
> >> > > >
> >> > > >> Venki and I did some investigation for DRILL-4323. The issue
> >> reported
> >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems to
> us
> >> > > >> this is not a regression from 1.4.0; it's a regression from 1.3.0
> >> > > >> probably.
> >> > > >>
> >> > > >> DRILL-4083 makes the planner to use DrillHiveNativeReader in
> stead
> >> of
> >> > > >> HiveReader for "select count(*) from hive_table" query.  However,
> >> the
> >> > > >> Project after the scan produces empty schema.  Before DRILL-4083,
> >> > > >> Drill uses HiveScan, which works fine.
> >> > > >>
> >> > > >>
> >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
> >> > > >> <al...@gmail.com> wrote:
> >> > > >> > We could revert the change to the StoragePlugin interface to
> add
> >> the
> >> > > >> > getLogicalOptimizerRules
> >> > > >> > and getPhysicalOptimizerRules, and only put those in
> >> > > >> AbstractStoragePlugin
> >> > > >> > with default implementations.
> >> > > >> >
> >> > > >> > We could then update the docs to tell users to not implement
> the
> >> > > >> interface
> >> > > >> > directly, and instead extend the abstract class, which I
> believe
> >> > would
> >> > > >> > allow us to make changes like this in the future without
> breaking
> >> > > >> plugins.
> >> > > >> > At a major release version we could then just pull up all of
> the
> >> new
> >> > > >> > methods we added to the abstract class into the interface, or
> just
> >> > get
> >> > > >> rid
> >> > > >> > of it all-together.
> >> > > >> >
> >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
> >> > > >> venki.korukanti@gmail.com>
> >> > > >> > wrote:
> >> > > >> >
> >> > > >> >> Here is the partial fix:
> >> > > >> >>
> >> > > >> >>
> >> > > >>
> >> > >
> >> >
> >>
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> >> > > >> >>
> >> > > >> >> If the existing StragePlugins are implementing
> >> > AbstractStoragePlugin
> >> > > >> class
> >> > > >> >> this fix should resolve the backward compat issue. If they are
> >> > > >> implementing
> >> > > >> >> StoragePlugin interface directly then it won't resolve the
> issue.
> >> > Not
> >> > > >> sure
> >> > > >> >> how to handle it in that case. Let me know if there any ways
> to
> >> > > resolve
> >> > > >> >> this issue completely.
> >> > > >> >>
> >> > > >> >> Thanks
> >> > > >> >> Venki
> >> > > >> >>
> >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <
> asinha@maprtech.com
> >> >
> >> > > >> wrote:
> >> > > >> >>
> >> > > >> >> > I was able to do the build with maven 3.3.9 (after doing
> 'brew
> >> > > install
> >> > > >> >> > maven').  Here are my env variables related to maven:
> >> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> >> > > >> >> >
> >> > > >> >> >
> >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <
> >> > > jacques@dremio.com>
> >> > > >> >> > wrote:
> >> > > >> >> >
> >> > > >> >> > > Aman, for reference on the Maven version issue. You can
> see
> >> the
> >> > > >> output
> >> > > >> >> of
> >> > > >> >> > > the shade plugin between the two Apache Builds. If you
> search
> >> > for
> >> > > >> >> > > maven-shade-plugin you'll see that build 647 (maven 3.0.5)
> >> > fails
> >> > > >> >> because
> >> > > >> >> > it
> >> > > >> >> > > includes a bunch of hadoop (and other libraries that are
> >> listed
> >> > > as
> >> > > >> >> being
> >> > > >> >> > > excluded). If you look at build 648 (maven 3.3.3), those
> >> items
> >> > > are
> >> > > >> >> > included
> >> > > >> >> > > so that the build completes successfully.
> >> > > >> >> > >
> >> > > >> >> > > https://builds.apache.org/job/drill-scm/647/consoleText
> >> > > >> >> > > https://builds.apache.org/job/drill-scm/648/consoleText
> >> > > >> >> > >
> >> > > >> >> > > --
> >> > > >> >> > > Jacques Nadeau
> >> > > >> >> > > CTO and Co-Founder, Dremio
> >> > > >> >> > >
> >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
> >> > > >> >> > > challapallirahul@gmail.com> wrote:
> >> > > >> >> > >
> >> > > >> >> > > > Jason,
> >> > > >> >> > > >
> >> > > >> >> > > > As of now I haven't narrowed down the exact commit which
> >> > caused
> >> > > >> >> > > DRILL-4323
> >> > > >> >> > > > and I do not know the scope of the fix.
> >> > > >> >> > > >
> >> > > >> >> > > > - Rahul
> >> > > >> >> > > >
> >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
> >> > > >> >> > > altekrusejason@gmail.com
> >> > > >> >> > > > >
> >> > > >> >> > > > wrote:
> >> > > >> >> > > >
> >> > > >> >> > > > > I think it makes sense to wait for Venki's fix of the
> >> API,
> >> > as
> >> > > >> >> > > releasing a
> >> > > >> >> > > > > version with the change would make
> reverting/refactoring
> >> > the
> >> > > API
> >> > > >> >> > > breaking
> >> > > >> >> > > > > change less useful.
> >> > > >> >> > > > >
> >> > > >> >> > > > > Do you have any idea about the scope of fixing the
> hive
> >> > > native
> >> > > >> >> > reader,
> >> > > >> >> > > or
> >> > > >> >> > > > > which change since the 1.4 release broke it? Is
> someone
> >> > > >> available
> >> > > >> >> to
> >> > > >> >> > > look
> >> > > >> >> > > > > at it today?
> >> > > >> >> > > > >
> >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> >> > > >> >> > > > > challapallirahul@gmail.com> wrote:
> >> > > >> >> > > > >
> >> > > >> >> > > > > > Just found an issue with hive native parquet reader
> >> > > >> (DRILL-4323).
> >> > > >> >> > > This
> >> > > >> >> > > > > is a
> >> > > >> >> > > > > > regression from 1.4 and in my opinion should be
> treated
> >> > as
> >> > > a
> >> > > >> >> > blocker.
> >> > > >> >> > > > > >
> >> > > >> >> > > > > > - Rahul
> >> > > >> >> > > > > >
> >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> >> > > >> >> > > > > > venki.korukanti@gmail.com>
> >> > > >> >> > > > > > wrote:
> >> > > >> >> > > > > >
> >> > > >> >> > > > > > > Sorry I didn't realize I was breaking the public
> >> > > interface
> >> > > >> API.
> >> > > >> >> > Let
> >> > > >> >> > > > me
> >> > > >> >> > > > > > work
> >> > > >> >> > > > > > > on a patch to make it backward compatible.
> >> > > >> >> > > > > > >
> >> > > >> >> > > > > > > Thanks
> >> > > >> >> > > > > > > Venki
> >> > > >> >> > > > > > >
> >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips
> <
> >> > > >> >> > > steven@dremio.com
> >> > > >> >> > > > >
> >> > > >> >> > > > > > > wrote:
> >> > > >> >> > > > > > >
> >> > > >> >> > > > > > > > I just wanted to bring up an issue that I just
> now
> >> > > >> >> discovered,
> >> > > >> >> > > that
> >> > > >> >> > > > > has
> >> > > >> >> > > > > > > > caused me a fair amount of grief.
> >> > > >> >> > > > > > > >
> >> > > >> >> > > > > > > >
> https://github.com/apache/drill/pull/300/commits
> >> > > >> >> > > > > > > >
> >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing API, and causes
> >> > > >> >> StoragePlugins
> >> > > >> >> > > > that
> >> > > >> >> > > > > > were
> >> > > >> >> > > > > > > > compiled against currently released versions of
> >> Drill
> >> > > to
> >> > > >> no
> >> > > >> >> > > longer
> >> > > >> >> > > > > > > > functional properly. I would prefer that this
> >> > breaking
> >> > > >> change
> >> > > >> >> > be
> >> > > >> >> > > > > > modified
> >> > > >> >> > > > > > > > to be backward compatible if possible.
> >> > > >> >> > > > > > > >
> >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason
> Altekruse <
> >> > > >> >> > > > > > > > altekrusejason@gmail.com>
> >> > > >> >> > > > > > > > wrote:
> >> > > >> >> > > > > > > >
> >> > > >> >> > > > > > > > > Hi Aman,
> >> > > >> >> > > > > > > > >
> >> > > >> >> > > > > > > > > This is the failure that he was seeing. He
> >> figured
> >> > > out
> >> > > >> that
> >> > > >> >> > the
> >> > > >> >> > > > new
> >> > > >> >> > > > > > > > > exclusions in jdbc-all were not being
> respected
> >> > when
> >> > > the
> >> > > >> >> > build
> >> > > >> >> > > > was
> >> > > >> >> > > > > > run
> >> > > >> >> > > > > > > > with
> >> > > >> >> > > > > > > > > an older Maven version, causing the jar size
> to
> >> > > increase
> >> > > >> >> > > > > > significantly.
> >> > > >> >> > > > > > > > He
> >> > > >> >> > > > > > > > > added an enforcer to make sure the JAR didn't
> >> grow
> >> > > >> >> > > unexpectedly.
> >> > > >> >> > > > > Can
> >> > > >> >> > > > > > > you
> >> > > >> >> > > > > > > > > try to update your maven version and re-run
> the
> >> > > build?
> >> > > >> >> > > > > > > > >
> >> > > >> >> > > > > > > > > - Jason
> >> > > >> >> > > > > > > > >
> >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
> >> > > >> >> > > > asinha@maprtech.com>
> >> > > >> >> > > > > > > > wrote:
> >> > > >> >> > > > > > > > >
> >> > > >> >> > > > > > > > > > Jacques, I am getting the following build
> >> failure
> >> > > on
> >> > > >> the
> >> > > >> >> > > latest
> >> > > >> >> > > > > > > master
> >> > > >> >> > > > > > > > > > branch...is this what you saw for the Apache
> >> > build
> >> > > ?
> >> > > >> My
> >> > > >> >> > mvn
> >> > > >> >> > > > > > version
> >> > > >> >> > > > > > > > > output
> >> > > >> >> > > > > > > > > > is shown below.  Should we all be upgrading
> to
> >> a
> >> > > newer
> >> > > >> >> mvn
> >> > > >> >> > ?
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > > > [INFO] ---
> maven-enforcer-plugin:1.3.1:enforce
> >> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @
> drill-jdbc-all
> >> > ---
> >> > > >> >> > > > > > > > > > [WARNING] Rule 0:
> >> > > >> >> > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
> >> > > >> >> > > > > > > > > failed
> >> > > >> >> > > > > > > > > > with message:
> >> > > >> >> > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar
> is
> >> > > outside
> >> > > >> the
> >> > > >> >> > > > > expected
> >> > > >> >> > > > > > > size
> >> > > >> >> > > > > > > > > > range.
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > > >                   This is likely due to you
> >> > adding
> >> > > new
> >> > > >> >> > > > > dependencies
> >> > > >> >> > > > > > > to
> >> > > >> >> > > > > > > > a
> >> > > >> >> > > > > > > > > > java-exec and not updating the excludes in
> this
> >> > > >> module.
> >> > > >> >> > This
> >> > > >> >> > > is
> >> > > >> >> > > > > > > > important
> >> > > >> >> > > > > > > > > > as it minimizes the size of the dependency
> of
> >> > Drill
> >> > > >> >> > > application
> >> > > >> >> > > > > > > users.
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > >
> >> > > >> >> > > > > > > >
> >> > > >> >> > > > > > >
> >> > > >> >> > > > > >
> >> > > >> >> > > > >
> >> > > >> >> > > >
> >> > > >> >> > >
> >> > > >> >> >
> >> > > >> >>
> >> > > >>
> >> > >
> >> >
> >>
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> > > >> >> > > > > > > > > > size (44664121) too large. Max. is
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > >
> >> > > >> >> > > > > > > >
> >> > > >> >> > > > > > >
> >> > > >> >> > > > > >
> >> > > >> >> > > > >
> >> > > >> >> > > >
> >> > > >> >> > >
> >> > > >> >> >
> >> > > >> >>
> >> > > >>
> >> > >
> >> >
> >>
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > > >
> Administrators-MacBook-Pro-144:incubator-drill
> >> > > asinha$
> >> > > >> >> mvn
> >> > > >> >> > > > > > --version
> >> > > >> >> > > > > > > > > > Apache Maven 3.0.5
> >> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> >> > > >> >> > > > > > > > 2013-02-19
> >> > > >> >> > > > > > > > > > 05:51:28-0800)
> >> > > >> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3
> >> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle
> >> > Corporation
> >> > > >> >> > > > > > > > > > Java home:
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > >
> >> > > >>
> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> >> > > >> >> > > > > > > > > > Default locale: en_US, platform encoding:
> UTF-8
> >> > > >> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5",
> arch:
> >> > > >> "x86_64",
> >> > > >> >> > > family:
> >> > > >> >> > > > > > "mac"
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques
> >> Nadeau <
> >> > > >> >> > > > > > jacques@dremio.com>
> >> > > >> >> > > > > > > > > > wrote:
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > > > > Build back to normal. It looks like the
> >> Apache
> >> > > >> server
> >> > > >> >> was
> >> > > >> >> > > > using
> >> > > >> >> > > > > > an
> >> > > >> >> > > > > > > > old
> >> > > >> >> > > > > > > > > > > version of Maven. Once I switched to
> >> something
> >> > > more
> >> > > >> >> > recent,
> >> > > >> >> > > > the
> >> > > >> >> > > > > > > build
> >> > > >> >> > > > > > > > > > > passed.
> >> > > >> >> > > > > > > > > > >
> >> > > >> >> > > > > > > > > > > --
> >> > > >> >> > > > > > > > > > > Jacques Nadeau
> >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
> >> > > >> >> > > > > > > > > > >
> >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques
> >> > Nadeau <
> >> > > >> >> > > > > > > jacques@dremio.com>
> >> > > >> >> > > > > > > > > > > wrote:
> >> > > >> >> > > > > > > > > > >
> >> > > >> >> > > > > > > > > > > > Hmm... this merge caused the Apache
> build
> >> to
> >> > > fail.
> >> > > >> >> > > > > > > Investigating...
> >> > > >> >> > > > > > > > > > > >
> >> > > >> >> > > > > > > > > > > > --
> >> > > >> >> > > > > > > > > > > > Jacques Nadeau
> >> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
> >> > > >> >> > > > > > > > > > > >
> >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques
> >> > > Nadeau <
> >> > > >> >> > > > > > > > jacques@dremio.com>
> >> > > >> >> > > > > > > > > > > > wrote:
> >> > > >> >> > > > > > > > > > > >
> >> > > >> >> > > > > > > > > > > >> I got clean regression runs as well.
> I've
> >> > > merged
> >> > > >> the
> >> > > >> >> > > > patch.
> >> > > >> >> > > > > > > > > > > >>
> >> > > >> >> > > > > > > > > > > >> Jason, you want to start the release
> >> > process?
> >> > > >> >> > > > > > > > > > > >>
> >> > > >> >> > > > > > > > > > > >> --
> >> > > >> >> > > > > > > > > > > >> Jacques Nadeau
> >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
> >> > > >> >> > > > > > > > > > > >>
> >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM,
> Abhishek
> >> > > >> Girish <
> >> > > >> >> > > > > > > > agirish@mapr.com
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > > > > >> wrote:
> >> > > >> >> > > > > > > > > > > >>
> >> > > >> >> > > > > > > > > > > >>> Had two clean Functional runs. TPC-H
> >> SF100
> >> > > was
> >> > > >> also
> >> > > >> >> > > > > > successful.
> >> > > >> >> > > > > > > > > > > >>>
> >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM,
> rahul
> >> > > >> >> challapalli <
> >> > > >> >> > > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
> >> > > >> >> > > > > > > > > > > >>>
> >> > > >> >> > > > > > > > > > > >>> > Kicked off a functional run with
> your
> >> > > branch.
> >> > > >> >> Will
> >> > > >> >> > > let
> >> > > >> >> > > > > you
> >> > > >> >> > > > > > > know
> >> > > >> >> > > > > > > > > > once
> >> > > >> >> > > > > > > > > > > it
> >> > > >> >> > > > > > > > > > > >>> > finishes
> >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> >> > > > > > > > > > > >>> > - Rahul
> >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM,
> >> Jacques
> >> > > >> Nadeau <
> >> > > >> >> > > > > > > > > > jacques@dremio.com>
> >> > > >> >> > > > > > > > > > > >>> > wrote:
> >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I have an
> >> > updated
> >> > > >> patch
> >> > > >> >> > for
> >> > > >> >> > > > 4291
> >> > > >> >> > > > > > > that
> >> > > >> >> > > > > > > > is
> >> > > >> >> > > > > > > > > > > >>> ready.
> >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems that
> >> something
> >> > > isn't
> >> > > >> >> > > working
> >> > > >> >> > > > > with
> >> > > >> >> > > > > > > our
> >> > > >> >> > > > > > > > > > > >>> extended
> >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't been able to
> run
> >> an
> >> > > >> extended
> >> > > >> >> > > > > > regression.
> >> > > >> >> > > > > > > > > Unit
> >> > > >> >> > > > > > > > > > > >>> tests
> >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else possibly
> able
> >> to
> >> > > run a
> >> > > >> >> > > > regression
> >> > > >> >> > > > > > > suite
> >> > > >> >> > > > > > > > > > > against
> >> > > >> >> > > > > > > > > > > >>> > this
> >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can confirm
> things
> >> > look
> >> > > >> good
> >> > > >> >> and
> >> > > >> >> > > > start
> >> > > >> >> > > > > > the
> >> > > >> >> > > > > > > > > > release
> >> > > >> >> > > > > > > > > > > >>> > > process?
> >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> >> > > > > > > > > > > >>> > > thanks,
> >> > > >> >> > > > > > > > > > > >>> > > Jacques
> >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> >> > > > > > > > > > > >>> > > [1]
> >> > > >> >> > > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> >> > > > > > > > > > > >>> > > --
> >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
> >> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
> >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM,
> >> > Jacques
> >> > > >> >> Nadeau
> >> > > >> >> > <
> >> > > >> >> > > > > > > > > > > jacques@dremio.com
> >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> >> > > > > > > > > > > >>> > > wrote:
> >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> >> > > > > > > > > > > >>> > > > I think the main things are 4196
> >> and
> >> > > 4291
> >> > > >> >> > should
> >> > > >> >> > > be
> >> > > >> >> > > > > > > > > completed.
> >> > > >> >> > > > > > > > > > I
> >> > > >> >> > > > > > > > > > > >>> know
> >> > > >> >> > > > > > > > > > > >>> > > Amit
> >> > > >> >> > > > > > > > > > > >>> > > > was able to reproduce 4196
> locally
> >> > this
> >> > > >> >> weekend
> >> > > >> >> > > so
> >> > > >> >> > > > I
> >> > > >> >> > > > > > > think
> >> > > >> >> > > > > > > > > > we're
> >> > > >> >> > > > > > > > > > > >>> close
> >> > > >> >> > > > > > > > > > > >>> > on
> >> > > >> >> > > > > > > > > > > >>> > > > that.
> >> > > >> >> > > > > > > > > > > >>> > > >
> >> > > >> >> > > > > > > > > > > >>> > > > --
> >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
> >> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
> >> > > >> >> > > > > > > > > > > >>> > > >
> >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14
> AM,
> >> > > Zelaine
> >> > > >> >> Fong
> >> > > >> >> > <
> >> > > >> >> > > > > > > > > > > zfong@maprtech.com
> >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> >> > > > > > > > > > > >>> > > wrote:
> >> > > >> >> > > > > > > > > > > >>> > > >
> >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on this?  What's
> >> > blocking
> >> > > us
> >> > > >> >> from
> >> > > >> >> > > > taking
> >> > > >> >> > > > > > > this
> >> > > >> >> > > > > > > > > to a
> >> > > >> >> > > > > > > > > > > >>> vote --
> >> > > >> >> > > > > > > > > > > >>> > > the
> >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join issues?
> >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
> >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18
> PM,
> >> > > Jacques
> >> > > >> >> > Nadeau
> >> > > >> >> > > <
> >> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
> >> > > >> >> > > > > > > > > > > >>> > > >> wrote:
> >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
> >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the issues that were
> >> > > >> mentioned in
> >> > > >> >> > > this
> >> > > >> >> > > > > > thread
> >> > > >> >> > > > > > > > > along
> >> > > >> >> > > > > > > > > > > >>> with a
> >> > > >> >> > > > > > > > > > > >>> > > >> > proposed categorization:
> >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
> >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> >> > > >> >> > > > > > > > > > > >>>
> >> > > >> https://issues.apache.org/jira/browse/DRILL-4190
> >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> >> > > >> >> > > > > > > > > > > >>>
> >> > > >> https://issues.apache.org/jira/browse/DRILL-4196
> >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246
> >> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
> >> > > >> >> > > > > > > > > > > >>>
> >> > > >> https://issues.apache.org/jira/browse/DRILL-4256
> >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
> >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> https://issues.apache.org/jira/browse/DRILL-4278
> >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
> >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285
> >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
> >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
> >> > > >> >> > > > > > > > > > > >>>
> >> > > >> https://issues.apache.org/jira/browse/DRILL-4247
> >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
> >> > > >> >> > > > > > > > > > > >>>
> >> > > >> https://issues.apache.org/jira/browse/DRILL-4203
> >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
> >> > > >> >> > > > > > > > > >
> >> https://issues.apache.org/jira/browse/DRILL-4266
> >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131
> >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> >> > > > > > > > > > > >>> > > >> > What do others think? Let's
> try
> >> to
> >> > > get
> >> > > >> the
> >> > > >> >> > > > > blockers
> >> > > >> >> > > > > > > > > wrapped
> >> > > >> >> > > > > > > > > > up
> >> > > >> >> > > > > > > > > > > >>> in
> >> > > >> >> > > > > > > > > > > >>> > the
> >> > > >> >> > > > > > > > > > > >>> > > >> next
> >> > > >> >> > > > > > > > > > > >>> > > >> > day or two and start a
> release
> >> > > vote...
> >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> >> > > > > > > > > > > >>> > > >> > --
> >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
> >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48
> PM,
> >> > > Jason
> >> > > >> >> > > Altekruse
> >> > > >> >> > > > <
> >> > > >> >> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
> >> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
> >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
> >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> >> > > > > > > > > > > >>> > > >> > > With the allocator changes
> >> > merged
> >> > > and
> >> > > >> >> > about
> >> > > >> >> > > a
> >> > > >> >> > > > > > month
> >> > > >> >> > > > > > > > > since
> >> > > >> >> > > > > > > > > > > the
> >> > > >> >> > > > > > > > > > > >>> last
> >> > > >> >> > > > > > > > > > > >>> > > >> > release
> >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it would be good to
> >> > start
> >> > > a
> >> > > >> vote
> >> > > >> >> > > > soon. I
> >> > > >> >> > > > > > > would
> >> > > >> >> > > > > > > > > > like
> >> > > >> >> > > > > > > > > > > to
> >> > > >> >> > > > > > > > > > > >>> > > >> volunteer
> >> > > >> >> > > > > > > > > > > >>> > > >> > to
> >> > > >> >> > > > > > > > > > > >>> > > >> > > be release manager.
> >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that there were some
> >> > issues
> >> > > >> that
> >> > > >> >> > were
> >> > > >> >> > > > > > > > identified
> >> > > >> >> > > > > > > > > > > after
> >> > > >> >> > > > > > > > > > > >>> the
> >> > > >> >> > > > > > > > > > > >>> > > >> > transfer
> >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I think
> that
> >> > > these
> >> > > >> >> > issues
> >> > > >> >> > > > > should
> >> > > >> >> > > > > > > be
> >> > > >> >> > > > > > > > > > fixed
> >> > > >> >> > > > > > > > > > > >>> before
> >> > > >> >> > > > > > > > > > > >>> > > we
> >> > > >> >> > > > > > > > > > > >>> > > >> > cut a
> >> > > >> >> > > > > > > > > > > >>> > > >> > > release candidate.
> >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at the
> associated
> >> > > JIRAs
> >> > > >> it
> >> > > >> >> > > looked
> >> > > >> >> > > > > > like
> >> > > >> >> > > > > > > > > there
> >> > > >> >> > > > > > > > > > > was
> >> > > >> >> > > > > > > > > > > >>> a
> >> > > >> >> > > > > > > > > > > >>> > > >> possible
> >> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix just
> adjusting
> >> > the
> >> > > >> >> > > > > > > > > > max_query_memory_per_node
> >> > > >> >> > > > > > > > > > > >>> > option,
> >> > > >> >> > > > > > > > > > > >>> > > >> and
> >> > > >> >> > > > > > > > > > > >>> > > >> > > some more involved work to
> >> > change
> >> > > >> how we
> >> > > >> >> > > > > determine
> >> > > >> >> > > > > > > the
> >> > > >> >> > > > > > > > > > > correct
> >> > > >> >> > > > > > > > > > > >>> > time
> >> > > >> >> > > > > > > > > > > >>> > > to
> >> > > >> >> > > > > > > > > > > >>> > > >> > > spill during external
> sort. I
> >> > > >> believe it
> >> > > >> >> > > makes
> >> > > >> >> > > > > > sense
> >> > > >> >> > > > > > > > to
> >> > > >> >> > > > > > > > > > make
> >> > > >> >> > > > > > > > > > > >>> > > external
> >> > > >> >> > > > > > > > > > > >>> > > >> > sort
> >> > > >> >> > > > > > > > > > > >>> > > >> > > work well with the newly
> >> > improved
> >> > > >> memory
> >> > > >> >> > > > > > accounting
> >> > > >> >> > > > > > > > > before
> >> > > >> >> > > > > > > > > > > >>> > cutting a
> >> > > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not sure
> how
> >> > much
> >> > > >> work
> >> > > >> >> is
> >> > > >> >> > > > left
> >> > > >> >> > > > > to
> >> > > >> >> > > > > > > be
> >> > > >> >> > > > > > > > > done
> >> > > >> >> > > > > > > > > > > >>> there.
> >> > > >> >> > > > > > > > > > > >>> > > [1]
> >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond with your
> >> > thoughts
> >> > > on
> >> > > >> a
> >> > > >> >> > > release
> >> > > >> >> > > > > > soon
> >> > > >> >> > > > > > > > and
> >> > > >> >> > > > > > > > > > any
> >> > > >> >> > > > > > > > > > > >>> JIRAs
> >> > > >> >> > > > > > > > > > > >>> > > you
> >> > > >> >> > > > > > > > > > > >>> > > >> > would
> >> > > >> >> > > > > > > > > > > >>> > > >> > > like to include in the
> >> release.
> >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
> >> > > >> >> > > > > > > >
> https://issues.apache.org/jira/browse/DRILL-4243
> >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
> >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
> >> > > >> >> > > > > > > > > > > >>> > > >> > >
> >> > > >> >> > > > > > > > > > > >>> > > >> >
> >> > > >> >> > > > > > > > > > > >>> > > >>
> >> > > >> >> > > > > > > > > > > >>> > > >
> >> > > >> >> > > > > > > > > > > >>> > > >
> >> > > >> >> > > > > > > > > > > >>> > >
> >> > > >> >> > > > > > > > > > > >>> >
> >> > > >> >> > > > > > > > > > > >>>
> >> > > >> >> > > > > > > > > > > >>
> >> > > >> >> > > > > > > > > > > >>
> >> > > >> >> > > > > > > > > > > >
> >> > > >> >> > > > > > > > > > >
> >> > > >> >> > > > > > > > > >
> >> > > >> >> > > > > > > > >
> >> > > >> >> > > > > > > >
> >> > > >> >> > > > > > >
> >> > > >> >> > > > > >
> >> > > >> >> > > > >
> >> > > >> >> > > >
> >> > > >> >> > >
> >> > > >> >> >
> >> > > >> >>
> >> > > >>
> >> > >
> >> >
> >>
>

Re: Time for a 1.5 release?

Posted by Jinfeng Ni <ji...@gmail.com>.

Sean probably is quite close to find a fix for this issue. Let's see
whether he can post a patch soon.


On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
<ch...@gmail.com> wrote:
> DRILL-4323 only happens when assertions are turned on (my install script
> automatically enables assertions).
>
> - Rahul
>
> On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <zf...@maprtech.com> wrote:
>
>> Rahul,
>>
>> Let us know if you can reproduce the issue with assertions turned OFF.  If
>> you cannot, then I think this should not be a blocker for 1.5, and Hsuan
>> will continue to look into fixing this.
>>
>> -- Zelaine
>>
>> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <kn...@maprtech.com>
>> wrote:
>>
>> > I do see the same error with assertion turned on.
>> >
>> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <ji...@gmail.com>
>> wrote:
>> >
>> > > Krystal, by any chance, did you turn on assertion when you start
>> > drillbit?
>> > >
>> > > I think the error in drill-4323 will be raised when assertion = on.
>> > >
>> > >
>> > >
>> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <kn...@maprtech.com>
>> > > wrote:
>> > > > I am running drill-1.4 and is able to run count(*) successfully.
>> > > >
>> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session set
>> > > > `store.hive.optimize_scan_with_native_readers` = true;
>> > > > +-------+--------------------------------------------------------+
>> > > > |  ok   |                        summary                         |
>> > > > +-------+--------------------------------------------------------+
>> > > > | true  | store.hive.optimize_scan_with_native_readers updated.  |
>> > > > +-------+--------------------------------------------------------+
>> > > > 1 row selected (0.374 seconds)
>> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select count(*) from
>> > > > lineitem_parquet_partitioned_hive_string;
>> > > > +---------+
>> > > > | EXPR$0  |
>> > > > +---------+
>> > > > | 60175   |
>> > > > +---------+
>> > > > 1 row selected (2.986 seconds)
>> > > >
>> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <ji...@gmail.com>
>> > > wrote:
>> > > >
>> > > >> Venki and I did some investigation for DRILL-4323. The issue
>> reported
>> > > >> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems to us
>> > > >> this is not a regression from 1.4.0; it's a regression from 1.3.0
>> > > >> probably.
>> > > >>
>> > > >> DRILL-4083 makes the planner to use DrillHiveNativeReader in stead
>> of
>> > > >> HiveReader for "select count(*) from hive_table" query.  However,
>> the
>> > > >> Project after the scan produces empty schema.  Before DRILL-4083,
>> > > >> Drill uses HiveScan, which works fine.
>> > > >>
>> > > >>
>> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
>> > > >> <al...@gmail.com> wrote:
>> > > >> > We could revert the change to the StoragePlugin interface to add
>> the
>> > > >> > getLogicalOptimizerRules
>> > > >> > and getPhysicalOptimizerRules, and only put those in
>> > > >> AbstractStoragePlugin
>> > > >> > with default implementations.
>> > > >> >
>> > > >> > We could then update the docs to tell users to not implement the
>> > > >> interface
>> > > >> > directly, and instead extend the abstract class, which I believe
>> > would
>> > > >> > allow us to make changes like this in the future without breaking
>> > > >> plugins.
>> > > >> > At a major release version we could then just pull up all of the
>> new
>> > > >> > methods we added to the abstract class into the interface, or just
>> > get
>> > > >> rid
>> > > >> > of it all-together.
>> > > >> >
>> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
>> > > >> venki.korukanti@gmail.com>
>> > > >> > wrote:
>> > > >> >
>> > > >> >> Here is the partial fix:
>> > > >> >>
>> > > >> >>
>> > > >>
>> > >
>> >
>> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
>> > > >> >>
>> > > >> >> If the existing StragePlugins are implementing
>> > AbstractStoragePlugin
>> > > >> class
>> > > >> >> this fix should resolve the backward compat issue. If they are
>> > > >> implementing
>> > > >> >> StoragePlugin interface directly then it won't resolve the issue.
>> > Not
>> > > >> sure
>> > > >> >> how to handle it in that case. Let me know if there any ways to
>> > > resolve
>> > > >> >> this issue completely.
>> > > >> >>
>> > > >> >> Thanks
>> > > >> >> Venki
>> > > >> >>
>> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <asinha@maprtech.com
>> >
>> > > >> wrote:
>> > > >> >>
>> > > >> >> > I was able to do the build with maven 3.3.9 (after doing 'brew
>> > > install
>> > > >> >> > maven').  Here are my env variables related to maven:
>> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
>> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
>> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
>> > > >> >> >
>> > > >> >> >
>> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <
>> > > jacques@dremio.com>
>> > > >> >> > wrote:
>> > > >> >> >
>> > > >> >> > > Aman, for reference on the Maven version issue. You can see
>> the
>> > > >> output
>> > > >> >> of
>> > > >> >> > > the shade plugin between the two Apache Builds. If you search
>> > for
>> > > >> >> > > maven-shade-plugin you'll see that build 647 (maven 3.0.5)
>> > fails
>> > > >> >> because
>> > > >> >> > it
>> > > >> >> > > includes a bunch of hadoop (and other libraries that are
>> listed
>> > > as
>> > > >> >> being
>> > > >> >> > > excluded). If you look at build 648 (maven 3.3.3), those
>> items
>> > > are
>> > > >> >> > included
>> > > >> >> > > so that the build completes successfully.
>> > > >> >> > >
>> > > >> >> > > https://builds.apache.org/job/drill-scm/647/consoleText
>> > > >> >> > > https://builds.apache.org/job/drill-scm/648/consoleText
>> > > >> >> > >
>> > > >> >> > > --
>> > > >> >> > > Jacques Nadeau
>> > > >> >> > > CTO and Co-Founder, Dremio
>> > > >> >> > >
>> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
>> > > >> >> > > challapallirahul@gmail.com> wrote:
>> > > >> >> > >
>> > > >> >> > > > Jason,
>> > > >> >> > > >
>> > > >> >> > > > As of now I haven't narrowed down the exact commit which
>> > caused
>> > > >> >> > > DRILL-4323
>> > > >> >> > > > and I do not know the scope of the fix.
>> > > >> >> > > >
>> > > >> >> > > > - Rahul
>> > > >> >> > > >
>> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
>> > > >> >> > > altekrusejason@gmail.com
>> > > >> >> > > > >
>> > > >> >> > > > wrote:
>> > > >> >> > > >
>> > > >> >> > > > > I think it makes sense to wait for Venki's fix of the
>> API,
>> > as
>> > > >> >> > > releasing a
>> > > >> >> > > > > version with the change would make reverting/refactoring
>> > the
>> > > API
>> > > >> >> > > breaking
>> > > >> >> > > > > change less useful.
>> > > >> >> > > > >
>> > > >> >> > > > > Do you have any idea about the scope of fixing the hive
>> > > native
>> > > >> >> > reader,
>> > > >> >> > > or
>> > > >> >> > > > > which change since the 1.4 release broke it? Is someone
>> > > >> available
>> > > >> >> to
>> > > >> >> > > look
>> > > >> >> > > > > at it today?
>> > > >> >> > > > >
>> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
>> > > >> >> > > > > challapallirahul@gmail.com> wrote:
>> > > >> >> > > > >
>> > > >> >> > > > > > Just found an issue with hive native parquet reader
>> > > >> (DRILL-4323).
>> > > >> >> > > This
>> > > >> >> > > > > is a
>> > > >> >> > > > > > regression from 1.4 and in my opinion should be treated
>> > as
>> > > a
>> > > >> >> > blocker.
>> > > >> >> > > > > >
>> > > >> >> > > > > > - Rahul
>> > > >> >> > > > > >
>> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
>> > > >> >> > > > > > venki.korukanti@gmail.com>
>> > > >> >> > > > > > wrote:
>> > > >> >> > > > > >
>> > > >> >> > > > > > > Sorry I didn't realize I was breaking the public
>> > > interface
>> > > >> API.
>> > > >> >> > Let
>> > > >> >> > > > me
>> > > >> >> > > > > > work
>> > > >> >> > > > > > > on a patch to make it backward compatible.
>> > > >> >> > > > > > >
>> > > >> >> > > > > > > Thanks
>> > > >> >> > > > > > > Venki
>> > > >> >> > > > > > >
>> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <
>> > > >> >> > > steven@dremio.com
>> > > >> >> > > > >
>> > > >> >> > > > > > > wrote:
>> > > >> >> > > > > > >
>> > > >> >> > > > > > > > I just wanted to bring up an issue that I just now
>> > > >> >> discovered,
>> > > >> >> > > that
>> > > >> >> > > > > has
>> > > >> >> > > > > > > > caused me a fair amount of grief.
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > > > https://github.com/apache/drill/pull/300/commits
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > > > DRILL-4198 changes a user-facing API, and causes
>> > > >> >> StoragePlugins
>> > > >> >> > > > that
>> > > >> >> > > > > > were
>> > > >> >> > > > > > > > compiled against currently released versions of
>> Drill
>> > > to
>> > > >> no
>> > > >> >> > > longer
>> > > >> >> > > > > > > > functional properly. I would prefer that this
>> > breaking
>> > > >> change
>> > > >> >> > be
>> > > >> >> > > > > > modified
>> > > >> >> > > > > > > > to be backward compatible if possible.
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
>> > > >> >> > > > > > > > altekrusejason@gmail.com>
>> > > >> >> > > > > > > > wrote:
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > > > > Hi Aman,
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > > > This is the failure that he was seeing. He
>> figured
>> > > out
>> > > >> that
>> > > >> >> > the
>> > > >> >> > > > new
>> > > >> >> > > > > > > > > exclusions in jdbc-all were not being respected
>> > when
>> > > the
>> > > >> >> > build
>> > > >> >> > > > was
>> > > >> >> > > > > > run
>> > > >> >> > > > > > > > with
>> > > >> >> > > > > > > > > an older Maven version, causing the jar size to
>> > > increase
>> > > >> >> > > > > > significantly.
>> > > >> >> > > > > > > > He
>> > > >> >> > > > > > > > > added an enforcer to make sure the JAR didn't
>> grow
>> > > >> >> > > unexpectedly.
>> > > >> >> > > > > Can
>> > > >> >> > > > > > > you
>> > > >> >> > > > > > > > > try to update your maven version and re-run the
>> > > build?
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > > > - Jason
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
>> > > >> >> > > > asinha@maprtech.com>
>> > > >> >> > > > > > > > wrote:
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > > > > Jacques, I am getting the following build
>> failure
>> > > on
>> > > >> the
>> > > >> >> > > latest
>> > > >> >> > > > > > > master
>> > > >> >> > > > > > > > > > branch...is this what you saw for the Apache
>> > build
>> > > ?
>> > > >> My
>> > > >> >> > mvn
>> > > >> >> > > > > > version
>> > > >> >> > > > > > > > > output
>> > > >> >> > > > > > > > > > is shown below.  Should we all be upgrading to
>> a
>> > > newer
>> > > >> >> mvn
>> > > >> >> > ?
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
>> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all
>> > ---
>> > > >> >> > > > > > > > > > [WARNING] Rule 0:
>> > > >> >> > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
>> > > >> >> > > > > > > > > failed
>> > > >> >> > > > > > > > > > with message:
>> > > >> >> > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is
>> > > outside
>> > > >> the
>> > > >> >> > > > > expected
>> > > >> >> > > > > > > size
>> > > >> >> > > > > > > > > > range.
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > >                   This is likely due to you
>> > adding
>> > > new
>> > > >> >> > > > > dependencies
>> > > >> >> > > > > > > to
>> > > >> >> > > > > > > > a
>> > > >> >> > > > > > > > > > java-exec and not updating the excludes in this
>> > > >> module.
>> > > >> >> > This
>> > > >> >> > > is
>> > > >> >> > > > > > > > important
>> > > >> >> > > > > > > > > > as it minimizes the size of the dependency of
>> > Drill
>> > > >> >> > > application
>> > > >> >> > > > > > > users.
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > >
>> > > >> >> > > > > >
>> > > >> >> > > > >
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> > > >> >> > > > > > > > > > size (44664121) too large. Max. is
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > >
>> > > >> >> > > > > >
>> > > >> >> > > > >
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill
>> > > asinha$
>> > > >> >> mvn
>> > > >> >> > > > > > --version
>> > > >> >> > > > > > > > > > Apache Maven 3.0.5
>> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
>> > > >> >> > > > > > > > 2013-02-19
>> > > >> >> > > > > > > > > > 05:51:28-0800)
>> > > >> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3
>> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle
>> > Corporation
>> > > >> >> > > > > > > > > > Java home:
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > >
>> > > >> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
>> > > >> >> > > > > > > > > > Default locale: en_US, platform encoding: UTF-8
>> > > >> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5", arch:
>> > > >> "x86_64",
>> > > >> >> > > family:
>> > > >> >> > > > > > "mac"
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques
>> Nadeau <
>> > > >> >> > > > > > jacques@dremio.com>
>> > > >> >> > > > > > > > > > wrote:
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > > > Build back to normal. It looks like the
>> Apache
>> > > >> server
>> > > >> >> was
>> > > >> >> > > > using
>> > > >> >> > > > > > an
>> > > >> >> > > > > > > > old
>> > > >> >> > > > > > > > > > > version of Maven. Once I switched to
>> something
>> > > more
>> > > >> >> > recent,
>> > > >> >> > > > the
>> > > >> >> > > > > > > build
>> > > >> >> > > > > > > > > > > passed.
>> > > >> >> > > > > > > > > > >
>> > > >> >> > > > > > > > > > > --
>> > > >> >> > > > > > > > > > > Jacques Nadeau
>> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
>> > > >> >> > > > > > > > > > >
>> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques
>> > Nadeau <
>> > > >> >> > > > > > > jacques@dremio.com>
>> > > >> >> > > > > > > > > > > wrote:
>> > > >> >> > > > > > > > > > >
>> > > >> >> > > > > > > > > > > > Hmm... this merge caused the Apache build
>> to
>> > > fail.
>> > > >> >> > > > > > > Investigating...
>> > > >> >> > > > > > > > > > > >
>> > > >> >> > > > > > > > > > > > --
>> > > >> >> > > > > > > > > > > > Jacques Nadeau
>> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
>> > > >> >> > > > > > > > > > > >
>> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques
>> > > Nadeau <
>> > > >> >> > > > > > > > jacques@dremio.com>
>> > > >> >> > > > > > > > > > > > wrote:
>> > > >> >> > > > > > > > > > > >
>> > > >> >> > > > > > > > > > > >> I got clean regression runs as well. I've
>> > > merged
>> > > >> the
>> > > >> >> > > > patch.
>> > > >> >> > > > > > > > > > > >>
>> > > >> >> > > > > > > > > > > >> Jason, you want to start the release
>> > process?
>> > > >> >> > > > > > > > > > > >>
>> > > >> >> > > > > > > > > > > >> --
>> > > >> >> > > > > > > > > > > >> Jacques Nadeau
>> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
>> > > >> >> > > > > > > > > > > >>
>> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek
>> > > >> Girish <
>> > > >> >> > > > > > > > agirish@mapr.com
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > > > >> wrote:
>> > > >> >> > > > > > > > > > > >>
>> > > >> >> > > > > > > > > > > >>> Had two clean Functional runs. TPC-H
>> SF100
>> > > was
>> > > >> also
>> > > >> >> > > > > > successful.
>> > > >> >> > > > > > > > > > > >>>
>> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul
>> > > >> >> challapalli <
>> > > >> >> > > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
>> > > >> >> > > > > > > > > > > >>>
>> > > >> >> > > > > > > > > > > >>> > Kicked off a functional run with your
>> > > branch.
>> > > >> >> Will
>> > > >> >> > > let
>> > > >> >> > > > > you
>> > > >> >> > > > > > > know
>> > > >> >> > > > > > > > > > once
>> > > >> >> > > > > > > > > > > it
>> > > >> >> > > > > > > > > > > >>> > finishes
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> >> > > > > > > > > > > >>> > - Rahul
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM,
>> Jacques
>> > > >> Nadeau <
>> > > >> >> > > > > > > > > > jacques@dremio.com>
>> > > >> >> > > > > > > > > > > >>> > wrote:
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I have an
>> > updated
>> > > >> patch
>> > > >> >> > for
>> > > >> >> > > > 4291
>> > > >> >> > > > > > > that
>> > > >> >> > > > > > > > is
>> > > >> >> > > > > > > > > > > >>> ready.
>> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems that
>> something
>> > > isn't
>> > > >> >> > > working
>> > > >> >> > > > > with
>> > > >> >> > > > > > > our
>> > > >> >> > > > > > > > > > > >>> extended
>> > > >> >> > > > > > > > > > > >>> > > tests so I haven't been able to run
>> an
>> > > >> extended
>> > > >> >> > > > > > regression.
>> > > >> >> > > > > > > > > Unit
>> > > >> >> > > > > > > > > > > >>> tests
>> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else possibly able
>> to
>> > > run a
>> > > >> >> > > > regression
>> > > >> >> > > > > > > suite
>> > > >> >> > > > > > > > > > > against
>> > > >> >> > > > > > > > > > > >>> > this
>> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can confirm things
>> > look
>> > > >> good
>> > > >> >> and
>> > > >> >> > > > start
>> > > >> >> > > > > > the
>> > > >> >> > > > > > > > > > release
>> > > >> >> > > > > > > > > > > >>> > > process?
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> > > > > > > > > > > >>> > > thanks,
>> > > >> >> > > > > > > > > > > >>> > > Jacques
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> > > > > > > > > > > >>> > > [1]
>> > > >> >> > > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> > > > > > > > > > > >>> > > --
>> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
>> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM,
>> > Jacques
>> > > >> >> Nadeau
>> > > >> >> > <
>> > > >> >> > > > > > > > > > > jacques@dremio.com
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> >> > > > > > > > > > > >>> > > wrote:
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> > > > > > > > > > > >>> > > > I think the main things are 4196
>> and
>> > > 4291
>> > > >> >> > should
>> > > >> >> > > be
>> > > >> >> > > > > > > > > completed.
>> > > >> >> > > > > > > > > > I
>> > > >> >> > > > > > > > > > > >>> know
>> > > >> >> > > > > > > > > > > >>> > > Amit
>> > > >> >> > > > > > > > > > > >>> > > > was able to reproduce 4196 locally
>> > this
>> > > >> >> weekend
>> > > >> >> > > so
>> > > >> >> > > > I
>> > > >> >> > > > > > > think
>> > > >> >> > > > > > > > > > we're
>> > > >> >> > > > > > > > > > > >>> close
>> > > >> >> > > > > > > > > > > >>> > on
>> > > >> >> > > > > > > > > > > >>> > > > that.
>> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> >> > > > > > > > > > > >>> > > > --
>> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
>> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
>> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM,
>> > > Zelaine
>> > > >> >> Fong
>> > > >> >> > <
>> > > >> >> > > > > > > > > > > zfong@maprtech.com
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> >> > > > > > > > > > > >>> > > wrote:
>> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> >> > > > > > > > > > > >>> > > >> Any updates on this?  What's
>> > blocking
>> > > us
>> > > >> >> from
>> > > >> >> > > > taking
>> > > >> >> > > > > > > this
>> > > >> >> > > > > > > > > to a
>> > > >> >> > > > > > > > > > > >>> vote --
>> > > >> >> > > > > > > > > > > >>> > > the
>> > > >> >> > > > > > > > > > > >>> > > >> sort merge join issues?
>> > > >> >> > > > > > > > > > > >>> > > >>
>> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
>> > > >> >> > > > > > > > > > > >>> > > >>
>> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM,
>> > > Jacques
>> > > >> >> > Nadeau
>> > > >> >> > > <
>> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
>> > > >> >> > > > > > > > > > > >>> > > >> wrote:
>> > > >> >> > > > > > > > > > > >>> > > >>
>> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> > Here are the issues that were
>> > > >> mentioned in
>> > > >> >> > > this
>> > > >> >> > > > > > thread
>> > > >> >> > > > > > > > > along
>> > > >> >> > > > > > > > > > > >>> with a
>> > > >> >> > > > > > > > > > > >>> > > >> > proposed categorization:
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
>> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
>> > > >> >> > > > > > > > > > > >>>
>> > > >> https://issues.apache.org/jira/browse/DRILL-4190
>> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
>> > > >> >> > > > > > > > > > > >>>
>> > > >> https://issues.apache.org/jira/browse/DRILL-4196
>> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246
>> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
>> > > >> >> > > > > > > > > > > >>>
>> > > >> https://issues.apache.org/jira/browse/DRILL-4256
>> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> https://issues.apache.org/jira/browse/DRILL-4278
>> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285
>> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
>> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
>> > > >> >> > > > > > > > > > > >>>
>> > > >> https://issues.apache.org/jira/browse/DRILL-4247
>> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
>> > > >> >> > > > > > > > > > > >>>
>> > > >> https://issues.apache.org/jira/browse/DRILL-4203
>> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
>> > > >> >> > > > > > > > > >
>> https://issues.apache.org/jira/browse/DRILL-4266
>> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> > What do others think? Let's try
>> to
>> > > get
>> > > >> the
>> > > >> >> > > > > blockers
>> > > >> >> > > > > > > > > wrapped
>> > > >> >> > > > > > > > > > up
>> > > >> >> > > > > > > > > > > >>> in
>> > > >> >> > > > > > > > > > > >>> > the
>> > > >> >> > > > > > > > > > > >>> > > >> next
>> > > >> >> > > > > > > > > > > >>> > > >> > day or two and start a release
>> > > vote...
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> > --
>> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
>> > > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM,
>> > > Jason
>> > > >> >> > > Altekruse
>> > > >> >> > > > <
>> > > >> >> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
>> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> > > With the allocator changes
>> > merged
>> > > and
>> > > >> >> > about
>> > > >> >> > > a
>> > > >> >> > > > > > month
>> > > >> >> > > > > > > > > since
>> > > >> >> > > > > > > > > > > the
>> > > >> >> > > > > > > > > > > >>> last
>> > > >> >> > > > > > > > > > > >>> > > >> > release
>> > > >> >> > > > > > > > > > > >>> > > >> > > I think it would be good to
>> > start
>> > > a
>> > > >> vote
>> > > >> >> > > > soon. I
>> > > >> >> > > > > > > would
>> > > >> >> > > > > > > > > > like
>> > > >> >> > > > > > > > > > > to
>> > > >> >> > > > > > > > > > > >>> > > >> volunteer
>> > > >> >> > > > > > > > > > > >>> > > >> > to
>> > > >> >> > > > > > > > > > > >>> > > >> > > be release manager.
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> > > I know that there were some
>> > issues
>> > > >> that
>> > > >> >> > were
>> > > >> >> > > > > > > > identified
>> > > >> >> > > > > > > > > > > after
>> > > >> >> > > > > > > > > > > >>> the
>> > > >> >> > > > > > > > > > > >>> > > >> > transfer
>> > > >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I think that
>> > > these
>> > > >> >> > issues
>> > > >> >> > > > > should
>> > > >> >> > > > > > > be
>> > > >> >> > > > > > > > > > fixed
>> > > >> >> > > > > > > > > > > >>> before
>> > > >> >> > > > > > > > > > > >>> > > we
>> > > >> >> > > > > > > > > > > >>> > > >> > cut a
>> > > >> >> > > > > > > > > > > >>> > > >> > > release candidate.
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at the associated
>> > > JIRAs
>> > > >> it
>> > > >> >> > > looked
>> > > >> >> > > > > > like
>> > > >> >> > > > > > > > > there
>> > > >> >> > > > > > > > > > > was
>> > > >> >> > > > > > > > > > > >>> a
>> > > >> >> > > > > > > > > > > >>> > > >> possible
>> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix just adjusting
>> > the
>> > > >> >> > > > > > > > > > max_query_memory_per_node
>> > > >> >> > > > > > > > > > > >>> > option,
>> > > >> >> > > > > > > > > > > >>> > > >> and
>> > > >> >> > > > > > > > > > > >>> > > >> > > some more involved work to
>> > change
>> > > >> how we
>> > > >> >> > > > > determine
>> > > >> >> > > > > > > the
>> > > >> >> > > > > > > > > > > correct
>> > > >> >> > > > > > > > > > > >>> > time
>> > > >> >> > > > > > > > > > > >>> > > to
>> > > >> >> > > > > > > > > > > >>> > > >> > > spill during external sort. I
>> > > >> believe it
>> > > >> >> > > makes
>> > > >> >> > > > > > sense
>> > > >> >> > > > > > > > to
>> > > >> >> > > > > > > > > > make
>> > > >> >> > > > > > > > > > > >>> > > external
>> > > >> >> > > > > > > > > > > >>> > > >> > sort
>> > > >> >> > > > > > > > > > > >>> > > >> > > work well with the newly
>> > improved
>> > > >> memory
>> > > >> >> > > > > > accounting
>> > > >> >> > > > > > > > > before
>> > > >> >> > > > > > > > > > > >>> > cutting a
>> > > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not sure how
>> > much
>> > > >> work
>> > > >> >> is
>> > > >> >> > > > left
>> > > >> >> > > > > to
>> > > >> >> > > > > > > be
>> > > >> >> > > > > > > > > done
>> > > >> >> > > > > > > > > > > >>> there.
>> > > >> >> > > > > > > > > > > >>> > > [1]
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond with your
>> > thoughts
>> > > on
>> > > >> a
>> > > >> >> > > release
>> > > >> >> > > > > > soon
>> > > >> >> > > > > > > > and
>> > > >> >> > > > > > > > > > any
>> > > >> >> > > > > > > > > > > >>> JIRAs
>> > > >> >> > > > > > > > > > > >>> > > you
>> > > >> >> > > > > > > > > > > >>> > > >> > would
>> > > >> >> > > > > > > > > > > >>> > > >> > > like to include in the
>> release.
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
>> > > >> >> > > > > > > > https://issues.apache.org/jira/browse/DRILL-4243
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
>> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >>
>> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> >> > > > > > > > > > > >>>
>> > > >> >> > > > > > > > > > > >>
>> > > >> >> > > > > > > > > > > >>
>> > > >> >> > > > > > > > > > > >
>> > > >> >> > > > > > > > > > >
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > >
>> > > >> >> > > > > >
>> > > >> >> > > > >
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>>

Re: Time for a 1.5 release?

Posted by rahul challapalli <ch...@gmail.com>.

DRILL-4323 only happens when assertions are turned on (my install script
automatically enables assertions).

- Rahul

On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <zf...@maprtech.com> wrote:

> Rahul,
>
> Let us know if you can reproduce the issue with assertions turned OFF.  If
> you cannot, then I think this should not be a blocker for 1.5, and Hsuan
> will continue to look into fixing this.
>
> -- Zelaine
>
> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <kn...@maprtech.com>
> wrote:
>
> > I do see the same error with assertion turned on.
> >
> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <ji...@gmail.com>
> wrote:
> >
> > > Krystal, by any chance, did you turn on assertion when you start
> > drillbit?
> > >
> > > I think the error in drill-4323 will be raised when assertion = on.
> > >
> > >
> > >
> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <kn...@maprtech.com>
> > > wrote:
> > > > I am running drill-1.4 and is able to run count(*) successfully.
> > > >
> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session set
> > > > `store.hive.optimize_scan_with_native_readers` = true;
> > > > +-------+--------------------------------------------------------+
> > > > |  ok   |                        summary                         |
> > > > +-------+--------------------------------------------------------+
> > > > | true  | store.hive.optimize_scan_with_native_readers updated.  |
> > > > +-------+--------------------------------------------------------+
> > > > 1 row selected (0.374 seconds)
> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select count(*) from
> > > > lineitem_parquet_partitioned_hive_string;
> > > > +---------+
> > > > | EXPR$0  |
> > > > +---------+
> > > > | 60175   |
> > > > +---------+
> > > > 1 row selected (2.986 seconds)
> > > >
> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <ji...@gmail.com>
> > > wrote:
> > > >
> > > >> Venki and I did some investigation for DRILL-4323. The issue
> reported
> > > >> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems to us
> > > >> this is not a regression from 1.4.0; it's a regression from 1.3.0
> > > >> probably.
> > > >>
> > > >> DRILL-4083 makes the planner to use DrillHiveNativeReader in stead
> of
> > > >> HiveReader for "select count(*) from hive_table" query.  However,
> the
> > > >> Project after the scan produces empty schema.  Before DRILL-4083,
> > > >> Drill uses HiveScan, which works fine.
> > > >>
> > > >>
> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
> > > >> <al...@gmail.com> wrote:
> > > >> > We could revert the change to the StoragePlugin interface to add
> the
> > > >> > getLogicalOptimizerRules
> > > >> > and getPhysicalOptimizerRules, and only put those in
> > > >> AbstractStoragePlugin
> > > >> > with default implementations.
> > > >> >
> > > >> > We could then update the docs to tell users to not implement the
> > > >> interface
> > > >> > directly, and instead extend the abstract class, which I believe
> > would
> > > >> > allow us to make changes like this in the future without breaking
> > > >> plugins.
> > > >> > At a major release version we could then just pull up all of the
> new
> > > >> > methods we added to the abstract class into the interface, or just
> > get
> > > >> rid
> > > >> > of it all-together.
> > > >> >
> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
> > > >> venki.korukanti@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> >> Here is the partial fix:
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> > > >> >>
> > > >> >> If the existing StragePlugins are implementing
> > AbstractStoragePlugin
> > > >> class
> > > >> >> this fix should resolve the backward compat issue. If they are
> > > >> implementing
> > > >> >> StoragePlugin interface directly then it won't resolve the issue.
> > Not
> > > >> sure
> > > >> >> how to handle it in that case. Let me know if there any ways to
> > > resolve
> > > >> >> this issue completely.
> > > >> >>
> > > >> >> Thanks
> > > >> >> Venki
> > > >> >>
> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <asinha@maprtech.com
> >
> > > >> wrote:
> > > >> >>
> > > >> >> > I was able to do the build with maven 3.3.9 (after doing 'brew
> > > install
> > > >> >> > maven').  Here are my env variables related to maven:
> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> > > >> >> >
> > > >> >> >
> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <
> > > jacques@dremio.com>
> > > >> >> > wrote:
> > > >> >> >
> > > >> >> > > Aman, for reference on the Maven version issue. You can see
> the
> > > >> output
> > > >> >> of
> > > >> >> > > the shade plugin between the two Apache Builds. If you search
> > for
> > > >> >> > > maven-shade-plugin you'll see that build 647 (maven 3.0.5)
> > fails
> > > >> >> because
> > > >> >> > it
> > > >> >> > > includes a bunch of hadoop (and other libraries that are
> listed
> > > as
> > > >> >> being
> > > >> >> > > excluded). If you look at build 648 (maven 3.3.3), those
> items
> > > are
> > > >> >> > included
> > > >> >> > > so that the build completes successfully.
> > > >> >> > >
> > > >> >> > > https://builds.apache.org/job/drill-scm/647/consoleText
> > > >> >> > > https://builds.apache.org/job/drill-scm/648/consoleText
> > > >> >> > >
> > > >> >> > > --
> > > >> >> > > Jacques Nadeau
> > > >> >> > > CTO and Co-Founder, Dremio
> > > >> >> > >
> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
> > > >> >> > > challapallirahul@gmail.com> wrote:
> > > >> >> > >
> > > >> >> > > > Jason,
> > > >> >> > > >
> > > >> >> > > > As of now I haven't narrowed down the exact commit which
> > caused
> > > >> >> > > DRILL-4323
> > > >> >> > > > and I do not know the scope of the fix.
> > > >> >> > > >
> > > >> >> > > > - Rahul
> > > >> >> > > >
> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
> > > >> >> > > altekrusejason@gmail.com
> > > >> >> > > > >
> > > >> >> > > > wrote:
> > > >> >> > > >
> > > >> >> > > > > I think it makes sense to wait for Venki's fix of the
> API,
> > as
> > > >> >> > > releasing a
> > > >> >> > > > > version with the change would make reverting/refactoring
> > the
> > > API
> > > >> >> > > breaking
> > > >> >> > > > > change less useful.
> > > >> >> > > > >
> > > >> >> > > > > Do you have any idea about the scope of fixing the hive
> > > native
> > > >> >> > reader,
> > > >> >> > > or
> > > >> >> > > > > which change since the 1.4 release broke it? Is someone
> > > >> available
> > > >> >> to
> > > >> >> > > look
> > > >> >> > > > > at it today?
> > > >> >> > > > >
> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> > > >> >> > > > > challapallirahul@gmail.com> wrote:
> > > >> >> > > > >
> > > >> >> > > > > > Just found an issue with hive native parquet reader
> > > >> (DRILL-4323).
> > > >> >> > > This
> > > >> >> > > > > is a
> > > >> >> > > > > > regression from 1.4 and in my opinion should be treated
> > as
> > > a
> > > >> >> > blocker.
> > > >> >> > > > > >
> > > >> >> > > > > > - Rahul
> > > >> >> > > > > >
> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> > > >> >> > > > > > venki.korukanti@gmail.com>
> > > >> >> > > > > > wrote:
> > > >> >> > > > > >
> > > >> >> > > > > > > Sorry I didn't realize I was breaking the public
> > > interface
> > > >> API.
> > > >> >> > Let
> > > >> >> > > > me
> > > >> >> > > > > > work
> > > >> >> > > > > > > on a patch to make it backward compatible.
> > > >> >> > > > > > >
> > > >> >> > > > > > > Thanks
> > > >> >> > > > > > > Venki
> > > >> >> > > > > > >
> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <
> > > >> >> > > steven@dremio.com
> > > >> >> > > > >
> > > >> >> > > > > > > wrote:
> > > >> >> > > > > > >
> > > >> >> > > > > > > > I just wanted to bring up an issue that I just now
> > > >> >> discovered,
> > > >> >> > > that
> > > >> >> > > > > has
> > > >> >> > > > > > > > caused me a fair amount of grief.
> > > >> >> > > > > > > >
> > > >> >> > > > > > > > https://github.com/apache/drill/pull/300/commits
> > > >> >> > > > > > > >
> > > >> >> > > > > > > > DRILL-4198 changes a user-facing API, and causes
> > > >> >> StoragePlugins
> > > >> >> > > > that
> > > >> >> > > > > > were
> > > >> >> > > > > > > > compiled against currently released versions of
> Drill
> > > to
> > > >> no
> > > >> >> > > longer
> > > >> >> > > > > > > > functional properly. I would prefer that this
> > breaking
> > > >> change
> > > >> >> > be
> > > >> >> > > > > > modified
> > > >> >> > > > > > > > to be backward compatible if possible.
> > > >> >> > > > > > > >
> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> > > >> >> > > > > > > > altekrusejason@gmail.com>
> > > >> >> > > > > > > > wrote:
> > > >> >> > > > > > > >
> > > >> >> > > > > > > > > Hi Aman,
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > This is the failure that he was seeing. He
> figured
> > > out
> > > >> that
> > > >> >> > the
> > > >> >> > > > new
> > > >> >> > > > > > > > > exclusions in jdbc-all were not being respected
> > when
> > > the
> > > >> >> > build
> > > >> >> > > > was
> > > >> >> > > > > > run
> > > >> >> > > > > > > > with
> > > >> >> > > > > > > > > an older Maven version, causing the jar size to
> > > increase
> > > >> >> > > > > > significantly.
> > > >> >> > > > > > > > He
> > > >> >> > > > > > > > > added an enforcer to make sure the JAR didn't
> grow
> > > >> >> > > unexpectedly.
> > > >> >> > > > > Can
> > > >> >> > > > > > > you
> > > >> >> > > > > > > > > try to update your maven version and re-run the
> > > build?
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > - Jason
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
> > > >> >> > > > asinha@maprtech.com>
> > > >> >> > > > > > > > wrote:
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > > Jacques, I am getting the following build
> failure
> > > on
> > > >> the
> > > >> >> > > latest
> > > >> >> > > > > > > master
> > > >> >> > > > > > > > > > branch...is this what you saw for the Apache
> > build
> > > ?
> > > >> My
> > > >> >> > mvn
> > > >> >> > > > > > version
> > > >> >> > > > > > > > > output
> > > >> >> > > > > > > > > > is shown below.  Should we all be upgrading to
> a
> > > newer
> > > >> >> mvn
> > > >> >> > ?
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all
> > ---
> > > >> >> > > > > > > > > > [WARNING] Rule 0:
> > > >> >> > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
> > > >> >> > > > > > > > > failed
> > > >> >> > > > > > > > > > with message:
> > > >> >> > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is
> > > outside
> > > >> the
> > > >> >> > > > > expected
> > > >> >> > > > > > > size
> > > >> >> > > > > > > > > > range.
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > >                   This is likely due to you
> > adding
> > > new
> > > >> >> > > > > dependencies
> > > >> >> > > > > > > to
> > > >> >> > > > > > > > a
> > > >> >> > > > > > > > > > java-exec and not updating the excludes in this
> > > >> module.
> > > >> >> > This
> > > >> >> > > is
> > > >> >> > > > > > > > important
> > > >> >> > > > > > > > > > as it minimizes the size of the dependency of
> > Drill
> > > >> >> > > application
> > > >> >> > > > > > > users.
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > >
> > > >> >> > > > > > >
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > >> >> > > > > > > > > > size (44664121) too large. Max. is
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > >
> > > >> >> > > > > > >
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill
> > > asinha$
> > > >> >> mvn
> > > >> >> > > > > > --version
> > > >> >> > > > > > > > > > Apache Maven 3.0.5
> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > > >> >> > > > > > > > 2013-02-19
> > > >> >> > > > > > > > > > 05:51:28-0800)
> > > >> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3
> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle
> > Corporation
> > > >> >> > > > > > > > > > Java home:
> > > >> >> > > > > > > > > >
> > > >> >> > > > >
> > > >> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > > >> >> > > > > > > > > > Default locale: en_US, platform encoding: UTF-8
> > > >> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5", arch:
> > > >> "x86_64",
> > > >> >> > > family:
> > > >> >> > > > > > "mac"
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques
> Nadeau <
> > > >> >> > > > > > jacques@dremio.com>
> > > >> >> > > > > > > > > > wrote:
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > > > Build back to normal. It looks like the
> Apache
> > > >> server
> > > >> >> was
> > > >> >> > > > using
> > > >> >> > > > > > an
> > > >> >> > > > > > > > old
> > > >> >> > > > > > > > > > > version of Maven. Once I switched to
> something
> > > more
> > > >> >> > recent,
> > > >> >> > > > the
> > > >> >> > > > > > > build
> > > >> >> > > > > > > > > > > passed.
> > > >> >> > > > > > > > > > >
> > > >> >> > > > > > > > > > > --
> > > >> >> > > > > > > > > > > Jacques Nadeau
> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
> > > >> >> > > > > > > > > > >
> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques
> > Nadeau <
> > > >> >> > > > > > > jacques@dremio.com>
> > > >> >> > > > > > > > > > > wrote:
> > > >> >> > > > > > > > > > >
> > > >> >> > > > > > > > > > > > Hmm... this merge caused the Apache build
> to
> > > fail.
> > > >> >> > > > > > > Investigating...
> > > >> >> > > > > > > > > > > >
> > > >> >> > > > > > > > > > > > --
> > > >> >> > > > > > > > > > > > Jacques Nadeau
> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
> > > >> >> > > > > > > > > > > >
> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques
> > > Nadeau <
> > > >> >> > > > > > > > jacques@dremio.com>
> > > >> >> > > > > > > > > > > > wrote:
> > > >> >> > > > > > > > > > > >
> > > >> >> > > > > > > > > > > >> I got clean regression runs as well. I've
> > > merged
> > > >> the
> > > >> >> > > > patch.
> > > >> >> > > > > > > > > > > >>
> > > >> >> > > > > > > > > > > >> Jason, you want to start the release
> > process?
> > > >> >> > > > > > > > > > > >>
> > > >> >> > > > > > > > > > > >> --
> > > >> >> > > > > > > > > > > >> Jacques Nadeau
> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
> > > >> >> > > > > > > > > > > >>
> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek
> > > >> Girish <
> > > >> >> > > > > > > > agirish@mapr.com
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > > > >> wrote:
> > > >> >> > > > > > > > > > > >>
> > > >> >> > > > > > > > > > > >>> Had two clean Functional runs. TPC-H
> SF100
> > > was
> > > >> also
> > > >> >> > > > > > successful.
> > > >> >> > > > > > > > > > > >>>
> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul
> > > >> >> challapalli <
> > > >> >> > > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
> > > >> >> > > > > > > > > > > >>>
> > > >> >> > > > > > > > > > > >>> > Kicked off a functional run with your
> > > branch.
> > > >> >> Will
> > > >> >> > > let
> > > >> >> > > > > you
> > > >> >> > > > > > > know
> > > >> >> > > > > > > > > > once
> > > >> >> > > > > > > > > > > it
> > > >> >> > > > > > > > > > > >>> > finishes
> > > >> >> > > > > > > > > > > >>> >
> > > >> >> > > > > > > > > > > >>> > - Rahul
> > > >> >> > > > > > > > > > > >>> >
> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM,
> Jacques
> > > >> Nadeau <
> > > >> >> > > > > > > > > > jacques@dremio.com>
> > > >> >> > > > > > > > > > > >>> > wrote:
> > > >> >> > > > > > > > > > > >>> >
> > > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I have an
> > updated
> > > >> patch
> > > >> >> > for
> > > >> >> > > > 4291
> > > >> >> > > > > > > that
> > > >> >> > > > > > > > is
> > > >> >> > > > > > > > > > > >>> ready.
> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems that
> something
> > > isn't
> > > >> >> > > working
> > > >> >> > > > > with
> > > >> >> > > > > > > our
> > > >> >> > > > > > > > > > > >>> extended
> > > >> >> > > > > > > > > > > >>> > > tests so I haven't been able to run
> an
> > > >> extended
> > > >> >> > > > > > regression.
> > > >> >> > > > > > > > > Unit
> > > >> >> > > > > > > > > > > >>> tests
> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else possibly able
> to
> > > run a
> > > >> >> > > > regression
> > > >> >> > > > > > > suite
> > > >> >> > > > > > > > > > > against
> > > >> >> > > > > > > > > > > >>> > this
> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can confirm things
> > look
> > > >> good
> > > >> >> and
> > > >> >> > > > start
> > > >> >> > > > > > the
> > > >> >> > > > > > > > > > release
> > > >> >> > > > > > > > > > > >>> > > process?
> > > >> >> > > > > > > > > > > >>> > >
> > > >> >> > > > > > > > > > > >>> > > thanks,
> > > >> >> > > > > > > > > > > >>> > > Jacques
> > > >> >> > > > > > > > > > > >>> > >
> > > >> >> > > > > > > > > > > >>> > > [1]
> > > >> >> > > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > >> >> > > > > > > > > > > >>> > >
> > > >> >> > > > > > > > > > > >>> > > --
> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
> > > >> >> > > > > > > > > > > >>> > >
> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM,
> > Jacques
> > > >> >> Nadeau
> > > >> >> > <
> > > >> >> > > > > > > > > > > jacques@dremio.com
> > > >> >> > > > > > > > > > > >>> >
> > > >> >> > > > > > > > > > > >>> > > wrote:
> > > >> >> > > > > > > > > > > >>> > >
> > > >> >> > > > > > > > > > > >>> > > > I think the main things are 4196
> and
> > > 4291
> > > >> >> > should
> > > >> >> > > be
> > > >> >> > > > > > > > > completed.
> > > >> >> > > > > > > > > > I
> > > >> >> > > > > > > > > > > >>> know
> > > >> >> > > > > > > > > > > >>> > > Amit
> > > >> >> > > > > > > > > > > >>> > > > was able to reproduce 4196 locally
> > this
> > > >> >> weekend
> > > >> >> > > so
> > > >> >> > > > I
> > > >> >> > > > > > > think
> > > >> >> > > > > > > > > > we're
> > > >> >> > > > > > > > > > > >>> close
> > > >> >> > > > > > > > > > > >>> > on
> > > >> >> > > > > > > > > > > >>> > > > that.
> > > >> >> > > > > > > > > > > >>> > > >
> > > >> >> > > > > > > > > > > >>> > > > --
> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
> > > >> >> > > > > > > > > > > >>> > > >
> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM,
> > > Zelaine
> > > >> >> Fong
> > > >> >> > <
> > > >> >> > > > > > > > > > > zfong@maprtech.com
> > > >> >> > > > > > > > > > > >>> >
> > > >> >> > > > > > > > > > > >>> > > wrote:
> > > >> >> > > > > > > > > > > >>> > > >
> > > >> >> > > > > > > > > > > >>> > > >> Any updates on this?  What's
> > blocking
> > > us
> > > >> >> from
> > > >> >> > > > taking
> > > >> >> > > > > > > this
> > > >> >> > > > > > > > > to a
> > > >> >> > > > > > > > > > > >>> vote --
> > > >> >> > > > > > > > > > > >>> > > the
> > > >> >> > > > > > > > > > > >>> > > >> sort merge join issues?
> > > >> >> > > > > > > > > > > >>> > > >>
> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
> > > >> >> > > > > > > > > > > >>> > > >>
> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM,
> > > Jacques
> > > >> >> > Nadeau
> > > >> >> > > <
> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
> > > >> >> > > > > > > > > > > >>> > > >> wrote:
> > > >> >> > > > > > > > > > > >>> > > >>
> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> >> > > > > > > > > > > >>> > > >> > Here are the issues that were
> > > >> mentioned in
> > > >> >> > > this
> > > >> >> > > > > > thread
> > > >> >> > > > > > > > > along
> > > >> >> > > > > > > > > > > >>> with a
> > > >> >> > > > > > > > > > > >>> > > >> > proposed categorization:
> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> > > >> >> > > > > > > > > > > >>>
> > > >> https://issues.apache.org/jira/browse/DRILL-4190
> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> > > >> >> > > > > > > > > > > >>>
> > > >> https://issues.apache.org/jira/browse/DRILL-4196
> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > > >> >> > > > > > > > > > > >>> > >
> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246
> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
> > > >> >> > > > > > > > > > > >>>
> > > >> https://issues.apache.org/jira/browse/DRILL-4256
> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
> > > >> >> > > > > > > > > > > >>> >
> > > >> https://issues.apache.org/jira/browse/DRILL-4278
> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
> > > >> >> > > > > > > > > > > >>> > >
> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285
> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
> > > >> >> > > > > > > > > > > >>>
> > > >> https://issues.apache.org/jira/browse/DRILL-4247
> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
> > > >> >> > > > > > > > > > > >>>
> > > >> https://issues.apache.org/jira/browse/DRILL-4203
> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
> > > >> >> > > > > > > > > >
> https://issues.apache.org/jira/browse/DRILL-4266
> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > > >> >> > > > > > > > > > > >>> > >
> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131
> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> >> > > > > > > > > > > >>> > > >> > What do others think? Let's try
> to
> > > get
> > > >> the
> > > >> >> > > > > blockers
> > > >> >> > > > > > > > > wrapped
> > > >> >> > > > > > > > > > up
> > > >> >> > > > > > > > > > > >>> in
> > > >> >> > > > > > > > > > > >>> > the
> > > >> >> > > > > > > > > > > >>> > > >> next
> > > >> >> > > > > > > > > > > >>> > > >> > day or two and start a release
> > > vote...
> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> >> > > > > > > > > > > >>> > > >> > --
> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
> > > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM,
> > > Jason
> > > >> >> > > Altekruse
> > > >> >> > > > <
> > > >> >> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> >> > > > > > > > > > > >>> > > >> > > With the allocator changes
> > merged
> > > and
> > > >> >> > about
> > > >> >> > > a
> > > >> >> > > > > > month
> > > >> >> > > > > > > > > since
> > > >> >> > > > > > > > > > > the
> > > >> >> > > > > > > > > > > >>> last
> > > >> >> > > > > > > > > > > >>> > > >> > release
> > > >> >> > > > > > > > > > > >>> > > >> > > I think it would be good to
> > start
> > > a
> > > >> vote
> > > >> >> > > > soon. I
> > > >> >> > > > > > > would
> > > >> >> > > > > > > > > > like
> > > >> >> > > > > > > > > > > to
> > > >> >> > > > > > > > > > > >>> > > >> volunteer
> > > >> >> > > > > > > > > > > >>> > > >> > to
> > > >> >> > > > > > > > > > > >>> > > >> > > be release manager.
> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> >> > > > > > > > > > > >>> > > >> > > I know that there were some
> > issues
> > > >> that
> > > >> >> > were
> > > >> >> > > > > > > > identified
> > > >> >> > > > > > > > > > > after
> > > >> >> > > > > > > > > > > >>> the
> > > >> >> > > > > > > > > > > >>> > > >> > transfer
> > > >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I think that
> > > these
> > > >> >> > issues
> > > >> >> > > > > should
> > > >> >> > > > > > > be
> > > >> >> > > > > > > > > > fixed
> > > >> >> > > > > > > > > > > >>> before
> > > >> >> > > > > > > > > > > >>> > > we
> > > >> >> > > > > > > > > > > >>> > > >> > cut a
> > > >> >> > > > > > > > > > > >>> > > >> > > release candidate.
> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at the associated
> > > JIRAs
> > > >> it
> > > >> >> > > looked
> > > >> >> > > > > > like
> > > >> >> > > > > > > > > there
> > > >> >> > > > > > > > > > > was
> > > >> >> > > > > > > > > > > >>> a
> > > >> >> > > > > > > > > > > >>> > > >> possible
> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix just adjusting
> > the
> > > >> >> > > > > > > > > > max_query_memory_per_node
> > > >> >> > > > > > > > > > > >>> > option,
> > > >> >> > > > > > > > > > > >>> > > >> and
> > > >> >> > > > > > > > > > > >>> > > >> > > some more involved work to
> > change
> > > >> how we
> > > >> >> > > > > determine
> > > >> >> > > > > > > the
> > > >> >> > > > > > > > > > > correct
> > > >> >> > > > > > > > > > > >>> > time
> > > >> >> > > > > > > > > > > >>> > > to
> > > >> >> > > > > > > > > > > >>> > > >> > > spill during external sort. I
> > > >> believe it
> > > >> >> > > makes
> > > >> >> > > > > > sense
> > > >> >> > > > > > > > to
> > > >> >> > > > > > > > > > make
> > > >> >> > > > > > > > > > > >>> > > external
> > > >> >> > > > > > > > > > > >>> > > >> > sort
> > > >> >> > > > > > > > > > > >>> > > >> > > work well with the newly
> > improved
> > > >> memory
> > > >> >> > > > > > accounting
> > > >> >> > > > > > > > > before
> > > >> >> > > > > > > > > > > >>> > cutting a
> > > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not sure how
> > much
> > > >> work
> > > >> >> is
> > > >> >> > > > left
> > > >> >> > > > > to
> > > >> >> > > > > > > be
> > > >> >> > > > > > > > > done
> > > >> >> > > > > > > > > > > >>> there.
> > > >> >> > > > > > > > > > > >>> > > [1]
> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond with your
> > thoughts
> > > on
> > > >> a
> > > >> >> > > release
> > > >> >> > > > > > soon
> > > >> >> > > > > > > > and
> > > >> >> > > > > > > > > > any
> > > >> >> > > > > > > > > > > >>> JIRAs
> > > >> >> > > > > > > > > > > >>> > > you
> > > >> >> > > > > > > > > > > >>> > > >> > would
> > > >> >> > > > > > > > > > > >>> > > >> > > like to include in the
> release.
> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
> > > >> >> > > > > > > > https://issues.apache.org/jira/browse/DRILL-4243
> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
> > > >> >> > > > > > > > > > > >>> > > >> > >
> > > >> >> > > > > > > > > > > >>> > > >> >
> > > >> >> > > > > > > > > > > >>> > > >>
> > > >> >> > > > > > > > > > > >>> > > >
> > > >> >> > > > > > > > > > > >>> > > >
> > > >> >> > > > > > > > > > > >>> > >
> > > >> >> > > > > > > > > > > >>> >
> > > >> >> > > > > > > > > > > >>>
> > > >> >> > > > > > > > > > > >>
> > > >> >> > > > > > > > > > > >>
> > > >> >> > > > > > > > > > > >
> > > >> >> > > > > > > > > > >
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > >
> > > >> >> > > > > > >
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Zelaine Fong <zf...@maprtech.com>.

Rahul,

Let us know if you can reproduce the issue with assertions turned OFF.  If
you cannot, then I think this should not be a blocker for 1.5, and Hsuan
will continue to look into fixing this.

-- Zelaine

On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <kn...@maprtech.com>
wrote:

> I do see the same error with assertion turned on.
>
> On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <ji...@gmail.com> wrote:
>
> > Krystal, by any chance, did you turn on assertion when you start
> drillbit?
> >
> > I think the error in drill-4323 will be raised when assertion = on.
> >
> >
> >
> > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <kn...@maprtech.com>
> > wrote:
> > > I am running drill-1.4 and is able to run count(*) successfully.
> > >
> > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session set
> > > `store.hive.optimize_scan_with_native_readers` = true;
> > > +-------+--------------------------------------------------------+
> > > |  ok   |                        summary                         |
> > > +-------+--------------------------------------------------------+
> > > | true  | store.hive.optimize_scan_with_native_readers updated.  |
> > > +-------+--------------------------------------------------------+
> > > 1 row selected (0.374 seconds)
> > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select count(*) from
> > > lineitem_parquet_partitioned_hive_string;
> > > +---------+
> > > | EXPR$0  |
> > > +---------+
> > > | 60175   |
> > > +---------+
> > > 1 row selected (2.986 seconds)
> > >
> > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <ji...@gmail.com>
> > wrote:
> > >
> > >> Venki and I did some investigation for DRILL-4323. The issue reported
> > >> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems to us
> > >> this is not a regression from 1.4.0; it's a regression from 1.3.0
> > >> probably.
> > >>
> > >> DRILL-4083 makes the planner to use DrillHiveNativeReader in stead of
> > >> HiveReader for "select count(*) from hive_table" query.  However, the
> > >> Project after the scan produces empty schema.  Before DRILL-4083,
> > >> Drill uses HiveScan, which works fine.
> > >>
> > >>
> > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
> > >> <al...@gmail.com> wrote:
> > >> > We could revert the change to the StoragePlugin interface to add the
> > >> > getLogicalOptimizerRules
> > >> > and getPhysicalOptimizerRules, and only put those in
> > >> AbstractStoragePlugin
> > >> > with default implementations.
> > >> >
> > >> > We could then update the docs to tell users to not implement the
> > >> interface
> > >> > directly, and instead extend the abstract class, which I believe
> would
> > >> > allow us to make changes like this in the future without breaking
> > >> plugins.
> > >> > At a major release version we could then just pull up all of the new
> > >> > methods we added to the abstract class into the interface, or just
> get
> > >> rid
> > >> > of it all-together.
> > >> >
> > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
> > >> venki.korukanti@gmail.com>
> > >> > wrote:
> > >> >
> > >> >> Here is the partial fix:
> > >> >>
> > >> >>
> > >>
> >
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> > >> >>
> > >> >> If the existing StragePlugins are implementing
> AbstractStoragePlugin
> > >> class
> > >> >> this fix should resolve the backward compat issue. If they are
> > >> implementing
> > >> >> StoragePlugin interface directly then it won't resolve the issue.
> Not
> > >> sure
> > >> >> how to handle it in that case. Let me know if there any ways to
> > resolve
> > >> >> this issue completely.
> > >> >>
> > >> >> Thanks
> > >> >> Venki
> > >> >>
> > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <as...@maprtech.com>
> > >> wrote:
> > >> >>
> > >> >> > I was able to do the build with maven 3.3.9 (after doing 'brew
> > install
> > >> >> > maven').  Here are my env variables related to maven:
> > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> > >> >> >
> > >> >> >
> > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <
> > jacques@dremio.com>
> > >> >> > wrote:
> > >> >> >
> > >> >> > > Aman, for reference on the Maven version issue. You can see the
> > >> output
> > >> >> of
> > >> >> > > the shade plugin between the two Apache Builds. If you search
> for
> > >> >> > > maven-shade-plugin you'll see that build 647 (maven 3.0.5)
> fails
> > >> >> because
> > >> >> > it
> > >> >> > > includes a bunch of hadoop (and other libraries that are listed
> > as
> > >> >> being
> > >> >> > > excluded). If you look at build 648 (maven 3.3.3), those items
> > are
> > >> >> > included
> > >> >> > > so that the build completes successfully.
> > >> >> > >
> > >> >> > > https://builds.apache.org/job/drill-scm/647/consoleText
> > >> >> > > https://builds.apache.org/job/drill-scm/648/consoleText
> > >> >> > >
> > >> >> > > --
> > >> >> > > Jacques Nadeau
> > >> >> > > CTO and Co-Founder, Dremio
> > >> >> > >
> > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
> > >> >> > > challapallirahul@gmail.com> wrote:
> > >> >> > >
> > >> >> > > > Jason,
> > >> >> > > >
> > >> >> > > > As of now I haven't narrowed down the exact commit which
> caused
> > >> >> > > DRILL-4323
> > >> >> > > > and I do not know the scope of the fix.
> > >> >> > > >
> > >> >> > > > - Rahul
> > >> >> > > >
> > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
> > >> >> > > altekrusejason@gmail.com
> > >> >> > > > >
> > >> >> > > > wrote:
> > >> >> > > >
> > >> >> > > > > I think it makes sense to wait for Venki's fix of the API,
> as
> > >> >> > > releasing a
> > >> >> > > > > version with the change would make reverting/refactoring
> the
> > API
> > >> >> > > breaking
> > >> >> > > > > change less useful.
> > >> >> > > > >
> > >> >> > > > > Do you have any idea about the scope of fixing the hive
> > native
> > >> >> > reader,
> > >> >> > > or
> > >> >> > > > > which change since the 1.4 release broke it? Is someone
> > >> available
> > >> >> to
> > >> >> > > look
> > >> >> > > > > at it today?
> > >> >> > > > >
> > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> > >> >> > > > > challapallirahul@gmail.com> wrote:
> > >> >> > > > >
> > >> >> > > > > > Just found an issue with hive native parquet reader
> > >> (DRILL-4323).
> > >> >> > > This
> > >> >> > > > > is a
> > >> >> > > > > > regression from 1.4 and in my opinion should be treated
> as
> > a
> > >> >> > blocker.
> > >> >> > > > > >
> > >> >> > > > > > - Rahul
> > >> >> > > > > >
> > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> > >> >> > > > > > venki.korukanti@gmail.com>
> > >> >> > > > > > wrote:
> > >> >> > > > > >
> > >> >> > > > > > > Sorry I didn't realize I was breaking the public
> > interface
> > >> API.
> > >> >> > Let
> > >> >> > > > me
> > >> >> > > > > > work
> > >> >> > > > > > > on a patch to make it backward compatible.
> > >> >> > > > > > >
> > >> >> > > > > > > Thanks
> > >> >> > > > > > > Venki
> > >> >> > > > > > >
> > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <
> > >> >> > > steven@dremio.com
> > >> >> > > > >
> > >> >> > > > > > > wrote:
> > >> >> > > > > > >
> > >> >> > > > > > > > I just wanted to bring up an issue that I just now
> > >> >> discovered,
> > >> >> > > that
> > >> >> > > > > has
> > >> >> > > > > > > > caused me a fair amount of grief.
> > >> >> > > > > > > >
> > >> >> > > > > > > > https://github.com/apache/drill/pull/300/commits
> > >> >> > > > > > > >
> > >> >> > > > > > > > DRILL-4198 changes a user-facing API, and causes
> > >> >> StoragePlugins
> > >> >> > > > that
> > >> >> > > > > > were
> > >> >> > > > > > > > compiled against currently released versions of Drill
> > to
> > >> no
> > >> >> > > longer
> > >> >> > > > > > > > functional properly. I would prefer that this
> breaking
> > >> change
> > >> >> > be
> > >> >> > > > > > modified
> > >> >> > > > > > > > to be backward compatible if possible.
> > >> >> > > > > > > >
> > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> > >> >> > > > > > > > altekrusejason@gmail.com>
> > >> >> > > > > > > > wrote:
> > >> >> > > > > > > >
> > >> >> > > > > > > > > Hi Aman,
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > This is the failure that he was seeing. He figured
> > out
> > >> that
> > >> >> > the
> > >> >> > > > new
> > >> >> > > > > > > > > exclusions in jdbc-all were not being respected
> when
> > the
> > >> >> > build
> > >> >> > > > was
> > >> >> > > > > > run
> > >> >> > > > > > > > with
> > >> >> > > > > > > > > an older Maven version, causing the jar size to
> > increase
> > >> >> > > > > > significantly.
> > >> >> > > > > > > > He
> > >> >> > > > > > > > > added an enforcer to make sure the JAR didn't grow
> > >> >> > > unexpectedly.
> > >> >> > > > > Can
> > >> >> > > > > > > you
> > >> >> > > > > > > > > try to update your maven version and re-run the
> > build?
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > - Jason
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
> > >> >> > > > asinha@maprtech.com>
> > >> >> > > > > > > > wrote:
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > > Jacques, I am getting the following build failure
> > on
> > >> the
> > >> >> > > latest
> > >> >> > > > > > > master
> > >> >> > > > > > > > > > branch...is this what you saw for the Apache
> build
> > ?
> > >> My
> > >> >> > mvn
> > >> >> > > > > > version
> > >> >> > > > > > > > > output
> > >> >> > > > > > > > > > is shown below.  Should we all be upgrading to a
> > newer
> > >> >> mvn
> > >> >> > ?
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all
> ---
> > >> >> > > > > > > > > > [WARNING] Rule 0:
> > >> >> > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
> > >> >> > > > > > > > > failed
> > >> >> > > > > > > > > > with message:
> > >> >> > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is
> > outside
> > >> the
> > >> >> > > > > expected
> > >> >> > > > > > > size
> > >> >> > > > > > > > > > range.
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > >                   This is likely due to you
> adding
> > new
> > >> >> > > > > dependencies
> > >> >> > > > > > > to
> > >> >> > > > > > > > a
> > >> >> > > > > > > > > > java-exec and not updating the excludes in this
> > >> module.
> > >> >> > This
> > >> >> > > is
> > >> >> > > > > > > > important
> > >> >> > > > > > > > > > as it minimizes the size of the dependency of
> Drill
> > >> >> > > application
> > >> >> > > > > > > users.
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > >
> > >> >> > > > > > > >
> > >> >> > > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > >> >> > > > > > > > > > size (44664121) too large. Max. is
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > >
> > >> >> > > > > > > >
> > >> >> > > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill
> > asinha$
> > >> >> mvn
> > >> >> > > > > > --version
> > >> >> > > > > > > > > > Apache Maven 3.0.5
> > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > >> >> > > > > > > > 2013-02-19
> > >> >> > > > > > > > > > 05:51:28-0800)
> > >> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3
> > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle
> Corporation
> > >> >> > > > > > > > > > Java home:
> > >> >> > > > > > > > > >
> > >> >> > > > >
> > >> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > >> >> > > > > > > > > > Default locale: en_US, platform encoding: UTF-8
> > >> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5", arch:
> > >> "x86_64",
> > >> >> > > family:
> > >> >> > > > > > "mac"
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <
> > >> >> > > > > > jacques@dremio.com>
> > >> >> > > > > > > > > > wrote:
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > > > Build back to normal. It looks like the Apache
> > >> server
> > >> >> was
> > >> >> > > > using
> > >> >> > > > > > an
> > >> >> > > > > > > > old
> > >> >> > > > > > > > > > > version of Maven. Once I switched to something
> > more
> > >> >> > recent,
> > >> >> > > > the
> > >> >> > > > > > > build
> > >> >> > > > > > > > > > > passed.
> > >> >> > > > > > > > > > >
> > >> >> > > > > > > > > > > --
> > >> >> > > > > > > > > > > Jacques Nadeau
> > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
> > >> >> > > > > > > > > > >
> > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques
> Nadeau <
> > >> >> > > > > > > jacques@dremio.com>
> > >> >> > > > > > > > > > > wrote:
> > >> >> > > > > > > > > > >
> > >> >> > > > > > > > > > > > Hmm... this merge caused the Apache build to
> > fail.
> > >> >> > > > > > > Investigating...
> > >> >> > > > > > > > > > > >
> > >> >> > > > > > > > > > > > --
> > >> >> > > > > > > > > > > > Jacques Nadeau
> > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
> > >> >> > > > > > > > > > > >
> > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques
> > Nadeau <
> > >> >> > > > > > > > jacques@dremio.com>
> > >> >> > > > > > > > > > > > wrote:
> > >> >> > > > > > > > > > > >
> > >> >> > > > > > > > > > > >> I got clean regression runs as well. I've
> > merged
> > >> the
> > >> >> > > > patch.
> > >> >> > > > > > > > > > > >>
> > >> >> > > > > > > > > > > >> Jason, you want to start the release
> process?
> > >> >> > > > > > > > > > > >>
> > >> >> > > > > > > > > > > >> --
> > >> >> > > > > > > > > > > >> Jacques Nadeau
> > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
> > >> >> > > > > > > > > > > >>
> > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek
> > >> Girish <
> > >> >> > > > > > > > agirish@mapr.com
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > > > >> wrote:
> > >> >> > > > > > > > > > > >>
> > >> >> > > > > > > > > > > >>> Had two clean Functional runs. TPC-H SF100
> > was
> > >> also
> > >> >> > > > > > successful.
> > >> >> > > > > > > > > > > >>>
> > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul
> > >> >> challapalli <
> > >> >> > > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
> > >> >> > > > > > > > > > > >>>
> > >> >> > > > > > > > > > > >>> > Kicked off a functional run with your
> > branch.
> > >> >> Will
> > >> >> > > let
> > >> >> > > > > you
> > >> >> > > > > > > know
> > >> >> > > > > > > > > > once
> > >> >> > > > > > > > > > > it
> > >> >> > > > > > > > > > > >>> > finishes
> > >> >> > > > > > > > > > > >>> >
> > >> >> > > > > > > > > > > >>> > - Rahul
> > >> >> > > > > > > > > > > >>> >
> > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques
> > >> Nadeau <
> > >> >> > > > > > > > > > jacques@dremio.com>
> > >> >> > > > > > > > > > > >>> > wrote:
> > >> >> > > > > > > > > > > >>> >
> > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I have an
> updated
> > >> patch
> > >> >> > for
> > >> >> > > > 4291
> > >> >> > > > > > > that
> > >> >> > > > > > > > is
> > >> >> > > > > > > > > > > >>> ready.
> > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems that something
> > isn't
> > >> >> > > working
> > >> >> > > > > with
> > >> >> > > > > > > our
> > >> >> > > > > > > > > > > >>> extended
> > >> >> > > > > > > > > > > >>> > > tests so I haven't been able to run an
> > >> extended
> > >> >> > > > > > regression.
> > >> >> > > > > > > > > Unit
> > >> >> > > > > > > > > > > >>> tests
> > >> >> > > > > > > > > > > >>> > > pass. Is someone else possibly able to
> > run a
> > >> >> > > > regression
> > >> >> > > > > > > suite
> > >> >> > > > > > > > > > > against
> > >> >> > > > > > > > > > > >>> > this
> > >> >> > > > > > > > > > > >>> > > branch [1] so we can confirm things
> look
> > >> good
> > >> >> and
> > >> >> > > > start
> > >> >> > > > > > the
> > >> >> > > > > > > > > > release
> > >> >> > > > > > > > > > > >>> > > process?
> > >> >> > > > > > > > > > > >>> > >
> > >> >> > > > > > > > > > > >>> > > thanks,
> > >> >> > > > > > > > > > > >>> > > Jacques
> > >> >> > > > > > > > > > > >>> > >
> > >> >> > > > > > > > > > > >>> > > [1]
> > >> >> > > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > >> >> > > > > > > > > > > >>> > >
> > >> >> > > > > > > > > > > >>> > > --
> > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
> > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
> > >> >> > > > > > > > > > > >>> > >
> > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM,
> Jacques
> > >> >> Nadeau
> > >> >> > <
> > >> >> > > > > > > > > > > jacques@dremio.com
> > >> >> > > > > > > > > > > >>> >
> > >> >> > > > > > > > > > > >>> > > wrote:
> > >> >> > > > > > > > > > > >>> > >
> > >> >> > > > > > > > > > > >>> > > > I think the main things are 4196 and
> > 4291
> > >> >> > should
> > >> >> > > be
> > >> >> > > > > > > > > completed.
> > >> >> > > > > > > > > > I
> > >> >> > > > > > > > > > > >>> know
> > >> >> > > > > > > > > > > >>> > > Amit
> > >> >> > > > > > > > > > > >>> > > > was able to reproduce 4196 locally
> this
> > >> >> weekend
> > >> >> > > so
> > >> >> > > > I
> > >> >> > > > > > > think
> > >> >> > > > > > > > > > we're
> > >> >> > > > > > > > > > > >>> close
> > >> >> > > > > > > > > > > >>> > on
> > >> >> > > > > > > > > > > >>> > > > that.
> > >> >> > > > > > > > > > > >>> > > >
> > >> >> > > > > > > > > > > >>> > > > --
> > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
> > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
> > >> >> > > > > > > > > > > >>> > > >
> > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM,
> > Zelaine
> > >> >> Fong
> > >> >> > <
> > >> >> > > > > > > > > > > zfong@maprtech.com
> > >> >> > > > > > > > > > > >>> >
> > >> >> > > > > > > > > > > >>> > > wrote:
> > >> >> > > > > > > > > > > >>> > > >
> > >> >> > > > > > > > > > > >>> > > >> Any updates on this?  What's
> blocking
> > us
> > >> >> from
> > >> >> > > > taking
> > >> >> > > > > > > this
> > >> >> > > > > > > > > to a
> > >> >> > > > > > > > > > > >>> vote --
> > >> >> > > > > > > > > > > >>> > > the
> > >> >> > > > > > > > > > > >>> > > >> sort merge join issues?
> > >> >> > > > > > > > > > > >>> > > >>
> > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
> > >> >> > > > > > > > > > > >>> > > >>
> > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM,
> > Jacques
> > >> >> > Nadeau
> > >> >> > > <
> > >> >> > > > > > > > > > > >>> jacques@dremio.com>
> > >> >> > > > > > > > > > > >>> > > >> wrote:
> > >> >> > > > > > > > > > > >>> > > >>
> > >> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
> > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > > > > > > > > > >>> > > >> > Here are the issues that were
> > >> mentioned in
> > >> >> > > this
> > >> >> > > > > > thread
> > >> >> > > > > > > > > along
> > >> >> > > > > > > > > > > >>> with a
> > >> >> > > > > > > > > > > >>> > > >> > proposed categorization:
> > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
> > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> > >> >> > > > > > > > > > > >>>
> > >> https://issues.apache.org/jira/browse/DRILL-4190
> > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> > >> >> > > > > > > > > > > >>>
> > >> https://issues.apache.org/jira/browse/DRILL-4196
> > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > >> >> > > > > > > > > > > >>> > >
> > >> >> https://issues.apache.org/jira/browse/DRILL-4246
> > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
> > >> >> > > > > > > > > > > >>>
> > >> https://issues.apache.org/jira/browse/DRILL-4256
> > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
> > >> >> > > > > > > > > > > >>> >
> > >> https://issues.apache.org/jira/browse/DRILL-4278
> > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
> > >> >> > > > > > > > > > > >>> > >
> > >> >> https://issues.apache.org/jira/browse/DRILL-4285
> > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
> > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
> > >> >> > > > > > > > > > > >>>
> > >> https://issues.apache.org/jira/browse/DRILL-4247
> > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
> > >> >> > > > > > > > > > > >>>
> > >> https://issues.apache.org/jira/browse/DRILL-4203
> > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
> > >> >> > > > > > > > > > https://issues.apache.org/jira/browse/DRILL-4266
> > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > >> >> > > > > > > > > > > >>> > >
> > >> >> https://issues.apache.org/jira/browse/DRILL-4131
> > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > > > > > > > > > >>> > > >> > What do others think? Let's try to
> > get
> > >> the
> > >> >> > > > > blockers
> > >> >> > > > > > > > > wrapped
> > >> >> > > > > > > > > > up
> > >> >> > > > > > > > > > > >>> in
> > >> >> > > > > > > > > > > >>> > the
> > >> >> > > > > > > > > > > >>> > > >> next
> > >> >> > > > > > > > > > > >>> > > >> > day or two and start a release
> > vote...
> > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > > > > > > > > > >>> > > >> > --
> > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
> > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM,
> > Jason
> > >> >> > > Altekruse
> > >> >> > > > <
> > >> >> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
> > >> >> > > > > > > > > > > >>> > > >> > wrote:
> > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
> > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > > > > > > > > > >>> > > >> > > With the allocator changes
> merged
> > and
> > >> >> > about
> > >> >> > > a
> > >> >> > > > > > month
> > >> >> > > > > > > > > since
> > >> >> > > > > > > > > > > the
> > >> >> > > > > > > > > > > >>> last
> > >> >> > > > > > > > > > > >>> > > >> > release
> > >> >> > > > > > > > > > > >>> > > >> > > I think it would be good to
> start
> > a
> > >> vote
> > >> >> > > > soon. I
> > >> >> > > > > > > would
> > >> >> > > > > > > > > > like
> > >> >> > > > > > > > > > > to
> > >> >> > > > > > > > > > > >>> > > >> volunteer
> > >> >> > > > > > > > > > > >>> > > >> > to
> > >> >> > > > > > > > > > > >>> > > >> > > be release manager.
> > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > > > > > > > > > >>> > > >> > > I know that there were some
> issues
> > >> that
> > >> >> > were
> > >> >> > > > > > > > identified
> > >> >> > > > > > > > > > > after
> > >> >> > > > > > > > > > > >>> the
> > >> >> > > > > > > > > > > >>> > > >> > transfer
> > >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I think that
> > these
> > >> >> > issues
> > >> >> > > > > should
> > >> >> > > > > > > be
> > >> >> > > > > > > > > > fixed
> > >> >> > > > > > > > > > > >>> before
> > >> >> > > > > > > > > > > >>> > > we
> > >> >> > > > > > > > > > > >>> > > >> > cut a
> > >> >> > > > > > > > > > > >>> > > >> > > release candidate.
> > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > > > > > > > > > >>> > > >> > > From looking at the associated
> > JIRAs
> > >> it
> > >> >> > > looked
> > >> >> > > > > > like
> > >> >> > > > > > > > > there
> > >> >> > > > > > > > > > > was
> > >> >> > > > > > > > > > > >>> a
> > >> >> > > > > > > > > > > >>> > > >> possible
> > >> >> > > > > > > > > > > >>> > > >> > > short term fix just adjusting
> the
> > >> >> > > > > > > > > > max_query_memory_per_node
> > >> >> > > > > > > > > > > >>> > option,
> > >> >> > > > > > > > > > > >>> > > >> and
> > >> >> > > > > > > > > > > >>> > > >> > > some more involved work to
> change
> > >> how we
> > >> >> > > > > determine
> > >> >> > > > > > > the
> > >> >> > > > > > > > > > > correct
> > >> >> > > > > > > > > > > >>> > time
> > >> >> > > > > > > > > > > >>> > > to
> > >> >> > > > > > > > > > > >>> > > >> > > spill during external sort. I
> > >> believe it
> > >> >> > > makes
> > >> >> > > > > > sense
> > >> >> > > > > > > > to
> > >> >> > > > > > > > > > make
> > >> >> > > > > > > > > > > >>> > > external
> > >> >> > > > > > > > > > > >>> > > >> > sort
> > >> >> > > > > > > > > > > >>> > > >> > > work well with the newly
> improved
> > >> memory
> > >> >> > > > > > accounting
> > >> >> > > > > > > > > before
> > >> >> > > > > > > > > > > >>> > cutting a
> > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not sure how
> much
> > >> work
> > >> >> is
> > >> >> > > > left
> > >> >> > > > > to
> > >> >> > > > > > > be
> > >> >> > > > > > > > > done
> > >> >> > > > > > > > > > > >>> there.
> > >> >> > > > > > > > > > > >>> > > [1]
> > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > > > > > > > > > >>> > > >> > > Please respond with your
> thoughts
> > on
> > >> a
> > >> >> > > release
> > >> >> > > > > > soon
> > >> >> > > > > > > > and
> > >> >> > > > > > > > > > any
> > >> >> > > > > > > > > > > >>> JIRAs
> > >> >> > > > > > > > > > > >>> > > you
> > >> >> > > > > > > > > > > >>> > > >> > would
> > >> >> > > > > > > > > > > >>> > > >> > > like to include in the release.
> > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > > > > > > > > > >>> > > >> > > [1] -
> > >> >> > > > > > > > https://issues.apache.org/jira/browse/DRILL-4243
> > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
> > >> >> > > > > > > > > > > >>> > > >> > > Jason
> > >> >> > > > > > > > > > > >>> > > >> > >
> > >> >> > > > > > > > > > > >>> > > >> >
> > >> >> > > > > > > > > > > >>> > > >>
> > >> >> > > > > > > > > > > >>> > > >
> > >> >> > > > > > > > > > > >>> > > >
> > >> >> > > > > > > > > > > >>> > >
> > >> >> > > > > > > > > > > >>> >
> > >> >> > > > > > > > > > > >>>
> > >> >> > > > > > > > > > > >>
> > >> >> > > > > > > > > > > >>
> > >> >> > > > > > > > > > > >
> > >> >> > > > > > > > > > >
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > >
> > >> >> > > > > > > >
> > >> >> > > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
>

Re: Time for a 1.5 release?

Posted by Krystal Nguyen <kn...@maprtech.com>.

I do see the same error with assertion turned on.

On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <ji...@gmail.com> wrote:

> Krystal, by any chance, did you turn on assertion when you start drillbit?
>
> I think the error in drill-4323 will be raised when assertion = on.
>
>
>
> On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <kn...@maprtech.com>
> wrote:
> > I am running drill-1.4 and is able to run count(*) successfully.
> >
> > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session set
> > `store.hive.optimize_scan_with_native_readers` = true;
> > +-------+--------------------------------------------------------+
> > |  ok   |                        summary                         |
> > +-------+--------------------------------------------------------+
> > | true  | store.hive.optimize_scan_with_native_readers updated.  |
> > +-------+--------------------------------------------------------+
> > 1 row selected (0.374 seconds)
> > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select count(*) from
> > lineitem_parquet_partitioned_hive_string;
> > +---------+
> > | EXPR$0  |
> > +---------+
> > | 60175   |
> > +---------+
> > 1 row selected (2.986 seconds)
> >
> > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <ji...@gmail.com>
> wrote:
> >
> >> Venki and I did some investigation for DRILL-4323. The issue reported
> >> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems to us
> >> this is not a regression from 1.4.0; it's a regression from 1.3.0
> >> probably.
> >>
> >> DRILL-4083 makes the planner to use DrillHiveNativeReader in stead of
> >> HiveReader for "select count(*) from hive_table" query.  However, the
> >> Project after the scan produces empty schema.  Before DRILL-4083,
> >> Drill uses HiveScan, which works fine.
> >>
> >>
> >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
> >> <al...@gmail.com> wrote:
> >> > We could revert the change to the StoragePlugin interface to add the
> >> > getLogicalOptimizerRules
> >> > and getPhysicalOptimizerRules, and only put those in
> >> AbstractStoragePlugin
> >> > with default implementations.
> >> >
> >> > We could then update the docs to tell users to not implement the
> >> interface
> >> > directly, and instead extend the abstract class, which I believe would
> >> > allow us to make changes like this in the future without breaking
> >> plugins.
> >> > At a major release version we could then just pull up all of the new
> >> > methods we added to the abstract class into the interface, or just get
> >> rid
> >> > of it all-together.
> >> >
> >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
> >> venki.korukanti@gmail.com>
> >> > wrote:
> >> >
> >> >> Here is the partial fix:
> >> >>
> >> >>
> >>
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> >> >>
> >> >> If the existing StragePlugins are implementing AbstractStoragePlugin
> >> class
> >> >> this fix should resolve the backward compat issue. If they are
> >> implementing
> >> >> StoragePlugin interface directly then it won't resolve the issue. Not
> >> sure
> >> >> how to handle it in that case. Let me know if there any ways to
> resolve
> >> >> this issue completely.
> >> >>
> >> >> Thanks
> >> >> Venki
> >> >>
> >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <as...@maprtech.com>
> >> wrote:
> >> >>
> >> >> > I was able to do the build with maven 3.3.9 (after doing 'brew
> install
> >> >> > maven').  Here are my env variables related to maven:
> >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> >> >> >
> >> >> >
> >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <
> jacques@dremio.com>
> >> >> > wrote:
> >> >> >
> >> >> > > Aman, for reference on the Maven version issue. You can see the
> >> output
> >> >> of
> >> >> > > the shade plugin between the two Apache Builds. If you search for
> >> >> > > maven-shade-plugin you'll see that build 647 (maven 3.0.5) fails
> >> >> because
> >> >> > it
> >> >> > > includes a bunch of hadoop (and other libraries that are listed
> as
> >> >> being
> >> >> > > excluded). If you look at build 648 (maven 3.3.3), those items
> are
> >> >> > included
> >> >> > > so that the build completes successfully.
> >> >> > >
> >> >> > > https://builds.apache.org/job/drill-scm/647/consoleText
> >> >> > > https://builds.apache.org/job/drill-scm/648/consoleText
> >> >> > >
> >> >> > > --
> >> >> > > Jacques Nadeau
> >> >> > > CTO and Co-Founder, Dremio
> >> >> > >
> >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
> >> >> > > challapallirahul@gmail.com> wrote:
> >> >> > >
> >> >> > > > Jason,
> >> >> > > >
> >> >> > > > As of now I haven't narrowed down the exact commit which caused
> >> >> > > DRILL-4323
> >> >> > > > and I do not know the scope of the fix.
> >> >> > > >
> >> >> > > > - Rahul
> >> >> > > >
> >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
> >> >> > > altekrusejason@gmail.com
> >> >> > > > >
> >> >> > > > wrote:
> >> >> > > >
> >> >> > > > > I think it makes sense to wait for Venki's fix of the API, as
> >> >> > > releasing a
> >> >> > > > > version with the change would make reverting/refactoring the
> API
> >> >> > > breaking
> >> >> > > > > change less useful.
> >> >> > > > >
> >> >> > > > > Do you have any idea about the scope of fixing the hive
> native
> >> >> > reader,
> >> >> > > or
> >> >> > > > > which change since the 1.4 release broke it? Is someone
> >> available
> >> >> to
> >> >> > > look
> >> >> > > > > at it today?
> >> >> > > > >
> >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> >> >> > > > > challapallirahul@gmail.com> wrote:
> >> >> > > > >
> >> >> > > > > > Just found an issue with hive native parquet reader
> >> (DRILL-4323).
> >> >> > > This
> >> >> > > > > is a
> >> >> > > > > > regression from 1.4 and in my opinion should be treated as
> a
> >> >> > blocker.
> >> >> > > > > >
> >> >> > > > > > - Rahul
> >> >> > > > > >
> >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> >> >> > > > > > venki.korukanti@gmail.com>
> >> >> > > > > > wrote:
> >> >> > > > > >
> >> >> > > > > > > Sorry I didn't realize I was breaking the public
> interface
> >> API.
> >> >> > Let
> >> >> > > > me
> >> >> > > > > > work
> >> >> > > > > > > on a patch to make it backward compatible.
> >> >> > > > > > >
> >> >> > > > > > > Thanks
> >> >> > > > > > > Venki
> >> >> > > > > > >
> >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <
> >> >> > > steven@dremio.com
> >> >> > > > >
> >> >> > > > > > > wrote:
> >> >> > > > > > >
> >> >> > > > > > > > I just wanted to bring up an issue that I just now
> >> >> discovered,
> >> >> > > that
> >> >> > > > > has
> >> >> > > > > > > > caused me a fair amount of grief.
> >> >> > > > > > > >
> >> >> > > > > > > > https://github.com/apache/drill/pull/300/commits
> >> >> > > > > > > >
> >> >> > > > > > > > DRILL-4198 changes a user-facing API, and causes
> >> >> StoragePlugins
> >> >> > > > that
> >> >> > > > > > were
> >> >> > > > > > > > compiled against currently released versions of Drill
> to
> >> no
> >> >> > > longer
> >> >> > > > > > > > functional properly. I would prefer that this breaking
> >> change
> >> >> > be
> >> >> > > > > > modified
> >> >> > > > > > > > to be backward compatible if possible.
> >> >> > > > > > > >
> >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> >> >> > > > > > > > altekrusejason@gmail.com>
> >> >> > > > > > > > wrote:
> >> >> > > > > > > >
> >> >> > > > > > > > > Hi Aman,
> >> >> > > > > > > > >
> >> >> > > > > > > > > This is the failure that he was seeing. He figured
> out
> >> that
> >> >> > the
> >> >> > > > new
> >> >> > > > > > > > > exclusions in jdbc-all were not being respected when
> the
> >> >> > build
> >> >> > > > was
> >> >> > > > > > run
> >> >> > > > > > > > with
> >> >> > > > > > > > > an older Maven version, causing the jar size to
> increase
> >> >> > > > > > significantly.
> >> >> > > > > > > > He
> >> >> > > > > > > > > added an enforcer to make sure the JAR didn't grow
> >> >> > > unexpectedly.
> >> >> > > > > Can
> >> >> > > > > > > you
> >> >> > > > > > > > > try to update your maven version and re-run the
> build?
> >> >> > > > > > > > >
> >> >> > > > > > > > > - Jason
> >> >> > > > > > > > >
> >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
> >> >> > > > asinha@maprtech.com>
> >> >> > > > > > > > wrote:
> >> >> > > > > > > > >
> >> >> > > > > > > > > > Jacques, I am getting the following build failure
> on
> >> the
> >> >> > > latest
> >> >> > > > > > > master
> >> >> > > > > > > > > > branch...is this what you saw for the Apache build
> ?
> >> My
> >> >> > mvn
> >> >> > > > > > version
> >> >> > > > > > > > > output
> >> >> > > > > > > > > > is shown below.  Should we all be upgrading to a
> newer
> >> >> mvn
> >> >> > ?
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> >> >> > > > > > > > > > [WARNING] Rule 0:
> >> >> > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
> >> >> > > > > > > > > failed
> >> >> > > > > > > > > > with message:
> >> >> > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is
> outside
> >> the
> >> >> > > > > expected
> >> >> > > > > > > size
> >> >> > > > > > > > > > range.
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >                   This is likely due to you adding
> new
> >> >> > > > > dependencies
> >> >> > > > > > > to
> >> >> > > > > > > > a
> >> >> > > > > > > > > > java-exec and not updating the excludes in this
> >> module.
> >> >> > This
> >> >> > > is
> >> >> > > > > > > > important
> >> >> > > > > > > > > > as it minimizes the size of the dependency of Drill
> >> >> > > application
> >> >> > > > > > > users.
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> >> > > > > > > > > > size (44664121) too large. Max. is
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill
> asinha$
> >> >> mvn
> >> >> > > > > > --version
> >> >> > > > > > > > > > Apache Maven 3.0.5
> >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> >> >> > > > > > > > 2013-02-19
> >> >> > > > > > > > > > 05:51:28-0800)
> >> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3
> >> >> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle Corporation
> >> >> > > > > > > > > > Java home:
> >> >> > > > > > > > > >
> >> >> > > > >
> >> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> >> >> > > > > > > > > > Default locale: en_US, platform encoding: UTF-8
> >> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5", arch:
> >> "x86_64",
> >> >> > > family:
> >> >> > > > > > "mac"
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <
> >> >> > > > > > jacques@dremio.com>
> >> >> > > > > > > > > > wrote:
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > > Build back to normal. It looks like the Apache
> >> server
> >> >> was
> >> >> > > > using
> >> >> > > > > > an
> >> >> > > > > > > > old
> >> >> > > > > > > > > > > version of Maven. Once I switched to something
> more
> >> >> > recent,
> >> >> > > > the
> >> >> > > > > > > build
> >> >> > > > > > > > > > > passed.
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > > > --
> >> >> > > > > > > > > > > Jacques Nadeau
> >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <
> >> >> > > > > > > jacques@dremio.com>
> >> >> > > > > > > > > > > wrote:
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > > > > Hmm... this merge caused the Apache build to
> fail.
> >> >> > > > > > > Investigating...
> >> >> > > > > > > > > > > >
> >> >> > > > > > > > > > > > --
> >> >> > > > > > > > > > > > Jacques Nadeau
> >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
> >> >> > > > > > > > > > > >
> >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques
> Nadeau <
> >> >> > > > > > > > jacques@dremio.com>
> >> >> > > > > > > > > > > > wrote:
> >> >> > > > > > > > > > > >
> >> >> > > > > > > > > > > >> I got clean regression runs as well. I've
> merged
> >> the
> >> >> > > > patch.
> >> >> > > > > > > > > > > >>
> >> >> > > > > > > > > > > >> Jason, you want to start the release process?
> >> >> > > > > > > > > > > >>
> >> >> > > > > > > > > > > >> --
> >> >> > > > > > > > > > > >> Jacques Nadeau
> >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
> >> >> > > > > > > > > > > >>
> >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek
> >> Girish <
> >> >> > > > > > > > agirish@mapr.com
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > > >> wrote:
> >> >> > > > > > > > > > > >>
> >> >> > > > > > > > > > > >>> Had two clean Functional runs. TPC-H SF100
> was
> >> also
> >> >> > > > > > successful.
> >> >> > > > > > > > > > > >>>
> >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul
> >> >> challapalli <
> >> >> > > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
> >> >> > > > > > > > > > > >>>
> >> >> > > > > > > > > > > >>> > Kicked off a functional run with your
> branch.
> >> >> Will
> >> >> > > let
> >> >> > > > > you
> >> >> > > > > > > know
> >> >> > > > > > > > > > once
> >> >> > > > > > > > > > > it
> >> >> > > > > > > > > > > >>> > finishes
> >> >> > > > > > > > > > > >>> >
> >> >> > > > > > > > > > > >>> > - Rahul
> >> >> > > > > > > > > > > >>> >
> >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques
> >> Nadeau <
> >> >> > > > > > > > > > jacques@dremio.com>
> >> >> > > > > > > > > > > >>> > wrote:
> >> >> > > > > > > > > > > >>> >
> >> >> > > > > > > > > > > >>> > > 4196 was merged today. I have an updated
> >> patch
> >> >> > for
> >> >> > > > 4291
> >> >> > > > > > > that
> >> >> > > > > > > > is
> >> >> > > > > > > > > > > >>> ready.
> >> >> > > > > > > > > > > >>> > > Unfortunately, it seems that something
> isn't
> >> >> > > working
> >> >> > > > > with
> >> >> > > > > > > our
> >> >> > > > > > > > > > > >>> extended
> >> >> > > > > > > > > > > >>> > > tests so I haven't been able to run an
> >> extended
> >> >> > > > > > regression.
> >> >> > > > > > > > > Unit
> >> >> > > > > > > > > > > >>> tests
> >> >> > > > > > > > > > > >>> > > pass. Is someone else possibly able to
> run a
> >> >> > > > regression
> >> >> > > > > > > suite
> >> >> > > > > > > > > > > against
> >> >> > > > > > > > > > > >>> > this
> >> >> > > > > > > > > > > >>> > > branch [1] so we can confirm things look
> >> good
> >> >> and
> >> >> > > > start
> >> >> > > > > > the
> >> >> > > > > > > > > > release
> >> >> > > > > > > > > > > >>> > > process?
> >> >> > > > > > > > > > > >>> > >
> >> >> > > > > > > > > > > >>> > > thanks,
> >> >> > > > > > > > > > > >>> > > Jacques
> >> >> > > > > > > > > > > >>> > >
> >> >> > > > > > > > > > > >>> > > [1]
> >> >> > > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> >> >> > > > > > > > > > > >>> > >
> >> >> > > > > > > > > > > >>> > > --
> >> >> > > > > > > > > > > >>> > > Jacques Nadeau
> >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
> >> >> > > > > > > > > > > >>> > >
> >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques
> >> >> Nadeau
> >> >> > <
> >> >> > > > > > > > > > > jacques@dremio.com
> >> >> > > > > > > > > > > >>> >
> >> >> > > > > > > > > > > >>> > > wrote:
> >> >> > > > > > > > > > > >>> > >
> >> >> > > > > > > > > > > >>> > > > I think the main things are 4196 and
> 4291
> >> >> > should
> >> >> > > be
> >> >> > > > > > > > > completed.
> >> >> > > > > > > > > > I
> >> >> > > > > > > > > > > >>> know
> >> >> > > > > > > > > > > >>> > > Amit
> >> >> > > > > > > > > > > >>> > > > was able to reproduce 4196 locally this
> >> >> weekend
> >> >> > > so
> >> >> > > > I
> >> >> > > > > > > think
> >> >> > > > > > > > > > we're
> >> >> > > > > > > > > > > >>> close
> >> >> > > > > > > > > > > >>> > on
> >> >> > > > > > > > > > > >>> > > > that.
> >> >> > > > > > > > > > > >>> > > >
> >> >> > > > > > > > > > > >>> > > > --
> >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
> >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
> >> >> > > > > > > > > > > >>> > > >
> >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM,
> Zelaine
> >> >> Fong
> >> >> > <
> >> >> > > > > > > > > > > zfong@maprtech.com
> >> >> > > > > > > > > > > >>> >
> >> >> > > > > > > > > > > >>> > > wrote:
> >> >> > > > > > > > > > > >>> > > >
> >> >> > > > > > > > > > > >>> > > >> Any updates on this?  What's blocking
> us
> >> >> from
> >> >> > > > taking
> >> >> > > > > > > this
> >> >> > > > > > > > > to a
> >> >> > > > > > > > > > > >>> vote --
> >> >> > > > > > > > > > > >>> > > the
> >> >> > > > > > > > > > > >>> > > >> sort merge join issues?
> >> >> > > > > > > > > > > >>> > > >>
> >> >> > > > > > > > > > > >>> > > >> -- Zelaine
> >> >> > > > > > > > > > > >>> > > >>
> >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM,
> Jacques
> >> >> > Nadeau
> >> >> > > <
> >> >> > > > > > > > > > > >>> jacques@dremio.com>
> >> >> > > > > > > > > > > >>> > > >> wrote:
> >> >> > > > > > > > > > > >>> > > >>
> >> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
> >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > > > > > > > > > >>> > > >> > Here are the issues that were
> >> mentioned in
> >> >> > > this
> >> >> > > > > > thread
> >> >> > > > > > > > > along
> >> >> > > > > > > > > > > >>> with a
> >> >> > > > > > > > > > > >>> > > >> > proposed categorization:
> >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > > > > > > > > > >>> > > >> > Release Blockers
> >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> >> >> > > > > > > > > > > >>>
> >> https://issues.apache.org/jira/browse/DRILL-4190
> >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> >> >> > > > > > > > > > > >>>
> >> https://issues.apache.org/jira/browse/DRILL-4196
> >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> >> >> > > > > > > > > > > >>> > >
> >> >> https://issues.apache.org/jira/browse/DRILL-4246
> >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
> >> >> > > > > > > > > > > >>>
> >> https://issues.apache.org/jira/browse/DRILL-4256
> >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
> >> >> > > > > > > > > > > >>> >
> >> https://issues.apache.org/jira/browse/DRILL-4278
> >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
> >> >> > > > > > > > > > > >>> > >
> >> >> https://issues.apache.org/jira/browse/DRILL-4285
> >> >> > > > > > > > > > > >>> > > >> > Nice to Have
> >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
> >> >> > > > > > > > > > > >>>
> >> https://issues.apache.org/jira/browse/DRILL-4247
> >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
> >> >> > > > > > > > > > > >>>
> >> https://issues.apache.org/jira/browse/DRILL-4203
> >> >> > > > > > > > > > > >>> > > >> > Open Jacques
> >> >> > > > > > > > > > https://issues.apache.org/jira/browse/DRILL-4266
> >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> >> >> > > > > > > > > > > >>> > >
> >> >> https://issues.apache.org/jira/browse/DRILL-4131
> >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > > > > > > > > > >>> > > >> > What do others think? Let's try to
> get
> >> the
> >> >> > > > > blockers
> >> >> > > > > > > > > wrapped
> >> >> > > > > > > > > > up
> >> >> > > > > > > > > > > >>> in
> >> >> > > > > > > > > > > >>> > the
> >> >> > > > > > > > > > > >>> > > >> next
> >> >> > > > > > > > > > > >>> > > >> > day or two and start a release
> vote...
> >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > > > > > > > > > >>> > > >> > --
> >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
> >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM,
> Jason
> >> >> > > Altekruse
> >> >> > > > <
> >> >> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
> >> >> > > > > > > > > > > >>> > > >> > wrote:
> >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > > > > > > > > > >>> > > >> > > Hello All,
> >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > > > > > > > > > >>> > > >> > > With the allocator changes merged
> and
> >> >> > about
> >> >> > > a
> >> >> > > > > > month
> >> >> > > > > > > > > since
> >> >> > > > > > > > > > > the
> >> >> > > > > > > > > > > >>> last
> >> >> > > > > > > > > > > >>> > > >> > release
> >> >> > > > > > > > > > > >>> > > >> > > I think it would be good to start
> a
> >> vote
> >> >> > > > soon. I
> >> >> > > > > > > would
> >> >> > > > > > > > > > like
> >> >> > > > > > > > > > > to
> >> >> > > > > > > > > > > >>> > > >> volunteer
> >> >> > > > > > > > > > > >>> > > >> > to
> >> >> > > > > > > > > > > >>> > > >> > > be release manager.
> >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > > > > > > > > > >>> > > >> > > I know that there were some issues
> >> that
> >> >> > were
> >> >> > > > > > > > identified
> >> >> > > > > > > > > > > after
> >> >> > > > > > > > > > > >>> the
> >> >> > > > > > > > > > > >>> > > >> > transfer
> >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I think that
> these
> >> >> > issues
> >> >> > > > > should
> >> >> > > > > > > be
> >> >> > > > > > > > > > fixed
> >> >> > > > > > > > > > > >>> before
> >> >> > > > > > > > > > > >>> > > we
> >> >> > > > > > > > > > > >>> > > >> > cut a
> >> >> > > > > > > > > > > >>> > > >> > > release candidate.
> >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > > > > > > > > > >>> > > >> > > From looking at the associated
> JIRAs
> >> it
> >> >> > > looked
> >> >> > > > > > like
> >> >> > > > > > > > > there
> >> >> > > > > > > > > > > was
> >> >> > > > > > > > > > > >>> a
> >> >> > > > > > > > > > > >>> > > >> possible
> >> >> > > > > > > > > > > >>> > > >> > > short term fix just adjusting the
> >> >> > > > > > > > > > max_query_memory_per_node
> >> >> > > > > > > > > > > >>> > option,
> >> >> > > > > > > > > > > >>> > > >> and
> >> >> > > > > > > > > > > >>> > > >> > > some more involved work to change
> >> how we
> >> >> > > > > determine
> >> >> > > > > > > the
> >> >> > > > > > > > > > > correct
> >> >> > > > > > > > > > > >>> > time
> >> >> > > > > > > > > > > >>> > > to
> >> >> > > > > > > > > > > >>> > > >> > > spill during external sort. I
> >> believe it
> >> >> > > makes
> >> >> > > > > > sense
> >> >> > > > > > > > to
> >> >> > > > > > > > > > make
> >> >> > > > > > > > > > > >>> > > external
> >> >> > > > > > > > > > > >>> > > >> > sort
> >> >> > > > > > > > > > > >>> > > >> > > work well with the newly improved
> >> memory
> >> >> > > > > > accounting
> >> >> > > > > > > > > before
> >> >> > > > > > > > > > > >>> > cutting a
> >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not sure how much
> >> work
> >> >> is
> >> >> > > > left
> >> >> > > > > to
> >> >> > > > > > > be
> >> >> > > > > > > > > done
> >> >> > > > > > > > > > > >>> there.
> >> >> > > > > > > > > > > >>> > > [1]
> >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > > > > > > > > > >>> > > >> > > Please respond with your thoughts
> on
> >> a
> >> >> > > release
> >> >> > > > > > soon
> >> >> > > > > > > > and
> >> >> > > > > > > > > > any
> >> >> > > > > > > > > > > >>> JIRAs
> >> >> > > > > > > > > > > >>> > > you
> >> >> > > > > > > > > > > >>> > > >> > would
> >> >> > > > > > > > > > > >>> > > >> > > like to include in the release.
> >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > > > > > > > > > >>> > > >> > > [1] -
> >> >> > > > > > > > https://issues.apache.org/jira/browse/DRILL-4243
> >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > > > > > > > > > >>> > > >> > > Thanks,
> >> >> > > > > > > > > > > >>> > > >> > > Jason
> >> >> > > > > > > > > > > >>> > > >> > >
> >> >> > > > > > > > > > > >>> > > >> >
> >> >> > > > > > > > > > > >>> > > >>
> >> >> > > > > > > > > > > >>> > > >
> >> >> > > > > > > > > > > >>> > > >
> >> >> > > > > > > > > > > >>> > >
> >> >> > > > > > > > > > > >>> >
> >> >> > > > > > > > > > > >>>
> >> >> > > > > > > > > > > >>
> >> >> > > > > > > > > > > >>
> >> >> > > > > > > > > > > >
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > >
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
>

Re: Time for a 1.5 release?

Posted by Jinfeng Ni <ji...@gmail.com>.

Krystal, by any chance, did you turn on assertion when you start drillbit?

I think the error in drill-4323 will be raised when assertion = on.



On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <kn...@maprtech.com> wrote:
> I am running drill-1.4 and is able to run count(*) successfully.
>
> 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session set
> `store.hive.optimize_scan_with_native_readers` = true;
> +-------+--------------------------------------------------------+
> |  ok   |                        summary                         |
> +-------+--------------------------------------------------------+
> | true  | store.hive.optimize_scan_with_native_readers updated.  |
> +-------+--------------------------------------------------------+
> 1 row selected (0.374 seconds)
> 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select count(*) from
> lineitem_parquet_partitioned_hive_string;
> +---------+
> | EXPR$0  |
> +---------+
> | 60175   |
> +---------+
> 1 row selected (2.986 seconds)
>
> On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <ji...@gmail.com> wrote:
>
>> Venki and I did some investigation for DRILL-4323. The issue reported
>> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems to us
>> this is not a regression from 1.4.0; it's a regression from 1.3.0
>> probably.
>>
>> DRILL-4083 makes the planner to use DrillHiveNativeReader in stead of
>> HiveReader for "select count(*) from hive_table" query.  However, the
>> Project after the scan produces empty schema.  Before DRILL-4083,
>> Drill uses HiveScan, which works fine.
>>
>>
>> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
>> <al...@gmail.com> wrote:
>> > We could revert the change to the StoragePlugin interface to add the
>> > getLogicalOptimizerRules
>> > and getPhysicalOptimizerRules, and only put those in
>> AbstractStoragePlugin
>> > with default implementations.
>> >
>> > We could then update the docs to tell users to not implement the
>> interface
>> > directly, and instead extend the abstract class, which I believe would
>> > allow us to make changes like this in the future without breaking
>> plugins.
>> > At a major release version we could then just pull up all of the new
>> > methods we added to the abstract class into the interface, or just get
>> rid
>> > of it all-together.
>> >
>> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
>> venki.korukanti@gmail.com>
>> > wrote:
>> >
>> >> Here is the partial fix:
>> >>
>> >>
>> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
>> >>
>> >> If the existing StragePlugins are implementing AbstractStoragePlugin
>> class
>> >> this fix should resolve the backward compat issue. If they are
>> implementing
>> >> StoragePlugin interface directly then it won't resolve the issue. Not
>> sure
>> >> how to handle it in that case. Let me know if there any ways to resolve
>> >> this issue completely.
>> >>
>> >> Thanks
>> >> Venki
>> >>
>> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <as...@maprtech.com>
>> wrote:
>> >>
>> >> > I was able to do the build with maven 3.3.9 (after doing 'brew install
>> >> > maven').  Here are my env variables related to maven:
>> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
>> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
>> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
>> >> >
>> >> >
>> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <ja...@dremio.com>
>> >> > wrote:
>> >> >
>> >> > > Aman, for reference on the Maven version issue. You can see the
>> output
>> >> of
>> >> > > the shade plugin between the two Apache Builds. If you search for
>> >> > > maven-shade-plugin you'll see that build 647 (maven 3.0.5) fails
>> >> because
>> >> > it
>> >> > > includes a bunch of hadoop (and other libraries that are listed as
>> >> being
>> >> > > excluded). If you look at build 648 (maven 3.3.3), those items are
>> >> > included
>> >> > > so that the build completes successfully.
>> >> > >
>> >> > > https://builds.apache.org/job/drill-scm/647/consoleText
>> >> > > https://builds.apache.org/job/drill-scm/648/consoleText
>> >> > >
>> >> > > --
>> >> > > Jacques Nadeau
>> >> > > CTO and Co-Founder, Dremio
>> >> > >
>> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
>> >> > > challapallirahul@gmail.com> wrote:
>> >> > >
>> >> > > > Jason,
>> >> > > >
>> >> > > > As of now I haven't narrowed down the exact commit which caused
>> >> > > DRILL-4323
>> >> > > > and I do not know the scope of the fix.
>> >> > > >
>> >> > > > - Rahul
>> >> > > >
>> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
>> >> > > altekrusejason@gmail.com
>> >> > > > >
>> >> > > > wrote:
>> >> > > >
>> >> > > > > I think it makes sense to wait for Venki's fix of the API, as
>> >> > > releasing a
>> >> > > > > version with the change would make reverting/refactoring the API
>> >> > > breaking
>> >> > > > > change less useful.
>> >> > > > >
>> >> > > > > Do you have any idea about the scope of fixing the hive native
>> >> > reader,
>> >> > > or
>> >> > > > > which change since the 1.4 release broke it? Is someone
>> available
>> >> to
>> >> > > look
>> >> > > > > at it today?
>> >> > > > >
>> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
>> >> > > > > challapallirahul@gmail.com> wrote:
>> >> > > > >
>> >> > > > > > Just found an issue with hive native parquet reader
>> (DRILL-4323).
>> >> > > This
>> >> > > > > is a
>> >> > > > > > regression from 1.4 and in my opinion should be treated as a
>> >> > blocker.
>> >> > > > > >
>> >> > > > > > - Rahul
>> >> > > > > >
>> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
>> >> > > > > > venki.korukanti@gmail.com>
>> >> > > > > > wrote:
>> >> > > > > >
>> >> > > > > > > Sorry I didn't realize I was breaking the public interface
>> API.
>> >> > Let
>> >> > > > me
>> >> > > > > > work
>> >> > > > > > > on a patch to make it backward compatible.
>> >> > > > > > >
>> >> > > > > > > Thanks
>> >> > > > > > > Venki
>> >> > > > > > >
>> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <
>> >> > > steven@dremio.com
>> >> > > > >
>> >> > > > > > > wrote:
>> >> > > > > > >
>> >> > > > > > > > I just wanted to bring up an issue that I just now
>> >> discovered,
>> >> > > that
>> >> > > > > has
>> >> > > > > > > > caused me a fair amount of grief.
>> >> > > > > > > >
>> >> > > > > > > > https://github.com/apache/drill/pull/300/commits
>> >> > > > > > > >
>> >> > > > > > > > DRILL-4198 changes a user-facing API, and causes
>> >> StoragePlugins
>> >> > > > that
>> >> > > > > > were
>> >> > > > > > > > compiled against currently released versions of Drill to
>> no
>> >> > > longer
>> >> > > > > > > > functional properly. I would prefer that this breaking
>> change
>> >> > be
>> >> > > > > > modified
>> >> > > > > > > > to be backward compatible if possible.
>> >> > > > > > > >
>> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
>> >> > > > > > > > altekrusejason@gmail.com>
>> >> > > > > > > > wrote:
>> >> > > > > > > >
>> >> > > > > > > > > Hi Aman,
>> >> > > > > > > > >
>> >> > > > > > > > > This is the failure that he was seeing. He figured out
>> that
>> >> > the
>> >> > > > new
>> >> > > > > > > > > exclusions in jdbc-all were not being respected when the
>> >> > build
>> >> > > > was
>> >> > > > > > run
>> >> > > > > > > > with
>> >> > > > > > > > > an older Maven version, causing the jar size to increase
>> >> > > > > > significantly.
>> >> > > > > > > > He
>> >> > > > > > > > > added an enforcer to make sure the JAR didn't grow
>> >> > > unexpectedly.
>> >> > > > > Can
>> >> > > > > > > you
>> >> > > > > > > > > try to update your maven version and re-run the build?
>> >> > > > > > > > >
>> >> > > > > > > > > - Jason
>> >> > > > > > > > >
>> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
>> >> > > > asinha@maprtech.com>
>> >> > > > > > > > wrote:
>> >> > > > > > > > >
>> >> > > > > > > > > > Jacques, I am getting the following build failure on
>> the
>> >> > > latest
>> >> > > > > > > master
>> >> > > > > > > > > > branch...is this what you saw for the Apache build ?
>> My
>> >> > mvn
>> >> > > > > > version
>> >> > > > > > > > > output
>> >> > > > > > > > > > is shown below.  Should we all be upgrading to a newer
>> >> mvn
>> >> > ?
>> >> > > > > > > > > >
>> >> > > > > > > > > >
>> >> > > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
>> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
>> >> > > > > > > > > > [WARNING] Rule 0:
>> >> > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
>> >> > > > > > > > > failed
>> >> > > > > > > > > > with message:
>> >> > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside
>> the
>> >> > > > > expected
>> >> > > > > > > size
>> >> > > > > > > > > > range.
>> >> > > > > > > > > >
>> >> > > > > > > > > >                   This is likely due to you adding new
>> >> > > > > dependencies
>> >> > > > > > > to
>> >> > > > > > > > a
>> >> > > > > > > > > > java-exec and not updating the excludes in this
>> module.
>> >> > This
>> >> > > is
>> >> > > > > > > > important
>> >> > > > > > > > > > as it minimizes the size of the dependency of Drill
>> >> > > application
>> >> > > > > > > users.
>> >> > > > > > > > > >
>> >> > > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> >> > > > > > > > > > size (44664121) too large. Max. is
>> >> > > > > > > > > >
>> >> > > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> >> > > > > > > > > >
>> >> > > > > > > > > >
>> >> > > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill asinha$
>> >> mvn
>> >> > > > > > --version
>> >> > > > > > > > > > Apache Maven 3.0.5
>> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
>> >> > > > > > > > 2013-02-19
>> >> > > > > > > > > > 05:51:28-0800)
>> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3
>> >> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle Corporation
>> >> > > > > > > > > > Java home:
>> >> > > > > > > > > >
>> >> > > > >
>> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
>> >> > > > > > > > > > Default locale: en_US, platform encoding: UTF-8
>> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5", arch:
>> "x86_64",
>> >> > > family:
>> >> > > > > > "mac"
>> >> > > > > > > > > >
>> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <
>> >> > > > > > jacques@dremio.com>
>> >> > > > > > > > > > wrote:
>> >> > > > > > > > > >
>> >> > > > > > > > > > > Build back to normal. It looks like the Apache
>> server
>> >> was
>> >> > > > using
>> >> > > > > > an
>> >> > > > > > > > old
>> >> > > > > > > > > > > version of Maven. Once I switched to something more
>> >> > recent,
>> >> > > > the
>> >> > > > > > > build
>> >> > > > > > > > > > > passed.
>> >> > > > > > > > > > >
>> >> > > > > > > > > > > --
>> >> > > > > > > > > > > Jacques Nadeau
>> >> > > > > > > > > > > CTO and Co-Founder, Dremio
>> >> > > > > > > > > > >
>> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <
>> >> > > > > > > jacques@dremio.com>
>> >> > > > > > > > > > > wrote:
>> >> > > > > > > > > > >
>> >> > > > > > > > > > > > Hmm... this merge caused the Apache build to fail.
>> >> > > > > > > Investigating...
>> >> > > > > > > > > > > >
>> >> > > > > > > > > > > > --
>> >> > > > > > > > > > > > Jacques Nadeau
>> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
>> >> > > > > > > > > > > >
>> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <
>> >> > > > > > > > jacques@dremio.com>
>> >> > > > > > > > > > > > wrote:
>> >> > > > > > > > > > > >
>> >> > > > > > > > > > > >> I got clean regression runs as well. I've merged
>> the
>> >> > > > patch.
>> >> > > > > > > > > > > >>
>> >> > > > > > > > > > > >> Jason, you want to start the release process?
>> >> > > > > > > > > > > >>
>> >> > > > > > > > > > > >> --
>> >> > > > > > > > > > > >> Jacques Nadeau
>> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
>> >> > > > > > > > > > > >>
>> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek
>> Girish <
>> >> > > > > > > > agirish@mapr.com
>> >> > > > > > > > > >
>> >> > > > > > > > > > > >> wrote:
>> >> > > > > > > > > > > >>
>> >> > > > > > > > > > > >>> Had two clean Functional runs. TPC-H SF100 was
>> also
>> >> > > > > > successful.
>> >> > > > > > > > > > > >>>
>> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul
>> >> challapalli <
>> >> > > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
>> >> > > > > > > > > > > >>>
>> >> > > > > > > > > > > >>> > Kicked off a functional run with your branch.
>> >> Will
>> >> > > let
>> >> > > > > you
>> >> > > > > > > know
>> >> > > > > > > > > > once
>> >> > > > > > > > > > > it
>> >> > > > > > > > > > > >>> > finishes
>> >> > > > > > > > > > > >>> >
>> >> > > > > > > > > > > >>> > - Rahul
>> >> > > > > > > > > > > >>> >
>> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques
>> Nadeau <
>> >> > > > > > > > > > jacques@dremio.com>
>> >> > > > > > > > > > > >>> > wrote:
>> >> > > > > > > > > > > >>> >
>> >> > > > > > > > > > > >>> > > 4196 was merged today. I have an updated
>> patch
>> >> > for
>> >> > > > 4291
>> >> > > > > > > that
>> >> > > > > > > > is
>> >> > > > > > > > > > > >>> ready.
>> >> > > > > > > > > > > >>> > > Unfortunately, it seems that something isn't
>> >> > > working
>> >> > > > > with
>> >> > > > > > > our
>> >> > > > > > > > > > > >>> extended
>> >> > > > > > > > > > > >>> > > tests so I haven't been able to run an
>> extended
>> >> > > > > > regression.
>> >> > > > > > > > > Unit
>> >> > > > > > > > > > > >>> tests
>> >> > > > > > > > > > > >>> > > pass. Is someone else possibly able to run a
>> >> > > > regression
>> >> > > > > > > suite
>> >> > > > > > > > > > > against
>> >> > > > > > > > > > > >>> > this
>> >> > > > > > > > > > > >>> > > branch [1] so we can confirm things look
>> good
>> >> and
>> >> > > > start
>> >> > > > > > the
>> >> > > > > > > > > > release
>> >> > > > > > > > > > > >>> > > process?
>> >> > > > > > > > > > > >>> > >
>> >> > > > > > > > > > > >>> > > thanks,
>> >> > > > > > > > > > > >>> > > Jacques
>> >> > > > > > > > > > > >>> > >
>> >> > > > > > > > > > > >>> > > [1]
>> >> > > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
>> >> > > > > > > > > > > >>> > >
>> >> > > > > > > > > > > >>> > > --
>> >> > > > > > > > > > > >>> > > Jacques Nadeau
>> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
>> >> > > > > > > > > > > >>> > >
>> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques
>> >> Nadeau
>> >> > <
>> >> > > > > > > > > > > jacques@dremio.com
>> >> > > > > > > > > > > >>> >
>> >> > > > > > > > > > > >>> > > wrote:
>> >> > > > > > > > > > > >>> > >
>> >> > > > > > > > > > > >>> > > > I think the main things are 4196 and 4291
>> >> > should
>> >> > > be
>> >> > > > > > > > > completed.
>> >> > > > > > > > > > I
>> >> > > > > > > > > > > >>> know
>> >> > > > > > > > > > > >>> > > Amit
>> >> > > > > > > > > > > >>> > > > was able to reproduce 4196 locally this
>> >> weekend
>> >> > > so
>> >> > > > I
>> >> > > > > > > think
>> >> > > > > > > > > > we're
>> >> > > > > > > > > > > >>> close
>> >> > > > > > > > > > > >>> > on
>> >> > > > > > > > > > > >>> > > > that.
>> >> > > > > > > > > > > >>> > > >
>> >> > > > > > > > > > > >>> > > > --
>> >> > > > > > > > > > > >>> > > > Jacques Nadeau
>> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
>> >> > > > > > > > > > > >>> > > >
>> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine
>> >> Fong
>> >> > <
>> >> > > > > > > > > > > zfong@maprtech.com
>> >> > > > > > > > > > > >>> >
>> >> > > > > > > > > > > >>> > > wrote:
>> >> > > > > > > > > > > >>> > > >
>> >> > > > > > > > > > > >>> > > >> Any updates on this?  What's blocking us
>> >> from
>> >> > > > taking
>> >> > > > > > > this
>> >> > > > > > > > > to a
>> >> > > > > > > > > > > >>> vote --
>> >> > > > > > > > > > > >>> > > the
>> >> > > > > > > > > > > >>> > > >> sort merge join issues?
>> >> > > > > > > > > > > >>> > > >>
>> >> > > > > > > > > > > >>> > > >> -- Zelaine
>> >> > > > > > > > > > > >>> > > >>
>> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques
>> >> > Nadeau
>> >> > > <
>> >> > > > > > > > > > > >>> jacques@dremio.com>
>> >> > > > > > > > > > > >>> > > >> wrote:
>> >> > > > > > > > > > > >>> > > >>
>> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
>> >> > > > > > > > > > > >>> > > >> >
>> >> > > > > > > > > > > >>> > > >> > Here are the issues that were
>> mentioned in
>> >> > > this
>> >> > > > > > thread
>> >> > > > > > > > > along
>> >> > > > > > > > > > > >>> with a
>> >> > > > > > > > > > > >>> > > >> > proposed categorization:
>> >> > > > > > > > > > > >>> > > >> >
>> >> > > > > > > > > > > >>> > > >> > Release Blockers
>> >> > > > > > > > > > > >>> > > >> > In-progress Amit
>> >> > > > > > > > > > > >>>
>> https://issues.apache.org/jira/browse/DRILL-4190
>> >> > > > > > > > > > > >>> > > >> > In-progress Amit
>> >> > > > > > > > > > > >>>
>> https://issues.apache.org/jira/browse/DRILL-4196
>> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
>> >> > > > > > > > > > > >>> > >
>> >> https://issues.apache.org/jira/browse/DRILL-4246
>> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
>> >> > > > > > > > > > > >>>
>> https://issues.apache.org/jira/browse/DRILL-4256
>> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
>> >> > > > > > > > > > > >>> >
>> https://issues.apache.org/jira/browse/DRILL-4278
>> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
>> >> > > > > > > > > > > >>> > >
>> >> https://issues.apache.org/jira/browse/DRILL-4285
>> >> > > > > > > > > > > >>> > > >> > Nice to Have
>> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
>> >> > > > > > > > > > > >>>
>> https://issues.apache.org/jira/browse/DRILL-4247
>> >> > > > > > > > > > > >>> > > >> > In-progress Jason
>> >> > > > > > > > > > > >>>
>> https://issues.apache.org/jira/browse/DRILL-4203
>> >> > > > > > > > > > > >>> > > >> > Open Jacques
>> >> > > > > > > > > > https://issues.apache.org/jira/browse/DRILL-4266
>> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
>> >> > > > > > > > > > > >>> > >
>> >> https://issues.apache.org/jira/browse/DRILL-4131
>> >> > > > > > > > > > > >>> > > >> >
>> >> > > > > > > > > > > >>> > > >> > What do others think? Let's try to get
>> the
>> >> > > > > blockers
>> >> > > > > > > > > wrapped
>> >> > > > > > > > > > up
>> >> > > > > > > > > > > >>> in
>> >> > > > > > > > > > > >>> > the
>> >> > > > > > > > > > > >>> > > >> next
>> >> > > > > > > > > > > >>> > > >> > day or two and start a release vote...
>> >> > > > > > > > > > > >>> > > >> >
>> >> > > > > > > > > > > >>> > > >> >
>> >> > > > > > > > > > > >>> > > >> >
>> >> > > > > > > > > > > >>> > > >> > --
>> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
>> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
>> >> > > > > > > > > > > >>> > > >> >
>> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason
>> >> > > Altekruse
>> >> > > > <
>> >> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
>> >> > > > > > > > > > > >>> > > >> > wrote:
>> >> > > > > > > > > > > >>> > > >> >
>> >> > > > > > > > > > > >>> > > >> > > Hello All,
>> >> > > > > > > > > > > >>> > > >> > >
>> >> > > > > > > > > > > >>> > > >> > > With the allocator changes merged and
>> >> > about
>> >> > > a
>> >> > > > > > month
>> >> > > > > > > > > since
>> >> > > > > > > > > > > the
>> >> > > > > > > > > > > >>> last
>> >> > > > > > > > > > > >>> > > >> > release
>> >> > > > > > > > > > > >>> > > >> > > I think it would be good to start a
>> vote
>> >> > > > soon. I
>> >> > > > > > > would
>> >> > > > > > > > > > like
>> >> > > > > > > > > > > to
>> >> > > > > > > > > > > >>> > > >> volunteer
>> >> > > > > > > > > > > >>> > > >> > to
>> >> > > > > > > > > > > >>> > > >> > > be release manager.
>> >> > > > > > > > > > > >>> > > >> > >
>> >> > > > > > > > > > > >>> > > >> > > I know that there were some issues
>> that
>> >> > were
>> >> > > > > > > > identified
>> >> > > > > > > > > > > after
>> >> > > > > > > > > > > >>> the
>> >> > > > > > > > > > > >>> > > >> > transfer
>> >> > > > > > > > > > > >>> > > >> > > patch was merged. I think that these
>> >> > issues
>> >> > > > > should
>> >> > > > > > > be
>> >> > > > > > > > > > fixed
>> >> > > > > > > > > > > >>> before
>> >> > > > > > > > > > > >>> > > we
>> >> > > > > > > > > > > >>> > > >> > cut a
>> >> > > > > > > > > > > >>> > > >> > > release candidate.
>> >> > > > > > > > > > > >>> > > >> > >
>> >> > > > > > > > > > > >>> > > >> > > From looking at the associated JIRAs
>> it
>> >> > > looked
>> >> > > > > > like
>> >> > > > > > > > > there
>> >> > > > > > > > > > > was
>> >> > > > > > > > > > > >>> a
>> >> > > > > > > > > > > >>> > > >> possible
>> >> > > > > > > > > > > >>> > > >> > > short term fix just adjusting the
>> >> > > > > > > > > > max_query_memory_per_node
>> >> > > > > > > > > > > >>> > option,
>> >> > > > > > > > > > > >>> > > >> and
>> >> > > > > > > > > > > >>> > > >> > > some more involved work to change
>> how we
>> >> > > > > determine
>> >> > > > > > > the
>> >> > > > > > > > > > > correct
>> >> > > > > > > > > > > >>> > time
>> >> > > > > > > > > > > >>> > > to
>> >> > > > > > > > > > > >>> > > >> > > spill during external sort. I
>> believe it
>> >> > > makes
>> >> > > > > > sense
>> >> > > > > > > > to
>> >> > > > > > > > > > make
>> >> > > > > > > > > > > >>> > > external
>> >> > > > > > > > > > > >>> > > >> > sort
>> >> > > > > > > > > > > >>> > > >> > > work well with the newly improved
>> memory
>> >> > > > > > accounting
>> >> > > > > > > > > before
>> >> > > > > > > > > > > >>> > cutting a
>> >> > > > > > > > > > > >>> > > >> > > release, but I'm not sure how much
>> work
>> >> is
>> >> > > > left
>> >> > > > > to
>> >> > > > > > > be
>> >> > > > > > > > > done
>> >> > > > > > > > > > > >>> there.
>> >> > > > > > > > > > > >>> > > [1]
>> >> > > > > > > > > > > >>> > > >> > >
>> >> > > > > > > > > > > >>> > > >> > > Please respond with your thoughts on
>> a
>> >> > > release
>> >> > > > > > soon
>> >> > > > > > > > and
>> >> > > > > > > > > > any
>> >> > > > > > > > > > > >>> JIRAs
>> >> > > > > > > > > > > >>> > > you
>> >> > > > > > > > > > > >>> > > >> > would
>> >> > > > > > > > > > > >>> > > >> > > like to include in the release.
>> >> > > > > > > > > > > >>> > > >> > >
>> >> > > > > > > > > > > >>> > > >> > > [1] -
>> >> > > > > > > > https://issues.apache.org/jira/browse/DRILL-4243
>> >> > > > > > > > > > > >>> > > >> > >
>> >> > > > > > > > > > > >>> > > >> > > Thanks,
>> >> > > > > > > > > > > >>> > > >> > > Jason
>> >> > > > > > > > > > > >>> > > >> > >
>> >> > > > > > > > > > > >>> > > >> >
>> >> > > > > > > > > > > >>> > > >>
>> >> > > > > > > > > > > >>> > > >
>> >> > > > > > > > > > > >>> > > >
>> >> > > > > > > > > > > >>> > >
>> >> > > > > > > > > > > >>> >
>> >> > > > > > > > > > > >>>
>> >> > > > > > > > > > > >>
>> >> > > > > > > > > > > >>
>> >> > > > > > > > > > > >
>> >> > > > > > > > > > >
>> >> > > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>>

Re: Time for a 1.5 release?

Posted by Krystal Nguyen <kn...@maprtech.com>.

I am running drill-1.4 and is able to run count(*) successfully.

0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session set
`store.hive.optimize_scan_with_native_readers` = true;
+-------+--------------------------------------------------------+
|  ok   |                        summary                         |
+-------+--------------------------------------------------------+
| true  | store.hive.optimize_scan_with_native_readers updated.  |
+-------+--------------------------------------------------------+
1 row selected (0.374 seconds)
0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select count(*) from
lineitem_parquet_partitioned_hive_string;
+---------+
| EXPR$0  |
+---------+
| 60175   |
+---------+
1 row selected (2.986 seconds)

On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <ji...@gmail.com> wrote:

> Venki and I did some investigation for DRILL-4323. The issue reported
> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems to us
> this is not a regression from 1.4.0; it's a regression from 1.3.0
> probably.
>
> DRILL-4083 makes the planner to use DrillHiveNativeReader in stead of
> HiveReader for "select count(*) from hive_table" query.  However, the
> Project after the scan produces empty schema.  Before DRILL-4083,
> Drill uses HiveScan, which works fine.
>
>
> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
> <al...@gmail.com> wrote:
> > We could revert the change to the StoragePlugin interface to add the
> > getLogicalOptimizerRules
> > and getPhysicalOptimizerRules, and only put those in
> AbstractStoragePlugin
> > with default implementations.
> >
> > We could then update the docs to tell users to not implement the
> interface
> > directly, and instead extend the abstract class, which I believe would
> > allow us to make changes like this in the future without breaking
> plugins.
> > At a major release version we could then just pull up all of the new
> > methods we added to the abstract class into the interface, or just get
> rid
> > of it all-together.
> >
> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
> venki.korukanti@gmail.com>
> > wrote:
> >
> >> Here is the partial fix:
> >>
> >>
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> >>
> >> If the existing StragePlugins are implementing AbstractStoragePlugin
> class
> >> this fix should resolve the backward compat issue. If they are
> implementing
> >> StoragePlugin interface directly then it won't resolve the issue. Not
> sure
> >> how to handle it in that case. Let me know if there any ways to resolve
> >> this issue completely.
> >>
> >> Thanks
> >> Venki
> >>
> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <as...@maprtech.com>
> wrote:
> >>
> >> > I was able to do the build with maven 3.3.9 (after doing 'brew install
> >> > maven').  Here are my env variables related to maven:
> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> >> >
> >> >
> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <ja...@dremio.com>
> >> > wrote:
> >> >
> >> > > Aman, for reference on the Maven version issue. You can see the
> output
> >> of
> >> > > the shade plugin between the two Apache Builds. If you search for
> >> > > maven-shade-plugin you'll see that build 647 (maven 3.0.5) fails
> >> because
> >> > it
> >> > > includes a bunch of hadoop (and other libraries that are listed as
> >> being
> >> > > excluded). If you look at build 648 (maven 3.3.3), those items are
> >> > included
> >> > > so that the build completes successfully.
> >> > >
> >> > > https://builds.apache.org/job/drill-scm/647/consoleText
> >> > > https://builds.apache.org/job/drill-scm/648/consoleText
> >> > >
> >> > > --
> >> > > Jacques Nadeau
> >> > > CTO and Co-Founder, Dremio
> >> > >
> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
> >> > > challapallirahul@gmail.com> wrote:
> >> > >
> >> > > > Jason,
> >> > > >
> >> > > > As of now I haven't narrowed down the exact commit which caused
> >> > > DRILL-4323
> >> > > > and I do not know the scope of the fix.
> >> > > >
> >> > > > - Rahul
> >> > > >
> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
> >> > > altekrusejason@gmail.com
> >> > > > >
> >> > > > wrote:
> >> > > >
> >> > > > > I think it makes sense to wait for Venki's fix of the API, as
> >> > > releasing a
> >> > > > > version with the change would make reverting/refactoring the API
> >> > > breaking
> >> > > > > change less useful.
> >> > > > >
> >> > > > > Do you have any idea about the scope of fixing the hive native
> >> > reader,
> >> > > or
> >> > > > > which change since the 1.4 release broke it? Is someone
> available
> >> to
> >> > > look
> >> > > > > at it today?
> >> > > > >
> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> >> > > > > challapallirahul@gmail.com> wrote:
> >> > > > >
> >> > > > > > Just found an issue with hive native parquet reader
> (DRILL-4323).
> >> > > This
> >> > > > > is a
> >> > > > > > regression from 1.4 and in my opinion should be treated as a
> >> > blocker.
> >> > > > > >
> >> > > > > > - Rahul
> >> > > > > >
> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> >> > > > > > venki.korukanti@gmail.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Sorry I didn't realize I was breaking the public interface
> API.
> >> > Let
> >> > > > me
> >> > > > > > work
> >> > > > > > > on a patch to make it backward compatible.
> >> > > > > > >
> >> > > > > > > Thanks
> >> > > > > > > Venki
> >> > > > > > >
> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <
> >> > > steven@dremio.com
> >> > > > >
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > I just wanted to bring up an issue that I just now
> >> discovered,
> >> > > that
> >> > > > > has
> >> > > > > > > > caused me a fair amount of grief.
> >> > > > > > > >
> >> > > > > > > > https://github.com/apache/drill/pull/300/commits
> >> > > > > > > >
> >> > > > > > > > DRILL-4198 changes a user-facing API, and causes
> >> StoragePlugins
> >> > > > that
> >> > > > > > were
> >> > > > > > > > compiled against currently released versions of Drill to
> no
> >> > > longer
> >> > > > > > > > functional properly. I would prefer that this breaking
> change
> >> > be
> >> > > > > > modified
> >> > > > > > > > to be backward compatible if possible.
> >> > > > > > > >
> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> >> > > > > > > > altekrusejason@gmail.com>
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > Hi Aman,
> >> > > > > > > > >
> >> > > > > > > > > This is the failure that he was seeing. He figured out
> that
> >> > the
> >> > > > new
> >> > > > > > > > > exclusions in jdbc-all were not being respected when the
> >> > build
> >> > > > was
> >> > > > > > run
> >> > > > > > > > with
> >> > > > > > > > > an older Maven version, causing the jar size to increase
> >> > > > > > significantly.
> >> > > > > > > > He
> >> > > > > > > > > added an enforcer to make sure the JAR didn't grow
> >> > > unexpectedly.
> >> > > > > Can
> >> > > > > > > you
> >> > > > > > > > > try to update your maven version and re-run the build?
> >> > > > > > > > >
> >> > > > > > > > > - Jason
> >> > > > > > > > >
> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
> >> > > > asinha@maprtech.com>
> >> > > > > > > > wrote:
> >> > > > > > > > >
> >> > > > > > > > > > Jacques, I am getting the following build failure on
> the
> >> > > latest
> >> > > > > > > master
> >> > > > > > > > > > branch...is this what you saw for the Apache build ?
> My
> >> > mvn
> >> > > > > > version
> >> > > > > > > > > output
> >> > > > > > > > > > is shown below.  Should we all be upgrading to a newer
> >> mvn
> >> > ?
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> >> > > > > > > > > > [WARNING] Rule 0:
> >> > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
> >> > > > > > > > > failed
> >> > > > > > > > > > with message:
> >> > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside
> the
> >> > > > > expected
> >> > > > > > > size
> >> > > > > > > > > > range.
> >> > > > > > > > > >
> >> > > > > > > > > >                   This is likely due to you adding new
> >> > > > > dependencies
> >> > > > > > > to
> >> > > > > > > > a
> >> > > > > > > > > > java-exec and not updating the excludes in this
> module.
> >> > This
> >> > > is
> >> > > > > > > > important
> >> > > > > > > > > > as it minimizes the size of the dependency of Drill
> >> > > application
> >> > > > > > > users.
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> > > > > > > > > > size (44664121) too large. Max. is
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill asinha$
> >> mvn
> >> > > > > > --version
> >> > > > > > > > > > Apache Maven 3.0.5
> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> >> > > > > > > > 2013-02-19
> >> > > > > > > > > > 05:51:28-0800)
> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3
> >> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle Corporation
> >> > > > > > > > > > Java home:
> >> > > > > > > > > >
> >> > > > >
> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> >> > > > > > > > > > Default locale: en_US, platform encoding: UTF-8
> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5", arch:
> "x86_64",
> >> > > family:
> >> > > > > > "mac"
> >> > > > > > > > > >
> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <
> >> > > > > > jacques@dremio.com>
> >> > > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > Build back to normal. It looks like the Apache
> server
> >> was
> >> > > > using
> >> > > > > > an
> >> > > > > > > > old
> >> > > > > > > > > > > version of Maven. Once I switched to something more
> >> > recent,
> >> > > > the
> >> > > > > > > build
> >> > > > > > > > > > > passed.
> >> > > > > > > > > > >
> >> > > > > > > > > > > --
> >> > > > > > > > > > > Jacques Nadeau
> >> > > > > > > > > > > CTO and Co-Founder, Dremio
> >> > > > > > > > > > >
> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <
> >> > > > > > > jacques@dremio.com>
> >> > > > > > > > > > > wrote:
> >> > > > > > > > > > >
> >> > > > > > > > > > > > Hmm... this merge caused the Apache build to fail.
> >> > > > > > > Investigating...
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > --
> >> > > > > > > > > > > > Jacques Nadeau
> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <
> >> > > > > > > > jacques@dremio.com>
> >> > > > > > > > > > > > wrote:
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >> I got clean regression runs as well. I've merged
> the
> >> > > > patch.
> >> > > > > > > > > > > >>
> >> > > > > > > > > > > >> Jason, you want to start the release process?
> >> > > > > > > > > > > >>
> >> > > > > > > > > > > >> --
> >> > > > > > > > > > > >> Jacques Nadeau
> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
> >> > > > > > > > > > > >>
> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek
> Girish <
> >> > > > > > > > agirish@mapr.com
> >> > > > > > > > > >
> >> > > > > > > > > > > >> wrote:
> >> > > > > > > > > > > >>
> >> > > > > > > > > > > >>> Had two clean Functional runs. TPC-H SF100 was
> also
> >> > > > > > successful.
> >> > > > > > > > > > > >>>
> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul
> >> challapalli <
> >> > > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
> >> > > > > > > > > > > >>>
> >> > > > > > > > > > > >>> > Kicked off a functional run with your branch.
> >> Will
> >> > > let
> >> > > > > you
> >> > > > > > > know
> >> > > > > > > > > > once
> >> > > > > > > > > > > it
> >> > > > > > > > > > > >>> > finishes
> >> > > > > > > > > > > >>> >
> >> > > > > > > > > > > >>> > - Rahul
> >> > > > > > > > > > > >>> >
> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques
> Nadeau <
> >> > > > > > > > > > jacques@dremio.com>
> >> > > > > > > > > > > >>> > wrote:
> >> > > > > > > > > > > >>> >
> >> > > > > > > > > > > >>> > > 4196 was merged today. I have an updated
> patch
> >> > for
> >> > > > 4291
> >> > > > > > > that
> >> > > > > > > > is
> >> > > > > > > > > > > >>> ready.
> >> > > > > > > > > > > >>> > > Unfortunately, it seems that something isn't
> >> > > working
> >> > > > > with
> >> > > > > > > our
> >> > > > > > > > > > > >>> extended
> >> > > > > > > > > > > >>> > > tests so I haven't been able to run an
> extended
> >> > > > > > regression.
> >> > > > > > > > > Unit
> >> > > > > > > > > > > >>> tests
> >> > > > > > > > > > > >>> > > pass. Is someone else possibly able to run a
> >> > > > regression
> >> > > > > > > suite
> >> > > > > > > > > > > against
> >> > > > > > > > > > > >>> > this
> >> > > > > > > > > > > >>> > > branch [1] so we can confirm things look
> good
> >> and
> >> > > > start
> >> > > > > > the
> >> > > > > > > > > > release
> >> > > > > > > > > > > >>> > > process?
> >> > > > > > > > > > > >>> > >
> >> > > > > > > > > > > >>> > > thanks,
> >> > > > > > > > > > > >>> > > Jacques
> >> > > > > > > > > > > >>> > >
> >> > > > > > > > > > > >>> > > [1]
> >> > > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> >> > > > > > > > > > > >>> > >
> >> > > > > > > > > > > >>> > > --
> >> > > > > > > > > > > >>> > > Jacques Nadeau
> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
> >> > > > > > > > > > > >>> > >
> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques
> >> Nadeau
> >> > <
> >> > > > > > > > > > > jacques@dremio.com
> >> > > > > > > > > > > >>> >
> >> > > > > > > > > > > >>> > > wrote:
> >> > > > > > > > > > > >>> > >
> >> > > > > > > > > > > >>> > > > I think the main things are 4196 and 4291
> >> > should
> >> > > be
> >> > > > > > > > > completed.
> >> > > > > > > > > > I
> >> > > > > > > > > > > >>> know
> >> > > > > > > > > > > >>> > > Amit
> >> > > > > > > > > > > >>> > > > was able to reproduce 4196 locally this
> >> weekend
> >> > > so
> >> > > > I
> >> > > > > > > think
> >> > > > > > > > > > we're
> >> > > > > > > > > > > >>> close
> >> > > > > > > > > > > >>> > on
> >> > > > > > > > > > > >>> > > > that.
> >> > > > > > > > > > > >>> > > >
> >> > > > > > > > > > > >>> > > > --
> >> > > > > > > > > > > >>> > > > Jacques Nadeau
> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
> >> > > > > > > > > > > >>> > > >
> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine
> >> Fong
> >> > <
> >> > > > > > > > > > > zfong@maprtech.com
> >> > > > > > > > > > > >>> >
> >> > > > > > > > > > > >>> > > wrote:
> >> > > > > > > > > > > >>> > > >
> >> > > > > > > > > > > >>> > > >> Any updates on this?  What's blocking us
> >> from
> >> > > > taking
> >> > > > > > > this
> >> > > > > > > > > to a
> >> > > > > > > > > > > >>> vote --
> >> > > > > > > > > > > >>> > > the
> >> > > > > > > > > > > >>> > > >> sort merge join issues?
> >> > > > > > > > > > > >>> > > >>
> >> > > > > > > > > > > >>> > > >> -- Zelaine
> >> > > > > > > > > > > >>> > > >>
> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques
> >> > Nadeau
> >> > > <
> >> > > > > > > > > > > >>> jacques@dremio.com>
> >> > > > > > > > > > > >>> > > >> wrote:
> >> > > > > > > > > > > >>> > > >>
> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
> >> > > > > > > > > > > >>> > > >> >
> >> > > > > > > > > > > >>> > > >> > Here are the issues that were
> mentioned in
> >> > > this
> >> > > > > > thread
> >> > > > > > > > > along
> >> > > > > > > > > > > >>> with a
> >> > > > > > > > > > > >>> > > >> > proposed categorization:
> >> > > > > > > > > > > >>> > > >> >
> >> > > > > > > > > > > >>> > > >> > Release Blockers
> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> >> > > > > > > > > > > >>>
> https://issues.apache.org/jira/browse/DRILL-4190
> >> > > > > > > > > > > >>> > > >> > In-progress Amit
> >> > > > > > > > > > > >>>
> https://issues.apache.org/jira/browse/DRILL-4196
> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> >> > > > > > > > > > > >>> > >
> >> https://issues.apache.org/jira/browse/DRILL-4246
> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
> >> > > > > > > > > > > >>>
> https://issues.apache.org/jira/browse/DRILL-4256
> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
> >> > > > > > > > > > > >>> >
> https://issues.apache.org/jira/browse/DRILL-4278
> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
> >> > > > > > > > > > > >>> > >
> >> https://issues.apache.org/jira/browse/DRILL-4285
> >> > > > > > > > > > > >>> > > >> > Nice to Have
> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
> >> > > > > > > > > > > >>>
> https://issues.apache.org/jira/browse/DRILL-4247
> >> > > > > > > > > > > >>> > > >> > In-progress Jason
> >> > > > > > > > > > > >>>
> https://issues.apache.org/jira/browse/DRILL-4203
> >> > > > > > > > > > > >>> > > >> > Open Jacques
> >> > > > > > > > > > https://issues.apache.org/jira/browse/DRILL-4266
> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> >> > > > > > > > > > > >>> > >
> >> https://issues.apache.org/jira/browse/DRILL-4131
> >> > > > > > > > > > > >>> > > >> >
> >> > > > > > > > > > > >>> > > >> > What do others think? Let's try to get
> the
> >> > > > > blockers
> >> > > > > > > > > wrapped
> >> > > > > > > > > > up
> >> > > > > > > > > > > >>> in
> >> > > > > > > > > > > >>> > the
> >> > > > > > > > > > > >>> > > >> next
> >> > > > > > > > > > > >>> > > >> > day or two and start a release vote...
> >> > > > > > > > > > > >>> > > >> >
> >> > > > > > > > > > > >>> > > >> >
> >> > > > > > > > > > > >>> > > >> >
> >> > > > > > > > > > > >>> > > >> > --
> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> >> > > > > > > > > > > >>> > > >> >
> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason
> >> > > Altekruse
> >> > > > <
> >> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
> >> > > > > > > > > > > >>> > > >> > wrote:
> >> > > > > > > > > > > >>> > > >> >
> >> > > > > > > > > > > >>> > > >> > > Hello All,
> >> > > > > > > > > > > >>> > > >> > >
> >> > > > > > > > > > > >>> > > >> > > With the allocator changes merged and
> >> > about
> >> > > a
> >> > > > > > month
> >> > > > > > > > > since
> >> > > > > > > > > > > the
> >> > > > > > > > > > > >>> last
> >> > > > > > > > > > > >>> > > >> > release
> >> > > > > > > > > > > >>> > > >> > > I think it would be good to start a
> vote
> >> > > > soon. I
> >> > > > > > > would
> >> > > > > > > > > > like
> >> > > > > > > > > > > to
> >> > > > > > > > > > > >>> > > >> volunteer
> >> > > > > > > > > > > >>> > > >> > to
> >> > > > > > > > > > > >>> > > >> > > be release manager.
> >> > > > > > > > > > > >>> > > >> > >
> >> > > > > > > > > > > >>> > > >> > > I know that there were some issues
> that
> >> > were
> >> > > > > > > > identified
> >> > > > > > > > > > > after
> >> > > > > > > > > > > >>> the
> >> > > > > > > > > > > >>> > > >> > transfer
> >> > > > > > > > > > > >>> > > >> > > patch was merged. I think that these
> >> > issues
> >> > > > > should
> >> > > > > > > be
> >> > > > > > > > > > fixed
> >> > > > > > > > > > > >>> before
> >> > > > > > > > > > > >>> > > we
> >> > > > > > > > > > > >>> > > >> > cut a
> >> > > > > > > > > > > >>> > > >> > > release candidate.
> >> > > > > > > > > > > >>> > > >> > >
> >> > > > > > > > > > > >>> > > >> > > From looking at the associated JIRAs
> it
> >> > > looked
> >> > > > > > like
> >> > > > > > > > > there
> >> > > > > > > > > > > was
> >> > > > > > > > > > > >>> a
> >> > > > > > > > > > > >>> > > >> possible
> >> > > > > > > > > > > >>> > > >> > > short term fix just adjusting the
> >> > > > > > > > > > max_query_memory_per_node
> >> > > > > > > > > > > >>> > option,
> >> > > > > > > > > > > >>> > > >> and
> >> > > > > > > > > > > >>> > > >> > > some more involved work to change
> how we
> >> > > > > determine
> >> > > > > > > the
> >> > > > > > > > > > > correct
> >> > > > > > > > > > > >>> > time
> >> > > > > > > > > > > >>> > > to
> >> > > > > > > > > > > >>> > > >> > > spill during external sort. I
> believe it
> >> > > makes
> >> > > > > > sense
> >> > > > > > > > to
> >> > > > > > > > > > make
> >> > > > > > > > > > > >>> > > external
> >> > > > > > > > > > > >>> > > >> > sort
> >> > > > > > > > > > > >>> > > >> > > work well with the newly improved
> memory
> >> > > > > > accounting
> >> > > > > > > > > before
> >> > > > > > > > > > > >>> > cutting a
> >> > > > > > > > > > > >>> > > >> > > release, but I'm not sure how much
> work
> >> is
> >> > > > left
> >> > > > > to
> >> > > > > > > be
> >> > > > > > > > > done
> >> > > > > > > > > > > >>> there.
> >> > > > > > > > > > > >>> > > [1]
> >> > > > > > > > > > > >>> > > >> > >
> >> > > > > > > > > > > >>> > > >> > > Please respond with your thoughts on
> a
> >> > > release
> >> > > > > > soon
> >> > > > > > > > and
> >> > > > > > > > > > any
> >> > > > > > > > > > > >>> JIRAs
> >> > > > > > > > > > > >>> > > you
> >> > > > > > > > > > > >>> > > >> > would
> >> > > > > > > > > > > >>> > > >> > > like to include in the release.
> >> > > > > > > > > > > >>> > > >> > >
> >> > > > > > > > > > > >>> > > >> > > [1] -
> >> > > > > > > > https://issues.apache.org/jira/browse/DRILL-4243
> >> > > > > > > > > > > >>> > > >> > >
> >> > > > > > > > > > > >>> > > >> > > Thanks,
> >> > > > > > > > > > > >>> > > >> > > Jason
> >> > > > > > > > > > > >>> > > >> > >
> >> > > > > > > > > > > >>> > > >> >
> >> > > > > > > > > > > >>> > > >>
> >> > > > > > > > > > > >>> > > >
> >> > > > > > > > > > > >>> > > >
> >> > > > > > > > > > > >>> > >
> >> > > > > > > > > > > >>> >
> >> > > > > > > > > > > >>>
> >> > > > > > > > > > > >>
> >> > > > > > > > > > > >>
> >> > > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>

Re: Time for a 1.5 release?

Posted by Jinfeng Ni <ji...@gmail.com>.

Venki and I did some investigation for DRILL-4323. The issue reported
in DRILL-4323 seems to happen on 1.4.0 release as well. Seems to us
this is not a regression from 1.4.0; it's a regression from 1.3.0
probably.

DRILL-4083 makes the planner to use DrillHiveNativeReader in stead of
HiveReader for "select count(*) from hive_table" query.  However, the
Project after the scan produces empty schema.  Before DRILL-4083,
Drill uses HiveScan, which works fine.


On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
<al...@gmail.com> wrote:
> We could revert the change to the StoragePlugin interface to add the
> getLogicalOptimizerRules
> and getPhysicalOptimizerRules, and only put those in AbstractStoragePlugin
> with default implementations.
>
> We could then update the docs to tell users to not implement the interface
> directly, and instead extend the abstract class, which I believe would
> allow us to make changes like this in the future without breaking plugins.
> At a major release version we could then just pull up all of the new
> methods we added to the abstract class into the interface, or just get rid
> of it all-together.
>
> On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <ve...@gmail.com>
> wrote:
>
>> Here is the partial fix:
>>
>> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
>>
>> If the existing StragePlugins are implementing AbstractStoragePlugin class
>> this fix should resolve the backward compat issue. If they are implementing
>> StoragePlugin interface directly then it won't resolve the issue. Not sure
>> how to handle it in that case. Let me know if there any ways to resolve
>> this issue completely.
>>
>> Thanks
>> Venki
>>
>> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <as...@maprtech.com> wrote:
>>
>> > I was able to do the build with maven 3.3.9 (after doing 'brew install
>> > maven').  Here are my env variables related to maven:
>> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
>> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
>> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
>> >
>> >
>> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <ja...@dremio.com>
>> > wrote:
>> >
>> > > Aman, for reference on the Maven version issue. You can see the output
>> of
>> > > the shade plugin between the two Apache Builds. If you search for
>> > > maven-shade-plugin you'll see that build 647 (maven 3.0.5) fails
>> because
>> > it
>> > > includes a bunch of hadoop (and other libraries that are listed as
>> being
>> > > excluded). If you look at build 648 (maven 3.3.3), those items are
>> > included
>> > > so that the build completes successfully.
>> > >
>> > > https://builds.apache.org/job/drill-scm/647/consoleText
>> > > https://builds.apache.org/job/drill-scm/648/consoleText
>> > >
>> > > --
>> > > Jacques Nadeau
>> > > CTO and Co-Founder, Dremio
>> > >
>> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
>> > > challapallirahul@gmail.com> wrote:
>> > >
>> > > > Jason,
>> > > >
>> > > > As of now I haven't narrowed down the exact commit which caused
>> > > DRILL-4323
>> > > > and I do not know the scope of the fix.
>> > > >
>> > > > - Rahul
>> > > >
>> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
>> > > altekrusejason@gmail.com
>> > > > >
>> > > > wrote:
>> > > >
>> > > > > I think it makes sense to wait for Venki's fix of the API, as
>> > > releasing a
>> > > > > version with the change would make reverting/refactoring the API
>> > > breaking
>> > > > > change less useful.
>> > > > >
>> > > > > Do you have any idea about the scope of fixing the hive native
>> > reader,
>> > > or
>> > > > > which change since the 1.4 release broke it? Is someone available
>> to
>> > > look
>> > > > > at it today?
>> > > > >
>> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
>> > > > > challapallirahul@gmail.com> wrote:
>> > > > >
>> > > > > > Just found an issue with hive native parquet reader (DRILL-4323).
>> > > This
>> > > > > is a
>> > > > > > regression from 1.4 and in my opinion should be treated as a
>> > blocker.
>> > > > > >
>> > > > > > - Rahul
>> > > > > >
>> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
>> > > > > > venki.korukanti@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Sorry I didn't realize I was breaking the public interface API.
>> > Let
>> > > > me
>> > > > > > work
>> > > > > > > on a patch to make it backward compatible.
>> > > > > > >
>> > > > > > > Thanks
>> > > > > > > Venki
>> > > > > > >
>> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <
>> > > steven@dremio.com
>> > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > I just wanted to bring up an issue that I just now
>> discovered,
>> > > that
>> > > > > has
>> > > > > > > > caused me a fair amount of grief.
>> > > > > > > >
>> > > > > > > > https://github.com/apache/drill/pull/300/commits
>> > > > > > > >
>> > > > > > > > DRILL-4198 changes a user-facing API, and causes
>> StoragePlugins
>> > > > that
>> > > > > > were
>> > > > > > > > compiled against currently released versions of Drill to no
>> > > longer
>> > > > > > > > functional properly. I would prefer that this breaking change
>> > be
>> > > > > > modified
>> > > > > > > > to be backward compatible if possible.
>> > > > > > > >
>> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
>> > > > > > > > altekrusejason@gmail.com>
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hi Aman,
>> > > > > > > > >
>> > > > > > > > > This is the failure that he was seeing. He figured out that
>> > the
>> > > > new
>> > > > > > > > > exclusions in jdbc-all were not being respected when the
>> > build
>> > > > was
>> > > > > > run
>> > > > > > > > with
>> > > > > > > > > an older Maven version, causing the jar size to increase
>> > > > > > significantly.
>> > > > > > > > He
>> > > > > > > > > added an enforcer to make sure the JAR didn't grow
>> > > unexpectedly.
>> > > > > Can
>> > > > > > > you
>> > > > > > > > > try to update your maven version and re-run the build?
>> > > > > > > > >
>> > > > > > > > > - Jason
>> > > > > > > > >
>> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
>> > > > asinha@maprtech.com>
>> > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Jacques, I am getting the following build failure on the
>> > > latest
>> > > > > > > master
>> > > > > > > > > > branch...is this what you saw for the Apache build ?  My
>> > mvn
>> > > > > > version
>> > > > > > > > > output
>> > > > > > > > > > is shown below.  Should we all be upgrading to a newer
>> mvn
>> > ?
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
>> > > > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
>> > > > > > > > > > [WARNING] Rule 0:
>> > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
>> > > > > > > > > failed
>> > > > > > > > > > with message:
>> > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the
>> > > > > expected
>> > > > > > > size
>> > > > > > > > > > range.
>> > > > > > > > > >
>> > > > > > > > > >                   This is likely due to you adding new
>> > > > > dependencies
>> > > > > > > to
>> > > > > > > > a
>> > > > > > > > > > java-exec and not updating the excludes in this module.
>> > This
>> > > is
>> > > > > > > > important
>> > > > > > > > > > as it minimizes the size of the dependency of Drill
>> > > application
>> > > > > > > users.
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> > > > > > > > > > size (44664121) too large. Max. is
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill asinha$
>> mvn
>> > > > > > --version
>> > > > > > > > > > Apache Maven 3.0.5
>> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
>> > > > > > > > 2013-02-19
>> > > > > > > > > > 05:51:28-0800)
>> > > > > > > > > > Maven home: /opt/local/share/java/maven3
>> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle Corporation
>> > > > > > > > > > Java home:
>> > > > > > > > > >
>> > > > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
>> > > > > > > > > > Default locale: en_US, platform encoding: UTF-8
>> > > > > > > > > > OS name: "mac os x", version: "10.9.5", arch: "x86_64",
>> > > family:
>> > > > > > "mac"
>> > > > > > > > > >
>> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <
>> > > > > > jacques@dremio.com>
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > Build back to normal. It looks like the Apache server
>> was
>> > > > using
>> > > > > > an
>> > > > > > > > old
>> > > > > > > > > > > version of Maven. Once I switched to something more
>> > recent,
>> > > > the
>> > > > > > > build
>> > > > > > > > > > > passed.
>> > > > > > > > > > >
>> > > > > > > > > > > --
>> > > > > > > > > > > Jacques Nadeau
>> > > > > > > > > > > CTO and Co-Founder, Dremio
>> > > > > > > > > > >
>> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <
>> > > > > > > jacques@dremio.com>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > Hmm... this merge caused the Apache build to fail.
>> > > > > > > Investigating...
>> > > > > > > > > > > >
>> > > > > > > > > > > > --
>> > > > > > > > > > > > Jacques Nadeau
>> > > > > > > > > > > > CTO and Co-Founder, Dremio
>> > > > > > > > > > > >
>> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <
>> > > > > > > > jacques@dremio.com>
>> > > > > > > > > > > > wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > >> I got clean regression runs as well. I've merged the
>> > > > patch.
>> > > > > > > > > > > >>
>> > > > > > > > > > > >> Jason, you want to start the release process?
>> > > > > > > > > > > >>
>> > > > > > > > > > > >> --
>> > > > > > > > > > > >> Jacques Nadeau
>> > > > > > > > > > > >> CTO and Co-Founder, Dremio
>> > > > > > > > > > > >>
>> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <
>> > > > > > > > agirish@mapr.com
>> > > > > > > > > >
>> > > > > > > > > > > >> wrote:
>> > > > > > > > > > > >>
>> > > > > > > > > > > >>> Had two clean Functional runs. TPC-H SF100 was also
>> > > > > > successful.
>> > > > > > > > > > > >>>
>> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul
>> challapalli <
>> > > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
>> > > > > > > > > > > >>>
>> > > > > > > > > > > >>> > Kicked off a functional run with your branch.
>> Will
>> > > let
>> > > > > you
>> > > > > > > know
>> > > > > > > > > > once
>> > > > > > > > > > > it
>> > > > > > > > > > > >>> > finishes
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>> > - Rahul
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <
>> > > > > > > > > > jacques@dremio.com>
>> > > > > > > > > > > >>> > wrote:
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>> > > 4196 was merged today. I have an updated patch
>> > for
>> > > > 4291
>> > > > > > > that
>> > > > > > > > is
>> > > > > > > > > > > >>> ready.
>> > > > > > > > > > > >>> > > Unfortunately, it seems that something isn't
>> > > working
>> > > > > with
>> > > > > > > our
>> > > > > > > > > > > >>> extended
>> > > > > > > > > > > >>> > > tests so I haven't been able to run an extended
>> > > > > > regression.
>> > > > > > > > > Unit
>> > > > > > > > > > > >>> tests
>> > > > > > > > > > > >>> > > pass. Is someone else possibly able to run a
>> > > > regression
>> > > > > > > suite
>> > > > > > > > > > > against
>> > > > > > > > > > > >>> > this
>> > > > > > > > > > > >>> > > branch [1] so we can confirm things look good
>> and
>> > > > start
>> > > > > > the
>> > > > > > > > > > release
>> > > > > > > > > > > >>> > > process?
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > thanks,
>> > > > > > > > > > > >>> > > Jacques
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > [1]
>> > > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > --
>> > > > > > > > > > > >>> > > Jacques Nadeau
>> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques
>> Nadeau
>> > <
>> > > > > > > > > > > jacques@dremio.com
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>> > > wrote:
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > > I think the main things are 4196 and 4291
>> > should
>> > > be
>> > > > > > > > > completed.
>> > > > > > > > > > I
>> > > > > > > > > > > >>> know
>> > > > > > > > > > > >>> > > Amit
>> > > > > > > > > > > >>> > > > was able to reproduce 4196 locally this
>> weekend
>> > > so
>> > > > I
>> > > > > > > think
>> > > > > > > > > > we're
>> > > > > > > > > > > >>> close
>> > > > > > > > > > > >>> > on
>> > > > > > > > > > > >>> > > > that.
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > > --
>> > > > > > > > > > > >>> > > > Jacques Nadeau
>> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine
>> Fong
>> > <
>> > > > > > > > > > > zfong@maprtech.com
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>> > > wrote:
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > >> Any updates on this?  What's blocking us
>> from
>> > > > taking
>> > > > > > > this
>> > > > > > > > > to a
>> > > > > > > > > > > >>> vote --
>> > > > > > > > > > > >>> > > the
>> > > > > > > > > > > >>> > > >> sort merge join issues?
>> > > > > > > > > > > >>> > > >>
>> > > > > > > > > > > >>> > > >> -- Zelaine
>> > > > > > > > > > > >>> > > >>
>> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques
>> > Nadeau
>> > > <
>> > > > > > > > > > > >>> jacques@dremio.com>
>> > > > > > > > > > > >>> > > >> wrote:
>> > > > > > > > > > > >>> > > >>
>> > > > > > > > > > > >>> > > >> > Bumping this thread...
>> > > > > > > > > > > >>> > > >> >
>> > > > > > > > > > > >>> > > >> > Here are the issues that were mentioned in
>> > > this
>> > > > > > thread
>> > > > > > > > > along
>> > > > > > > > > > > >>> with a
>> > > > > > > > > > > >>> > > >> > proposed categorization:
>> > > > > > > > > > > >>> > > >> >
>> > > > > > > > > > > >>> > > >> > Release Blockers
>> > > > > > > > > > > >>> > > >> > In-progress Amit
>> > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4190
>> > > > > > > > > > > >>> > > >> > In-progress Amit
>> > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4196
>> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
>> > > > > > > > > > > >>> > >
>> https://issues.apache.org/jira/browse/DRILL-4246
>> > > > > > > > > > > >>> > > >> > In-review Jinfeng
>> > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4256
>> > > > > > > > > > > >>> > > >> > In-progress Jacques
>> > > > > > > > > > > >>> > https://issues.apache.org/jira/browse/DRILL-4278
>> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
>> > > > > > > > > > > >>> > >
>> https://issues.apache.org/jira/browse/DRILL-4285
>> > > > > > > > > > > >>> > > >> > Nice to Have
>> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
>> > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4247
>> > > > > > > > > > > >>> > > >> > In-progress Jason
>> > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4203
>> > > > > > > > > > > >>> > > >> > Open Jacques
>> > > > > > > > > > https://issues.apache.org/jira/browse/DRILL-4266
>> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
>> > > > > > > > > > > >>> > >
>> https://issues.apache.org/jira/browse/DRILL-4131
>> > > > > > > > > > > >>> > > >> >
>> > > > > > > > > > > >>> > > >> > What do others think? Let's try to get the
>> > > > > blockers
>> > > > > > > > > wrapped
>> > > > > > > > > > up
>> > > > > > > > > > > >>> in
>> > > > > > > > > > > >>> > the
>> > > > > > > > > > > >>> > > >> next
>> > > > > > > > > > > >>> > > >> > day or two and start a release vote...
>> > > > > > > > > > > >>> > > >> >
>> > > > > > > > > > > >>> > > >> >
>> > > > > > > > > > > >>> > > >> >
>> > > > > > > > > > > >>> > > >> > --
>> > > > > > > > > > > >>> > > >> > Jacques Nadeau
>> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
>> > > > > > > > > > > >>> > > >> >
>> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason
>> > > Altekruse
>> > > > <
>> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
>> > > > > > > > > > > >>> > > >> > wrote:
>> > > > > > > > > > > >>> > > >> >
>> > > > > > > > > > > >>> > > >> > > Hello All,
>> > > > > > > > > > > >>> > > >> > >
>> > > > > > > > > > > >>> > > >> > > With the allocator changes merged and
>> > about
>> > > a
>> > > > > > month
>> > > > > > > > > since
>> > > > > > > > > > > the
>> > > > > > > > > > > >>> last
>> > > > > > > > > > > >>> > > >> > release
>> > > > > > > > > > > >>> > > >> > > I think it would be good to start a vote
>> > > > soon. I
>> > > > > > > would
>> > > > > > > > > > like
>> > > > > > > > > > > to
>> > > > > > > > > > > >>> > > >> volunteer
>> > > > > > > > > > > >>> > > >> > to
>> > > > > > > > > > > >>> > > >> > > be release manager.
>> > > > > > > > > > > >>> > > >> > >
>> > > > > > > > > > > >>> > > >> > > I know that there were some issues that
>> > were
>> > > > > > > > identified
>> > > > > > > > > > > after
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > >> > transfer
>> > > > > > > > > > > >>> > > >> > > patch was merged. I think that these
>> > issues
>> > > > > should
>> > > > > > > be
>> > > > > > > > > > fixed
>> > > > > > > > > > > >>> before
>> > > > > > > > > > > >>> > > we
>> > > > > > > > > > > >>> > > >> > cut a
>> > > > > > > > > > > >>> > > >> > > release candidate.
>> > > > > > > > > > > >>> > > >> > >
>> > > > > > > > > > > >>> > > >> > > From looking at the associated JIRAs it
>> > > looked
>> > > > > > like
>> > > > > > > > > there
>> > > > > > > > > > > was
>> > > > > > > > > > > >>> a
>> > > > > > > > > > > >>> > > >> possible
>> > > > > > > > > > > >>> > > >> > > short term fix just adjusting the
>> > > > > > > > > > max_query_memory_per_node
>> > > > > > > > > > > >>> > option,
>> > > > > > > > > > > >>> > > >> and
>> > > > > > > > > > > >>> > > >> > > some more involved work to change how we
>> > > > > determine
>> > > > > > > the
>> > > > > > > > > > > correct
>> > > > > > > > > > > >>> > time
>> > > > > > > > > > > >>> > > to
>> > > > > > > > > > > >>> > > >> > > spill during external sort. I believe it
>> > > makes
>> > > > > > sense
>> > > > > > > > to
>> > > > > > > > > > make
>> > > > > > > > > > > >>> > > external
>> > > > > > > > > > > >>> > > >> > sort
>> > > > > > > > > > > >>> > > >> > > work well with the newly improved memory
>> > > > > > accounting
>> > > > > > > > > before
>> > > > > > > > > > > >>> > cutting a
>> > > > > > > > > > > >>> > > >> > > release, but I'm not sure how much work
>> is
>> > > > left
>> > > > > to
>> > > > > > > be
>> > > > > > > > > done
>> > > > > > > > > > > >>> there.
>> > > > > > > > > > > >>> > > [1]
>> > > > > > > > > > > >>> > > >> > >
>> > > > > > > > > > > >>> > > >> > > Please respond with your thoughts on a
>> > > release
>> > > > > > soon
>> > > > > > > > and
>> > > > > > > > > > any
>> > > > > > > > > > > >>> JIRAs
>> > > > > > > > > > > >>> > > you
>> > > > > > > > > > > >>> > > >> > would
>> > > > > > > > > > > >>> > > >> > > like to include in the release.
>> > > > > > > > > > > >>> > > >> > >
>> > > > > > > > > > > >>> > > >> > > [1] -
>> > > > > > > > https://issues.apache.org/jira/browse/DRILL-4243
>> > > > > > > > > > > >>> > > >> > >
>> > > > > > > > > > > >>> > > >> > > Thanks,
>> > > > > > > > > > > >>> > > >> > > Jason
>> > > > > > > > > > > >>> > > >> > >
>> > > > > > > > > > > >>> > > >> >
>> > > > > > > > > > > >>> > > >>
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>>
>> > > > > > > > > > > >>
>> > > > > > > > > > > >>
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>

Re: Time for a 1.5 release?

Posted by Jason Altekruse <al...@gmail.com>.

We could revert the change to the StoragePlugin interface to add the
getLogicalOptimizerRules
and getPhysicalOptimizerRules, and only put those in AbstractStoragePlugin
with default implementations.

We could then update the docs to tell users to not implement the interface
directly, and instead extend the abstract class, which I believe would
allow us to make changes like this in the future without breaking plugins.
At a major release version we could then just pull up all of the new
methods we added to the abstract class into the interface, or just get rid
of it all-together.

On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <ve...@gmail.com>
wrote:

> Here is the partial fix:
>
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
>
> If the existing StragePlugins are implementing AbstractStoragePlugin class
> this fix should resolve the backward compat issue. If they are implementing
> StoragePlugin interface directly then it won't resolve the issue. Not sure
> how to handle it in that case. Let me know if there any ways to resolve
> this issue completely.
>
> Thanks
> Venki
>
> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <as...@maprtech.com> wrote:
>
> > I was able to do the build with maven 3.3.9 (after doing 'brew install
> > maven').  Here are my env variables related to maven:
> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> >
> >
> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <ja...@dremio.com>
> > wrote:
> >
> > > Aman, for reference on the Maven version issue. You can see the output
> of
> > > the shade plugin between the two Apache Builds. If you search for
> > > maven-shade-plugin you'll see that build 647 (maven 3.0.5) fails
> because
> > it
> > > includes a bunch of hadoop (and other libraries that are listed as
> being
> > > excluded). If you look at build 648 (maven 3.3.3), those items are
> > included
> > > so that the build completes successfully.
> > >
> > > https://builds.apache.org/job/drill-scm/647/consoleText
> > > https://builds.apache.org/job/drill-scm/648/consoleText
> > >
> > > --
> > > Jacques Nadeau
> > > CTO and Co-Founder, Dremio
> > >
> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
> > > challapallirahul@gmail.com> wrote:
> > >
> > > > Jason,
> > > >
> > > > As of now I haven't narrowed down the exact commit which caused
> > > DRILL-4323
> > > > and I do not know the scope of the fix.
> > > >
> > > > - Rahul
> > > >
> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
> > > altekrusejason@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > I think it makes sense to wait for Venki's fix of the API, as
> > > releasing a
> > > > > version with the change would make reverting/refactoring the API
> > > breaking
> > > > > change less useful.
> > > > >
> > > > > Do you have any idea about the scope of fixing the hive native
> > reader,
> > > or
> > > > > which change since the 1.4 release broke it? Is someone available
> to
> > > look
> > > > > at it today?
> > > > >
> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> > > > > challapallirahul@gmail.com> wrote:
> > > > >
> > > > > > Just found an issue with hive native parquet reader (DRILL-4323).
> > > This
> > > > > is a
> > > > > > regression from 1.4 and in my opinion should be treated as a
> > blocker.
> > > > > >
> > > > > > - Rahul
> > > > > >
> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> > > > > > venki.korukanti@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Sorry I didn't realize I was breaking the public interface API.
> > Let
> > > > me
> > > > > > work
> > > > > > > on a patch to make it backward compatible.
> > > > > > >
> > > > > > > Thanks
> > > > > > > Venki
> > > > > > >
> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <
> > > steven@dremio.com
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I just wanted to bring up an issue that I just now
> discovered,
> > > that
> > > > > has
> > > > > > > > caused me a fair amount of grief.
> > > > > > > >
> > > > > > > > https://github.com/apache/drill/pull/300/commits
> > > > > > > >
> > > > > > > > DRILL-4198 changes a user-facing API, and causes
> StoragePlugins
> > > > that
> > > > > > were
> > > > > > > > compiled against currently released versions of Drill to no
> > > longer
> > > > > > > > functional properly. I would prefer that this breaking change
> > be
> > > > > > modified
> > > > > > > > to be backward compatible if possible.
> > > > > > > >
> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> > > > > > > > altekrusejason@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Aman,
> > > > > > > > >
> > > > > > > > > This is the failure that he was seeing. He figured out that
> > the
> > > > new
> > > > > > > > > exclusions in jdbc-all were not being respected when the
> > build
> > > > was
> > > > > > run
> > > > > > > > with
> > > > > > > > > an older Maven version, causing the jar size to increase
> > > > > > significantly.
> > > > > > > > He
> > > > > > > > > added an enforcer to make sure the JAR didn't grow
> > > unexpectedly.
> > > > > Can
> > > > > > > you
> > > > > > > > > try to update your maven version and re-run the build?
> > > > > > > > >
> > > > > > > > > - Jason
> > > > > > > > >
> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
> > > > asinha@maprtech.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Jacques, I am getting the following build failure on the
> > > latest
> > > > > > > master
> > > > > > > > > > branch...is this what you saw for the Apache build ?  My
> > mvn
> > > > > > version
> > > > > > > > > output
> > > > > > > > > > is shown below.  Should we all be upgrading to a newer
> mvn
> > ?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > > > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> > > > > > > > > > [WARNING] Rule 0:
> > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
> > > > > > > > > failed
> > > > > > > > > > with message:
> > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the
> > > > > expected
> > > > > > > size
> > > > > > > > > > range.
> > > > > > > > > >
> > > > > > > > > >                   This is likely due to you adding new
> > > > > dependencies
> > > > > > > to
> > > > > > > > a
> > > > > > > > > > java-exec and not updating the excludes in this module.
> > This
> > > is
> > > > > > > > important
> > > > > > > > > > as it minimizes the size of the dependency of Drill
> > > application
> > > > > > > users.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > > > > > > size (44664121) too large. Max. is
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill asinha$
> mvn
> > > > > > --version
> > > > > > > > > > Apache Maven 3.0.5
> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > > > > > > > 2013-02-19
> > > > > > > > > > 05:51:28-0800)
> > > > > > > > > > Maven home: /opt/local/share/java/maven3
> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle Corporation
> > > > > > > > > > Java home:
> > > > > > > > > >
> > > > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > > > > > > > > > Default locale: en_US, platform encoding: UTF-8
> > > > > > > > > > OS name: "mac os x", version: "10.9.5", arch: "x86_64",
> > > family:
> > > > > > "mac"
> > > > > > > > > >
> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <
> > > > > > jacques@dremio.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Build back to normal. It looks like the Apache server
> was
> > > > using
> > > > > > an
> > > > > > > > old
> > > > > > > > > > > version of Maven. Once I switched to something more
> > recent,
> > > > the
> > > > > > > build
> > > > > > > > > > > passed.
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Jacques Nadeau
> > > > > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <
> > > > > > > jacques@dremio.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hmm... this merge caused the Apache build to fail.
> > > > > > > Investigating...
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Jacques Nadeau
> > > > > > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <
> > > > > > > > jacques@dremio.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > >> I got clean regression runs as well. I've merged the
> > > > patch.
> > > > > > > > > > > >>
> > > > > > > > > > > >> Jason, you want to start the release process?
> > > > > > > > > > > >>
> > > > > > > > > > > >> --
> > > > > > > > > > > >> Jacques Nadeau
> > > > > > > > > > > >> CTO and Co-Founder, Dremio
> > > > > > > > > > > >>
> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <
> > > > > > > > agirish@mapr.com
> > > > > > > > > >
> > > > > > > > > > > >> wrote:
> > > > > > > > > > > >>
> > > > > > > > > > > >>> Had two clean Functional runs. TPC-H SF100 was also
> > > > > > successful.
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul
> challapalli <
> > > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> > Kicked off a functional run with your branch.
> Will
> > > let
> > > > > you
> > > > > > > know
> > > > > > > > > > once
> > > > > > > > > > > it
> > > > > > > > > > > >>> > finishes
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>> > - Rahul
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <
> > > > > > > > > > jacques@dremio.com>
> > > > > > > > > > > >>> > wrote:
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>> > > 4196 was merged today. I have an updated patch
> > for
> > > > 4291
> > > > > > > that
> > > > > > > > is
> > > > > > > > > > > >>> ready.
> > > > > > > > > > > >>> > > Unfortunately, it seems that something isn't
> > > working
> > > > > with
> > > > > > > our
> > > > > > > > > > > >>> extended
> > > > > > > > > > > >>> > > tests so I haven't been able to run an extended
> > > > > > regression.
> > > > > > > > > Unit
> > > > > > > > > > > >>> tests
> > > > > > > > > > > >>> > > pass. Is someone else possibly able to run a
> > > > regression
> > > > > > > suite
> > > > > > > > > > > against
> > > > > > > > > > > >>> > this
> > > > > > > > > > > >>> > > branch [1] so we can confirm things look good
> and
> > > > start
> > > > > > the
> > > > > > > > > > release
> > > > > > > > > > > >>> > > process?
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > thanks,
> > > > > > > > > > > >>> > > Jacques
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > [1]
> > > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > --
> > > > > > > > > > > >>> > > Jacques Nadeau
> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques
> Nadeau
> > <
> > > > > > > > > > > jacques@dremio.com
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>> > > wrote:
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > > I think the main things are 4196 and 4291
> > should
> > > be
> > > > > > > > > completed.
> > > > > > > > > > I
> > > > > > > > > > > >>> know
> > > > > > > > > > > >>> > > Amit
> > > > > > > > > > > >>> > > > was able to reproduce 4196 locally this
> weekend
> > > so
> > > > I
> > > > > > > think
> > > > > > > > > > we're
> > > > > > > > > > > >>> close
> > > > > > > > > > > >>> > on
> > > > > > > > > > > >>> > > > that.
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > > --
> > > > > > > > > > > >>> > > > Jacques Nadeau
> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine
> Fong
> > <
> > > > > > > > > > > zfong@maprtech.com
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>> > > wrote:
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > >> Any updates on this?  What's blocking us
> from
> > > > taking
> > > > > > > this
> > > > > > > > > to a
> > > > > > > > > > > >>> vote --
> > > > > > > > > > > >>> > > the
> > > > > > > > > > > >>> > > >> sort merge join issues?
> > > > > > > > > > > >>> > > >>
> > > > > > > > > > > >>> > > >> -- Zelaine
> > > > > > > > > > > >>> > > >>
> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques
> > Nadeau
> > > <
> > > > > > > > > > > >>> jacques@dremio.com>
> > > > > > > > > > > >>> > > >> wrote:
> > > > > > > > > > > >>> > > >>
> > > > > > > > > > > >>> > > >> > Bumping this thread...
> > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > >>> > > >> > Here are the issues that were mentioned in
> > > this
> > > > > > thread
> > > > > > > > > along
> > > > > > > > > > > >>> with a
> > > > > > > > > > > >>> > > >> > proposed categorization:
> > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > >>> > > >> > Release Blockers
> > > > > > > > > > > >>> > > >> > In-progress Amit
> > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4190
> > > > > > > > > > > >>> > > >> > In-progress Amit
> > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4196
> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > > > > > > >>> > >
> https://issues.apache.org/jira/browse/DRILL-4246
> > > > > > > > > > > >>> > > >> > In-review Jinfeng
> > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4256
> > > > > > > > > > > >>> > > >> > In-progress Jacques
> > > > > > > > > > > >>> > https://issues.apache.org/jira/browse/DRILL-4278
> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
> > > > > > > > > > > >>> > >
> https://issues.apache.org/jira/browse/DRILL-4285
> > > > > > > > > > > >>> > > >> > Nice to Have
> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
> > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4247
> > > > > > > > > > > >>> > > >> > In-progress Jason
> > > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4203
> > > > > > > > > > > >>> > > >> > Open Jacques
> > > > > > > > > > https://issues.apache.org/jira/browse/DRILL-4266
> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > > > > > > >>> > >
> https://issues.apache.org/jira/browse/DRILL-4131
> > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > >>> > > >> > What do others think? Let's try to get the
> > > > > blockers
> > > > > > > > > wrapped
> > > > > > > > > > up
> > > > > > > > > > > >>> in
> > > > > > > > > > > >>> > the
> > > > > > > > > > > >>> > > >> next
> > > > > > > > > > > >>> > > >> > day or two and start a release vote...
> > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > >>> > > >> > --
> > > > > > > > > > > >>> > > >> > Jacques Nadeau
> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason
> > > Altekruse
> > > > <
> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
> > > > > > > > > > > >>> > > >> > wrote:
> > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > >>> > > >> > > Hello All,
> > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > >>> > > >> > > With the allocator changes merged and
> > about
> > > a
> > > > > > month
> > > > > > > > > since
> > > > > > > > > > > the
> > > > > > > > > > > >>> last
> > > > > > > > > > > >>> > > >> > release
> > > > > > > > > > > >>> > > >> > > I think it would be good to start a vote
> > > > soon. I
> > > > > > > would
> > > > > > > > > > like
> > > > > > > > > > > to
> > > > > > > > > > > >>> > > >> volunteer
> > > > > > > > > > > >>> > > >> > to
> > > > > > > > > > > >>> > > >> > > be release manager.
> > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > >>> > > >> > > I know that there were some issues that
> > were
> > > > > > > > identified
> > > > > > > > > > > after
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > >> > transfer
> > > > > > > > > > > >>> > > >> > > patch was merged. I think that these
> > issues
> > > > > should
> > > > > > > be
> > > > > > > > > > fixed
> > > > > > > > > > > >>> before
> > > > > > > > > > > >>> > > we
> > > > > > > > > > > >>> > > >> > cut a
> > > > > > > > > > > >>> > > >> > > release candidate.
> > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > >>> > > >> > > From looking at the associated JIRAs it
> > > looked
> > > > > > like
> > > > > > > > > there
> > > > > > > > > > > was
> > > > > > > > > > > >>> a
> > > > > > > > > > > >>> > > >> possible
> > > > > > > > > > > >>> > > >> > > short term fix just adjusting the
> > > > > > > > > > max_query_memory_per_node
> > > > > > > > > > > >>> > option,
> > > > > > > > > > > >>> > > >> and
> > > > > > > > > > > >>> > > >> > > some more involved work to change how we
> > > > > determine
> > > > > > > the
> > > > > > > > > > > correct
> > > > > > > > > > > >>> > time
> > > > > > > > > > > >>> > > to
> > > > > > > > > > > >>> > > >> > > spill during external sort. I believe it
> > > makes
> > > > > > sense
> > > > > > > > to
> > > > > > > > > > make
> > > > > > > > > > > >>> > > external
> > > > > > > > > > > >>> > > >> > sort
> > > > > > > > > > > >>> > > >> > > work well with the newly improved memory
> > > > > > accounting
> > > > > > > > > before
> > > > > > > > > > > >>> > cutting a
> > > > > > > > > > > >>> > > >> > > release, but I'm not sure how much work
> is
> > > > left
> > > > > to
> > > > > > > be
> > > > > > > > > done
> > > > > > > > > > > >>> there.
> > > > > > > > > > > >>> > > [1]
> > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > >>> > > >> > > Please respond with your thoughts on a
> > > release
> > > > > > soon
> > > > > > > > and
> > > > > > > > > > any
> > > > > > > > > > > >>> JIRAs
> > > > > > > > > > > >>> > > you
> > > > > > > > > > > >>> > > >> > would
> > > > > > > > > > > >>> > > >> > > like to include in the release.
> > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > >>> > > >> > > [1] -
> > > > > > > > https://issues.apache.org/jira/browse/DRILL-4243
> > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > >>> > > >> > > Thanks,
> > > > > > > > > > > >>> > > >> > > Jason
> > > > > > > > > > > >>> > > >> > >
> > > > > > > > > > > >>> > > >> >
> > > > > > > > > > > >>> > > >>
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>>
> > > > > > > > > > > >>
> > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Venki Korukanti <ve...@gmail.com>.

Here is the partial fix:
https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b

If the existing StragePlugins are implementing AbstractStoragePlugin class
this fix should resolve the backward compat issue. If they are implementing
StoragePlugin interface directly then it won't resolve the issue. Not sure
how to handle it in that case. Let me know if there any ways to resolve
this issue completely.

Thanks
Venki

On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <as...@maprtech.com> wrote:

> I was able to do the build with maven 3.3.9 (after doing 'brew install
> maven').  Here are my env variables related to maven:
> M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
>
>
> On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <ja...@dremio.com>
> wrote:
>
> > Aman, for reference on the Maven version issue. You can see the output of
> > the shade plugin between the two Apache Builds. If you search for
> > maven-shade-plugin you'll see that build 647 (maven 3.0.5) fails because
> it
> > includes a bunch of hadoop (and other libraries that are listed as being
> > excluded). If you look at build 648 (maven 3.3.3), those items are
> included
> > so that the build completes successfully.
> >
> > https://builds.apache.org/job/drill-scm/647/consoleText
> > https://builds.apache.org/job/drill-scm/648/consoleText
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
> > challapallirahul@gmail.com> wrote:
> >
> > > Jason,
> > >
> > > As of now I haven't narrowed down the exact commit which caused
> > DRILL-4323
> > > and I do not know the scope of the fix.
> > >
> > > - Rahul
> > >
> > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
> > altekrusejason@gmail.com
> > > >
> > > wrote:
> > >
> > > > I think it makes sense to wait for Venki's fix of the API, as
> > releasing a
> > > > version with the change would make reverting/refactoring the API
> > breaking
> > > > change less useful.
> > > >
> > > > Do you have any idea about the scope of fixing the hive native
> reader,
> > or
> > > > which change since the 1.4 release broke it? Is someone available to
> > look
> > > > at it today?
> > > >
> > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> > > > challapallirahul@gmail.com> wrote:
> > > >
> > > > > Just found an issue with hive native parquet reader (DRILL-4323).
> > This
> > > > is a
> > > > > regression from 1.4 and in my opinion should be treated as a
> blocker.
> > > > >
> > > > > - Rahul
> > > > >
> > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> > > > > venki.korukanti@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Sorry I didn't realize I was breaking the public interface API.
> Let
> > > me
> > > > > work
> > > > > > on a patch to make it backward compatible.
> > > > > >
> > > > > > Thanks
> > > > > > Venki
> > > > > >
> > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <
> > steven@dremio.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > I just wanted to bring up an issue that I just now discovered,
> > that
> > > > has
> > > > > > > caused me a fair amount of grief.
> > > > > > >
> > > > > > > https://github.com/apache/drill/pull/300/commits
> > > > > > >
> > > > > > > DRILL-4198 changes a user-facing API, and causes StoragePlugins
> > > that
> > > > > were
> > > > > > > compiled against currently released versions of Drill to no
> > longer
> > > > > > > functional properly. I would prefer that this breaking change
> be
> > > > > modified
> > > > > > > to be backward compatible if possible.
> > > > > > >
> > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> > > > > > > altekrusejason@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Aman,
> > > > > > > >
> > > > > > > > This is the failure that he was seeing. He figured out that
> the
> > > new
> > > > > > > > exclusions in jdbc-all were not being respected when the
> build
> > > was
> > > > > run
> > > > > > > with
> > > > > > > > an older Maven version, causing the jar size to increase
> > > > > significantly.
> > > > > > > He
> > > > > > > > added an enforcer to make sure the JAR didn't grow
> > unexpectedly.
> > > > Can
> > > > > > you
> > > > > > > > try to update your maven version and re-run the build?
> > > > > > > >
> > > > > > > > - Jason
> > > > > > > >
> > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
> > > asinha@maprtech.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Jacques, I am getting the following build failure on the
> > latest
> > > > > > master
> > > > > > > > > branch...is this what you saw for the Apache build ?  My
> mvn
> > > > > version
> > > > > > > > output
> > > > > > > > > is shown below.  Should we all be upgrading to a newer mvn
> ?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> > > > > > > > > [WARNING] Rule 0:
> > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
> > > > > > > > failed
> > > > > > > > > with message:
> > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the
> > > > expected
> > > > > > size
> > > > > > > > > range.
> > > > > > > > >
> > > > > > > > >                   This is likely due to you adding new
> > > > dependencies
> > > > > > to
> > > > > > > a
> > > > > > > > > java-exec and not updating the excludes in this module.
> This
> > is
> > > > > > > important
> > > > > > > > > as it minimizes the size of the dependency of Drill
> > application
> > > > > > users.
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > > > > > size (44664121) too large. Max. is
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill asinha$ mvn
> > > > > --version
> > > > > > > > > Apache Maven 3.0.5
> > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > > > > > > 2013-02-19
> > > > > > > > > 05:51:28-0800)
> > > > > > > > > Maven home: /opt/local/share/java/maven3
> > > > > > > > > Java version: 1.7.0_45, vendor: Oracle Corporation
> > > > > > > > > Java home:
> > > > > > > > >
> > > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > > > > > > > > Default locale: en_US, platform encoding: UTF-8
> > > > > > > > > OS name: "mac os x", version: "10.9.5", arch: "x86_64",
> > family:
> > > > > "mac"
> > > > > > > > >
> > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <
> > > > > jacques@dremio.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Build back to normal. It looks like the Apache server was
> > > using
> > > > > an
> > > > > > > old
> > > > > > > > > > version of Maven. Once I switched to something more
> recent,
> > > the
> > > > > > build
> > > > > > > > > > passed.
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Jacques Nadeau
> > > > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > > > >
> > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <
> > > > > > jacques@dremio.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hmm... this merge caused the Apache build to fail.
> > > > > > Investigating...
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Jacques Nadeau
> > > > > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <
> > > > > > > jacques@dremio.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > >> I got clean regression runs as well. I've merged the
> > > patch.
> > > > > > > > > > >>
> > > > > > > > > > >> Jason, you want to start the release process?
> > > > > > > > > > >>
> > > > > > > > > > >> --
> > > > > > > > > > >> Jacques Nadeau
> > > > > > > > > > >> CTO and Co-Founder, Dremio
> > > > > > > > > > >>
> > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <
> > > > > > > agirish@mapr.com
> > > > > > > > >
> > > > > > > > > > >> wrote:
> > > > > > > > > > >>
> > > > > > > > > > >>> Had two clean Functional runs. TPC-H SF100 was also
> > > > > successful.
> > > > > > > > > > >>>
> > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
> > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
> > > > > > > > > > >>>
> > > > > > > > > > >>> > Kicked off a functional run with your branch. Will
> > let
> > > > you
> > > > > > know
> > > > > > > > > once
> > > > > > > > > > it
> > > > > > > > > > >>> > finishes
> > > > > > > > > > >>> >
> > > > > > > > > > >>> > - Rahul
> > > > > > > > > > >>> >
> > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <
> > > > > > > > > jacques@dremio.com>
> > > > > > > > > > >>> > wrote:
> > > > > > > > > > >>> >
> > > > > > > > > > >>> > > 4196 was merged today. I have an updated patch
> for
> > > 4291
> > > > > > that
> > > > > > > is
> > > > > > > > > > >>> ready.
> > > > > > > > > > >>> > > Unfortunately, it seems that something isn't
> > working
> > > > with
> > > > > > our
> > > > > > > > > > >>> extended
> > > > > > > > > > >>> > > tests so I haven't been able to run an extended
> > > > > regression.
> > > > > > > > Unit
> > > > > > > > > > >>> tests
> > > > > > > > > > >>> > > pass. Is someone else possibly able to run a
> > > regression
> > > > > > suite
> > > > > > > > > > against
> > > > > > > > > > >>> > this
> > > > > > > > > > >>> > > branch [1] so we can confirm things look good and
> > > start
> > > > > the
> > > > > > > > > release
> > > > > > > > > > >>> > > process?
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > thanks,
> > > > > > > > > > >>> > > Jacques
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > [1]
> > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > --
> > > > > > > > > > >>> > > Jacques Nadeau
> > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau
> <
> > > > > > > > > > jacques@dremio.com
> > > > > > > > > > >>> >
> > > > > > > > > > >>> > > wrote:
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > > I think the main things are 4196 and 4291
> should
> > be
> > > > > > > > completed.
> > > > > > > > > I
> > > > > > > > > > >>> know
> > > > > > > > > > >>> > > Amit
> > > > > > > > > > >>> > > > was able to reproduce 4196 locally this weekend
> > so
> > > I
> > > > > > think
> > > > > > > > > we're
> > > > > > > > > > >>> close
> > > > > > > > > > >>> > on
> > > > > > > > > > >>> > > > that.
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > > --
> > > > > > > > > > >>> > > > Jacques Nadeau
> > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong
> <
> > > > > > > > > > zfong@maprtech.com
> > > > > > > > > > >>> >
> > > > > > > > > > >>> > > wrote:
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > >> Any updates on this?  What's blocking us from
> > > taking
> > > > > > this
> > > > > > > > to a
> > > > > > > > > > >>> vote --
> > > > > > > > > > >>> > > the
> > > > > > > > > > >>> > > >> sort merge join issues?
> > > > > > > > > > >>> > > >>
> > > > > > > > > > >>> > > >> -- Zelaine
> > > > > > > > > > >>> > > >>
> > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques
> Nadeau
> > <
> > > > > > > > > > >>> jacques@dremio.com>
> > > > > > > > > > >>> > > >> wrote:
> > > > > > > > > > >>> > > >>
> > > > > > > > > > >>> > > >> > Bumping this thread...
> > > > > > > > > > >>> > > >> >
> > > > > > > > > > >>> > > >> > Here are the issues that were mentioned in
> > this
> > > > > thread
> > > > > > > > along
> > > > > > > > > > >>> with a
> > > > > > > > > > >>> > > >> > proposed categorization:
> > > > > > > > > > >>> > > >> >
> > > > > > > > > > >>> > > >> > Release Blockers
> > > > > > > > > > >>> > > >> > In-progress Amit
> > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4190
> > > > > > > > > > >>> > > >> > In-progress Amit
> > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4196
> > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4246
> > > > > > > > > > >>> > > >> > In-review Jinfeng
> > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4256
> > > > > > > > > > >>> > > >> > In-progress Jacques
> > > > > > > > > > >>> > https://issues.apache.org/jira/browse/DRILL-4278
> > > > > > > > > > >>> > > >> > Ready to merge Laurent
> > > > > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4285
> > > > > > > > > > >>> > > >> > Nice to Have
> > > > > > > > > > >>> > > >> > Open Jason/Hakim
> > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4247
> > > > > > > > > > >>> > > >> > In-progress Jason
> > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4203
> > > > > > > > > > >>> > > >> > Open Jacques
> > > > > > > > > https://issues.apache.org/jira/browse/DRILL-4266
> > > > > > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4131
> > > > > > > > > > >>> > > >> >
> > > > > > > > > > >>> > > >> > What do others think? Let's try to get the
> > > > blockers
> > > > > > > > wrapped
> > > > > > > > > up
> > > > > > > > > > >>> in
> > > > > > > > > > >>> > the
> > > > > > > > > > >>> > > >> next
> > > > > > > > > > >>> > > >> > day or two and start a release vote...
> > > > > > > > > > >>> > > >> >
> > > > > > > > > > >>> > > >> >
> > > > > > > > > > >>> > > >> >
> > > > > > > > > > >>> > > >> > --
> > > > > > > > > > >>> > > >> > Jacques Nadeau
> > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> > > > > > > > > > >>> > > >> >
> > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason
> > Altekruse
> > > <
> > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
> > > > > > > > > > >>> > > >> > wrote:
> > > > > > > > > > >>> > > >> >
> > > > > > > > > > >>> > > >> > > Hello All,
> > > > > > > > > > >>> > > >> > >
> > > > > > > > > > >>> > > >> > > With the allocator changes merged and
> about
> > a
> > > > > month
> > > > > > > > since
> > > > > > > > > > the
> > > > > > > > > > >>> last
> > > > > > > > > > >>> > > >> > release
> > > > > > > > > > >>> > > >> > > I think it would be good to start a vote
> > > soon. I
> > > > > > would
> > > > > > > > > like
> > > > > > > > > > to
> > > > > > > > > > >>> > > >> volunteer
> > > > > > > > > > >>> > > >> > to
> > > > > > > > > > >>> > > >> > > be release manager.
> > > > > > > > > > >>> > > >> > >
> > > > > > > > > > >>> > > >> > > I know that there were some issues that
> were
> > > > > > > identified
> > > > > > > > > > after
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > >> > transfer
> > > > > > > > > > >>> > > >> > > patch was merged. I think that these
> issues
> > > > should
> > > > > > be
> > > > > > > > > fixed
> > > > > > > > > > >>> before
> > > > > > > > > > >>> > > we
> > > > > > > > > > >>> > > >> > cut a
> > > > > > > > > > >>> > > >> > > release candidate.
> > > > > > > > > > >>> > > >> > >
> > > > > > > > > > >>> > > >> > > From looking at the associated JIRAs it
> > looked
> > > > > like
> > > > > > > > there
> > > > > > > > > > was
> > > > > > > > > > >>> a
> > > > > > > > > > >>> > > >> possible
> > > > > > > > > > >>> > > >> > > short term fix just adjusting the
> > > > > > > > > max_query_memory_per_node
> > > > > > > > > > >>> > option,
> > > > > > > > > > >>> > > >> and
> > > > > > > > > > >>> > > >> > > some more involved work to change how we
> > > > determine
> > > > > > the
> > > > > > > > > > correct
> > > > > > > > > > >>> > time
> > > > > > > > > > >>> > > to
> > > > > > > > > > >>> > > >> > > spill during external sort. I believe it
> > makes
> > > > > sense
> > > > > > > to
> > > > > > > > > make
> > > > > > > > > > >>> > > external
> > > > > > > > > > >>> > > >> > sort
> > > > > > > > > > >>> > > >> > > work well with the newly improved memory
> > > > > accounting
> > > > > > > > before
> > > > > > > > > > >>> > cutting a
> > > > > > > > > > >>> > > >> > > release, but I'm not sure how much work is
> > > left
> > > > to
> > > > > > be
> > > > > > > > done
> > > > > > > > > > >>> there.
> > > > > > > > > > >>> > > [1]
> > > > > > > > > > >>> > > >> > >
> > > > > > > > > > >>> > > >> > > Please respond with your thoughts on a
> > release
> > > > > soon
> > > > > > > and
> > > > > > > > > any
> > > > > > > > > > >>> JIRAs
> > > > > > > > > > >>> > > you
> > > > > > > > > > >>> > > >> > would
> > > > > > > > > > >>> > > >> > > like to include in the release.
> > > > > > > > > > >>> > > >> > >
> > > > > > > > > > >>> > > >> > > [1] -
> > > > > > > https://issues.apache.org/jira/browse/DRILL-4243
> > > > > > > > > > >>> > > >> > >
> > > > > > > > > > >>> > > >> > > Thanks,
> > > > > > > > > > >>> > > >> > > Jason
> > > > > > > > > > >>> > > >> > >
> > > > > > > > > > >>> > > >> >
> > > > > > > > > > >>> > > >>
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> >
> > > > > > > > > > >>>
> > > > > > > > > > >>
> > > > > > > > > > >>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Aman Sinha <as...@maprtech.com>.

I was able to do the build with maven 3.3.9 (after doing 'brew install
maven').  Here are my env variables related to maven:
M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m


On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <ja...@dremio.com> wrote:

> Aman, for reference on the Maven version issue. You can see the output of
> the shade plugin between the two Apache Builds. If you search for
> maven-shade-plugin you'll see that build 647 (maven 3.0.5) fails because it
> includes a bunch of hadoop (and other libraries that are listed as being
> excluded). If you look at build 648 (maven 3.3.3), those items are included
> so that the build completes successfully.
>
> https://builds.apache.org/job/drill-scm/647/consoleText
> https://builds.apache.org/job/drill-scm/648/consoleText
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
> challapallirahul@gmail.com> wrote:
>
> > Jason,
> >
> > As of now I haven't narrowed down the exact commit which caused
> DRILL-4323
> > and I do not know the scope of the fix.
> >
> > - Rahul
> >
> > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
> altekrusejason@gmail.com
> > >
> > wrote:
> >
> > > I think it makes sense to wait for Venki's fix of the API, as
> releasing a
> > > version with the change would make reverting/refactoring the API
> breaking
> > > change less useful.
> > >
> > > Do you have any idea about the scope of fixing the hive native reader,
> or
> > > which change since the 1.4 release broke it? Is someone available to
> look
> > > at it today?
> > >
> > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> > > challapallirahul@gmail.com> wrote:
> > >
> > > > Just found an issue with hive native parquet reader (DRILL-4323).
> This
> > > is a
> > > > regression from 1.4 and in my opinion should be treated as a blocker.
> > > >
> > > > - Rahul
> > > >
> > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> > > > venki.korukanti@gmail.com>
> > > > wrote:
> > > >
> > > > > Sorry I didn't realize I was breaking the public interface API. Let
> > me
> > > > work
> > > > > on a patch to make it backward compatible.
> > > > >
> > > > > Thanks
> > > > > Venki
> > > > >
> > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <
> steven@dremio.com
> > >
> > > > > wrote:
> > > > >
> > > > > > I just wanted to bring up an issue that I just now discovered,
> that
> > > has
> > > > > > caused me a fair amount of grief.
> > > > > >
> > > > > > https://github.com/apache/drill/pull/300/commits
> > > > > >
> > > > > > DRILL-4198 changes a user-facing API, and causes StoragePlugins
> > that
> > > > were
> > > > > > compiled against currently released versions of Drill to no
> longer
> > > > > > functional properly. I would prefer that this breaking change be
> > > > modified
> > > > > > to be backward compatible if possible.
> > > > > >
> > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> > > > > > altekrusejason@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Aman,
> > > > > > >
> > > > > > > This is the failure that he was seeing. He figured out that the
> > new
> > > > > > > exclusions in jdbc-all were not being respected when the build
> > was
> > > > run
> > > > > > with
> > > > > > > an older Maven version, causing the jar size to increase
> > > > significantly.
> > > > > > He
> > > > > > > added an enforcer to make sure the JAR didn't grow
> unexpectedly.
> > > Can
> > > > > you
> > > > > > > try to update your maven version and re-run the build?
> > > > > > >
> > > > > > > - Jason
> > > > > > >
> > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
> > asinha@maprtech.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Jacques, I am getting the following build failure on the
> latest
> > > > > master
> > > > > > > > branch...is this what you saw for the Apache build ?  My mvn
> > > > version
> > > > > > > output
> > > > > > > > is shown below.  Should we all be upgrading to a newer mvn ?
> > > > > > > >
> > > > > > > >
> > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> > > > > > > > [WARNING] Rule 0:
> > > > org.apache.maven.plugins.enforcer.RequireFilesSize
> > > > > > > failed
> > > > > > > > with message:
> > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the
> > > expected
> > > > > size
> > > > > > > > range.
> > > > > > > >
> > > > > > > >                   This is likely due to you adding new
> > > dependencies
> > > > > to
> > > > > > a
> > > > > > > > java-exec and not updating the excludes in this module. This
> is
> > > > > > important
> > > > > > > > as it minimizes the size of the dependency of Drill
> application
> > > > > users.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > > > > size (44664121) too large. Max. is
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > > > >
> > > > > > > >
> > > > > > > > Administrators-MacBook-Pro-144:incubator-drill asinha$ mvn
> > > > --version
> > > > > > > > Apache Maven 3.0.5
> (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > > > > > 2013-02-19
> > > > > > > > 05:51:28-0800)
> > > > > > > > Maven home: /opt/local/share/java/maven3
> > > > > > > > Java version: 1.7.0_45, vendor: Oracle Corporation
> > > > > > > > Java home:
> > > > > > > >
> > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > > > > > > > Default locale: en_US, platform encoding: UTF-8
> > > > > > > > OS name: "mac os x", version: "10.9.5", arch: "x86_64",
> family:
> > > > "mac"
> > > > > > > >
> > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <
> > > > jacques@dremio.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Build back to normal. It looks like the Apache server was
> > using
> > > > an
> > > > > > old
> > > > > > > > > version of Maven. Once I switched to something more recent,
> > the
> > > > > build
> > > > > > > > > passed.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Jacques Nadeau
> > > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > > >
> > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <
> > > > > jacques@dremio.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hmm... this merge caused the Apache build to fail.
> > > > > Investigating...
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Jacques Nadeau
> > > > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > > > >
> > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <
> > > > > > jacques@dremio.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > >> I got clean regression runs as well. I've merged the
> > patch.
> > > > > > > > > >>
> > > > > > > > > >> Jason, you want to start the release process?
> > > > > > > > > >>
> > > > > > > > > >> --
> > > > > > > > > >> Jacques Nadeau
> > > > > > > > > >> CTO and Co-Founder, Dremio
> > > > > > > > > >>
> > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <
> > > > > > agirish@mapr.com
> > > > > > > >
> > > > > > > > > >> wrote:
> > > > > > > > > >>
> > > > > > > > > >>> Had two clean Functional runs. TPC-H SF100 was also
> > > > successful.
> > > > > > > > > >>>
> > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
> > > > > > > > > >>> challapallirahul@gmail.com> wrote:
> > > > > > > > > >>>
> > > > > > > > > >>> > Kicked off a functional run with your branch. Will
> let
> > > you
> > > > > know
> > > > > > > > once
> > > > > > > > > it
> > > > > > > > > >>> > finishes
> > > > > > > > > >>> >
> > > > > > > > > >>> > - Rahul
> > > > > > > > > >>> >
> > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <
> > > > > > > > jacques@dremio.com>
> > > > > > > > > >>> > wrote:
> > > > > > > > > >>> >
> > > > > > > > > >>> > > 4196 was merged today. I have an updated patch for
> > 4291
> > > > > that
> > > > > > is
> > > > > > > > > >>> ready.
> > > > > > > > > >>> > > Unfortunately, it seems that something isn't
> working
> > > with
> > > > > our
> > > > > > > > > >>> extended
> > > > > > > > > >>> > > tests so I haven't been able to run an extended
> > > > regression.
> > > > > > > Unit
> > > > > > > > > >>> tests
> > > > > > > > > >>> > > pass. Is someone else possibly able to run a
> > regression
> > > > > suite
> > > > > > > > > against
> > > > > > > > > >>> > this
> > > > > > > > > >>> > > branch [1] so we can confirm things look good and
> > start
> > > > the
> > > > > > > > release
> > > > > > > > > >>> > > process?
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > thanks,
> > > > > > > > > >>> > > Jacques
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > [1]
> > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > --
> > > > > > > > > >>> > > Jacques Nadeau
> > > > > > > > > >>> > > CTO and Co-Founder, Dremio
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <
> > > > > > > > > jacques@dremio.com
> > > > > > > > > >>> >
> > > > > > > > > >>> > > wrote:
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > > I think the main things are 4196 and 4291 should
> be
> > > > > > > completed.
> > > > > > > > I
> > > > > > > > > >>> know
> > > > > > > > > >>> > > Amit
> > > > > > > > > >>> > > > was able to reproduce 4196 locally this weekend
> so
> > I
> > > > > think
> > > > > > > > we're
> > > > > > > > > >>> close
> > > > > > > > > >>> > on
> > > > > > > > > >>> > > > that.
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > > --
> > > > > > > > > >>> > > > Jacques Nadeau
> > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <
> > > > > > > > > zfong@maprtech.com
> > > > > > > > > >>> >
> > > > > > > > > >>> > > wrote:
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > >> Any updates on this?  What's blocking us from
> > taking
> > > > > this
> > > > > > > to a
> > > > > > > > > >>> vote --
> > > > > > > > > >>> > > the
> > > > > > > > > >>> > > >> sort merge join issues?
> > > > > > > > > >>> > > >>
> > > > > > > > > >>> > > >> -- Zelaine
> > > > > > > > > >>> > > >>
> > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau
> <
> > > > > > > > > >>> jacques@dremio.com>
> > > > > > > > > >>> > > >> wrote:
> > > > > > > > > >>> > > >>
> > > > > > > > > >>> > > >> > Bumping this thread...
> > > > > > > > > >>> > > >> >
> > > > > > > > > >>> > > >> > Here are the issues that were mentioned in
> this
> > > > thread
> > > > > > > along
> > > > > > > > > >>> with a
> > > > > > > > > >>> > > >> > proposed categorization:
> > > > > > > > > >>> > > >> >
> > > > > > > > > >>> > > >> > Release Blockers
> > > > > > > > > >>> > > >> > In-progress Amit
> > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4190
> > > > > > > > > >>> > > >> > In-progress Amit
> > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4196
> > > > > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4246
> > > > > > > > > >>> > > >> > In-review Jinfeng
> > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4256
> > > > > > > > > >>> > > >> > In-progress Jacques
> > > > > > > > > >>> > https://issues.apache.org/jira/browse/DRILL-4278
> > > > > > > > > >>> > > >> > Ready to merge Laurent
> > > > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4285
> > > > > > > > > >>> > > >> > Nice to Have
> > > > > > > > > >>> > > >> > Open Jason/Hakim
> > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4247
> > > > > > > > > >>> > > >> > In-progress Jason
> > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4203
> > > > > > > > > >>> > > >> > Open Jacques
> > > > > > > > https://issues.apache.org/jira/browse/DRILL-4266
> > > > > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4131
> > > > > > > > > >>> > > >> >
> > > > > > > > > >>> > > >> > What do others think? Let's try to get the
> > > blockers
> > > > > > > wrapped
> > > > > > > > up
> > > > > > > > > >>> in
> > > > > > > > > >>> > the
> > > > > > > > > >>> > > >> next
> > > > > > > > > >>> > > >> > day or two and start a release vote...
> > > > > > > > > >>> > > >> >
> > > > > > > > > >>> > > >> >
> > > > > > > > > >>> > > >> >
> > > > > > > > > >>> > > >> > --
> > > > > > > > > >>> > > >> > Jacques Nadeau
> > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> > > > > > > > > >>> > > >> >
> > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason
> Altekruse
> > <
> > > > > > > > > >>> > > >> altekrusejason@gmail.com>
> > > > > > > > > >>> > > >> > wrote:
> > > > > > > > > >>> > > >> >
> > > > > > > > > >>> > > >> > > Hello All,
> > > > > > > > > >>> > > >> > >
> > > > > > > > > >>> > > >> > > With the allocator changes merged and about
> a
> > > > month
> > > > > > > since
> > > > > > > > > the
> > > > > > > > > >>> last
> > > > > > > > > >>> > > >> > release
> > > > > > > > > >>> > > >> > > I think it would be good to start a vote
> > soon. I
> > > > > would
> > > > > > > > like
> > > > > > > > > to
> > > > > > > > > >>> > > >> volunteer
> > > > > > > > > >>> > > >> > to
> > > > > > > > > >>> > > >> > > be release manager.
> > > > > > > > > >>> > > >> > >
> > > > > > > > > >>> > > >> > > I know that there were some issues that were
> > > > > > identified
> > > > > > > > > after
> > > > > > > > > >>> the
> > > > > > > > > >>> > > >> > transfer
> > > > > > > > > >>> > > >> > > patch was merged. I think that these issues
> > > should
> > > > > be
> > > > > > > > fixed
> > > > > > > > > >>> before
> > > > > > > > > >>> > > we
> > > > > > > > > >>> > > >> > cut a
> > > > > > > > > >>> > > >> > > release candidate.
> > > > > > > > > >>> > > >> > >
> > > > > > > > > >>> > > >> > > From looking at the associated JIRAs it
> looked
> > > > like
> > > > > > > there
> > > > > > > > > was
> > > > > > > > > >>> a
> > > > > > > > > >>> > > >> possible
> > > > > > > > > >>> > > >> > > short term fix just adjusting the
> > > > > > > > max_query_memory_per_node
> > > > > > > > > >>> > option,
> > > > > > > > > >>> > > >> and
> > > > > > > > > >>> > > >> > > some more involved work to change how we
> > > determine
> > > > > the
> > > > > > > > > correct
> > > > > > > > > >>> > time
> > > > > > > > > >>> > > to
> > > > > > > > > >>> > > >> > > spill during external sort. I believe it
> makes
> > > > sense
> > > > > > to
> > > > > > > > make
> > > > > > > > > >>> > > external
> > > > > > > > > >>> > > >> > sort
> > > > > > > > > >>> > > >> > > work well with the newly improved memory
> > > > accounting
> > > > > > > before
> > > > > > > > > >>> > cutting a
> > > > > > > > > >>> > > >> > > release, but I'm not sure how much work is
> > left
> > > to
> > > > > be
> > > > > > > done
> > > > > > > > > >>> there.
> > > > > > > > > >>> > > [1]
> > > > > > > > > >>> > > >> > >
> > > > > > > > > >>> > > >> > > Please respond with your thoughts on a
> release
> > > > soon
> > > > > > and
> > > > > > > > any
> > > > > > > > > >>> JIRAs
> > > > > > > > > >>> > > you
> > > > > > > > > >>> > > >> > would
> > > > > > > > > >>> > > >> > > like to include in the release.
> > > > > > > > > >>> > > >> > >
> > > > > > > > > >>> > > >> > > [1] -
> > > > > > https://issues.apache.org/jira/browse/DRILL-4243
> > > > > > > > > >>> > > >> > >
> > > > > > > > > >>> > > >> > > Thanks,
> > > > > > > > > >>> > > >> > > Jason
> > > > > > > > > >>> > > >> > >
> > > > > > > > > >>> > > >> >
> > > > > > > > > >>> > > >>
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > >
> > > > > > > > > >>> >
> > > > > > > > > >>>
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Jacques Nadeau <ja...@dremio.com>.

Aman, for reference on the Maven version issue. You can see the output of
the shade plugin between the two Apache Builds. If you search for
maven-shade-plugin you'll see that build 647 (maven 3.0.5) fails because it
includes a bunch of hadoop (and other libraries that are listed as being
excluded). If you look at build 648 (maven 3.3.3), those items are included
so that the build completes successfully.

https://builds.apache.org/job/drill-scm/647/consoleText
https://builds.apache.org/job/drill-scm/648/consoleText

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
challapallirahul@gmail.com> wrote:

> Jason,
>
> As of now I haven't narrowed down the exact commit which caused DRILL-4323
> and I do not know the scope of the fix.
>
> - Rahul
>
> On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <altekrusejason@gmail.com
> >
> wrote:
>
> > I think it makes sense to wait for Venki's fix of the API, as releasing a
> > version with the change would make reverting/refactoring the API breaking
> > change less useful.
> >
> > Do you have any idea about the scope of fixing the hive native reader, or
> > which change since the 1.4 release broke it? Is someone available to look
> > at it today?
> >
> > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> > challapallirahul@gmail.com> wrote:
> >
> > > Just found an issue with hive native parquet reader (DRILL-4323). This
> > is a
> > > regression from 1.4 and in my opinion should be treated as a blocker.
> > >
> > > - Rahul
> > >
> > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> > > venki.korukanti@gmail.com>
> > > wrote:
> > >
> > > > Sorry I didn't realize I was breaking the public interface API. Let
> me
> > > work
> > > > on a patch to make it backward compatible.
> > > >
> > > > Thanks
> > > > Venki
> > > >
> > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <steven@dremio.com
> >
> > > > wrote:
> > > >
> > > > > I just wanted to bring up an issue that I just now discovered, that
> > has
> > > > > caused me a fair amount of grief.
> > > > >
> > > > > https://github.com/apache/drill/pull/300/commits
> > > > >
> > > > > DRILL-4198 changes a user-facing API, and causes StoragePlugins
> that
> > > were
> > > > > compiled against currently released versions of Drill to no longer
> > > > > functional properly. I would prefer that this breaking change be
> > > modified
> > > > > to be backward compatible if possible.
> > > > >
> > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> > > > > altekrusejason@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Aman,
> > > > > >
> > > > > > This is the failure that he was seeing. He figured out that the
> new
> > > > > > exclusions in jdbc-all were not being respected when the build
> was
> > > run
> > > > > with
> > > > > > an older Maven version, causing the jar size to increase
> > > significantly.
> > > > > He
> > > > > > added an enforcer to make sure the JAR didn't grow unexpectedly.
> > Can
> > > > you
> > > > > > try to update your maven version and re-run the build?
> > > > > >
> > > > > > - Jason
> > > > > >
> > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
> asinha@maprtech.com>
> > > > > wrote:
> > > > > >
> > > > > > > Jacques, I am getting the following build failure on the latest
> > > > master
> > > > > > > branch...is this what you saw for the Apache build ?  My mvn
> > > version
> > > > > > output
> > > > > > > is shown below.  Should we all be upgrading to a newer mvn ?
> > > > > > >
> > > > > > >
> > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> > > > > > > [WARNING] Rule 0:
> > > org.apache.maven.plugins.enforcer.RequireFilesSize
> > > > > > failed
> > > > > > > with message:
> > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the
> > expected
> > > > size
> > > > > > > range.
> > > > > > >
> > > > > > >                   This is likely due to you adding new
> > dependencies
> > > > to
> > > > > a
> > > > > > > java-exec and not updating the excludes in this module. This is
> > > > > important
> > > > > > > as it minimizes the size of the dependency of Drill application
> > > > users.
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > > > size (44664121) too large. Max. is
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > > >
> > > > > > >
> > > > > > > Administrators-MacBook-Pro-144:incubator-drill asinha$ mvn
> > > --version
> > > > > > > Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > > > > 2013-02-19
> > > > > > > 05:51:28-0800)
> > > > > > > Maven home: /opt/local/share/java/maven3
> > > > > > > Java version: 1.7.0_45, vendor: Oracle Corporation
> > > > > > > Java home:
> > > > > > >
> > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > > > > > > Default locale: en_US, platform encoding: UTF-8
> > > > > > > OS name: "mac os x", version: "10.9.5", arch: "x86_64", family:
> > > "mac"
> > > > > > >
> > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <
> > > jacques@dremio.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Build back to normal. It looks like the Apache server was
> using
> > > an
> > > > > old
> > > > > > > > version of Maven. Once I switched to something more recent,
> the
> > > > build
> > > > > > > > passed.
> > > > > > > >
> > > > > > > > --
> > > > > > > > Jacques Nadeau
> > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > >
> > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <
> > > > jacques@dremio.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hmm... this merge caused the Apache build to fail.
> > > > Investigating...
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Jacques Nadeau
> > > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > > >
> > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <
> > > > > jacques@dremio.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> I got clean regression runs as well. I've merged the
> patch.
> > > > > > > > >>
> > > > > > > > >> Jason, you want to start the release process?
> > > > > > > > >>
> > > > > > > > >> --
> > > > > > > > >> Jacques Nadeau
> > > > > > > > >> CTO and Co-Founder, Dremio
> > > > > > > > >>
> > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <
> > > > > agirish@mapr.com
> > > > > > >
> > > > > > > > >> wrote:
> > > > > > > > >>
> > > > > > > > >>> Had two clean Functional runs. TPC-H SF100 was also
> > > successful.
> > > > > > > > >>>
> > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
> > > > > > > > >>> challapallirahul@gmail.com> wrote:
> > > > > > > > >>>
> > > > > > > > >>> > Kicked off a functional run with your branch. Will let
> > you
> > > > know
> > > > > > > once
> > > > > > > > it
> > > > > > > > >>> > finishes
> > > > > > > > >>> >
> > > > > > > > >>> > - Rahul
> > > > > > > > >>> >
> > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <
> > > > > > > jacques@dremio.com>
> > > > > > > > >>> > wrote:
> > > > > > > > >>> >
> > > > > > > > >>> > > 4196 was merged today. I have an updated patch for
> 4291
> > > > that
> > > > > is
> > > > > > > > >>> ready.
> > > > > > > > >>> > > Unfortunately, it seems that something isn't working
> > with
> > > > our
> > > > > > > > >>> extended
> > > > > > > > >>> > > tests so I haven't been able to run an extended
> > > regression.
> > > > > > Unit
> > > > > > > > >>> tests
> > > > > > > > >>> > > pass. Is someone else possibly able to run a
> regression
> > > > suite
> > > > > > > > against
> > > > > > > > >>> > this
> > > > > > > > >>> > > branch [1] so we can confirm things look good and
> start
> > > the
> > > > > > > release
> > > > > > > > >>> > > process?
> > > > > > > > >>> > >
> > > > > > > > >>> > > thanks,
> > > > > > > > >>> > > Jacques
> > > > > > > > >>> > >
> > > > > > > > >>> > > [1]
> > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > > > > > > >>> > >
> > > > > > > > >>> > > --
> > > > > > > > >>> > > Jacques Nadeau
> > > > > > > > >>> > > CTO and Co-Founder, Dremio
> > > > > > > > >>> > >
> > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <
> > > > > > > > jacques@dremio.com
> > > > > > > > >>> >
> > > > > > > > >>> > > wrote:
> > > > > > > > >>> > >
> > > > > > > > >>> > > > I think the main things are 4196 and 4291 should be
> > > > > > completed.
> > > > > > > I
> > > > > > > > >>> know
> > > > > > > > >>> > > Amit
> > > > > > > > >>> > > > was able to reproduce 4196 locally this weekend so
> I
> > > > think
> > > > > > > we're
> > > > > > > > >>> close
> > > > > > > > >>> > on
> > > > > > > > >>> > > > that.
> > > > > > > > >>> > > >
> > > > > > > > >>> > > > --
> > > > > > > > >>> > > > Jacques Nadeau
> > > > > > > > >>> > > > CTO and Co-Founder, Dremio
> > > > > > > > >>> > > >
> > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <
> > > > > > > > zfong@maprtech.com
> > > > > > > > >>> >
> > > > > > > > >>> > > wrote:
> > > > > > > > >>> > > >
> > > > > > > > >>> > > >> Any updates on this?  What's blocking us from
> taking
> > > > this
> > > > > > to a
> > > > > > > > >>> vote --
> > > > > > > > >>> > > the
> > > > > > > > >>> > > >> sort merge join issues?
> > > > > > > > >>> > > >>
> > > > > > > > >>> > > >> -- Zelaine
> > > > > > > > >>> > > >>
> > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <
> > > > > > > > >>> jacques@dremio.com>
> > > > > > > > >>> > > >> wrote:
> > > > > > > > >>> > > >>
> > > > > > > > >>> > > >> > Bumping this thread...
> > > > > > > > >>> > > >> >
> > > > > > > > >>> > > >> > Here are the issues that were mentioned in this
> > > thread
> > > > > > along
> > > > > > > > >>> with a
> > > > > > > > >>> > > >> > proposed categorization:
> > > > > > > > >>> > > >> >
> > > > > > > > >>> > > >> > Release Blockers
> > > > > > > > >>> > > >> > In-progress Amit
> > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4190
> > > > > > > > >>> > > >> > In-progress Amit
> > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4196
> > > > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4246
> > > > > > > > >>> > > >> > In-review Jinfeng
> > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4256
> > > > > > > > >>> > > >> > In-progress Jacques
> > > > > > > > >>> > https://issues.apache.org/jira/browse/DRILL-4278
> > > > > > > > >>> > > >> > Ready to merge Laurent
> > > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4285
> > > > > > > > >>> > > >> > Nice to Have
> > > > > > > > >>> > > >> > Open Jason/Hakim
> > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4247
> > > > > > > > >>> > > >> > In-progress Jason
> > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4203
> > > > > > > > >>> > > >> > Open Jacques
> > > > > > > https://issues.apache.org/jira/browse/DRILL-4266
> > > > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4131
> > > > > > > > >>> > > >> >
> > > > > > > > >>> > > >> > What do others think? Let's try to get the
> > blockers
> > > > > > wrapped
> > > > > > > up
> > > > > > > > >>> in
> > > > > > > > >>> > the
> > > > > > > > >>> > > >> next
> > > > > > > > >>> > > >> > day or two and start a release vote...
> > > > > > > > >>> > > >> >
> > > > > > > > >>> > > >> >
> > > > > > > > >>> > > >> >
> > > > > > > > >>> > > >> > --
> > > > > > > > >>> > > >> > Jacques Nadeau
> > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> > > > > > > > >>> > > >> >
> > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse
> <
> > > > > > > > >>> > > >> altekrusejason@gmail.com>
> > > > > > > > >>> > > >> > wrote:
> > > > > > > > >>> > > >> >
> > > > > > > > >>> > > >> > > Hello All,
> > > > > > > > >>> > > >> > >
> > > > > > > > >>> > > >> > > With the allocator changes merged and about a
> > > month
> > > > > > since
> > > > > > > > the
> > > > > > > > >>> last
> > > > > > > > >>> > > >> > release
> > > > > > > > >>> > > >> > > I think it would be good to start a vote
> soon. I
> > > > would
> > > > > > > like
> > > > > > > > to
> > > > > > > > >>> > > >> volunteer
> > > > > > > > >>> > > >> > to
> > > > > > > > >>> > > >> > > be release manager.
> > > > > > > > >>> > > >> > >
> > > > > > > > >>> > > >> > > I know that there were some issues that were
> > > > > identified
> > > > > > > > after
> > > > > > > > >>> the
> > > > > > > > >>> > > >> > transfer
> > > > > > > > >>> > > >> > > patch was merged. I think that these issues
> > should
> > > > be
> > > > > > > fixed
> > > > > > > > >>> before
> > > > > > > > >>> > > we
> > > > > > > > >>> > > >> > cut a
> > > > > > > > >>> > > >> > > release candidate.
> > > > > > > > >>> > > >> > >
> > > > > > > > >>> > > >> > > From looking at the associated JIRAs it looked
> > > like
> > > > > > there
> > > > > > > > was
> > > > > > > > >>> a
> > > > > > > > >>> > > >> possible
> > > > > > > > >>> > > >> > > short term fix just adjusting the
> > > > > > > max_query_memory_per_node
> > > > > > > > >>> > option,
> > > > > > > > >>> > > >> and
> > > > > > > > >>> > > >> > > some more involved work to change how we
> > determine
> > > > the
> > > > > > > > correct
> > > > > > > > >>> > time
> > > > > > > > >>> > > to
> > > > > > > > >>> > > >> > > spill during external sort. I believe it makes
> > > sense
> > > > > to
> > > > > > > make
> > > > > > > > >>> > > external
> > > > > > > > >>> > > >> > sort
> > > > > > > > >>> > > >> > > work well with the newly improved memory
> > > accounting
> > > > > > before
> > > > > > > > >>> > cutting a
> > > > > > > > >>> > > >> > > release, but I'm not sure how much work is
> left
> > to
> > > > be
> > > > > > done
> > > > > > > > >>> there.
> > > > > > > > >>> > > [1]
> > > > > > > > >>> > > >> > >
> > > > > > > > >>> > > >> > > Please respond with your thoughts on a release
> > > soon
> > > > > and
> > > > > > > any
> > > > > > > > >>> JIRAs
> > > > > > > > >>> > > you
> > > > > > > > >>> > > >> > would
> > > > > > > > >>> > > >> > > like to include in the release.
> > > > > > > > >>> > > >> > >
> > > > > > > > >>> > > >> > > [1] -
> > > > > https://issues.apache.org/jira/browse/DRILL-4243
> > > > > > > > >>> > > >> > >
> > > > > > > > >>> > > >> > > Thanks,
> > > > > > > > >>> > > >> > > Jason
> > > > > > > > >>> > > >> > >
> > > > > > > > >>> > > >> >
> > > > > > > > >>> > > >>
> > > > > > > > >>> > > >
> > > > > > > > >>> > > >
> > > > > > > > >>> > >
> > > > > > > > >>> >
> > > > > > > > >>>
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by rahul challapalli <ch...@gmail.com>.

Jason,

As of now I haven't narrowed down the exact commit which caused DRILL-4323
and I do not know the scope of the fix.

- Rahul

On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <al...@gmail.com>
wrote:

> I think it makes sense to wait for Venki's fix of the API, as releasing a
> version with the change would make reverting/refactoring the API breaking
> change less useful.
>
> Do you have any idea about the scope of fixing the hive native reader, or
> which change since the 1.4 release broke it? Is someone available to look
> at it today?
>
> On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> challapallirahul@gmail.com> wrote:
>
> > Just found an issue with hive native parquet reader (DRILL-4323). This
> is a
> > regression from 1.4 and in my opinion should be treated as a blocker.
> >
> > - Rahul
> >
> > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> > venki.korukanti@gmail.com>
> > wrote:
> >
> > > Sorry I didn't realize I was breaking the public interface API. Let me
> > work
> > > on a patch to make it backward compatible.
> > >
> > > Thanks
> > > Venki
> > >
> > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <st...@dremio.com>
> > > wrote:
> > >
> > > > I just wanted to bring up an issue that I just now discovered, that
> has
> > > > caused me a fair amount of grief.
> > > >
> > > > https://github.com/apache/drill/pull/300/commits
> > > >
> > > > DRILL-4198 changes a user-facing API, and causes StoragePlugins that
> > were
> > > > compiled against currently released versions of Drill to no longer
> > > > functional properly. I would prefer that this breaking change be
> > modified
> > > > to be backward compatible if possible.
> > > >
> > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> > > > altekrusejason@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Aman,
> > > > >
> > > > > This is the failure that he was seeing. He figured out that the new
> > > > > exclusions in jdbc-all were not being respected when the build was
> > run
> > > > with
> > > > > an older Maven version, causing the jar size to increase
> > significantly.
> > > > He
> > > > > added an enforcer to make sure the JAR didn't grow unexpectedly.
> Can
> > > you
> > > > > try to update your maven version and re-run the build?
> > > > >
> > > > > - Jason
> > > > >
> > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <as...@maprtech.com>
> > > > wrote:
> > > > >
> > > > > > Jacques, I am getting the following build failure on the latest
> > > master
> > > > > > branch...is this what you saw for the Apache build ?  My mvn
> > version
> > > > > output
> > > > > > is shown below.  Should we all be upgrading to a newer mvn ?
> > > > > >
> > > > > >
> > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> > > > > > [WARNING] Rule 0:
> > org.apache.maven.plugins.enforcer.RequireFilesSize
> > > > > failed
> > > > > > with message:
> > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the
> expected
> > > size
> > > > > > range.
> > > > > >
> > > > > >                   This is likely due to you adding new
> dependencies
> > > to
> > > > a
> > > > > > java-exec and not updating the excludes in this module. This is
> > > > important
> > > > > > as it minimizes the size of the dependency of Drill application
> > > users.
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > > size (44664121) too large. Max. is
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > >
> > > > > >
> > > > > > Administrators-MacBook-Pro-144:incubator-drill asinha$ mvn
> > --version
> > > > > > Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > > > 2013-02-19
> > > > > > 05:51:28-0800)
> > > > > > Maven home: /opt/local/share/java/maven3
> > > > > > Java version: 1.7.0_45, vendor: Oracle Corporation
> > > > > > Java home:
> > > > > >
> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > > > > > Default locale: en_US, platform encoding: UTF-8
> > > > > > OS name: "mac os x", version: "10.9.5", arch: "x86_64", family:
> > "mac"
> > > > > >
> > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <
> > jacques@dremio.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Build back to normal. It looks like the Apache server was using
> > an
> > > > old
> > > > > > > version of Maven. Once I switched to something more recent, the
> > > build
> > > > > > > passed.
> > > > > > >
> > > > > > > --
> > > > > > > Jacques Nadeau
> > > > > > > CTO and Co-Founder, Dremio
> > > > > > >
> > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <
> > > jacques@dremio.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hmm... this merge caused the Apache build to fail.
> > > Investigating...
> > > > > > > >
> > > > > > > > --
> > > > > > > > Jacques Nadeau
> > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > >
> > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <
> > > > jacques@dremio.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> I got clean regression runs as well. I've merged the patch.
> > > > > > > >>
> > > > > > > >> Jason, you want to start the release process?
> > > > > > > >>
> > > > > > > >> --
> > > > > > > >> Jacques Nadeau
> > > > > > > >> CTO and Co-Founder, Dremio
> > > > > > > >>
> > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <
> > > > agirish@mapr.com
> > > > > >
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >>> Had two clean Functional runs. TPC-H SF100 was also
> > successful.
> > > > > > > >>>
> > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
> > > > > > > >>> challapallirahul@gmail.com> wrote:
> > > > > > > >>>
> > > > > > > >>> > Kicked off a functional run with your branch. Will let
> you
> > > know
> > > > > > once
> > > > > > > it
> > > > > > > >>> > finishes
> > > > > > > >>> >
> > > > > > > >>> > - Rahul
> > > > > > > >>> >
> > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <
> > > > > > jacques@dremio.com>
> > > > > > > >>> > wrote:
> > > > > > > >>> >
> > > > > > > >>> > > 4196 was merged today. I have an updated patch for 4291
> > > that
> > > > is
> > > > > > > >>> ready.
> > > > > > > >>> > > Unfortunately, it seems that something isn't working
> with
> > > our
> > > > > > > >>> extended
> > > > > > > >>> > > tests so I haven't been able to run an extended
> > regression.
> > > > > Unit
> > > > > > > >>> tests
> > > > > > > >>> > > pass. Is someone else possibly able to run a regression
> > > suite
> > > > > > > against
> > > > > > > >>> > this
> > > > > > > >>> > > branch [1] so we can confirm things look good and start
> > the
> > > > > > release
> > > > > > > >>> > > process?
> > > > > > > >>> > >
> > > > > > > >>> > > thanks,
> > > > > > > >>> > > Jacques
> > > > > > > >>> > >
> > > > > > > >>> > > [1]
> https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > > > > > >>> > >
> > > > > > > >>> > > --
> > > > > > > >>> > > Jacques Nadeau
> > > > > > > >>> > > CTO and Co-Founder, Dremio
> > > > > > > >>> > >
> > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <
> > > > > > > jacques@dremio.com
> > > > > > > >>> >
> > > > > > > >>> > > wrote:
> > > > > > > >>> > >
> > > > > > > >>> > > > I think the main things are 4196 and 4291 should be
> > > > > completed.
> > > > > > I
> > > > > > > >>> know
> > > > > > > >>> > > Amit
> > > > > > > >>> > > > was able to reproduce 4196 locally this weekend so I
> > > think
> > > > > > we're
> > > > > > > >>> close
> > > > > > > >>> > on
> > > > > > > >>> > > > that.
> > > > > > > >>> > > >
> > > > > > > >>> > > > --
> > > > > > > >>> > > > Jacques Nadeau
> > > > > > > >>> > > > CTO and Co-Founder, Dremio
> > > > > > > >>> > > >
> > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <
> > > > > > > zfong@maprtech.com
> > > > > > > >>> >
> > > > > > > >>> > > wrote:
> > > > > > > >>> > > >
> > > > > > > >>> > > >> Any updates on this?  What's blocking us from taking
> > > this
> > > > > to a
> > > > > > > >>> vote --
> > > > > > > >>> > > the
> > > > > > > >>> > > >> sort merge join issues?
> > > > > > > >>> > > >>
> > > > > > > >>> > > >> -- Zelaine
> > > > > > > >>> > > >>
> > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <
> > > > > > > >>> jacques@dremio.com>
> > > > > > > >>> > > >> wrote:
> > > > > > > >>> > > >>
> > > > > > > >>> > > >> > Bumping this thread...
> > > > > > > >>> > > >> >
> > > > > > > >>> > > >> > Here are the issues that were mentioned in this
> > thread
> > > > > along
> > > > > > > >>> with a
> > > > > > > >>> > > >> > proposed categorization:
> > > > > > > >>> > > >> >
> > > > > > > >>> > > >> > Release Blockers
> > > > > > > >>> > > >> > In-progress Amit
> > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4190
> > > > > > > >>> > > >> > In-progress Amit
> > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4196
> > > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4246
> > > > > > > >>> > > >> > In-review Jinfeng
> > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4256
> > > > > > > >>> > > >> > In-progress Jacques
> > > > > > > >>> > https://issues.apache.org/jira/browse/DRILL-4278
> > > > > > > >>> > > >> > Ready to merge Laurent
> > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4285
> > > > > > > >>> > > >> > Nice to Have
> > > > > > > >>> > > >> > Open Jason/Hakim
> > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4247
> > > > > > > >>> > > >> > In-progress Jason
> > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4203
> > > > > > > >>> > > >> > Open Jacques
> > > > > > https://issues.apache.org/jira/browse/DRILL-4266
> > > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4131
> > > > > > > >>> > > >> >
> > > > > > > >>> > > >> > What do others think? Let's try to get the
> blockers
> > > > > wrapped
> > > > > > up
> > > > > > > >>> in
> > > > > > > >>> > the
> > > > > > > >>> > > >> next
> > > > > > > >>> > > >> > day or two and start a release vote...
> > > > > > > >>> > > >> >
> > > > > > > >>> > > >> >
> > > > > > > >>> > > >> >
> > > > > > > >>> > > >> > --
> > > > > > > >>> > > >> > Jacques Nadeau
> > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> > > > > > > >>> > > >> >
> > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> > > > > > > >>> > > >> altekrusejason@gmail.com>
> > > > > > > >>> > > >> > wrote:
> > > > > > > >>> > > >> >
> > > > > > > >>> > > >> > > Hello All,
> > > > > > > >>> > > >> > >
> > > > > > > >>> > > >> > > With the allocator changes merged and about a
> > month
> > > > > since
> > > > > > > the
> > > > > > > >>> last
> > > > > > > >>> > > >> > release
> > > > > > > >>> > > >> > > I think it would be good to start a vote soon. I
> > > would
> > > > > > like
> > > > > > > to
> > > > > > > >>> > > >> volunteer
> > > > > > > >>> > > >> > to
> > > > > > > >>> > > >> > > be release manager.
> > > > > > > >>> > > >> > >
> > > > > > > >>> > > >> > > I know that there were some issues that were
> > > > identified
> > > > > > > after
> > > > > > > >>> the
> > > > > > > >>> > > >> > transfer
> > > > > > > >>> > > >> > > patch was merged. I think that these issues
> should
> > > be
> > > > > > fixed
> > > > > > > >>> before
> > > > > > > >>> > > we
> > > > > > > >>> > > >> > cut a
> > > > > > > >>> > > >> > > release candidate.
> > > > > > > >>> > > >> > >
> > > > > > > >>> > > >> > > From looking at the associated JIRAs it looked
> > like
> > > > > there
> > > > > > > was
> > > > > > > >>> a
> > > > > > > >>> > > >> possible
> > > > > > > >>> > > >> > > short term fix just adjusting the
> > > > > > max_query_memory_per_node
> > > > > > > >>> > option,
> > > > > > > >>> > > >> and
> > > > > > > >>> > > >> > > some more involved work to change how we
> determine
> > > the
> > > > > > > correct
> > > > > > > >>> > time
> > > > > > > >>> > > to
> > > > > > > >>> > > >> > > spill during external sort. I believe it makes
> > sense
> > > > to
> > > > > > make
> > > > > > > >>> > > external
> > > > > > > >>> > > >> > sort
> > > > > > > >>> > > >> > > work well with the newly improved memory
> > accounting
> > > > > before
> > > > > > > >>> > cutting a
> > > > > > > >>> > > >> > > release, but I'm not sure how much work is left
> to
> > > be
> > > > > done
> > > > > > > >>> there.
> > > > > > > >>> > > [1]
> > > > > > > >>> > > >> > >
> > > > > > > >>> > > >> > > Please respond with your thoughts on a release
> > soon
> > > > and
> > > > > > any
> > > > > > > >>> JIRAs
> > > > > > > >>> > > you
> > > > > > > >>> > > >> > would
> > > > > > > >>> > > >> > > like to include in the release.
> > > > > > > >>> > > >> > >
> > > > > > > >>> > > >> > > [1] -
> > > > https://issues.apache.org/jira/browse/DRILL-4243
> > > > > > > >>> > > >> > >
> > > > > > > >>> > > >> > > Thanks,
> > > > > > > >>> > > >> > > Jason
> > > > > > > >>> > > >> > >
> > > > > > > >>> > > >> >
> > > > > > > >>> > > >>
> > > > > > > >>> > > >
> > > > > > > >>> > > >
> > > > > > > >>> > >
> > > > > > > >>> >
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Jason Altekruse <al...@gmail.com>.

I think it makes sense to wait for Venki's fix of the API, as releasing a
version with the change would make reverting/refactoring the API breaking
change less useful.

Do you have any idea about the scope of fixing the hive native reader, or
which change since the 1.4 release broke it? Is someone available to look
at it today?

On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
challapallirahul@gmail.com> wrote:

> Just found an issue with hive native parquet reader (DRILL-4323). This is a
> regression from 1.4 and in my opinion should be treated as a blocker.
>
> - Rahul
>
> On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> venki.korukanti@gmail.com>
> wrote:
>
> > Sorry I didn't realize I was breaking the public interface API. Let me
> work
> > on a patch to make it backward compatible.
> >
> > Thanks
> > Venki
> >
> > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <st...@dremio.com>
> > wrote:
> >
> > > I just wanted to bring up an issue that I just now discovered, that has
> > > caused me a fair amount of grief.
> > >
> > > https://github.com/apache/drill/pull/300/commits
> > >
> > > DRILL-4198 changes a user-facing API, and causes StoragePlugins that
> were
> > > compiled against currently released versions of Drill to no longer
> > > functional properly. I would prefer that this breaking change be
> modified
> > > to be backward compatible if possible.
> > >
> > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> > > altekrusejason@gmail.com>
> > > wrote:
> > >
> > > > Hi Aman,
> > > >
> > > > This is the failure that he was seeing. He figured out that the new
> > > > exclusions in jdbc-all were not being respected when the build was
> run
> > > with
> > > > an older Maven version, causing the jar size to increase
> significantly.
> > > He
> > > > added an enforcer to make sure the JAR didn't grow unexpectedly. Can
> > you
> > > > try to update your maven version and re-run the build?
> > > >
> > > > - Jason
> > > >
> > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <as...@maprtech.com>
> > > wrote:
> > > >
> > > > > Jacques, I am getting the following build failure on the latest
> > master
> > > > > branch...is this what you saw for the Apache build ?  My mvn
> version
> > > > output
> > > > > is shown below.  Should we all be upgrading to a newer mvn ?
> > > > >
> > > > >
> > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> > > > > [WARNING] Rule 0:
> org.apache.maven.plugins.enforcer.RequireFilesSize
> > > > failed
> > > > > with message:
> > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the expected
> > size
> > > > > range.
> > > > >
> > > > >                   This is likely due to you adding new dependencies
> > to
> > > a
> > > > > java-exec and not updating the excludes in this module. This is
> > > important
> > > > > as it minimizes the size of the dependency of Drill application
> > users.
> > > > >
> > > > >
> > > >
> > >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > > size (44664121) too large. Max. is
> > > > >
> > > > >
> > > >
> > >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > >
> > > > >
> > > > > Administrators-MacBook-Pro-144:incubator-drill asinha$ mvn
> --version
> > > > > Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > > 2013-02-19
> > > > > 05:51:28-0800)
> > > > > Maven home: /opt/local/share/java/maven3
> > > > > Java version: 1.7.0_45, vendor: Oracle Corporation
> > > > > Java home:
> > > > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > > > > Default locale: en_US, platform encoding: UTF-8
> > > > > OS name: "mac os x", version: "10.9.5", arch: "x86_64", family:
> "mac"
> > > > >
> > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <
> jacques@dremio.com>
> > > > > wrote:
> > > > >
> > > > > > Build back to normal. It looks like the Apache server was using
> an
> > > old
> > > > > > version of Maven. Once I switched to something more recent, the
> > build
> > > > > > passed.
> > > > > >
> > > > > > --
> > > > > > Jacques Nadeau
> > > > > > CTO and Co-Founder, Dremio
> > > > > >
> > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <
> > jacques@dremio.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hmm... this merge caused the Apache build to fail.
> > Investigating...
> > > > > > >
> > > > > > > --
> > > > > > > Jacques Nadeau
> > > > > > > CTO and Co-Founder, Dremio
> > > > > > >
> > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <
> > > jacques@dremio.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> I got clean regression runs as well. I've merged the patch.
> > > > > > >>
> > > > > > >> Jason, you want to start the release process?
> > > > > > >>
> > > > > > >> --
> > > > > > >> Jacques Nadeau
> > > > > > >> CTO and Co-Founder, Dremio
> > > > > > >>
> > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <
> > > agirish@mapr.com
> > > > >
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >>> Had two clean Functional runs. TPC-H SF100 was also
> successful.
> > > > > > >>>
> > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
> > > > > > >>> challapallirahul@gmail.com> wrote:
> > > > > > >>>
> > > > > > >>> > Kicked off a functional run with your branch. Will let you
> > know
> > > > > once
> > > > > > it
> > > > > > >>> > finishes
> > > > > > >>> >
> > > > > > >>> > - Rahul
> > > > > > >>> >
> > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <
> > > > > jacques@dremio.com>
> > > > > > >>> > wrote:
> > > > > > >>> >
> > > > > > >>> > > 4196 was merged today. I have an updated patch for 4291
> > that
> > > is
> > > > > > >>> ready.
> > > > > > >>> > > Unfortunately, it seems that something isn't working with
> > our
> > > > > > >>> extended
> > > > > > >>> > > tests so I haven't been able to run an extended
> regression.
> > > > Unit
> > > > > > >>> tests
> > > > > > >>> > > pass. Is someone else possibly able to run a regression
> > suite
> > > > > > against
> > > > > > >>> > this
> > > > > > >>> > > branch [1] so we can confirm things look good and start
> the
> > > > > release
> > > > > > >>> > > process?
> > > > > > >>> > >
> > > > > > >>> > > thanks,
> > > > > > >>> > > Jacques
> > > > > > >>> > >
> > > > > > >>> > > [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > > > > >>> > >
> > > > > > >>> > > --
> > > > > > >>> > > Jacques Nadeau
> > > > > > >>> > > CTO and Co-Founder, Dremio
> > > > > > >>> > >
> > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <
> > > > > > jacques@dremio.com
> > > > > > >>> >
> > > > > > >>> > > wrote:
> > > > > > >>> > >
> > > > > > >>> > > > I think the main things are 4196 and 4291 should be
> > > > completed.
> > > > > I
> > > > > > >>> know
> > > > > > >>> > > Amit
> > > > > > >>> > > > was able to reproduce 4196 locally this weekend so I
> > think
> > > > > we're
> > > > > > >>> close
> > > > > > >>> > on
> > > > > > >>> > > > that.
> > > > > > >>> > > >
> > > > > > >>> > > > --
> > > > > > >>> > > > Jacques Nadeau
> > > > > > >>> > > > CTO and Co-Founder, Dremio
> > > > > > >>> > > >
> > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <
> > > > > > zfong@maprtech.com
> > > > > > >>> >
> > > > > > >>> > > wrote:
> > > > > > >>> > > >
> > > > > > >>> > > >> Any updates on this?  What's blocking us from taking
> > this
> > > > to a
> > > > > > >>> vote --
> > > > > > >>> > > the
> > > > > > >>> > > >> sort merge join issues?
> > > > > > >>> > > >>
> > > > > > >>> > > >> -- Zelaine
> > > > > > >>> > > >>
> > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <
> > > > > > >>> jacques@dremio.com>
> > > > > > >>> > > >> wrote:
> > > > > > >>> > > >>
> > > > > > >>> > > >> > Bumping this thread...
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > Here are the issues that were mentioned in this
> thread
> > > > along
> > > > > > >>> with a
> > > > > > >>> > > >> > proposed categorization:
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > Release Blockers
> > > > > > >>> > > >> > In-progress Amit
> > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4190
> > > > > > >>> > > >> > In-progress Amit
> > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4196
> > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4246
> > > > > > >>> > > >> > In-review Jinfeng
> > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4256
> > > > > > >>> > > >> > In-progress Jacques
> > > > > > >>> > https://issues.apache.org/jira/browse/DRILL-4278
> > > > > > >>> > > >> > Ready to merge Laurent
> > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4285
> > > > > > >>> > > >> > Nice to Have
> > > > > > >>> > > >> > Open Jason/Hakim
> > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4247
> > > > > > >>> > > >> > In-progress Jason
> > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4203
> > > > > > >>> > > >> > Open Jacques
> > > > > https://issues.apache.org/jira/browse/DRILL-4266
> > > > > > >>> > > >> > Ready to merge Jacques
> > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4131
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > What do others think? Let's try to get the blockers
> > > > wrapped
> > > > > up
> > > > > > >>> in
> > > > > > >>> > the
> > > > > > >>> > > >> next
> > > > > > >>> > > >> > day or two and start a release vote...
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > --
> > > > > > >>> > > >> > Jacques Nadeau
> > > > > > >>> > > >> > CTO and Co-Founder, Dremio
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> > > > > > >>> > > >> altekrusejason@gmail.com>
> > > > > > >>> > > >> > wrote:
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > > Hello All,
> > > > > > >>> > > >> > >
> > > > > > >>> > > >> > > With the allocator changes merged and about a
> month
> > > > since
> > > > > > the
> > > > > > >>> last
> > > > > > >>> > > >> > release
> > > > > > >>> > > >> > > I think it would be good to start a vote soon. I
> > would
> > > > > like
> > > > > > to
> > > > > > >>> > > >> volunteer
> > > > > > >>> > > >> > to
> > > > > > >>> > > >> > > be release manager.
> > > > > > >>> > > >> > >
> > > > > > >>> > > >> > > I know that there were some issues that were
> > > identified
> > > > > > after
> > > > > > >>> the
> > > > > > >>> > > >> > transfer
> > > > > > >>> > > >> > > patch was merged. I think that these issues should
> > be
> > > > > fixed
> > > > > > >>> before
> > > > > > >>> > > we
> > > > > > >>> > > >> > cut a
> > > > > > >>> > > >> > > release candidate.
> > > > > > >>> > > >> > >
> > > > > > >>> > > >> > > From looking at the associated JIRAs it looked
> like
> > > > there
> > > > > > was
> > > > > > >>> a
> > > > > > >>> > > >> possible
> > > > > > >>> > > >> > > short term fix just adjusting the
> > > > > max_query_memory_per_node
> > > > > > >>> > option,
> > > > > > >>> > > >> and
> > > > > > >>> > > >> > > some more involved work to change how we determine
> > the
> > > > > > correct
> > > > > > >>> > time
> > > > > > >>> > > to
> > > > > > >>> > > >> > > spill during external sort. I believe it makes
> sense
> > > to
> > > > > make
> > > > > > >>> > > external
> > > > > > >>> > > >> > sort
> > > > > > >>> > > >> > > work well with the newly improved memory
> accounting
> > > > before
> > > > > > >>> > cutting a
> > > > > > >>> > > >> > > release, but I'm not sure how much work is left to
> > be
> > > > done
> > > > > > >>> there.
> > > > > > >>> > > [1]
> > > > > > >>> > > >> > >
> > > > > > >>> > > >> > > Please respond with your thoughts on a release
> soon
> > > and
> > > > > any
> > > > > > >>> JIRAs
> > > > > > >>> > > you
> > > > > > >>> > > >> > would
> > > > > > >>> > > >> > > like to include in the release.
> > > > > > >>> > > >> > >
> > > > > > >>> > > >> > > [1] -
> > > https://issues.apache.org/jira/browse/DRILL-4243
> > > > > > >>> > > >> > >
> > > > > > >>> > > >> > > Thanks,
> > > > > > >>> > > >> > > Jason
> > > > > > >>> > > >> > >
> > > > > > >>> > > >> >
> > > > > > >>> > > >>
> > > > > > >>> > > >
> > > > > > >>> > > >
> > > > > > >>> > >
> > > > > > >>> >
> > > > > > >>>
> > > > > > >>
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by rahul challapalli <ch...@gmail.com>.

Just found an issue with hive native parquet reader (DRILL-4323). This is a
regression from 1.4 and in my opinion should be treated as a blocker.

- Rahul

On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <ve...@gmail.com>
wrote:

> Sorry I didn't realize I was breaking the public interface API. Let me work
> on a patch to make it backward compatible.
>
> Thanks
> Venki
>
> On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <st...@dremio.com>
> wrote:
>
> > I just wanted to bring up an issue that I just now discovered, that has
> > caused me a fair amount of grief.
> >
> > https://github.com/apache/drill/pull/300/commits
> >
> > DRILL-4198 changes a user-facing API, and causes StoragePlugins that were
> > compiled against currently released versions of Drill to no longer
> > functional properly. I would prefer that this breaking change be modified
> > to be backward compatible if possible.
> >
> > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> > altekrusejason@gmail.com>
> > wrote:
> >
> > > Hi Aman,
> > >
> > > This is the failure that he was seeing. He figured out that the new
> > > exclusions in jdbc-all were not being respected when the build was run
> > with
> > > an older Maven version, causing the jar size to increase significantly.
> > He
> > > added an enforcer to make sure the JAR didn't grow unexpectedly. Can
> you
> > > try to update your maven version and re-run the build?
> > >
> > > - Jason
> > >
> > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <as...@maprtech.com>
> > wrote:
> > >
> > > > Jacques, I am getting the following build failure on the latest
> master
> > > > branch...is this what you saw for the Apache build ?  My mvn version
> > > output
> > > > is shown below.  Should we all be upgrading to a newer mvn ?
> > > >
> > > >
> > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> > > > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireFilesSize
> > > failed
> > > > with message:
> > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the expected
> size
> > > > range.
> > > >
> > > >                   This is likely due to you adding new dependencies
> to
> > a
> > > > java-exec and not updating the excludes in this module. This is
> > important
> > > > as it minimizes the size of the dependency of Drill application
> users.
> > > >
> > > >
> > >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > > size (44664121) too large. Max. is
> > > >
> > > >
> > >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > >
> > > >
> > > > Administrators-MacBook-Pro-144:incubator-drill asinha$ mvn --version
> > > > Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> > 2013-02-19
> > > > 05:51:28-0800)
> > > > Maven home: /opt/local/share/java/maven3
> > > > Java version: 1.7.0_45, vendor: Oracle Corporation
> > > > Java home:
> > > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > > > Default locale: en_US, platform encoding: UTF-8
> > > > OS name: "mac os x", version: "10.9.5", arch: "x86_64", family: "mac"
> > > >
> > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <ja...@dremio.com>
> > > > wrote:
> > > >
> > > > > Build back to normal. It looks like the Apache server was using an
> > old
> > > > > version of Maven. Once I switched to something more recent, the
> build
> > > > > passed.
> > > > >
> > > > > --
> > > > > Jacques Nadeau
> > > > > CTO and Co-Founder, Dremio
> > > > >
> > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <
> jacques@dremio.com>
> > > > > wrote:
> > > > >
> > > > > > Hmm... this merge caused the Apache build to fail.
> Investigating...
> > > > > >
> > > > > > --
> > > > > > Jacques Nadeau
> > > > > > CTO and Co-Founder, Dremio
> > > > > >
> > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <
> > jacques@dremio.com>
> > > > > > wrote:
> > > > > >
> > > > > >> I got clean regression runs as well. I've merged the patch.
> > > > > >>
> > > > > >> Jason, you want to start the release process?
> > > > > >>
> > > > > >> --
> > > > > >> Jacques Nadeau
> > > > > >> CTO and Co-Founder, Dremio
> > > > > >>
> > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <
> > agirish@mapr.com
> > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >>> Had two clean Functional runs. TPC-H SF100 was also successful.
> > > > > >>>
> > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
> > > > > >>> challapallirahul@gmail.com> wrote:
> > > > > >>>
> > > > > >>> > Kicked off a functional run with your branch. Will let you
> know
> > > > once
> > > > > it
> > > > > >>> > finishes
> > > > > >>> >
> > > > > >>> > - Rahul
> > > > > >>> >
> > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <
> > > > jacques@dremio.com>
> > > > > >>> > wrote:
> > > > > >>> >
> > > > > >>> > > 4196 was merged today. I have an updated patch for 4291
> that
> > is
> > > > > >>> ready.
> > > > > >>> > > Unfortunately, it seems that something isn't working with
> our
> > > > > >>> extended
> > > > > >>> > > tests so I haven't been able to run an extended regression.
> > > Unit
> > > > > >>> tests
> > > > > >>> > > pass. Is someone else possibly able to run a regression
> suite
> > > > > against
> > > > > >>> > this
> > > > > >>> > > branch [1] so we can confirm things look good and start the
> > > > release
> > > > > >>> > > process?
> > > > > >>> > >
> > > > > >>> > > thanks,
> > > > > >>> > > Jacques
> > > > > >>> > >
> > > > > >>> > > [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > > > >>> > >
> > > > > >>> > > --
> > > > > >>> > > Jacques Nadeau
> > > > > >>> > > CTO and Co-Founder, Dremio
> > > > > >>> > >
> > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <
> > > > > jacques@dremio.com
> > > > > >>> >
> > > > > >>> > > wrote:
> > > > > >>> > >
> > > > > >>> > > > I think the main things are 4196 and 4291 should be
> > > completed.
> > > > I
> > > > > >>> know
> > > > > >>> > > Amit
> > > > > >>> > > > was able to reproduce 4196 locally this weekend so I
> think
> > > > we're
> > > > > >>> close
> > > > > >>> > on
> > > > > >>> > > > that.
> > > > > >>> > > >
> > > > > >>> > > > --
> > > > > >>> > > > Jacques Nadeau
> > > > > >>> > > > CTO and Co-Founder, Dremio
> > > > > >>> > > >
> > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <
> > > > > zfong@maprtech.com
> > > > > >>> >
> > > > > >>> > > wrote:
> > > > > >>> > > >
> > > > > >>> > > >> Any updates on this?  What's blocking us from taking
> this
> > > to a
> > > > > >>> vote --
> > > > > >>> > > the
> > > > > >>> > > >> sort merge join issues?
> > > > > >>> > > >>
> > > > > >>> > > >> -- Zelaine
> > > > > >>> > > >>
> > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <
> > > > > >>> jacques@dremio.com>
> > > > > >>> > > >> wrote:
> > > > > >>> > > >>
> > > > > >>> > > >> > Bumping this thread...
> > > > > >>> > > >> >
> > > > > >>> > > >> > Here are the issues that were mentioned in this thread
> > > along
> > > > > >>> with a
> > > > > >>> > > >> > proposed categorization:
> > > > > >>> > > >> >
> > > > > >>> > > >> > Release Blockers
> > > > > >>> > > >> > In-progress Amit
> > > > > >>> https://issues.apache.org/jira/browse/DRILL-4190
> > > > > >>> > > >> > In-progress Amit
> > > > > >>> https://issues.apache.org/jira/browse/DRILL-4196
> > > > > >>> > > >> > Ready to merge Jacques
> > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4246
> > > > > >>> > > >> > In-review Jinfeng
> > > > > >>> https://issues.apache.org/jira/browse/DRILL-4256
> > > > > >>> > > >> > In-progress Jacques
> > > > > >>> > https://issues.apache.org/jira/browse/DRILL-4278
> > > > > >>> > > >> > Ready to merge Laurent
> > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4285
> > > > > >>> > > >> > Nice to Have
> > > > > >>> > > >> > Open Jason/Hakim
> > > > > >>> https://issues.apache.org/jira/browse/DRILL-4247
> > > > > >>> > > >> > In-progress Jason
> > > > > >>> https://issues.apache.org/jira/browse/DRILL-4203
> > > > > >>> > > >> > Open Jacques
> > > > https://issues.apache.org/jira/browse/DRILL-4266
> > > > > >>> > > >> > Ready to merge Jacques
> > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4131
> > > > > >>> > > >> >
> > > > > >>> > > >> > What do others think? Let's try to get the blockers
> > > wrapped
> > > > up
> > > > > >>> in
> > > > > >>> > the
> > > > > >>> > > >> next
> > > > > >>> > > >> > day or two and start a release vote...
> > > > > >>> > > >> >
> > > > > >>> > > >> >
> > > > > >>> > > >> >
> > > > > >>> > > >> > --
> > > > > >>> > > >> > Jacques Nadeau
> > > > > >>> > > >> > CTO and Co-Founder, Dremio
> > > > > >>> > > >> >
> > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> > > > > >>> > > >> altekrusejason@gmail.com>
> > > > > >>> > > >> > wrote:
> > > > > >>> > > >> >
> > > > > >>> > > >> > > Hello All,
> > > > > >>> > > >> > >
> > > > > >>> > > >> > > With the allocator changes merged and about a month
> > > since
> > > > > the
> > > > > >>> last
> > > > > >>> > > >> > release
> > > > > >>> > > >> > > I think it would be good to start a vote soon. I
> would
> > > > like
> > > > > to
> > > > > >>> > > >> volunteer
> > > > > >>> > > >> > to
> > > > > >>> > > >> > > be release manager.
> > > > > >>> > > >> > >
> > > > > >>> > > >> > > I know that there were some issues that were
> > identified
> > > > > after
> > > > > >>> the
> > > > > >>> > > >> > transfer
> > > > > >>> > > >> > > patch was merged. I think that these issues should
> be
> > > > fixed
> > > > > >>> before
> > > > > >>> > > we
> > > > > >>> > > >> > cut a
> > > > > >>> > > >> > > release candidate.
> > > > > >>> > > >> > >
> > > > > >>> > > >> > > From looking at the associated JIRAs it looked like
> > > there
> > > > > was
> > > > > >>> a
> > > > > >>> > > >> possible
> > > > > >>> > > >> > > short term fix just adjusting the
> > > > max_query_memory_per_node
> > > > > >>> > option,
> > > > > >>> > > >> and
> > > > > >>> > > >> > > some more involved work to change how we determine
> the
> > > > > correct
> > > > > >>> > time
> > > > > >>> > > to
> > > > > >>> > > >> > > spill during external sort. I believe it makes sense
> > to
> > > > make
> > > > > >>> > > external
> > > > > >>> > > >> > sort
> > > > > >>> > > >> > > work well with the newly improved memory accounting
> > > before
> > > > > >>> > cutting a
> > > > > >>> > > >> > > release, but I'm not sure how much work is left to
> be
> > > done
> > > > > >>> there.
> > > > > >>> > > [1]
> > > > > >>> > > >> > >
> > > > > >>> > > >> > > Please respond with your thoughts on a release soon
> > and
> > > > any
> > > > > >>> JIRAs
> > > > > >>> > > you
> > > > > >>> > > >> > would
> > > > > >>> > > >> > > like to include in the release.
> > > > > >>> > > >> > >
> > > > > >>> > > >> > > [1] -
> > https://issues.apache.org/jira/browse/DRILL-4243
> > > > > >>> > > >> > >
> > > > > >>> > > >> > > Thanks,
> > > > > >>> > > >> > > Jason
> > > > > >>> > > >> > >
> > > > > >>> > > >> >
> > > > > >>> > > >>
> > > > > >>> > > >
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>>
> > > > > >>
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Venki Korukanti <ve...@gmail.com>.

Sorry I didn't realize I was breaking the public interface API. Let me work
on a patch to make it backward compatible.

Thanks
Venki

On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <st...@dremio.com> wrote:

> I just wanted to bring up an issue that I just now discovered, that has
> caused me a fair amount of grief.
>
> https://github.com/apache/drill/pull/300/commits
>
> DRILL-4198 changes a user-facing API, and causes StoragePlugins that were
> compiled against currently released versions of Drill to no longer
> functional properly. I would prefer that this breaking change be modified
> to be backward compatible if possible.
>
> On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
> altekrusejason@gmail.com>
> wrote:
>
> > Hi Aman,
> >
> > This is the failure that he was seeing. He figured out that the new
> > exclusions in jdbc-all were not being respected when the build was run
> with
> > an older Maven version, causing the jar size to increase significantly.
> He
> > added an enforcer to make sure the JAR didn't grow unexpectedly. Can you
> > try to update your maven version and re-run the build?
> >
> > - Jason
> >
> > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <as...@maprtech.com>
> wrote:
> >
> > > Jacques, I am getting the following build failure on the latest master
> > > branch...is this what you saw for the Apache build ?  My mvn version
> > output
> > > is shown below.  Should we all be upgrading to a newer mvn ?
> > >
> > >
> > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> > > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireFilesSize
> > failed
> > > with message:
> > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the expected size
> > > range.
> > >
> > >                   This is likely due to you adding new dependencies to
> a
> > > java-exec and not updating the excludes in this module. This is
> important
> > > as it minimizes the size of the dependency of Drill application users.
> > >
> > >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > > size (44664121) too large. Max. is
> > >
> > >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > >
> > >
> > > Administrators-MacBook-Pro-144:incubator-drill asinha$ mvn --version
> > > Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> 2013-02-19
> > > 05:51:28-0800)
> > > Maven home: /opt/local/share/java/maven3
> > > Java version: 1.7.0_45, vendor: Oracle Corporation
> > > Java home:
> > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > > Default locale: en_US, platform encoding: UTF-8
> > > OS name: "mac os x", version: "10.9.5", arch: "x86_64", family: "mac"
> > >
> > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <ja...@dremio.com>
> > > wrote:
> > >
> > > > Build back to normal. It looks like the Apache server was using an
> old
> > > > version of Maven. Once I switched to something more recent, the build
> > > > passed.
> > > >
> > > > --
> > > > Jacques Nadeau
> > > > CTO and Co-Founder, Dremio
> > > >
> > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <ja...@dremio.com>
> > > > wrote:
> > > >
> > > > > Hmm... this merge caused the Apache build to fail. Investigating...
> > > > >
> > > > > --
> > > > > Jacques Nadeau
> > > > > CTO and Co-Founder, Dremio
> > > > >
> > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <
> jacques@dremio.com>
> > > > > wrote:
> > > > >
> > > > >> I got clean regression runs as well. I've merged the patch.
> > > > >>
> > > > >> Jason, you want to start the release process?
> > > > >>
> > > > >> --
> > > > >> Jacques Nadeau
> > > > >> CTO and Co-Founder, Dremio
> > > > >>
> > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <
> agirish@mapr.com
> > >
> > > > >> wrote:
> > > > >>
> > > > >>> Had two clean Functional runs. TPC-H SF100 was also successful.
> > > > >>>
> > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
> > > > >>> challapallirahul@gmail.com> wrote:
> > > > >>>
> > > > >>> > Kicked off a functional run with your branch. Will let you know
> > > once
> > > > it
> > > > >>> > finishes
> > > > >>> >
> > > > >>> > - Rahul
> > > > >>> >
> > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <
> > > jacques@dremio.com>
> > > > >>> > wrote:
> > > > >>> >
> > > > >>> > > 4196 was merged today. I have an updated patch for 4291 that
> is
> > > > >>> ready.
> > > > >>> > > Unfortunately, it seems that something isn't working with our
> > > > >>> extended
> > > > >>> > > tests so I haven't been able to run an extended regression.
> > Unit
> > > > >>> tests
> > > > >>> > > pass. Is someone else possibly able to run a regression suite
> > > > against
> > > > >>> > this
> > > > >>> > > branch [1] so we can confirm things look good and start the
> > > release
> > > > >>> > > process?
> > > > >>> > >
> > > > >>> > > thanks,
> > > > >>> > > Jacques
> > > > >>> > >
> > > > >>> > > [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > > >>> > >
> > > > >>> > > --
> > > > >>> > > Jacques Nadeau
> > > > >>> > > CTO and Co-Founder, Dremio
> > > > >>> > >
> > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <
> > > > jacques@dremio.com
> > > > >>> >
> > > > >>> > > wrote:
> > > > >>> > >
> > > > >>> > > > I think the main things are 4196 and 4291 should be
> > completed.
> > > I
> > > > >>> know
> > > > >>> > > Amit
> > > > >>> > > > was able to reproduce 4196 locally this weekend so I think
> > > we're
> > > > >>> close
> > > > >>> > on
> > > > >>> > > > that.
> > > > >>> > > >
> > > > >>> > > > --
> > > > >>> > > > Jacques Nadeau
> > > > >>> > > > CTO and Co-Founder, Dremio
> > > > >>> > > >
> > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <
> > > > zfong@maprtech.com
> > > > >>> >
> > > > >>> > > wrote:
> > > > >>> > > >
> > > > >>> > > >> Any updates on this?  What's blocking us from taking this
> > to a
> > > > >>> vote --
> > > > >>> > > the
> > > > >>> > > >> sort merge join issues?
> > > > >>> > > >>
> > > > >>> > > >> -- Zelaine
> > > > >>> > > >>
> > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <
> > > > >>> jacques@dremio.com>
> > > > >>> > > >> wrote:
> > > > >>> > > >>
> > > > >>> > > >> > Bumping this thread...
> > > > >>> > > >> >
> > > > >>> > > >> > Here are the issues that were mentioned in this thread
> > along
> > > > >>> with a
> > > > >>> > > >> > proposed categorization:
> > > > >>> > > >> >
> > > > >>> > > >> > Release Blockers
> > > > >>> > > >> > In-progress Amit
> > > > >>> https://issues.apache.org/jira/browse/DRILL-4190
> > > > >>> > > >> > In-progress Amit
> > > > >>> https://issues.apache.org/jira/browse/DRILL-4196
> > > > >>> > > >> > Ready to merge Jacques
> > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4246
> > > > >>> > > >> > In-review Jinfeng
> > > > >>> https://issues.apache.org/jira/browse/DRILL-4256
> > > > >>> > > >> > In-progress Jacques
> > > > >>> > https://issues.apache.org/jira/browse/DRILL-4278
> > > > >>> > > >> > Ready to merge Laurent
> > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4285
> > > > >>> > > >> > Nice to Have
> > > > >>> > > >> > Open Jason/Hakim
> > > > >>> https://issues.apache.org/jira/browse/DRILL-4247
> > > > >>> > > >> > In-progress Jason
> > > > >>> https://issues.apache.org/jira/browse/DRILL-4203
> > > > >>> > > >> > Open Jacques
> > > https://issues.apache.org/jira/browse/DRILL-4266
> > > > >>> > > >> > Ready to merge Jacques
> > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4131
> > > > >>> > > >> >
> > > > >>> > > >> > What do others think? Let's try to get the blockers
> > wrapped
> > > up
> > > > >>> in
> > > > >>> > the
> > > > >>> > > >> next
> > > > >>> > > >> > day or two and start a release vote...
> > > > >>> > > >> >
> > > > >>> > > >> >
> > > > >>> > > >> >
> > > > >>> > > >> > --
> > > > >>> > > >> > Jacques Nadeau
> > > > >>> > > >> > CTO and Co-Founder, Dremio
> > > > >>> > > >> >
> > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> > > > >>> > > >> altekrusejason@gmail.com>
> > > > >>> > > >> > wrote:
> > > > >>> > > >> >
> > > > >>> > > >> > > Hello All,
> > > > >>> > > >> > >
> > > > >>> > > >> > > With the allocator changes merged and about a month
> > since
> > > > the
> > > > >>> last
> > > > >>> > > >> > release
> > > > >>> > > >> > > I think it would be good to start a vote soon. I would
> > > like
> > > > to
> > > > >>> > > >> volunteer
> > > > >>> > > >> > to
> > > > >>> > > >> > > be release manager.
> > > > >>> > > >> > >
> > > > >>> > > >> > > I know that there were some issues that were
> identified
> > > > after
> > > > >>> the
> > > > >>> > > >> > transfer
> > > > >>> > > >> > > patch was merged. I think that these issues should be
> > > fixed
> > > > >>> before
> > > > >>> > > we
> > > > >>> > > >> > cut a
> > > > >>> > > >> > > release candidate.
> > > > >>> > > >> > >
> > > > >>> > > >> > > From looking at the associated JIRAs it looked like
> > there
> > > > was
> > > > >>> a
> > > > >>> > > >> possible
> > > > >>> > > >> > > short term fix just adjusting the
> > > max_query_memory_per_node
> > > > >>> > option,
> > > > >>> > > >> and
> > > > >>> > > >> > > some more involved work to change how we determine the
> > > > correct
> > > > >>> > time
> > > > >>> > > to
> > > > >>> > > >> > > spill during external sort. I believe it makes sense
> to
> > > make
> > > > >>> > > external
> > > > >>> > > >> > sort
> > > > >>> > > >> > > work well with the newly improved memory accounting
> > before
> > > > >>> > cutting a
> > > > >>> > > >> > > release, but I'm not sure how much work is left to be
> > done
> > > > >>> there.
> > > > >>> > > [1]
> > > > >>> > > >> > >
> > > > >>> > > >> > > Please respond with your thoughts on a release soon
> and
> > > any
> > > > >>> JIRAs
> > > > >>> > > you
> > > > >>> > > >> > would
> > > > >>> > > >> > > like to include in the release.
> > > > >>> > > >> > >
> > > > >>> > > >> > > [1] -
> https://issues.apache.org/jira/browse/DRILL-4243
> > > > >>> > > >> > >
> > > > >>> > > >> > > Thanks,
> > > > >>> > > >> > > Jason
> > > > >>> > > >> > >
> > > > >>> > > >> >
> > > > >>> > > >>
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > > >>
> > > > >>
> > > > >
> > > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Steven Phillips <st...@dremio.com>.

I just wanted to bring up an issue that I just now discovered, that has
caused me a fair amount of grief.

https://github.com/apache/drill/pull/300/commits

DRILL-4198 changes a user-facing API, and causes StoragePlugins that were
compiled against currently released versions of Drill to no longer
functional properly. I would prefer that this breaking change be modified
to be backward compatible if possible.

On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <al...@gmail.com>
wrote:

> Hi Aman,
>
> This is the failure that he was seeing. He figured out that the new
> exclusions in jdbc-all were not being respected when the build was run with
> an older Maven version, causing the jar size to increase significantly. He
> added an enforcer to make sure the JAR didn't grow unexpectedly. Can you
> try to update your maven version and re-run the build?
>
> - Jason
>
> On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <as...@maprtech.com> wrote:
>
> > Jacques, I am getting the following build failure on the latest master
> > branch...is this what you saw for the Apache build ?  My mvn version
> output
> > is shown below.  Should we all be upgrading to a newer mvn ?
> >
> >
> > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireFilesSize
> failed
> > with message:
> > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the expected size
> > range.
> >
> >                   This is likely due to you adding new dependencies to a
> > java-exec and not updating the excludes in this module. This is important
> > as it minimizes the size of the dependency of Drill application users.
> >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > size (44664121) too large. Max. is
> >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >
> >
> > Administrators-MacBook-Pro-144:incubator-drill asinha$ mvn --version
> > Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19
> > 05:51:28-0800)
> > Maven home: /opt/local/share/java/maven3
> > Java version: 1.7.0_45, vendor: Oracle Corporation
> > Java home:
> > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > Default locale: en_US, platform encoding: UTF-8
> > OS name: "mac os x", version: "10.9.5", arch: "x86_64", family: "mac"
> >
> > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <ja...@dremio.com>
> > wrote:
> >
> > > Build back to normal. It looks like the Apache server was using an old
> > > version of Maven. Once I switched to something more recent, the build
> > > passed.
> > >
> > > --
> > > Jacques Nadeau
> > > CTO and Co-Founder, Dremio
> > >
> > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <ja...@dremio.com>
> > > wrote:
> > >
> > > > Hmm... this merge caused the Apache build to fail. Investigating...
> > > >
> > > > --
> > > > Jacques Nadeau
> > > > CTO and Co-Founder, Dremio
> > > >
> > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <ja...@dremio.com>
> > > > wrote:
> > > >
> > > >> I got clean regression runs as well. I've merged the patch.
> > > >>
> > > >> Jason, you want to start the release process?
> > > >>
> > > >> --
> > > >> Jacques Nadeau
> > > >> CTO and Co-Founder, Dremio
> > > >>
> > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <agirish@mapr.com
> >
> > > >> wrote:
> > > >>
> > > >>> Had two clean Functional runs. TPC-H SF100 was also successful.
> > > >>>
> > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
> > > >>> challapallirahul@gmail.com> wrote:
> > > >>>
> > > >>> > Kicked off a functional run with your branch. Will let you know
> > once
> > > it
> > > >>> > finishes
> > > >>> >
> > > >>> > - Rahul
> > > >>> >
> > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <
> > jacques@dremio.com>
> > > >>> > wrote:
> > > >>> >
> > > >>> > > 4196 was merged today. I have an updated patch for 4291 that is
> > > >>> ready.
> > > >>> > > Unfortunately, it seems that something isn't working with our
> > > >>> extended
> > > >>> > > tests so I haven't been able to run an extended regression.
> Unit
> > > >>> tests
> > > >>> > > pass. Is someone else possibly able to run a regression suite
> > > against
> > > >>> > this
> > > >>> > > branch [1] so we can confirm things look good and start the
> > release
> > > >>> > > process?
> > > >>> > >
> > > >>> > > thanks,
> > > >>> > > Jacques
> > > >>> > >
> > > >>> > > [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > >>> > >
> > > >>> > > --
> > > >>> > > Jacques Nadeau
> > > >>> > > CTO and Co-Founder, Dremio
> > > >>> > >
> > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <
> > > jacques@dremio.com
> > > >>> >
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > > > I think the main things are 4196 and 4291 should be
> completed.
> > I
> > > >>> know
> > > >>> > > Amit
> > > >>> > > > was able to reproduce 4196 locally this weekend so I think
> > we're
> > > >>> close
> > > >>> > on
> > > >>> > > > that.
> > > >>> > > >
> > > >>> > > > --
> > > >>> > > > Jacques Nadeau
> > > >>> > > > CTO and Co-Founder, Dremio
> > > >>> > > >
> > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <
> > > zfong@maprtech.com
> > > >>> >
> > > >>> > > wrote:
> > > >>> > > >
> > > >>> > > >> Any updates on this?  What's blocking us from taking this
> to a
> > > >>> vote --
> > > >>> > > the
> > > >>> > > >> sort merge join issues?
> > > >>> > > >>
> > > >>> > > >> -- Zelaine
> > > >>> > > >>
> > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <
> > > >>> jacques@dremio.com>
> > > >>> > > >> wrote:
> > > >>> > > >>
> > > >>> > > >> > Bumping this thread...
> > > >>> > > >> >
> > > >>> > > >> > Here are the issues that were mentioned in this thread
> along
> > > >>> with a
> > > >>> > > >> > proposed categorization:
> > > >>> > > >> >
> > > >>> > > >> > Release Blockers
> > > >>> > > >> > In-progress Amit
> > > >>> https://issues.apache.org/jira/browse/DRILL-4190
> > > >>> > > >> > In-progress Amit
> > > >>> https://issues.apache.org/jira/browse/DRILL-4196
> > > >>> > > >> > Ready to merge Jacques
> > > >>> > > https://issues.apache.org/jira/browse/DRILL-4246
> > > >>> > > >> > In-review Jinfeng
> > > >>> https://issues.apache.org/jira/browse/DRILL-4256
> > > >>> > > >> > In-progress Jacques
> > > >>> > https://issues.apache.org/jira/browse/DRILL-4278
> > > >>> > > >> > Ready to merge Laurent
> > > >>> > > https://issues.apache.org/jira/browse/DRILL-4285
> > > >>> > > >> > Nice to Have
> > > >>> > > >> > Open Jason/Hakim
> > > >>> https://issues.apache.org/jira/browse/DRILL-4247
> > > >>> > > >> > In-progress Jason
> > > >>> https://issues.apache.org/jira/browse/DRILL-4203
> > > >>> > > >> > Open Jacques
> > https://issues.apache.org/jira/browse/DRILL-4266
> > > >>> > > >> > Ready to merge Jacques
> > > >>> > > https://issues.apache.org/jira/browse/DRILL-4131
> > > >>> > > >> >
> > > >>> > > >> > What do others think? Let's try to get the blockers
> wrapped
> > up
> > > >>> in
> > > >>> > the
> > > >>> > > >> next
> > > >>> > > >> > day or two and start a release vote...
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >> > --
> > > >>> > > >> > Jacques Nadeau
> > > >>> > > >> > CTO and Co-Founder, Dremio
> > > >>> > > >> >
> > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> > > >>> > > >> altekrusejason@gmail.com>
> > > >>> > > >> > wrote:
> > > >>> > > >> >
> > > >>> > > >> > > Hello All,
> > > >>> > > >> > >
> > > >>> > > >> > > With the allocator changes merged and about a month
> since
> > > the
> > > >>> last
> > > >>> > > >> > release
> > > >>> > > >> > > I think it would be good to start a vote soon. I would
> > like
> > > to
> > > >>> > > >> volunteer
> > > >>> > > >> > to
> > > >>> > > >> > > be release manager.
> > > >>> > > >> > >
> > > >>> > > >> > > I know that there were some issues that were identified
> > > after
> > > >>> the
> > > >>> > > >> > transfer
> > > >>> > > >> > > patch was merged. I think that these issues should be
> > fixed
> > > >>> before
> > > >>> > > we
> > > >>> > > >> > cut a
> > > >>> > > >> > > release candidate.
> > > >>> > > >> > >
> > > >>> > > >> > > From looking at the associated JIRAs it looked like
> there
> > > was
> > > >>> a
> > > >>> > > >> possible
> > > >>> > > >> > > short term fix just adjusting the
> > max_query_memory_per_node
> > > >>> > option,
> > > >>> > > >> and
> > > >>> > > >> > > some more involved work to change how we determine the
> > > correct
> > > >>> > time
> > > >>> > > to
> > > >>> > > >> > > spill during external sort. I believe it makes sense to
> > make
> > > >>> > > external
> > > >>> > > >> > sort
> > > >>> > > >> > > work well with the newly improved memory accounting
> before
> > > >>> > cutting a
> > > >>> > > >> > > release, but I'm not sure how much work is left to be
> done
> > > >>> there.
> > > >>> > > [1]
> > > >>> > > >> > >
> > > >>> > > >> > > Please respond with your thoughts on a release soon and
> > any
> > > >>> JIRAs
> > > >>> > > you
> > > >>> > > >> > would
> > > >>> > > >> > > like to include in the release.
> > > >>> > > >> > >
> > > >>> > > >> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> > > >>> > > >> > >
> > > >>> > > >> > > Thanks,
> > > >>> > > >> > > Jason
> > > >>> > > >> > >
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Aman Sinha <am...@apache.org>.

I upgraded to mvn 3.3.9 (using brew install maven) on my mac.  I ran into
some other issues .. running out of PermGen and when I bumped it up to
256M, ran into OutOfMemoryError during the build.   I will have more time
later today to look into it.

On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <al...@gmail.com>
wrote:

> Hi Aman,
>
> This is the failure that he was seeing. He figured out that the new
> exclusions in jdbc-all were not being respected when the build was run with
> an older Maven version, causing the jar size to increase significantly. He
> added an enforcer to make sure the JAR didn't grow unexpectedly. Can you
> try to update your maven version and re-run the build?
>
> - Jason
>
> On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <as...@maprtech.com> wrote:
>
> > Jacques, I am getting the following build failure on the latest master
> > branch...is this what you saw for the Apache build ?  My mvn version
> output
> > is shown below.  Should we all be upgrading to a newer mvn ?
> >
> >
> > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> > (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireFilesSize
> failed
> > with message:
> > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the expected size
> > range.
> >
> >                   This is likely due to you adding new dependencies to a
> > java-exec and not updating the excludes in this module. This is important
> > as it minimizes the size of the dependency of Drill application users.
> >
> >
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> > size (44664121) too large. Max. is
> >
> >
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >
> >
> > Administrators-MacBook-Pro-144:incubator-drill asinha$ mvn --version
> > Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19
> > 05:51:28-0800)
> > Maven home: /opt/local/share/java/maven3
> > Java version: 1.7.0_45, vendor: Oracle Corporation
> > Java home:
> > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> > Default locale: en_US, platform encoding: UTF-8
> > OS name: "mac os x", version: "10.9.5", arch: "x86_64", family: "mac"
> >
> > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <ja...@dremio.com>
> > wrote:
> >
> > > Build back to normal. It looks like the Apache server was using an old
> > > version of Maven. Once I switched to something more recent, the build
> > > passed.
> > >
> > > --
> > > Jacques Nadeau
> > > CTO and Co-Founder, Dremio
> > >
> > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <ja...@dremio.com>
> > > wrote:
> > >
> > > > Hmm... this merge caused the Apache build to fail. Investigating...
> > > >
> > > > --
> > > > Jacques Nadeau
> > > > CTO and Co-Founder, Dremio
> > > >
> > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <ja...@dremio.com>
> > > > wrote:
> > > >
> > > >> I got clean regression runs as well. I've merged the patch.
> > > >>
> > > >> Jason, you want to start the release process?
> > > >>
> > > >> --
> > > >> Jacques Nadeau
> > > >> CTO and Co-Founder, Dremio
> > > >>
> > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <agirish@mapr.com
> >
> > > >> wrote:
> > > >>
> > > >>> Had two clean Functional runs. TPC-H SF100 was also successful.
> > > >>>
> > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
> > > >>> challapallirahul@gmail.com> wrote:
> > > >>>
> > > >>> > Kicked off a functional run with your branch. Will let you know
> > once
> > > it
> > > >>> > finishes
> > > >>> >
> > > >>> > - Rahul
> > > >>> >
> > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <
> > jacques@dremio.com>
> > > >>> > wrote:
> > > >>> >
> > > >>> > > 4196 was merged today. I have an updated patch for 4291 that is
> > > >>> ready.
> > > >>> > > Unfortunately, it seems that something isn't working with our
> > > >>> extended
> > > >>> > > tests so I haven't been able to run an extended regression.
> Unit
> > > >>> tests
> > > >>> > > pass. Is someone else possibly able to run a regression suite
> > > against
> > > >>> > this
> > > >>> > > branch [1] so we can confirm things look good and start the
> > release
> > > >>> > > process?
> > > >>> > >
> > > >>> > > thanks,
> > > >>> > > Jacques
> > > >>> > >
> > > >>> > > [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > > >>> > >
> > > >>> > > --
> > > >>> > > Jacques Nadeau
> > > >>> > > CTO and Co-Founder, Dremio
> > > >>> > >
> > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <
> > > jacques@dremio.com
> > > >>> >
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > > > I think the main things are 4196 and 4291 should be
> completed.
> > I
> > > >>> know
> > > >>> > > Amit
> > > >>> > > > was able to reproduce 4196 locally this weekend so I think
> > we're
> > > >>> close
> > > >>> > on
> > > >>> > > > that.
> > > >>> > > >
> > > >>> > > > --
> > > >>> > > > Jacques Nadeau
> > > >>> > > > CTO and Co-Founder, Dremio
> > > >>> > > >
> > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <
> > > zfong@maprtech.com
> > > >>> >
> > > >>> > > wrote:
> > > >>> > > >
> > > >>> > > >> Any updates on this?  What's blocking us from taking this
> to a
> > > >>> vote --
> > > >>> > > the
> > > >>> > > >> sort merge join issues?
> > > >>> > > >>
> > > >>> > > >> -- Zelaine
> > > >>> > > >>
> > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <
> > > >>> jacques@dremio.com>
> > > >>> > > >> wrote:
> > > >>> > > >>
> > > >>> > > >> > Bumping this thread...
> > > >>> > > >> >
> > > >>> > > >> > Here are the issues that were mentioned in this thread
> along
> > > >>> with a
> > > >>> > > >> > proposed categorization:
> > > >>> > > >> >
> > > >>> > > >> > Release Blockers
> > > >>> > > >> > In-progress Amit
> > > >>> https://issues.apache.org/jira/browse/DRILL-4190
> > > >>> > > >> > In-progress Amit
> > > >>> https://issues.apache.org/jira/browse/DRILL-4196
> > > >>> > > >> > Ready to merge Jacques
> > > >>> > > https://issues.apache.org/jira/browse/DRILL-4246
> > > >>> > > >> > In-review Jinfeng
> > > >>> https://issues.apache.org/jira/browse/DRILL-4256
> > > >>> > > >> > In-progress Jacques
> > > >>> > https://issues.apache.org/jira/browse/DRILL-4278
> > > >>> > > >> > Ready to merge Laurent
> > > >>> > > https://issues.apache.org/jira/browse/DRILL-4285
> > > >>> > > >> > Nice to Have
> > > >>> > > >> > Open Jason/Hakim
> > > >>> https://issues.apache.org/jira/browse/DRILL-4247
> > > >>> > > >> > In-progress Jason
> > > >>> https://issues.apache.org/jira/browse/DRILL-4203
> > > >>> > > >> > Open Jacques
> > https://issues.apache.org/jira/browse/DRILL-4266
> > > >>> > > >> > Ready to merge Jacques
> > > >>> > > https://issues.apache.org/jira/browse/DRILL-4131
> > > >>> > > >> >
> > > >>> > > >> > What do others think? Let's try to get the blockers
> wrapped
> > up
> > > >>> in
> > > >>> > the
> > > >>> > > >> next
> > > >>> > > >> > day or two and start a release vote...
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >> > --
> > > >>> > > >> > Jacques Nadeau
> > > >>> > > >> > CTO and Co-Founder, Dremio
> > > >>> > > >> >
> > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> > > >>> > > >> altekrusejason@gmail.com>
> > > >>> > > >> > wrote:
> > > >>> > > >> >
> > > >>> > > >> > > Hello All,
> > > >>> > > >> > >
> > > >>> > > >> > > With the allocator changes merged and about a month
> since
> > > the
> > > >>> last
> > > >>> > > >> > release
> > > >>> > > >> > > I think it would be good to start a vote soon. I would
> > like
> > > to
> > > >>> > > >> volunteer
> > > >>> > > >> > to
> > > >>> > > >> > > be release manager.
> > > >>> > > >> > >
> > > >>> > > >> > > I know that there were some issues that were identified
> > > after
> > > >>> the
> > > >>> > > >> > transfer
> > > >>> > > >> > > patch was merged. I think that these issues should be
> > fixed
> > > >>> before
> > > >>> > > we
> > > >>> > > >> > cut a
> > > >>> > > >> > > release candidate.
> > > >>> > > >> > >
> > > >>> > > >> > > From looking at the associated JIRAs it looked like
> there
> > > was
> > > >>> a
> > > >>> > > >> possible
> > > >>> > > >> > > short term fix just adjusting the
> > max_query_memory_per_node
> > > >>> > option,
> > > >>> > > >> and
> > > >>> > > >> > > some more involved work to change how we determine the
> > > correct
> > > >>> > time
> > > >>> > > to
> > > >>> > > >> > > spill during external sort. I believe it makes sense to
> > make
> > > >>> > > external
> > > >>> > > >> > sort
> > > >>> > > >> > > work well with the newly improved memory accounting
> before
> > > >>> > cutting a
> > > >>> > > >> > > release, but I'm not sure how much work is left to be
> done
> > > >>> there.
> > > >>> > > [1]
> > > >>> > > >> > >
> > > >>> > > >> > > Please respond with your thoughts on a release soon and
> > any
> > > >>> JIRAs
> > > >>> > > you
> > > >>> > > >> > would
> > > >>> > > >> > > like to include in the release.
> > > >>> > > >> > >
> > > >>> > > >> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> > > >>> > > >> > >
> > > >>> > > >> > > Thanks,
> > > >>> > > >> > > Jason
> > > >>> > > >> > >
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Jason Altekruse <al...@gmail.com>.

Hi Aman,

This is the failure that he was seeing. He figured out that the new
exclusions in jdbc-all were not being respected when the build was run with
an older Maven version, causing the jar size to increase significantly. He
added an enforcer to make sure the JAR didn't grow unexpectedly. Can you
try to update your maven version and re-run the build?

- Jason

On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <as...@maprtech.com> wrote:

> Jacques, I am getting the following build failure on the latest master
> branch...is this what you saw for the Apache build ?  My mvn version output
> is shown below.  Should we all be upgrading to a newer mvn ?
>
>
> [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> (enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireFilesSize failed
> with message:
> The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the expected size
> range.
>
>                   This is likely due to you adding new dependencies to a
> java-exec and not updating the excludes in this module. This is important
> as it minimizes the size of the dependency of Drill application users.
>
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> size (44664121) too large. Max. is
>
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>
>
> Administrators-MacBook-Pro-144:incubator-drill asinha$ mvn --version
> Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19
> 05:51:28-0800)
> Maven home: /opt/local/share/java/maven3
> Java version: 1.7.0_45, vendor: Oracle Corporation
> Java home:
> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.9.5", arch: "x86_64", family: "mac"
>
> On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <ja...@dremio.com>
> wrote:
>
> > Build back to normal. It looks like the Apache server was using an old
> > version of Maven. Once I switched to something more recent, the build
> > passed.
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <ja...@dremio.com>
> > wrote:
> >
> > > Hmm... this merge caused the Apache build to fail. Investigating...
> > >
> > > --
> > > Jacques Nadeau
> > > CTO and Co-Founder, Dremio
> > >
> > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <ja...@dremio.com>
> > > wrote:
> > >
> > >> I got clean regression runs as well. I've merged the patch.
> > >>
> > >> Jason, you want to start the release process?
> > >>
> > >> --
> > >> Jacques Nadeau
> > >> CTO and Co-Founder, Dremio
> > >>
> > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <ag...@mapr.com>
> > >> wrote:
> > >>
> > >>> Had two clean Functional runs. TPC-H SF100 was also successful.
> > >>>
> > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
> > >>> challapallirahul@gmail.com> wrote:
> > >>>
> > >>> > Kicked off a functional run with your branch. Will let you know
> once
> > it
> > >>> > finishes
> > >>> >
> > >>> > - Rahul
> > >>> >
> > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <
> jacques@dremio.com>
> > >>> > wrote:
> > >>> >
> > >>> > > 4196 was merged today. I have an updated patch for 4291 that is
> > >>> ready.
> > >>> > > Unfortunately, it seems that something isn't working with our
> > >>> extended
> > >>> > > tests so I haven't been able to run an extended regression. Unit
> > >>> tests
> > >>> > > pass. Is someone else possibly able to run a regression suite
> > against
> > >>> > this
> > >>> > > branch [1] so we can confirm things look good and start the
> release
> > >>> > > process?
> > >>> > >
> > >>> > > thanks,
> > >>> > > Jacques
> > >>> > >
> > >>> > > [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > >>> > >
> > >>> > > --
> > >>> > > Jacques Nadeau
> > >>> > > CTO and Co-Founder, Dremio
> > >>> > >
> > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <
> > jacques@dremio.com
> > >>> >
> > >>> > > wrote:
> > >>> > >
> > >>> > > > I think the main things are 4196 and 4291 should be completed.
> I
> > >>> know
> > >>> > > Amit
> > >>> > > > was able to reproduce 4196 locally this weekend so I think
> we're
> > >>> close
> > >>> > on
> > >>> > > > that.
> > >>> > > >
> > >>> > > > --
> > >>> > > > Jacques Nadeau
> > >>> > > > CTO and Co-Founder, Dremio
> > >>> > > >
> > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <
> > zfong@maprtech.com
> > >>> >
> > >>> > > wrote:
> > >>> > > >
> > >>> > > >> Any updates on this?  What's blocking us from taking this to a
> > >>> vote --
> > >>> > > the
> > >>> > > >> sort merge join issues?
> > >>> > > >>
> > >>> > > >> -- Zelaine
> > >>> > > >>
> > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <
> > >>> jacques@dremio.com>
> > >>> > > >> wrote:
> > >>> > > >>
> > >>> > > >> > Bumping this thread...
> > >>> > > >> >
> > >>> > > >> > Here are the issues that were mentioned in this thread along
> > >>> with a
> > >>> > > >> > proposed categorization:
> > >>> > > >> >
> > >>> > > >> > Release Blockers
> > >>> > > >> > In-progress Amit
> > >>> https://issues.apache.org/jira/browse/DRILL-4190
> > >>> > > >> > In-progress Amit
> > >>> https://issues.apache.org/jira/browse/DRILL-4196
> > >>> > > >> > Ready to merge Jacques
> > >>> > > https://issues.apache.org/jira/browse/DRILL-4246
> > >>> > > >> > In-review Jinfeng
> > >>> https://issues.apache.org/jira/browse/DRILL-4256
> > >>> > > >> > In-progress Jacques
> > >>> > https://issues.apache.org/jira/browse/DRILL-4278
> > >>> > > >> > Ready to merge Laurent
> > >>> > > https://issues.apache.org/jira/browse/DRILL-4285
> > >>> > > >> > Nice to Have
> > >>> > > >> > Open Jason/Hakim
> > >>> https://issues.apache.org/jira/browse/DRILL-4247
> > >>> > > >> > In-progress Jason
> > >>> https://issues.apache.org/jira/browse/DRILL-4203
> > >>> > > >> > Open Jacques
> https://issues.apache.org/jira/browse/DRILL-4266
> > >>> > > >> > Ready to merge Jacques
> > >>> > > https://issues.apache.org/jira/browse/DRILL-4131
> > >>> > > >> >
> > >>> > > >> > What do others think? Let's try to get the blockers wrapped
> up
> > >>> in
> > >>> > the
> > >>> > > >> next
> > >>> > > >> > day or two and start a release vote...
> > >>> > > >> >
> > >>> > > >> >
> > >>> > > >> >
> > >>> > > >> > --
> > >>> > > >> > Jacques Nadeau
> > >>> > > >> > CTO and Co-Founder, Dremio
> > >>> > > >> >
> > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> > >>> > > >> altekrusejason@gmail.com>
> > >>> > > >> > wrote:
> > >>> > > >> >
> > >>> > > >> > > Hello All,
> > >>> > > >> > >
> > >>> > > >> > > With the allocator changes merged and about a month since
> > the
> > >>> last
> > >>> > > >> > release
> > >>> > > >> > > I think it would be good to start a vote soon. I would
> like
> > to
> > >>> > > >> volunteer
> > >>> > > >> > to
> > >>> > > >> > > be release manager.
> > >>> > > >> > >
> > >>> > > >> > > I know that there were some issues that were identified
> > after
> > >>> the
> > >>> > > >> > transfer
> > >>> > > >> > > patch was merged. I think that these issues should be
> fixed
> > >>> before
> > >>> > > we
> > >>> > > >> > cut a
> > >>> > > >> > > release candidate.
> > >>> > > >> > >
> > >>> > > >> > > From looking at the associated JIRAs it looked like there
> > was
> > >>> a
> > >>> > > >> possible
> > >>> > > >> > > short term fix just adjusting the
> max_query_memory_per_node
> > >>> > option,
> > >>> > > >> and
> > >>> > > >> > > some more involved work to change how we determine the
> > correct
> > >>> > time
> > >>> > > to
> > >>> > > >> > > spill during external sort. I believe it makes sense to
> make
> > >>> > > external
> > >>> > > >> > sort
> > >>> > > >> > > work well with the newly improved memory accounting before
> > >>> > cutting a
> > >>> > > >> > > release, but I'm not sure how much work is left to be done
> > >>> there.
> > >>> > > [1]
> > >>> > > >> > >
> > >>> > > >> > > Please respond with your thoughts on a release soon and
> any
> > >>> JIRAs
> > >>> > > you
> > >>> > > >> > would
> > >>> > > >> > > like to include in the release.
> > >>> > > >> > >
> > >>> > > >> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> > >>> > > >> > >
> > >>> > > >> > > Thanks,
> > >>> > > >> > > Jason
> > >>> > > >> > >
> > >>> > > >> >
> > >>> > > >>
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Aman Sinha <as...@maprtech.com>.

Jacques, I am getting the following build failure on the latest master
branch...is this what you saw for the Apache build ?  My mvn version output
is shown below.  Should we all be upgrading to a newer mvn ?


[INFO] --- maven-enforcer-plugin:1.3.1:enforce
(enforce-jdbc-jar-compactness) @ drill-jdbc-all ---
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireFilesSize failed
with message:
The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the expected size
range.

                  This is likely due to you adding new dependencies to a
java-exec and not updating the excludes in this module. This is important
as it minimizes the size of the dependency of Drill application users.
/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
size (44664121) too large. Max. is
20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar


Administrators-MacBook-Pro-144:incubator-drill asinha$ mvn --version
Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19
05:51:28-0800)
Maven home: /opt/local/share/java/maven3
Java version: 1.7.0_45, vendor: Oracle Corporation
Java home:
/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.9.5", arch: "x86_64", family: "mac"

On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau <ja...@dremio.com> wrote:

> Build back to normal. It looks like the Apache server was using an old
> version of Maven. Once I switched to something more recent, the build
> passed.
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <ja...@dremio.com>
> wrote:
>
> > Hmm... this merge caused the Apache build to fail. Investigating...
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <ja...@dremio.com>
> > wrote:
> >
> >> I got clean regression runs as well. I've merged the patch.
> >>
> >> Jason, you want to start the release process?
> >>
> >> --
> >> Jacques Nadeau
> >> CTO and Co-Founder, Dremio
> >>
> >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <ag...@mapr.com>
> >> wrote:
> >>
> >>> Had two clean Functional runs. TPC-H SF100 was also successful.
> >>>
> >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
> >>> challapallirahul@gmail.com> wrote:
> >>>
> >>> > Kicked off a functional run with your branch. Will let you know once
> it
> >>> > finishes
> >>> >
> >>> > - Rahul
> >>> >
> >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com>
> >>> > wrote:
> >>> >
> >>> > > 4196 was merged today. I have an updated patch for 4291 that is
> >>> ready.
> >>> > > Unfortunately, it seems that something isn't working with our
> >>> extended
> >>> > > tests so I haven't been able to run an extended regression. Unit
> >>> tests
> >>> > > pass. Is someone else possibly able to run a regression suite
> against
> >>> > this
> >>> > > branch [1] so we can confirm things look good and start the release
> >>> > > process?
> >>> > >
> >>> > > thanks,
> >>> > > Jacques
> >>> > >
> >>> > > [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
> >>> > >
> >>> > > --
> >>> > > Jacques Nadeau
> >>> > > CTO and Co-Founder, Dremio
> >>> > >
> >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <
> jacques@dremio.com
> >>> >
> >>> > > wrote:
> >>> > >
> >>> > > > I think the main things are 4196 and 4291 should be completed. I
> >>> know
> >>> > > Amit
> >>> > > > was able to reproduce 4196 locally this weekend so I think we're
> >>> close
> >>> > on
> >>> > > > that.
> >>> > > >
> >>> > > > --
> >>> > > > Jacques Nadeau
> >>> > > > CTO and Co-Founder, Dremio
> >>> > > >
> >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <
> zfong@maprtech.com
> >>> >
> >>> > > wrote:
> >>> > > >
> >>> > > >> Any updates on this?  What's blocking us from taking this to a
> >>> vote --
> >>> > > the
> >>> > > >> sort merge join issues?
> >>> > > >>
> >>> > > >> -- Zelaine
> >>> > > >>
> >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <
> >>> jacques@dremio.com>
> >>> > > >> wrote:
> >>> > > >>
> >>> > > >> > Bumping this thread...
> >>> > > >> >
> >>> > > >> > Here are the issues that were mentioned in this thread along
> >>> with a
> >>> > > >> > proposed categorization:
> >>> > > >> >
> >>> > > >> > Release Blockers
> >>> > > >> > In-progress Amit
> >>> https://issues.apache.org/jira/browse/DRILL-4190
> >>> > > >> > In-progress Amit
> >>> https://issues.apache.org/jira/browse/DRILL-4196
> >>> > > >> > Ready to merge Jacques
> >>> > > https://issues.apache.org/jira/browse/DRILL-4246
> >>> > > >> > In-review Jinfeng
> >>> https://issues.apache.org/jira/browse/DRILL-4256
> >>> > > >> > In-progress Jacques
> >>> > https://issues.apache.org/jira/browse/DRILL-4278
> >>> > > >> > Ready to merge Laurent
> >>> > > https://issues.apache.org/jira/browse/DRILL-4285
> >>> > > >> > Nice to Have
> >>> > > >> > Open Jason/Hakim
> >>> https://issues.apache.org/jira/browse/DRILL-4247
> >>> > > >> > In-progress Jason
> >>> https://issues.apache.org/jira/browse/DRILL-4203
> >>> > > >> > Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
> >>> > > >> > Ready to merge Jacques
> >>> > > https://issues.apache.org/jira/browse/DRILL-4131
> >>> > > >> >
> >>> > > >> > What do others think? Let's try to get the blockers wrapped up
> >>> in
> >>> > the
> >>> > > >> next
> >>> > > >> > day or two and start a release vote...
> >>> > > >> >
> >>> > > >> >
> >>> > > >> >
> >>> > > >> > --
> >>> > > >> > Jacques Nadeau
> >>> > > >> > CTO and Co-Founder, Dremio
> >>> > > >> >
> >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> >>> > > >> altekrusejason@gmail.com>
> >>> > > >> > wrote:
> >>> > > >> >
> >>> > > >> > > Hello All,
> >>> > > >> > >
> >>> > > >> > > With the allocator changes merged and about a month since
> the
> >>> last
> >>> > > >> > release
> >>> > > >> > > I think it would be good to start a vote soon. I would like
> to
> >>> > > >> volunteer
> >>> > > >> > to
> >>> > > >> > > be release manager.
> >>> > > >> > >
> >>> > > >> > > I know that there were some issues that were identified
> after
> >>> the
> >>> > > >> > transfer
> >>> > > >> > > patch was merged. I think that these issues should be fixed
> >>> before
> >>> > > we
> >>> > > >> > cut a
> >>> > > >> > > release candidate.
> >>> > > >> > >
> >>> > > >> > > From looking at the associated JIRAs it looked like there
> was
> >>> a
> >>> > > >> possible
> >>> > > >> > > short term fix just adjusting the max_query_memory_per_node
> >>> > option,
> >>> > > >> and
> >>> > > >> > > some more involved work to change how we determine the
> correct
> >>> > time
> >>> > > to
> >>> > > >> > > spill during external sort. I believe it makes sense to make
> >>> > > external
> >>> > > >> > sort
> >>> > > >> > > work well with the newly improved memory accounting before
> >>> > cutting a
> >>> > > >> > > release, but I'm not sure how much work is left to be done
> >>> there.
> >>> > > [1]
> >>> > > >> > >
> >>> > > >> > > Please respond with your thoughts on a release soon and any
> >>> JIRAs
> >>> > > you
> >>> > > >> > would
> >>> > > >> > > like to include in the release.
> >>> > > >> > >
> >>> > > >> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> >>> > > >> > >
> >>> > > >> > > Thanks,
> >>> > > >> > > Jason
> >>> > > >> > >
> >>> > > >> >
> >>> > > >>
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
>

Re: Time for a 1.5 release?

Posted by Jacques Nadeau <ja...@dremio.com>.

Build back to normal. It looks like the Apache server was using an old
version of Maven. Once I switched to something more recent, the build
passed.

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau <ja...@dremio.com> wrote:

> Hmm... this merge caused the Apache build to fail. Investigating...
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <ja...@dremio.com>
> wrote:
>
>> I got clean regression runs as well. I've merged the patch.
>>
>> Jason, you want to start the release process?
>>
>> --
>> Jacques Nadeau
>> CTO and Co-Founder, Dremio
>>
>> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <ag...@mapr.com>
>> wrote:
>>
>>> Had two clean Functional runs. TPC-H SF100 was also successful.
>>>
>>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
>>> challapallirahul@gmail.com> wrote:
>>>
>>> > Kicked off a functional run with your branch. Will let you know once it
>>> > finishes
>>> >
>>> > - Rahul
>>> >
>>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com>
>>> > wrote:
>>> >
>>> > > 4196 was merged today. I have an updated patch for 4291 that is
>>> ready.
>>> > > Unfortunately, it seems that something isn't working with our
>>> extended
>>> > > tests so I haven't been able to run an extended regression. Unit
>>> tests
>>> > > pass. Is someone else possibly able to run a regression suite against
>>> > this
>>> > > branch [1] so we can confirm things look good and start the release
>>> > > process?
>>> > >
>>> > > thanks,
>>> > > Jacques
>>> > >
>>> > > [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
>>> > >
>>> > > --
>>> > > Jacques Nadeau
>>> > > CTO and Co-Founder, Dremio
>>> > >
>>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <jacques@dremio.com
>>> >
>>> > > wrote:
>>> > >
>>> > > > I think the main things are 4196 and 4291 should be completed. I
>>> know
>>> > > Amit
>>> > > > was able to reproduce 4196 locally this weekend so I think we're
>>> close
>>> > on
>>> > > > that.
>>> > > >
>>> > > > --
>>> > > > Jacques Nadeau
>>> > > > CTO and Co-Founder, Dremio
>>> > > >
>>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <zfong@maprtech.com
>>> >
>>> > > wrote:
>>> > > >
>>> > > >> Any updates on this?  What's blocking us from taking this to a
>>> vote --
>>> > > the
>>> > > >> sort merge join issues?
>>> > > >>
>>> > > >> -- Zelaine
>>> > > >>
>>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <
>>> jacques@dremio.com>
>>> > > >> wrote:
>>> > > >>
>>> > > >> > Bumping this thread...
>>> > > >> >
>>> > > >> > Here are the issues that were mentioned in this thread along
>>> with a
>>> > > >> > proposed categorization:
>>> > > >> >
>>> > > >> > Release Blockers
>>> > > >> > In-progress Amit
>>> https://issues.apache.org/jira/browse/DRILL-4190
>>> > > >> > In-progress Amit
>>> https://issues.apache.org/jira/browse/DRILL-4196
>>> > > >> > Ready to merge Jacques
>>> > > https://issues.apache.org/jira/browse/DRILL-4246
>>> > > >> > In-review Jinfeng
>>> https://issues.apache.org/jira/browse/DRILL-4256
>>> > > >> > In-progress Jacques
>>> > https://issues.apache.org/jira/browse/DRILL-4278
>>> > > >> > Ready to merge Laurent
>>> > > https://issues.apache.org/jira/browse/DRILL-4285
>>> > > >> > Nice to Have
>>> > > >> > Open Jason/Hakim
>>> https://issues.apache.org/jira/browse/DRILL-4247
>>> > > >> > In-progress Jason
>>> https://issues.apache.org/jira/browse/DRILL-4203
>>> > > >> > Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
>>> > > >> > Ready to merge Jacques
>>> > > https://issues.apache.org/jira/browse/DRILL-4131
>>> > > >> >
>>> > > >> > What do others think? Let's try to get the blockers wrapped up
>>> in
>>> > the
>>> > > >> next
>>> > > >> > day or two and start a release vote...
>>> > > >> >
>>> > > >> >
>>> > > >> >
>>> > > >> > --
>>> > > >> > Jacques Nadeau
>>> > > >> > CTO and Co-Founder, Dremio
>>> > > >> >
>>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
>>> > > >> altekrusejason@gmail.com>
>>> > > >> > wrote:
>>> > > >> >
>>> > > >> > > Hello All,
>>> > > >> > >
>>> > > >> > > With the allocator changes merged and about a month since the
>>> last
>>> > > >> > release
>>> > > >> > > I think it would be good to start a vote soon. I would like to
>>> > > >> volunteer
>>> > > >> > to
>>> > > >> > > be release manager.
>>> > > >> > >
>>> > > >> > > I know that there were some issues that were identified after
>>> the
>>> > > >> > transfer
>>> > > >> > > patch was merged. I think that these issues should be fixed
>>> before
>>> > > we
>>> > > >> > cut a
>>> > > >> > > release candidate.
>>> > > >> > >
>>> > > >> > > From looking at the associated JIRAs it looked like there was
>>> a
>>> > > >> possible
>>> > > >> > > short term fix just adjusting the max_query_memory_per_node
>>> > option,
>>> > > >> and
>>> > > >> > > some more involved work to change how we determine the correct
>>> > time
>>> > > to
>>> > > >> > > spill during external sort. I believe it makes sense to make
>>> > > external
>>> > > >> > sort
>>> > > >> > > work well with the newly improved memory accounting before
>>> > cutting a
>>> > > >> > > release, but I'm not sure how much work is left to be done
>>> there.
>>> > > [1]
>>> > > >> > >
>>> > > >> > > Please respond with your thoughts on a release soon and any
>>> JIRAs
>>> > > you
>>> > > >> > would
>>> > > >> > > like to include in the release.
>>> > > >> > >
>>> > > >> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
>>> > > >> > >
>>> > > >> > > Thanks,
>>> > > >> > > Jason
>>> > > >> > >
>>> > > >> >
>>> > > >>
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>

Re: Time for a 1.5 release?

Posted by Jacques Nadeau <ja...@dremio.com>.

Hmm... this merge caused the Apache build to fail. Investigating...

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau <ja...@dremio.com> wrote:

> I got clean regression runs as well. I've merged the patch.
>
> Jason, you want to start the release process?
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <ag...@mapr.com>
> wrote:
>
>> Had two clean Functional runs. TPC-H SF100 was also successful.
>>
>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
>> challapallirahul@gmail.com> wrote:
>>
>> > Kicked off a functional run with your branch. Will let you know once it
>> > finishes
>> >
>> > - Rahul
>> >
>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com>
>> > wrote:
>> >
>> > > 4196 was merged today. I have an updated patch for 4291 that is ready.
>> > > Unfortunately, it seems that something isn't working with our extended
>> > > tests so I haven't been able to run an extended regression. Unit tests
>> > > pass. Is someone else possibly able to run a regression suite against
>> > this
>> > > branch [1] so we can confirm things look good and start the release
>> > > process?
>> > >
>> > > thanks,
>> > > Jacques
>> > >
>> > > [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
>> > >
>> > > --
>> > > Jacques Nadeau
>> > > CTO and Co-Founder, Dremio
>> > >
>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <ja...@dremio.com>
>> > > wrote:
>> > >
>> > > > I think the main things are 4196 and 4291 should be completed. I
>> know
>> > > Amit
>> > > > was able to reproduce 4196 locally this weekend so I think we're
>> close
>> > on
>> > > > that.
>> > > >
>> > > > --
>> > > > Jacques Nadeau
>> > > > CTO and Co-Founder, Dremio
>> > > >
>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <zf...@maprtech.com>
>> > > wrote:
>> > > >
>> > > >> Any updates on this?  What's blocking us from taking this to a
>> vote --
>> > > the
>> > > >> sort merge join issues?
>> > > >>
>> > > >> -- Zelaine
>> > > >>
>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <
>> jacques@dremio.com>
>> > > >> wrote:
>> > > >>
>> > > >> > Bumping this thread...
>> > > >> >
>> > > >> > Here are the issues that were mentioned in this thread along
>> with a
>> > > >> > proposed categorization:
>> > > >> >
>> > > >> > Release Blockers
>> > > >> > In-progress Amit
>> https://issues.apache.org/jira/browse/DRILL-4190
>> > > >> > In-progress Amit
>> https://issues.apache.org/jira/browse/DRILL-4196
>> > > >> > Ready to merge Jacques
>> > > https://issues.apache.org/jira/browse/DRILL-4246
>> > > >> > In-review Jinfeng
>> https://issues.apache.org/jira/browse/DRILL-4256
>> > > >> > In-progress Jacques
>> > https://issues.apache.org/jira/browse/DRILL-4278
>> > > >> > Ready to merge Laurent
>> > > https://issues.apache.org/jira/browse/DRILL-4285
>> > > >> > Nice to Have
>> > > >> > Open Jason/Hakim
>> https://issues.apache.org/jira/browse/DRILL-4247
>> > > >> > In-progress Jason
>> https://issues.apache.org/jira/browse/DRILL-4203
>> > > >> > Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
>> > > >> > Ready to merge Jacques
>> > > https://issues.apache.org/jira/browse/DRILL-4131
>> > > >> >
>> > > >> > What do others think? Let's try to get the blockers wrapped up in
>> > the
>> > > >> next
>> > > >> > day or two and start a release vote...
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > --
>> > > >> > Jacques Nadeau
>> > > >> > CTO and Co-Founder, Dremio
>> > > >> >
>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
>> > > >> altekrusejason@gmail.com>
>> > > >> > wrote:
>> > > >> >
>> > > >> > > Hello All,
>> > > >> > >
>> > > >> > > With the allocator changes merged and about a month since the
>> last
>> > > >> > release
>> > > >> > > I think it would be good to start a vote soon. I would like to
>> > > >> volunteer
>> > > >> > to
>> > > >> > > be release manager.
>> > > >> > >
>> > > >> > > I know that there were some issues that were identified after
>> the
>> > > >> > transfer
>> > > >> > > patch was merged. I think that these issues should be fixed
>> before
>> > > we
>> > > >> > cut a
>> > > >> > > release candidate.
>> > > >> > >
>> > > >> > > From looking at the associated JIRAs it looked like there was a
>> > > >> possible
>> > > >> > > short term fix just adjusting the max_query_memory_per_node
>> > option,
>> > > >> and
>> > > >> > > some more involved work to change how we determine the correct
>> > time
>> > > to
>> > > >> > > spill during external sort. I believe it makes sense to make
>> > > external
>> > > >> > sort
>> > > >> > > work well with the newly improved memory accounting before
>> > cutting a
>> > > >> > > release, but I'm not sure how much work is left to be done
>> there.
>> > > [1]
>> > > >> > >
>> > > >> > > Please respond with your thoughts on a release soon and any
>> JIRAs
>> > > you
>> > > >> > would
>> > > >> > > like to include in the release.
>> > > >> > >
>> > > >> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
>> > > >> > >
>> > > >> > > Thanks,
>> > > >> > > Jason
>> > > >> > >
>> > > >> >
>> > > >>
>> > > >
>> > > >
>> > >
>> >
>>
>
>

Re: Time for a 1.5 release?

Posted by Jacques Nadeau <ja...@dremio.com>.

I got clean regression runs as well. I've merged the patch.

Jason, you want to start the release process?

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish <ag...@mapr.com> wrote:

> Had two clean Functional runs. TPC-H SF100 was also successful.
>
> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
> challapallirahul@gmail.com> wrote:
>
> > Kicked off a functional run with your branch. Will let you know once it
> > finishes
> >
> > - Rahul
> >
> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com>
> > wrote:
> >
> > > 4196 was merged today. I have an updated patch for 4291 that is ready.
> > > Unfortunately, it seems that something isn't working with our extended
> > > tests so I haven't been able to run an extended regression. Unit tests
> > > pass. Is someone else possibly able to run a regression suite against
> > this
> > > branch [1] so we can confirm things look good and start the release
> > > process?
> > >
> > > thanks,
> > > Jacques
> > >
> > > [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
> > >
> > > --
> > > Jacques Nadeau
> > > CTO and Co-Founder, Dremio
> > >
> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <ja...@dremio.com>
> > > wrote:
> > >
> > > > I think the main things are 4196 and 4291 should be completed. I know
> > > Amit
> > > > was able to reproduce 4196 locally this weekend so I think we're
> close
> > on
> > > > that.
> > > >
> > > > --
> > > > Jacques Nadeau
> > > > CTO and Co-Founder, Dremio
> > > >
> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <zf...@maprtech.com>
> > > wrote:
> > > >
> > > >> Any updates on this?  What's blocking us from taking this to a vote
> --
> > > the
> > > >> sort merge join issues?
> > > >>
> > > >> -- Zelaine
> > > >>
> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <jacques@dremio.com
> >
> > > >> wrote:
> > > >>
> > > >> > Bumping this thread...
> > > >> >
> > > >> > Here are the issues that were mentioned in this thread along with
> a
> > > >> > proposed categorization:
> > > >> >
> > > >> > Release Blockers
> > > >> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4190
> > > >> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4196
> > > >> > Ready to merge Jacques
> > > https://issues.apache.org/jira/browse/DRILL-4246
> > > >> > In-review Jinfeng
> https://issues.apache.org/jira/browse/DRILL-4256
> > > >> > In-progress Jacques
> > https://issues.apache.org/jira/browse/DRILL-4278
> > > >> > Ready to merge Laurent
> > > https://issues.apache.org/jira/browse/DRILL-4285
> > > >> > Nice to Have
> > > >> > Open Jason/Hakim https://issues.apache.org/jira/browse/DRILL-4247
> > > >> > In-progress Jason
> https://issues.apache.org/jira/browse/DRILL-4203
> > > >> > Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
> > > >> > Ready to merge Jacques
> > > https://issues.apache.org/jira/browse/DRILL-4131
> > > >> >
> > > >> > What do others think? Let's try to get the blockers wrapped up in
> > the
> > > >> next
> > > >> > day or two and start a release vote...
> > > >> >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > Jacques Nadeau
> > > >> > CTO and Co-Founder, Dremio
> > > >> >
> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> > > >> altekrusejason@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > Hello All,
> > > >> > >
> > > >> > > With the allocator changes merged and about a month since the
> last
> > > >> > release
> > > >> > > I think it would be good to start a vote soon. I would like to
> > > >> volunteer
> > > >> > to
> > > >> > > be release manager.
> > > >> > >
> > > >> > > I know that there were some issues that were identified after
> the
> > > >> > transfer
> > > >> > > patch was merged. I think that these issues should be fixed
> before
> > > we
> > > >> > cut a
> > > >> > > release candidate.
> > > >> > >
> > > >> > > From looking at the associated JIRAs it looked like there was a
> > > >> possible
> > > >> > > short term fix just adjusting the max_query_memory_per_node
> > option,
> > > >> and
> > > >> > > some more involved work to change how we determine the correct
> > time
> > > to
> > > >> > > spill during external sort. I believe it makes sense to make
> > > external
> > > >> > sort
> > > >> > > work well with the newly improved memory accounting before
> > cutting a
> > > >> > > release, but I'm not sure how much work is left to be done
> there.
> > > [1]
> > > >> > >
> > > >> > > Please respond with your thoughts on a release soon and any
> JIRAs
> > > you
> > > >> > would
> > > >> > > like to include in the release.
> > > >> > >
> > > >> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> > > >> > >
> > > >> > > Thanks,
> > > >> > > Jason
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Abhishek Girish <ag...@mapr.com>.

Had two clean Functional runs. TPC-H SF100 was also successful.

On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli <
challapallirahul@gmail.com> wrote:

> Kicked off a functional run with your branch. Will let you know once it
> finishes
>
> - Rahul
>
> On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com>
> wrote:
>
> > 4196 was merged today. I have an updated patch for 4291 that is ready.
> > Unfortunately, it seems that something isn't working with our extended
> > tests so I haven't been able to run an extended regression. Unit tests
> > pass. Is someone else possibly able to run a regression suite against
> this
> > branch [1] so we can confirm things look good and start the release
> > process?
> >
> > thanks,
> > Jacques
> >
> > [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <ja...@dremio.com>
> > wrote:
> >
> > > I think the main things are 4196 and 4291 should be completed. I know
> > Amit
> > > was able to reproduce 4196 locally this weekend so I think we're close
> on
> > > that.
> > >
> > > --
> > > Jacques Nadeau
> > > CTO and Co-Founder, Dremio
> > >
> > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <zf...@maprtech.com>
> > wrote:
> > >
> > >> Any updates on this?  What's blocking us from taking this to a vote --
> > the
> > >> sort merge join issues?
> > >>
> > >> -- Zelaine
> > >>
> > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <ja...@dremio.com>
> > >> wrote:
> > >>
> > >> > Bumping this thread...
> > >> >
> > >> > Here are the issues that were mentioned in this thread along with a
> > >> > proposed categorization:
> > >> >
> > >> > Release Blockers
> > >> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4190
> > >> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4196
> > >> > Ready to merge Jacques
> > https://issues.apache.org/jira/browse/DRILL-4246
> > >> > In-review Jinfeng https://issues.apache.org/jira/browse/DRILL-4256
> > >> > In-progress Jacques
> https://issues.apache.org/jira/browse/DRILL-4278
> > >> > Ready to merge Laurent
> > https://issues.apache.org/jira/browse/DRILL-4285
> > >> > Nice to Have
> > >> > Open Jason/Hakim https://issues.apache.org/jira/browse/DRILL-4247
> > >> > In-progress Jason https://issues.apache.org/jira/browse/DRILL-4203
> > >> > Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
> > >> > Ready to merge Jacques
> > https://issues.apache.org/jira/browse/DRILL-4131
> > >> >
> > >> > What do others think? Let's try to get the blockers wrapped up in
> the
> > >> next
> > >> > day or two and start a release vote...
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Jacques Nadeau
> > >> > CTO and Co-Founder, Dremio
> > >> >
> > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> > >> altekrusejason@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hello All,
> > >> > >
> > >> > > With the allocator changes merged and about a month since the last
> > >> > release
> > >> > > I think it would be good to start a vote soon. I would like to
> > >> volunteer
> > >> > to
> > >> > > be release manager.
> > >> > >
> > >> > > I know that there were some issues that were identified after the
> > >> > transfer
> > >> > > patch was merged. I think that these issues should be fixed before
> > we
> > >> > cut a
> > >> > > release candidate.
> > >> > >
> > >> > > From looking at the associated JIRAs it looked like there was a
> > >> possible
> > >> > > short term fix just adjusting the max_query_memory_per_node
> option,
> > >> and
> > >> > > some more involved work to change how we determine the correct
> time
> > to
> > >> > > spill during external sort. I believe it makes sense to make
> > external
> > >> > sort
> > >> > > work well with the newly improved memory accounting before
> cutting a
> > >> > > release, but I'm not sure how much work is left to be done there.
> > [1]
> > >> > >
> > >> > > Please respond with your thoughts on a release soon and any JIRAs
> > you
> > >> > would
> > >> > > like to include in the release.
> > >> > >
> > >> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> > >> > >
> > >> > > Thanks,
> > >> > > Jason
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Re: Time for a 1.5 release?

Posted by rahul challapalli <ch...@gmail.com>.

Kicked off a functional run with your branch. Will let you know once it
finishes

- Rahul

On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau <ja...@dremio.com> wrote:

> 4196 was merged today. I have an updated patch for 4291 that is ready.
> Unfortunately, it seems that something isn't working with our extended
> tests so I haven't been able to run an extended regression. Unit tests
> pass. Is someone else possibly able to run a regression suite against this
> branch [1] so we can confirm things look good and start the release
> process?
>
> thanks,
> Jacques
>
> [1] https://github.com/jacques-n/drill/tree/DRILL-4291v2
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <ja...@dremio.com>
> wrote:
>
> > I think the main things are 4196 and 4291 should be completed. I know
> Amit
> > was able to reproduce 4196 locally this weekend so I think we're close on
> > that.
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <zf...@maprtech.com>
> wrote:
> >
> >> Any updates on this?  What's blocking us from taking this to a vote --
> the
> >> sort merge join issues?
> >>
> >> -- Zelaine
> >>
> >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <ja...@dremio.com>
> >> wrote:
> >>
> >> > Bumping this thread...
> >> >
> >> > Here are the issues that were mentioned in this thread along with a
> >> > proposed categorization:
> >> >
> >> > Release Blockers
> >> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4190
> >> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4196
> >> > Ready to merge Jacques
> https://issues.apache.org/jira/browse/DRILL-4246
> >> > In-review Jinfeng https://issues.apache.org/jira/browse/DRILL-4256
> >> > In-progress Jacques https://issues.apache.org/jira/browse/DRILL-4278
> >> > Ready to merge Laurent
> https://issues.apache.org/jira/browse/DRILL-4285
> >> > Nice to Have
> >> > Open Jason/Hakim https://issues.apache.org/jira/browse/DRILL-4247
> >> > In-progress Jason https://issues.apache.org/jira/browse/DRILL-4203
> >> > Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
> >> > Ready to merge Jacques
> https://issues.apache.org/jira/browse/DRILL-4131
> >> >
> >> > What do others think? Let's try to get the blockers wrapped up in the
> >> next
> >> > day or two and start a release vote...
> >> >
> >> >
> >> >
> >> > --
> >> > Jacques Nadeau
> >> > CTO and Co-Founder, Dremio
> >> >
> >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> >> altekrusejason@gmail.com>
> >> > wrote:
> >> >
> >> > > Hello All,
> >> > >
> >> > > With the allocator changes merged and about a month since the last
> >> > release
> >> > > I think it would be good to start a vote soon. I would like to
> >> volunteer
> >> > to
> >> > > be release manager.
> >> > >
> >> > > I know that there were some issues that were identified after the
> >> > transfer
> >> > > patch was merged. I think that these issues should be fixed before
> we
> >> > cut a
> >> > > release candidate.
> >> > >
> >> > > From looking at the associated JIRAs it looked like there was a
> >> possible
> >> > > short term fix just adjusting the max_query_memory_per_node option,
> >> and
> >> > > some more involved work to change how we determine the correct time
> to
> >> > > spill during external sort. I believe it makes sense to make
> external
> >> > sort
> >> > > work well with the newly improved memory accounting before cutting a
> >> > > release, but I'm not sure how much work is left to be done there.
> [1]
> >> > >
> >> > > Please respond with your thoughts on a release soon and any JIRAs
> you
> >> > would
> >> > > like to include in the release.
> >> > >
> >> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> >> > >
> >> > > Thanks,
> >> > > Jason
> >> > >
> >> >
> >>
> >
> >
>

Re: Time for a 1.5 release?

Posted by Jacques Nadeau <ja...@dremio.com>.

4196 was merged today. I have an updated patch for 4291 that is ready.
Unfortunately, it seems that something isn't working with our extended
tests so I haven't been able to run an extended regression. Unit tests
pass. Is someone else possibly able to run a regression suite against this
branch [1] so we can confirm things look good and start the release process?

thanks,
Jacques

[1] https://github.com/jacques-n/drill/tree/DRILL-4291v2

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau <ja...@dremio.com> wrote:

> I think the main things are 4196 and 4291 should be completed. I know Amit
> was able to reproduce 4196 locally this weekend so I think we're close on
> that.
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <zf...@maprtech.com> wrote:
>
>> Any updates on this?  What's blocking us from taking this to a vote -- the
>> sort merge join issues?
>>
>> -- Zelaine
>>
>> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <ja...@dremio.com>
>> wrote:
>>
>> > Bumping this thread...
>> >
>> > Here are the issues that were mentioned in this thread along with a
>> > proposed categorization:
>> >
>> > Release Blockers
>> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4190
>> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4196
>> > Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4246
>> > In-review Jinfeng https://issues.apache.org/jira/browse/DRILL-4256
>> > In-progress Jacques https://issues.apache.org/jira/browse/DRILL-4278
>> > Ready to merge Laurent https://issues.apache.org/jira/browse/DRILL-4285
>> > Nice to Have
>> > Open Jason/Hakim https://issues.apache.org/jira/browse/DRILL-4247
>> > In-progress Jason https://issues.apache.org/jira/browse/DRILL-4203
>> > Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
>> > Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4131
>> >
>> > What do others think? Let's try to get the blockers wrapped up in the
>> next
>> > day or two and start a release vote...
>> >
>> >
>> >
>> > --
>> > Jacques Nadeau
>> > CTO and Co-Founder, Dremio
>> >
>> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
>> altekrusejason@gmail.com>
>> > wrote:
>> >
>> > > Hello All,
>> > >
>> > > With the allocator changes merged and about a month since the last
>> > release
>> > > I think it would be good to start a vote soon. I would like to
>> volunteer
>> > to
>> > > be release manager.
>> > >
>> > > I know that there were some issues that were identified after the
>> > transfer
>> > > patch was merged. I think that these issues should be fixed before we
>> > cut a
>> > > release candidate.
>> > >
>> > > From looking at the associated JIRAs it looked like there was a
>> possible
>> > > short term fix just adjusting the max_query_memory_per_node option,
>> and
>> > > some more involved work to change how we determine the correct time to
>> > > spill during external sort. I believe it makes sense to make external
>> > sort
>> > > work well with the newly improved memory accounting before cutting a
>> > > release, but I'm not sure how much work is left to be done there. [1]
>> > >
>> > > Please respond with your thoughts on a release soon and any JIRAs you
>> > would
>> > > like to include in the release.
>> > >
>> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
>> > >
>> > > Thanks,
>> > > Jason
>> > >
>> >
>>
>
>

Re: Time for a 1.5 release?

Posted by Jacques Nadeau <ja...@dremio.com>.

I think the main things are 4196 and 4291 should be completed. I know Amit
was able to reproduce 4196 locally this weekend so I think we're close on
that.

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong <zf...@maprtech.com> wrote:

> Any updates on this?  What's blocking us from taking this to a vote -- the
> sort merge join issues?
>
> -- Zelaine
>
> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <ja...@dremio.com>
> wrote:
>
> > Bumping this thread...
> >
> > Here are the issues that were mentioned in this thread along with a
> > proposed categorization:
> >
> > Release Blockers
> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4190
> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4196
> > Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4246
> > In-review Jinfeng https://issues.apache.org/jira/browse/DRILL-4256
> > In-progress Jacques https://issues.apache.org/jira/browse/DRILL-4278
> > Ready to merge Laurent https://issues.apache.org/jira/browse/DRILL-4285
> > Nice to Have
> > Open Jason/Hakim https://issues.apache.org/jira/browse/DRILL-4247
> > In-progress Jason https://issues.apache.org/jira/browse/DRILL-4203
> > Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
> > Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4131
> >
> > What do others think? Let's try to get the blockers wrapped up in the
> next
> > day or two and start a release vote...
> >
> >
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> altekrusejason@gmail.com>
> > wrote:
> >
> > > Hello All,
> > >
> > > With the allocator changes merged and about a month since the last
> > release
> > > I think it would be good to start a vote soon. I would like to
> volunteer
> > to
> > > be release manager.
> > >
> > > I know that there were some issues that were identified after the
> > transfer
> > > patch was merged. I think that these issues should be fixed before we
> > cut a
> > > release candidate.
> > >
> > > From looking at the associated JIRAs it looked like there was a
> possible
> > > short term fix just adjusting the max_query_memory_per_node option, and
> > > some more involved work to change how we determine the correct time to
> > > spill during external sort. I believe it makes sense to make external
> > sort
> > > work well with the newly improved memory accounting before cutting a
> > > release, but I'm not sure how much work is left to be done there. [1]
> > >
> > > Please respond with your thoughts on a release soon and any JIRAs you
> > would
> > > like to include in the release.
> > >
> > > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> > >
> > > Thanks,
> > > Jason
> > >
> >
>

Re: Time for a 1.5 release?

Posted by Zelaine Fong <zf...@maprtech.com>.

Any updates on this?  What's blocking us from taking this to a vote -- the
sort merge join issues?

-- Zelaine

On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <ja...@dremio.com> wrote:

> Bumping this thread...
>
> Here are the issues that were mentioned in this thread along with a
> proposed categorization:
>
> Release Blockers
> In-progress Amit https://issues.apache.org/jira/browse/DRILL-4190
> In-progress Amit https://issues.apache.org/jira/browse/DRILL-4196
> Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4246
> In-review Jinfeng https://issues.apache.org/jira/browse/DRILL-4256
> In-progress Jacques https://issues.apache.org/jira/browse/DRILL-4278
> Ready to merge Laurent https://issues.apache.org/jira/browse/DRILL-4285
> Nice to Have
> Open Jason/Hakim https://issues.apache.org/jira/browse/DRILL-4247
> In-progress Jason https://issues.apache.org/jira/browse/DRILL-4203
> Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
> Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4131
>
> What do others think? Let's try to get the blockers wrapped up in the next
> day or two and start a release vote...
>
>
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <al...@gmail.com>
> wrote:
>
> > Hello All,
> >
> > With the allocator changes merged and about a month since the last
> release
> > I think it would be good to start a vote soon. I would like to volunteer
> to
> > be release manager.
> >
> > I know that there were some issues that were identified after the
> transfer
> > patch was merged. I think that these issues should be fixed before we
> cut a
> > release candidate.
> >
> > From looking at the associated JIRAs it looked like there was a possible
> > short term fix just adjusting the max_query_memory_per_node option, and
> > some more involved work to change how we determine the correct time to
> > spill during external sort. I believe it makes sense to make external
> sort
> > work well with the newly improved memory accounting before cutting a
> > release, but I'm not sure how much work is left to be done there. [1]
> >
> > Please respond with your thoughts on a release soon and any JIRAs you
> would
> > like to include in the release.
> >
> > [1] - https://issues.apache.org/jira/browse/DRILL-4243
> >
> > Thanks,
> > Jason
> >
>

Re: Time for a 1.5 release?

Posted by Steven Phillips <st...@dremio.com>.

I merged a patch yesterday that I believe addresses that issue. Can you see
if you still hit it?

On Thu, Jan 21, 2016 at 8:39 AM, Jacques Nadeau <ja...@dremio.com> wrote:

> Jinfeng, can you open a jira for the failing test if one isn't open?
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Wed, Jan 20, 2016 at 7:57 AM, Jinfeng Ni <ji...@gmail.com> wrote:
>
> > I still saw mvn full build failed on linux due to the following unit
> > test case sometime:
> >
> > Tests in error:
> >   TestTopNSchemaChanges.testMissingColumn:206 »  at position 0 column
> > '`kl1`' mi...
> >
> > Either we comment out that unit test, or we should fix the test case.
> > Otherwise, people may see maven build failure from the 1.5.0 release.
> >
> >
> >
> > On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <ja...@dremio.com>
> > wrote:
> > > Bumping this thread...
> > >
> > > Here are the issues that were mentioned in this thread along with a
> > > proposed categorization:
> > >
> > > Release Blockers
> > > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4190
> > > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4196
> > > Ready to merge Jacques
> https://issues.apache.org/jira/browse/DRILL-4246
> > > In-review Jinfeng https://issues.apache.org/jira/browse/DRILL-4256
> > > In-progress Jacques https://issues.apache.org/jira/browse/DRILL-4278
> > > Ready to merge Laurent
> https://issues.apache.org/jira/browse/DRILL-4285
> > > Nice to Have
> > > Open Jason/Hakim https://issues.apache.org/jira/browse/DRILL-4247
> > > In-progress Jason https://issues.apache.org/jira/browse/DRILL-4203
> > > Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
> > > Ready to merge Jacques
> https://issues.apache.org/jira/browse/DRILL-4131
> > >
> > > What do others think? Let's try to get the blockers wrapped up in the
> > next
> > > day or two and start a release vote...
> > >
> > >
> > >
> > > --
> > > Jacques Nadeau
> > > CTO and Co-Founder, Dremio
> > >
> > > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> > altekrusejason@gmail.com>
> > > wrote:
> > >
> > >> Hello All,
> > >>
> > >> With the allocator changes merged and about a month since the last
> > release
> > >> I think it would be good to start a vote soon. I would like to
> > volunteer to
> > >> be release manager.
> > >>
> > >> I know that there were some issues that were identified after the
> > transfer
> > >> patch was merged. I think that these issues should be fixed before we
> > cut a
> > >> release candidate.
> > >>
> > >> From looking at the associated JIRAs it looked like there was a
> possible
> > >> short term fix just adjusting the max_query_memory_per_node option,
> and
> > >> some more involved work to change how we determine the correct time to
> > >> spill during external sort. I believe it makes sense to make external
> > sort
> > >> work well with the newly improved memory accounting before cutting a
> > >> release, but I'm not sure how much work is left to be done there. [1]
> > >>
> > >> Please respond with your thoughts on a release soon and any JIRAs you
> > would
> > >> like to include in the release.
> > >>
> > >> [1] - https://issues.apache.org/jira/browse/DRILL-4243
> > >>
> > >> Thanks,
> > >> Jason
> > >>
> >
>

Re: Time for a 1.5 release?

Posted by Jinfeng Ni <ji...@gmail.com>.

Looks like there was a patch merged in yesterday, and it modified the
testcase [1].

While looking at the previous test, seems the expected results are not
right. (order by "DESC" should use NULL FIRST policy). I'm surprised
how it would happen with such random failure behavior.


[1] https://github.com/apache/drill/commit/34bf45f13c1fd23628ef39fb510df2b4f5750b3b#diff-f28bc46158476d9d13952888e777eca3L197

On Thu, Jan 21, 2016 at 8:39 AM, Jacques Nadeau <ja...@dremio.com> wrote:
> Jinfeng, can you open a jira for the failing test if one isn't open?
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Wed, Jan 20, 2016 at 7:57 AM, Jinfeng Ni <ji...@gmail.com> wrote:
>
>> I still saw mvn full build failed on linux due to the following unit
>> test case sometime:
>>
>> Tests in error:
>>   TestTopNSchemaChanges.testMissingColumn:206 »  at position 0 column
>> '`kl1`' mi...
>>
>> Either we comment out that unit test, or we should fix the test case.
>> Otherwise, people may see maven build failure from the 1.5.0 release.
>>
>>
>>
>> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <ja...@dremio.com>
>> wrote:
>> > Bumping this thread...
>> >
>> > Here are the issues that were mentioned in this thread along with a
>> > proposed categorization:
>> >
>> > Release Blockers
>> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4190
>> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4196
>> > Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4246
>> > In-review Jinfeng https://issues.apache.org/jira/browse/DRILL-4256
>> > In-progress Jacques https://issues.apache.org/jira/browse/DRILL-4278
>> > Ready to merge Laurent https://issues.apache.org/jira/browse/DRILL-4285
>> > Nice to Have
>> > Open Jason/Hakim https://issues.apache.org/jira/browse/DRILL-4247
>> > In-progress Jason https://issues.apache.org/jira/browse/DRILL-4203
>> > Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
>> > Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4131
>> >
>> > What do others think? Let's try to get the blockers wrapped up in the
>> next
>> > day or two and start a release vote...
>> >
>> >
>> >
>> > --
>> > Jacques Nadeau
>> > CTO and Co-Founder, Dremio
>> >
>> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
>> altekrusejason@gmail.com>
>> > wrote:
>> >
>> >> Hello All,
>> >>
>> >> With the allocator changes merged and about a month since the last
>> release
>> >> I think it would be good to start a vote soon. I would like to
>> volunteer to
>> >> be release manager.
>> >>
>> >> I know that there were some issues that were identified after the
>> transfer
>> >> patch was merged. I think that these issues should be fixed before we
>> cut a
>> >> release candidate.
>> >>
>> >> From looking at the associated JIRAs it looked like there was a possible
>> >> short term fix just adjusting the max_query_memory_per_node option, and
>> >> some more involved work to change how we determine the correct time to
>> >> spill during external sort. I believe it makes sense to make external
>> sort
>> >> work well with the newly improved memory accounting before cutting a
>> >> release, but I'm not sure how much work is left to be done there. [1]
>> >>
>> >> Please respond with your thoughts on a release soon and any JIRAs you
>> would
>> >> like to include in the release.
>> >>
>> >> [1] - https://issues.apache.org/jira/browse/DRILL-4243
>> >>
>> >> Thanks,
>> >> Jason
>> >>
>>

Re: Time for a 1.5 release?

Posted by Jacques Nadeau <ja...@dremio.com>.

Jinfeng, can you open a jira for the failing test if one isn't open?

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Wed, Jan 20, 2016 at 7:57 AM, Jinfeng Ni <ji...@gmail.com> wrote:

> I still saw mvn full build failed on linux due to the following unit
> test case sometime:
>
> Tests in error:
>   TestTopNSchemaChanges.testMissingColumn:206 »  at position 0 column
> '`kl1`' mi...
>
> Either we comment out that unit test, or we should fix the test case.
> Otherwise, people may see maven build failure from the 1.5.0 release.
>
>
>
> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <ja...@dremio.com>
> wrote:
> > Bumping this thread...
> >
> > Here are the issues that were mentioned in this thread along with a
> > proposed categorization:
> >
> > Release Blockers
> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4190
> > In-progress Amit https://issues.apache.org/jira/browse/DRILL-4196
> > Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4246
> > In-review Jinfeng https://issues.apache.org/jira/browse/DRILL-4256
> > In-progress Jacques https://issues.apache.org/jira/browse/DRILL-4278
> > Ready to merge Laurent https://issues.apache.org/jira/browse/DRILL-4285
> > Nice to Have
> > Open Jason/Hakim https://issues.apache.org/jira/browse/DRILL-4247
> > In-progress Jason https://issues.apache.org/jira/browse/DRILL-4203
> > Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
> > Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4131
> >
> > What do others think? Let's try to get the blockers wrapped up in the
> next
> > day or two and start a release vote...
> >
> >
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <
> altekrusejason@gmail.com>
> > wrote:
> >
> >> Hello All,
> >>
> >> With the allocator changes merged and about a month since the last
> release
> >> I think it would be good to start a vote soon. I would like to
> volunteer to
> >> be release manager.
> >>
> >> I know that there were some issues that were identified after the
> transfer
> >> patch was merged. I think that these issues should be fixed before we
> cut a
> >> release candidate.
> >>
> >> From looking at the associated JIRAs it looked like there was a possible
> >> short term fix just adjusting the max_query_memory_per_node option, and
> >> some more involved work to change how we determine the correct time to
> >> spill during external sort. I believe it makes sense to make external
> sort
> >> work well with the newly improved memory accounting before cutting a
> >> release, but I'm not sure how much work is left to be done there. [1]
> >>
> >> Please respond with your thoughts on a release soon and any JIRAs you
> would
> >> like to include in the release.
> >>
> >> [1] - https://issues.apache.org/jira/browse/DRILL-4243
> >>
> >> Thanks,
> >> Jason
> >>
>

Re: Time for a 1.5 release?

Posted by Jinfeng Ni <ji...@gmail.com>.

I still saw mvn full build failed on linux due to the following unit
test case sometime:

Tests in error:
  TestTopNSchemaChanges.testMissingColumn:206 »  at position 0 column
'`kl1`' mi...

Either we comment out that unit test, or we should fix the test case.
Otherwise, people may see maven build failure from the 1.5.0 release.



On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau <ja...@dremio.com> wrote:
> Bumping this thread...
>
> Here are the issues that were mentioned in this thread along with a
> proposed categorization:
>
> Release Blockers
> In-progress Amit https://issues.apache.org/jira/browse/DRILL-4190
> In-progress Amit https://issues.apache.org/jira/browse/DRILL-4196
> Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4246
> In-review Jinfeng https://issues.apache.org/jira/browse/DRILL-4256
> In-progress Jacques https://issues.apache.org/jira/browse/DRILL-4278
> Ready to merge Laurent https://issues.apache.org/jira/browse/DRILL-4285
> Nice to Have
> Open Jason/Hakim https://issues.apache.org/jira/browse/DRILL-4247
> In-progress Jason https://issues.apache.org/jira/browse/DRILL-4203
> Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
> Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4131
>
> What do others think? Let's try to get the blockers wrapped up in the next
> day or two and start a release vote...
>
>
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <al...@gmail.com>
> wrote:
>
>> Hello All,
>>
>> With the allocator changes merged and about a month since the last release
>> I think it would be good to start a vote soon. I would like to volunteer to
>> be release manager.
>>
>> I know that there were some issues that were identified after the transfer
>> patch was merged. I think that these issues should be fixed before we cut a
>> release candidate.
>>
>> From looking at the associated JIRAs it looked like there was a possible
>> short term fix just adjusting the max_query_memory_per_node option, and
>> some more involved work to change how we determine the correct time to
>> spill during external sort. I believe it makes sense to make external sort
>> work well with the newly improved memory accounting before cutting a
>> release, but I'm not sure how much work is left to be done there. [1]
>>
>> Please respond with your thoughts on a release soon and any JIRAs you would
>> like to include in the release.
>>
>> [1] - https://issues.apache.org/jira/browse/DRILL-4243
>>
>> Thanks,
>> Jason
>>

Re: Time for a 1.5 release?

Posted by Jacques Nadeau <ja...@dremio.com>.

Bumping this thread...

Here are the issues that were mentioned in this thread along with a
proposed categorization:

Release Blockers
In-progress Amit https://issues.apache.org/jira/browse/DRILL-4190
In-progress Amit https://issues.apache.org/jira/browse/DRILL-4196
Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4246
In-review Jinfeng https://issues.apache.org/jira/browse/DRILL-4256
In-progress Jacques https://issues.apache.org/jira/browse/DRILL-4278
Ready to merge Laurent https://issues.apache.org/jira/browse/DRILL-4285
Nice to Have
Open Jason/Hakim https://issues.apache.org/jira/browse/DRILL-4247
In-progress Jason https://issues.apache.org/jira/browse/DRILL-4203
Open Jacques https://issues.apache.org/jira/browse/DRILL-4266
Ready to merge Jacques https://issues.apache.org/jira/browse/DRILL-4131

What do others think? Let's try to get the blockers wrapped up in the next
day or two and start a release vote...



--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Mon, Jan 4, 2016 at 1:48 PM, Jason Altekruse <al...@gmail.com>
wrote:

> Hello All,
>
> With the allocator changes merged and about a month since the last release
> I think it would be good to start a vote soon. I would like to volunteer to
> be release manager.
>
> I know that there were some issues that were identified after the transfer
> patch was merged. I think that these issues should be fixed before we cut a
> release candidate.
>
> From looking at the associated JIRAs it looked like there was a possible
> short term fix just adjusting the max_query_memory_per_node option, and
> some more involved work to change how we determine the correct time to
> spill during external sort. I believe it makes sense to make external sort
> work well with the newly improved memory accounting before cutting a
> release, but I'm not sure how much work is left to be done there. [1]
>
> Please respond with your thoughts on a release soon and any JIRAs you would
> like to include in the release.
>
> [1] - https://issues.apache.org/jira/browse/DRILL-4243
>
> Thanks,
> Jason
>