You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Kevin Risden <kr...@apache.org> on 2018/06/21 19:24:36 UTC
Calcite - RelMetadataTest.testPullUpPredicatesForExprsItr - 20 second timeout
I have been hitting the RelMetadataTest.testPullUpPredicatesForExprsItr 20
second timeout more often recently and Apache build Jenkins has hit it
recently as well [1,2,3]. In one of the recent partial successes [4], the
RelMetadataTest suite takes just under 20 seconds at 19.45 seconds.
[INFO] Running org.apache.calcite.test.RelMetadataTest
[WARNING] Tests run: 140, Failures: 0, Errors: 0, Skipped: 6, Time
elapsed: 19.45 s - in org.apache.calcite.test.RelMetadataTest
I don't know much about what this test is trying to do. Is there something
that could be causing this test to run longer?
[1] https://builds.apache.org/job/Calcite-Master/306/
[2] https://builds.apache.org/job/Calcite-Master/307/
[3] https://builds.apache.org/job/Calcite-Master/308/
[4]
https://builds.apache.org/job/Calcite-Master/306/jdk=JDK%201.8%20(latest),label_exp=ubuntu&&!cloud-slave/consoleText
Kevin Risden
Re: Calcite - RelMetadataTest.testPullUpPredicatesForExprsItr - 20
second timeout
Posted by Julian Hyde <jh...@apache.org>.
Thanks Michael. I have logged https://issues.apache.org/jira/browse/CALCITE-2384 <https://issues.apache.org/jira/browse/CALCITE-2384>.
> On Jun 26, 2018, at 10:53 AM, Michael Mior <mm...@apache.org> wrote:
>
> I had some time to dig into this today. It looks like it was this commit
> which started slowing things down
>
> https://github.com/apache/calcite/commit/c3788056507206c50b8c700ab81abd4b511bb863
>
> I'm not too familiar with this part of the code, but if anyone has
> suggestions on improving, that would be great :)
>
> --
> Michael Mior
> mmior@apache.org
>
>
>
> Le jeu. 21 juin 2018 à 19:43, Julian Hyde <jh...@apache.org> a écrit :
>
>> I noticed this test slow down about a month ago. I hit the timeouts but I
>> assumed at the time that I was running on an underpowered/overtaxed machine.
>>
>> Can you log a JIRA case please, Kevin?
>>
>> It would be useful if someone could do some experiments to find out
>> whether this test was faster 6 months ago, and if so, which git commit
>> since then caused the slowdown.
>>
>> Julian
>>
>>
>>> On Jun 21, 2018, at 12:24 PM, Kevin Risden <kr...@apache.org> wrote:
>>>
>>> I have been hitting the RelMetadataTest.testPullUpPredicatesForExprsItr
>> 20
>>> second timeout more often recently and Apache build Jenkins has hit it
>>> recently as well [1,2,3]. In one of the recent partial successes [4], the
>>> RelMetadataTest suite takes just under 20 seconds at 19.45 seconds.
>>>
>>> [INFO] Running org.apache.calcite.test.RelMetadataTest
>>> [WARNING] Tests run: 140, Failures: 0, Errors: 0, Skipped: 6, Time
>>> elapsed: 19.45 s - in org.apache.calcite.test.RelMetadataTest
>>>
>>> I don't know much about what this test is trying to do. Is there
>> something
>>> that could be causing this test to run longer?
>>>
>>> [1] https://builds.apache.org/job/Calcite-Master/306/
>>> [2] https://builds.apache.org/job/Calcite-Master/307/
>>> [3] https://builds.apache.org/job/Calcite-Master/308/
>>> [4]
>>>
>> https://builds.apache.org/job/Calcite-Master/306/jdk=JDK%201.8%20(latest),label_exp=ubuntu&&!cloud-slave/consoleText
>>>
>>> Kevin Risden
>>
>>
Re: Calcite - RelMetadataTest.testPullUpPredicatesForExprsItr - 20 second timeout
Posted by Zoltan Haindrich <ki...@rxd.hu>.
Hello,
Yeah, I've also taken a look but it doesn't seemed connected - but after a while I've started suspecting that my I had too many back noise...
I instead went the other way; and looked at it with a profiler; and it seems to me that there are parts of an old 2-3 year old logic; which seems to be redundant with the current rexsimplify workings....I'm still confirming this..sorry for writing just now...I wanted to finish with investigation first...
cheers,
Zoltan
On 26 June 2018 19:53:57 CEST, Michael Mior <mm...@apache.org> wrote:
>I had some time to dig into this today. It looks like it was this
>commit
>which started slowing things down
>
>https://github.com/apache/calcite/commit/c3788056507206c50b8c700ab81abd4b511bb863
>
>I'm not too familiar with this part of the code, but if anyone has
>suggestions on improving, that would be great :)
>
>--
>Michael Mior
>mmior@apache.org
>
>
>
>Le jeu. 21 juin 2018 à 19:43, Julian Hyde <jh...@apache.org> a écrit :
>
>> I noticed this test slow down about a month ago. I hit the timeouts
>but I
>> assumed at the time that I was running on an underpowered/overtaxed
>machine.
>>
>> Can you log a JIRA case please, Kevin?
>>
>> It would be useful if someone could do some experiments to find out
>> whether this test was faster 6 months ago, and if so, which git
>commit
>> since then caused the slowdown.
>>
>> Julian
>>
>>
>> > On Jun 21, 2018, at 12:24 PM, Kevin Risden <kr...@apache.org>
>wrote:
>> >
>> > I have been hitting the
>RelMetadataTest.testPullUpPredicatesForExprsItr
>> 20
>> > second timeout more often recently and Apache build Jenkins has hit
>it
>> > recently as well [1,2,3]. In one of the recent partial successes
>[4], the
>> > RelMetadataTest suite takes just under 20 seconds at 19.45 seconds.
>> >
>> > [INFO] Running org.apache.calcite.test.RelMetadataTest
>> > [WARNING] Tests run: 140, Failures: 0, Errors: 0, Skipped: 6, Time
>> > elapsed: 19.45 s - in org.apache.calcite.test.RelMetadataTest
>> >
>> > I don't know much about what this test is trying to do. Is there
>> something
>> > that could be causing this test to run longer?
>> >
>> > [1] https://builds.apache.org/job/Calcite-Master/306/
>> > [2] https://builds.apache.org/job/Calcite-Master/307/
>> > [3] https://builds.apache.org/job/Calcite-Master/308/
>> > [4]
>> >
>>
>https://builds.apache.org/job/Calcite-Master/306/jdk=JDK%201.8%20(latest),label_exp=ubuntu&&!cloud-slave/consoleText
>> >
>> > Kevin Risden
>>
>>
Re: Calcite - RelMetadataTest.testPullUpPredicatesForExprsItr - 20
second timeout
Posted by Michael Mior <mm...@apache.org>.
I had some time to dig into this today. It looks like it was this commit
which started slowing things down
https://github.com/apache/calcite/commit/c3788056507206c50b8c700ab81abd4b511bb863
I'm not too familiar with this part of the code, but if anyone has
suggestions on improving, that would be great :)
--
Michael Mior
mmior@apache.org
Le jeu. 21 juin 2018 à 19:43, Julian Hyde <jh...@apache.org> a écrit :
> I noticed this test slow down about a month ago. I hit the timeouts but I
> assumed at the time that I was running on an underpowered/overtaxed machine.
>
> Can you log a JIRA case please, Kevin?
>
> It would be useful if someone could do some experiments to find out
> whether this test was faster 6 months ago, and if so, which git commit
> since then caused the slowdown.
>
> Julian
>
>
> > On Jun 21, 2018, at 12:24 PM, Kevin Risden <kr...@apache.org> wrote:
> >
> > I have been hitting the RelMetadataTest.testPullUpPredicatesForExprsItr
> 20
> > second timeout more often recently and Apache build Jenkins has hit it
> > recently as well [1,2,3]. In one of the recent partial successes [4], the
> > RelMetadataTest suite takes just under 20 seconds at 19.45 seconds.
> >
> > [INFO] Running org.apache.calcite.test.RelMetadataTest
> > [WARNING] Tests run: 140, Failures: 0, Errors: 0, Skipped: 6, Time
> > elapsed: 19.45 s - in org.apache.calcite.test.RelMetadataTest
> >
> > I don't know much about what this test is trying to do. Is there
> something
> > that could be causing this test to run longer?
> >
> > [1] https://builds.apache.org/job/Calcite-Master/306/
> > [2] https://builds.apache.org/job/Calcite-Master/307/
> > [3] https://builds.apache.org/job/Calcite-Master/308/
> > [4]
> >
> https://builds.apache.org/job/Calcite-Master/306/jdk=JDK%201.8%20(latest),label_exp=ubuntu&&!cloud-slave/consoleText
> >
> > Kevin Risden
>
>
Re: Calcite - RelMetadataTest.testPullUpPredicatesForExprsItr - 20
second timeout
Posted by Julian Hyde <jh...@apache.org>.
I noticed this test slow down about a month ago. I hit the timeouts but I assumed at the time that I was running on an underpowered/overtaxed machine.
Can you log a JIRA case please, Kevin?
It would be useful if someone could do some experiments to find out whether this test was faster 6 months ago, and if so, which git commit since then caused the slowdown.
Julian
> On Jun 21, 2018, at 12:24 PM, Kevin Risden <kr...@apache.org> wrote:
>
> I have been hitting the RelMetadataTest.testPullUpPredicatesForExprsItr 20
> second timeout more often recently and Apache build Jenkins has hit it
> recently as well [1,2,3]. In one of the recent partial successes [4], the
> RelMetadataTest suite takes just under 20 seconds at 19.45 seconds.
>
> [INFO] Running org.apache.calcite.test.RelMetadataTest
> [WARNING] Tests run: 140, Failures: 0, Errors: 0, Skipped: 6, Time
> elapsed: 19.45 s - in org.apache.calcite.test.RelMetadataTest
>
> I don't know much about what this test is trying to do. Is there something
> that could be causing this test to run longer?
>
> [1] https://builds.apache.org/job/Calcite-Master/306/
> [2] https://builds.apache.org/job/Calcite-Master/307/
> [3] https://builds.apache.org/job/Calcite-Master/308/
> [4]
> https://builds.apache.org/job/Calcite-Master/306/jdk=JDK%201.8%20(latest),label_exp=ubuntu&&!cloud-slave/consoleText
>
> Kevin Risden