You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2019/04/25 18:54:05 UTC
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2604
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2604/display/redirect?page=changes>
Changes:
[mxm] [BEAM-3863] Ensure correct firing of processing time timers
------------------------------------------
[...truncated 287.23 KB...]
0014 *** not run ***
==========================================================================================
2019-04-25T18:52:36.477Z Generating 100000 events in batch mode
Apr 25, 2019 6:52:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 25, 2019 6:52:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 25, 2019 6:52:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-25T18:52:36.818Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-25T18:52:36.819Z skipping since configuration is not implemented
2019-04-25T18:52:36.819Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-25T18:52:17.584Z and ran for PT19.234S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
Exception in thread "main" 0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.8 14762.3 92000
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
0001 3.4 29052.9 92000
0002 1.6 64102.6 351
0003 3.0 32840.7 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
2019-04-25T18:52:36.821Z Generating 100000 events in batch mode
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 25, 2019 6:52:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 25, 2019 6:52:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 25, 2019 6:52:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-25T18:53:52.334Z Waiting for main pipeline to 'finish'
2019-04-25T18:53:52.334Z DONE SqlQuery7
2019-04-25T18:53:52.335Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-25T18:53:52.335Z skipping since configuration is not implemented
2019-04-25T18:53:52.335Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-25T18:53:52.335Z skipping since configuration is not implemented
2019-04-25T18:53:52.335Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-25T18:53:52.336Z skipping since configuration is not implemented
2019-04-25T18:53:52.336Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-25T18:53:52.336Z skipping since configuration is not implemented
2019-04-25T18:53:52.336Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-25T18:53:52.336Z skipping since configuration is not implemented
2019-04-25T18:53:52.336Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-25T18:53:52.338Z Generating 100000 events in batch mode
Apr 25, 2019 6:53:52 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 25, 2019 6:53:52 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 25, 2019 6:53:52 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-25T18:54:03.004Z Waiting for main pipeline to 'finish'
2019-04-25T18:54:03.004Z DONE SqlBoundedSideInputJoin
2019-04-25T18:54:03.004Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-25T18:54:03.005Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 59s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/dbv3tjroxbkgi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PostCommit_Java_Nexmark_Direct #2646
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2646/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2645
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2645/display/redirect?page=changes>
Changes:
[relax] Ensure that all nested schemas are given ids, and fix bug where nullable
------------------------------------------
[...truncated 286.36 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T21:31:53.491Z Generating 100000 events in batch mode
Apr 29, 2019 9:31:53 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 9:31:53 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 9:31:53 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T21:31:53.828Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T21:31:53.829Z skipping since configuration is not implemented
2019-04-29T21:31:53.829Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T21:31:33.173Z and ran for PT20.656S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
Exception in thread "main" 0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 7.2 13964.5 92000
0001 3.8 26171.2 92000
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0002 1.7 58207.2 351
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0003 3.1 31786.4 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
2019-04-29T21:31:53.831Z Generating 100000 events in batch mode
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 9:31:53 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 9:31:53 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 9:31:53 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T21:33:10.843Z Waiting for main pipeline to 'finish'
2019-04-29T21:33:10.843Z DONE SqlQuery7
2019-04-29T21:33:10.844Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T21:33:10.844Z skipping since configuration is not implemented
2019-04-29T21:33:10.844Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T21:33:10.844Z skipping since configuration is not implemented
2019-04-29T21:33:10.845Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T21:33:10.845Z skipping since configuration is not implemented
2019-04-29T21:33:10.845Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T21:33:10.845Z skipping since configuration is not implemented
2019-04-29T21:33:10.845Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T21:33:10.846Z skipping since configuration is not implemented
2019-04-29T21:33:10.846Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T21:33:10.849Z Generating 100000 events in batch mode
Apr 29, 2019 9:33:10 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 9:33:10 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 9:33:10 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T21:33:22.173Z Waiting for main pipeline to 'finish'
2019-04-29T21:33:22.173Z DONE SqlBoundedSideInputJoin
2019-04-29T21:33:22.173Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T21:33:22.174Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 52s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/gn4n6yra7fb6i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2644
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2644/display/redirect?page=changes>
Changes:
[chamikara] Updates Dataflow runner to support external ParDos.
------------------------------------------
[...truncated 286.42 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T20:31:22.605Z Generating 100000 events in batch mode
Apr 29, 2019 8:31:22 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 8:31:22 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 8:31:22 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T20:31:22.900Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T20:31:22.900Z skipping since configuration is not implemented
2019-04-29T20:31:22.900Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T20:31:03.740Z and ran for PT19.160S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIException in thread "main" N; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.5 15394.1 92000
0001 3.8 25994.3 92000
0002 1.7 58548.0 351
0003 2.8 35816.6 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0008 *** not run ***
0009 *** not run ***
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0010 *** not run ***
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
2019-04-29T20:31:22.902Z Generating 100000 events in batch mode
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 8:31:22 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 8:31:22 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 8:31:22 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T20:32:37.663Z Waiting for main pipeline to 'finish'
2019-04-29T20:32:37.663Z DONE SqlQuery7
2019-04-29T20:32:37.664Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T20:32:37.664Z skipping since configuration is not implemented
2019-04-29T20:32:37.664Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T20:32:37.665Z skipping since configuration is not implemented
2019-04-29T20:32:37.665Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T20:32:37.665Z skipping since configuration is not implemented
2019-04-29T20:32:37.665Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T20:32:37.665Z skipping since configuration is not implemented
2019-04-29T20:32:37.665Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T20:32:37.665Z skipping since configuration is not implemented
2019-04-29T20:32:37.665Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T20:32:37.667Z Generating 100000 events in batch mode
Apr 29, 2019 8:32:37 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 8:32:37 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 8:32:37 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T20:32:48.608Z Waiting for main pipeline to 'finish'
2019-04-29T20:32:48.608Z DONE SqlBoundedSideInputJoin
2019-04-29T20:32:48.608Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T20:32:48.608Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 48s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/xk3qhz7aczkvi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2643
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2643/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-6526] Add ReadFiles and ParseFiles transforms for AvroIO
------------------------------------------
[...truncated 289.42 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T20:13:39.305Z Generating 100000 events in batch mode
Apr 29, 2019 8:13:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 8:13:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 8:13:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T20:13:39.615Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T20:13:39.615Z skipping since configuration is not implemented
2019-04-29T20:13:39.615Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T20:13:20.802Z and ran for PT18.813S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Exception in thread "main" Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0002 query:SELECTION; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0014 query:SESSION_SIDE_INPUT_JOI at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
N; exportSummaryToBigQuery:true
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
Performance:
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
0000 6.5 15309.2 92000
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
0001 3.3 29994.0 92000
0002 1.6 61804.7 351
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
0003 3.1 32594.5 580
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
0004 *** not run ***
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
0005 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
0009 *** not run ***
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
0010 *** not run ***
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
0011 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
0012 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
0013 *** not run ***
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
0014 *** not run ***
at java.util.Iterator.forEachRemaining(Iterator.java:116)
==========================================================================================
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
2019-04-29T20:13:39.617Z Generating 100000 events in batch mode
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 8:13:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 8:13:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 8:13:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T20:14:55.218Z Waiting for main pipeline to 'finish'
2019-04-29T20:14:55.218Z DONE SqlQuery7
2019-04-29T20:14:55.218Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T20:14:55.219Z skipping since configuration is not implemented
2019-04-29T20:14:55.219Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T20:14:55.219Z skipping since configuration is not implemented
2019-04-29T20:14:55.219Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T20:14:55.219Z skipping since configuration is not implemented
2019-04-29T20:14:55.219Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T20:14:55.219Z skipping since configuration is not implemented
2019-04-29T20:14:55.219Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T20:14:55.220Z skipping since configuration is not implemented
2019-04-29T20:14:55.220Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T20:14:55.221Z Generating 100000 events in batch mode
Apr 29, 2019 8:14:55 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 8:14:55 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 8:14:55 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T20:15:05.844Z Waiting for main pipeline to 'finish'
2019-04-29T20:15:05.844Z DONE SqlBoundedSideInputJoin
2019-04-29T20:15:05.845Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T20:15:05.845Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 48s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/ckf6orh4iosfa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2642
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2642/display/redirect?page=changes>
Changes:
[daniel.o.programmer] [BEAM-7087] Updating Go SDK errors in base beam directory
[daniel.o.programmer] [BEAM-7087] Moving "errors" package to its own subdirectory.
------------------------------------------
[...truncated 286.34 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T19:22:25.687Z Generating 100000 events in batch mode
Apr 29, 2019 7:22:25 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 7:22:25 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 7:22:25 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T19:22:26.012Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T19:22:26.012Z skipping since configuration is not implemented
2019-04-29T19:22:26.012Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T19:22:06.028Z and ran for PT19.984S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.6 15051.2 92000
Exception in thread "main" 0001 3.8 26406.1 92000
0002 1.7 59347.2 351
0003 3.0 33355.6 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0013 *** not run ***
0014 *** not run ***
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
==========================================================================================
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
2019-04-29T19:22:26.014Z Generating 100000 events in batch mode
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 7:22:26 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 7:22:26 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 7:22:26 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T19:23:36.399Z Waiting for main pipeline to 'finish'
2019-04-29T19:23:36.399Z DONE SqlQuery7
2019-04-29T19:23:36.400Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T19:23:36.401Z skipping since configuration is not implemented
2019-04-29T19:23:36.401Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T19:23:36.402Z skipping since configuration is not implemented
2019-04-29T19:23:36.402Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T19:23:36.402Z skipping since configuration is not implemented
2019-04-29T19:23:36.402Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T19:23:36.402Z skipping since configuration is not implemented
2019-04-29T19:23:36.403Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T19:23:36.403Z skipping since configuration is not implemented
2019-04-29T19:23:36.403Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T19:23:36.405Z Generating 100000 events in batch mode
Apr 29, 2019 7:23:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 7:23:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 7:23:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T19:23:47.223Z Waiting for main pipeline to 'finish'
2019-04-29T19:23:47.224Z DONE SqlBoundedSideInputJoin
2019-04-29T19:23:47.224Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T19:23:47.224Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 45s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/bgdwfbuq3omsg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2641
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2641/display/redirect?page=changes>
Changes:
[ajamato] Update dataflow container version to beam-master-20190426
------------------------------------------
[...truncated 286.95 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T19:05:36.404Z Generating 100000 events in batch mode
Apr 29, 2019 7:05:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 7:05:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 7:05:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T19:05:36.738Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T19:05:36.738Z skipping since configuration is not implemented
2019-04-29T19:05:36.738Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T19:05:16.247Z and ran for PT20.491S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
Exception in thread "main" 0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
0000 6.9 14453.0 92000
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
0001 4.1 24142.9 92000
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
0002 1.7 58616.6 351
0003 3.1 31756.1 580
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
0004 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
0005 *** not run ***
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
0006 *** not run ***
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
0007 *** not run ***
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
0008 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
0009 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
0010 *** not run ***
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
0011 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
2019-04-29T19:05:36.740Z Generating 100000 events in batch mode
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 7:05:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 7:05:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 7:05:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T19:06:51.989Z Waiting for main pipeline to 'finish'
2019-04-29T19:06:51.989Z DONE SqlQuery7
2019-04-29T19:06:51.990Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T19:06:51.990Z skipping since configuration is not implemented
2019-04-29T19:06:51.990Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T19:06:51.991Z skipping since configuration is not implemented
2019-04-29T19:06:51.991Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T19:06:51.991Z skipping since configuration is not implemented
2019-04-29T19:06:51.991Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T19:06:51.991Z skipping since configuration is not implemented
2019-04-29T19:06:51.991Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T19:06:51.991Z skipping since configuration is not implemented
2019-04-29T19:06:51.991Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T19:06:51.994Z Generating 100000 events in batch mode
Apr 29, 2019 7:06:52 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 7:06:52 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 7:06:52 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T19:07:03.395Z Waiting for main pipeline to 'finish'
2019-04-29T19:07:03.395Z DONE SqlBoundedSideInputJoin
2019-04-29T19:07:03.395Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T19:07:03.396Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 0s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/qogvqyxqxmxuo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2640
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2640/display/redirect>
------------------------------------------
[...truncated 286.38 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T18:12:33.483Z Generating 100000 events in batch mode
Apr 29, 2019 6:12:33 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 6:12:33 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 6:12:33 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T18:12:33.803Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T18:12:33.803Z skipping since configuration is not implemented
2019-04-29T18:12:33.803Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T18:12:13.455Z and ran for PT20.348S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Exception in thread "main" Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.8 14692.9 92000
0001 4.2 24032.7 92000
0002 1.7 60204.7 351
0003 3.0 33355.6 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
2019-04-29T18:12:33.804Z Generating 100000 events in batch mode
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 6:12:33 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 6:12:33 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 6:12:33 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T18:13:45.274Z Waiting for main pipeline to 'finish'
2019-04-29T18:13:45.274Z DONE SqlQuery7
2019-04-29T18:13:45.275Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T18:13:45.275Z skipping since configuration is not implemented
2019-04-29T18:13:45.275Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T18:13:45.276Z skipping since configuration is not implemented
2019-04-29T18:13:45.276Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T18:13:45.276Z skipping since configuration is not implemented
2019-04-29T18:13:45.276Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T18:13:45.276Z skipping since configuration is not implemented
2019-04-29T18:13:45.276Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T18:13:45.276Z skipping since configuration is not implemented
2019-04-29T18:13:45.276Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T18:13:45.279Z Generating 100000 events in batch mode
Apr 29, 2019 6:13:45 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 6:13:45 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 6:13:45 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T18:13:56.299Z Waiting for main pipeline to 'finish'
2019-04-29T18:13:56.299Z DONE SqlBoundedSideInputJoin
2019-04-29T18:13:56.300Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T18:13:56.300Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 46s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/2lzm3t25d6rm6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2639
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2639/display/redirect?page=changes>
Changes:
[valentyn] Adds a possibility to pass a project that hosts Cloud Spanner instance
------------------------------------------
[...truncated 286.41 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T17:48:20.497Z Generating 100000 events in batch mode
Apr 29, 2019 5:48:20 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 5:48:20 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 5:48:20 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T17:48:20.795Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T17:48:20.795Z skipping since configuration is not implemented
2019-04-29T17:48:20.795Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T17:48:01.212Z and ran for PT19.583S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
Exception in thread "main" 0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.8 14684.3 92000
0001 3.7 27137.0 92000
0002 1.6 62695.9 351
0003 3.1 32092.4 580
0004 *** not run ***
0005 *** not run ***
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0006 *** not run ***
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
2019-04-29T17:48:20.797Z Generating 100000 events in batch mode
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 5:48:20 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 5:48:20 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 5:48:20 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T17:49:33.010Z Waiting for main pipeline to 'finish'
2019-04-29T17:49:33.010Z DONE SqlQuery7
2019-04-29T17:49:33.011Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T17:49:33.011Z skipping since configuration is not implemented
2019-04-29T17:49:33.011Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T17:49:33.011Z skipping since configuration is not implemented
2019-04-29T17:49:33.011Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T17:49:33.011Z skipping since configuration is not implemented
2019-04-29T17:49:33.011Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T17:49:33.012Z skipping since configuration is not implemented
2019-04-29T17:49:33.012Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T17:49:33.012Z skipping since configuration is not implemented
2019-04-29T17:49:33.012Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T17:49:33.014Z Generating 100000 events in batch mode
Apr 29, 2019 5:49:33 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 5:49:33 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 5:49:33 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T17:49:43.839Z Waiting for main pipeline to 'finish'
2019-04-29T17:49:43.839Z DONE SqlBoundedSideInputJoin
2019-04-29T17:49:43.839Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T17:49:43.839Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 46s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/6sszopnj2pg32
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2638
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2638/display/redirect?page=changes>
Changes:
[ehudm] Python Datastore IO using google-cloud-datastore
------------------------------------------
[...truncated 286.98 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T17:31:00.222Z Generating 100000 events in batch mode
Apr 29, 2019 5:31:00 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 5:31:00 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 5:31:00 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T17:31:00.582Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T17:31:00.582Z skipping since configuration is not implemented
2019-04-29T17:31:00.582Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T17:30:40.401Z and ran for PT20.181S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIException in thread "main" N; exportSummaryToBigQuery:true
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0000 6.8 14624.2 92000
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0001 3.8 26150.6 92000
0002 1.7 60277.3 351
0003 3.2 31515.9 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0012 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
0013 *** not run ***
0014 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
==========================================================================================
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
2019-04-29T17:31:00.585Z Generating 100000 events in batch mode
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 5:31:00 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 5:31:00 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 5:31:00 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T17:32:14.436Z Waiting for main pipeline to 'finish'
2019-04-29T17:32:14.437Z DONE SqlQuery7
2019-04-29T17:32:14.437Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T17:32:14.438Z skipping since configuration is not implemented
2019-04-29T17:32:14.438Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T17:32:14.438Z skipping since configuration is not implemented
2019-04-29T17:32:14.438Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T17:32:14.438Z skipping since configuration is not implemented
2019-04-29T17:32:14.439Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T17:32:14.439Z skipping since configuration is not implemented
2019-04-29T17:32:14.439Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T17:32:14.439Z skipping since configuration is not implemented
2019-04-29T17:32:14.439Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T17:32:14.441Z Generating 100000 events in batch mode
Apr 29, 2019 5:32:14 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 5:32:14 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 5:32:14 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T17:32:25.487Z Waiting for main pipeline to 'finish'
2019-04-29T17:32:25.487Z DONE SqlBoundedSideInputJoin
2019-04-29T17:32:25.487Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T17:32:25.488Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 58s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/mjy4klr5gcrto
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2637
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2637/display/redirect?page=changes>
Changes:
[thw] [BEAM-7112] Timer race with state cleanup - take two
------------------------------------------
[...truncated 286.98 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T13:51:23.496Z Generating 100000 events in batch mode
Apr 29, 2019 1:51:23 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 1:51:23 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 1:51:23 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T13:51:23.858Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T13:51:23.859Z skipping since configuration is not implemented
2019-04-29T13:51:23.859Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T13:51:03.698Z and ran for PT20.160S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
Exception in thread "main" 0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 7.1 14052.8 92000
0001 3.6 27754.6 92000
0002 1.7 58651.0 351
0003 3.1 31959.1 580
0004 *** not run ***
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
0005 *** not run ***
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
0006 *** not run ***
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
0007 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
0008 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
0009 *** not run ***
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
0010 *** not run ***
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
0011 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
0012 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
0013 *** not run ***
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:4 0014 *** not run ***
==========================================================================================
71)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
2019-04-29T13:51:23.862Z Generating 100000 events in batch mode
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 1:51:23 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 1:51:23 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 1:51:23 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T13:52:38.490Z Waiting for main pipeline to 'finish'
2019-04-29T13:52:38.490Z DONE SqlQuery7
2019-04-29T13:52:38.491Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T13:52:38.491Z skipping since configuration is not implemented
2019-04-29T13:52:38.492Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T13:52:38.492Z skipping since configuration is not implemented
2019-04-29T13:52:38.492Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T13:52:38.492Z skipping since configuration is not implemented
2019-04-29T13:52:38.492Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T13:52:38.492Z skipping since configuration is not implemented
2019-04-29T13:52:38.492Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T13:52:38.492Z skipping since configuration is not implemented
2019-04-29T13:52:38.492Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T13:52:38.494Z Generating 100000 events in batch mode
Apr 29, 2019 1:52:38 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 1:52:38 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 1:52:38 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T13:52:49.453Z Waiting for main pipeline to 'finish'
2019-04-29T13:52:49.453Z DONE SqlBoundedSideInputJoin
2019-04-29T13:52:49.453Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T13:52:49.453Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 59s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/talmn5p42ix4w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2636
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2636/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-5311] Delete docker build images
------------------------------------------
[...truncated 286.37 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T13:31:56.042Z Generating 100000 events in batch mode
Apr 29, 2019 1:31:56 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 1:31:56 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 1:31:56 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T13:31:56.352Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T13:31:56.352Z skipping since configuration is not implemented
2019-04-29T13:31:56.352Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T13:31:36.980Z and ran for PT19.372S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.6 15193.0 92000
0001 3.5 28352.7 92000
0002 1.7 59417.7 351
0003 3.3 30581.0 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
Exception in thread "main" 0007 *** not run ***
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0008 *** not run ***
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
2019-04-29T13:31:56.354Z Generating 100000 events in batch mode
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 1:31:56 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 1:31:56 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 1:31:56 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T13:33:14.685Z Waiting for main pipeline to 'finish'
2019-04-29T13:33:14.685Z DONE SqlQuery7
2019-04-29T13:33:14.686Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T13:33:14.686Z skipping since configuration is not implemented
2019-04-29T13:33:14.686Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T13:33:14.686Z skipping since configuration is not implemented
2019-04-29T13:33:14.686Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T13:33:14.687Z skipping since configuration is not implemented
2019-04-29T13:33:14.687Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T13:33:14.687Z skipping since configuration is not implemented
2019-04-29T13:33:14.687Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T13:33:14.687Z skipping since configuration is not implemented
2019-04-29T13:33:14.687Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T13:33:14.689Z Generating 100000 events in batch mode
Apr 29, 2019 1:33:14 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 1:33:14 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 1:33:14 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T13:33:25.763Z Waiting for main pipeline to 'finish'
2019-04-29T13:33:25.763Z DONE SqlBoundedSideInputJoin
2019-04-29T13:33:25.763Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T13:33:25.764Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 52s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/sl3npdskod2ck
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2635
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2635/display/redirect?page=changes>
Changes:
[mxm] [BEAM-7170] Fix exception when retrieving ExpansionServer port after
------------------------------------------
[...truncated 288.12 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T12:55:22.078Z Generating 100000 events in batch mode
Apr 29, 2019 12:55:22 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 12:55:22 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 12:55:22 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T12:55:22.354Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T12:55:22.354Z skipping since configuration is not implemented
2019-04-29T12:55:22.354Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T12:55:03.342Z and ran for PT19.012S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.7 14923.1 92000
0001 3.6 27670.2 92000
0002 1.7 59136.6 351
0003 2.8 35174.1 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
2019-04-29T12:55:22.356Z at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
Generating 100000 events in batch mode
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 12:55:22 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 12:55:22 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 12:55:22 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T12:56:34.601Z Waiting for main pipeline to 'finish'
2019-04-29T12:56:34.601Z DONE SqlQuery7
2019-04-29T12:56:34.601Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T12:56:34.602Z skipping since configuration is not implemented
2019-04-29T12:56:34.602Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T12:56:34.602Z skipping since configuration is not implemented
2019-04-29T12:56:34.603Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T12:56:34.603Z skipping since configuration is not implemented
2019-04-29T12:56:34.603Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T12:56:34.603Z skipping since configuration is not implemented
2019-04-29T12:56:34.603Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T12:56:34.603Z skipping since configuration is not implemented
2019-04-29T12:56:34.603Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T12:56:34.605Z Generating 100000 events in batch mode
Apr 29, 2019 12:56:34 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 12:56:34 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 12:56:34 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T12:56:45.298Z Waiting for main pipeline to 'finish'
2019-04-29T12:56:45.298Z DONE SqlBoundedSideInputJoin
2019-04-29T12:56:45.298Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T12:56:45.299Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 45s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/23opadsw7y7ce
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2634
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2634/display/redirect>
------------------------------------------
[...truncated 286.54 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T12:12:35.626Z Generating 100000 events in batch mode
Apr 29, 2019 12:12:35 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 12:12:35 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 12:12:35 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T12:12:35.945Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T12:12:35.945Z skipping since configuration is not implemented
2019-04-29T12:12:35.945Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T12:12:17.047Z and ran for PT18.898S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
Exception in thread "main" 0000 6.7 15024.0 92000
0001 3.6 27901.8 92000
0002 1.6 61237.0 351
0003 2.7 36429.9 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0013 *** not run ***
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0014 *** not run ***
==========================================================================================
2019-04-29T12:12:35.947Z Generating 100000 events in batch mode
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 12:12:35 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 12:12:35 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 12:12:36 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T12:13:44.979Z Waiting for main pipeline to 'finish'
2019-04-29T12:13:44.980Z DONE SqlQuery7
2019-04-29T12:13:44.980Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T12:13:44.980Z skipping since configuration is not implemented
2019-04-29T12:13:44.980Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T12:13:44.981Z skipping since configuration is not implemented
2019-04-29T12:13:44.981Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T12:13:44.981Z skipping since configuration is not implemented
2019-04-29T12:13:44.981Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T12:13:44.981Z skipping since configuration is not implemented
2019-04-29T12:13:44.981Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T12:13:44.982Z skipping since configuration is not implemented
2019-04-29T12:13:44.982Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T12:13:44.984Z Generating 100000 events in batch mode
Apr 29, 2019 12:13:45 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 12:13:45 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 12:13:45 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T12:13:55.672Z Waiting for main pipeline to 'finish'
2019-04-29T12:13:55.673Z DONE SqlBoundedSideInputJoin
2019-04-29T12:13:55.673Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T12:13:55.674Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 42s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/cyr4liwklfkmg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2633
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2633/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-7076] Update Spark runner to use spark version 2.4.2
------------------------------------------
[...truncated 289.96 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T09:35:24.924Z Generating 100000 events in batch mode
Apr 29, 2019 9:35:24 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 9:35:24 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 9:35:25 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T09:35:25.258Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T09:35:25.258Z skipping since configuration is not implemented
2019-04-29T09:35:25.258Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T09:35:05.483Z and ran for PT19.775S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
Exception in thread "main" 0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.5 15330.4 92000
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0001 4.1 24432.0 92000
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0002 1.7 59737.2 351
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0003 3.1 32499.2 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0011 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
0012 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
0013 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
0014 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
==========================================================================================
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
2019-04-29T09:35:25.260Z Generating 100000 events in batch mode
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 9:35:25 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 9:35:25 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 9:35:25 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T09:36:41.674Z Waiting for main pipeline to 'finish'
2019-04-29T09:36:41.674Z DONE SqlQuery7
2019-04-29T09:36:41.675Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T09:36:41.675Z skipping since configuration is not implemented
2019-04-29T09:36:41.675Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T09:36:41.675Z skipping since configuration is not implemented
2019-04-29T09:36:41.675Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T09:36:41.676Z skipping since configuration is not implemented
2019-04-29T09:36:41.676Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T09:36:41.676Z skipping since configuration is not implemented
2019-04-29T09:36:41.676Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T09:36:41.676Z skipping since configuration is not implemented
2019-04-29T09:36:41.676Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T09:36:41.678Z Generating 100000 events in batch mode
Apr 29, 2019 9:36:41 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 9:36:41 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 9:36:41 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T09:36:52.701Z Waiting for main pipeline to 'finish'
2019-04-29T09:36:52.701Z DONE SqlBoundedSideInputJoin
2019-04-29T09:36:52.701Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T09:36:52.701Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 0s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/yoklt5yebnkbe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2632
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2632/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-2939] Support SDF expansion in the Flink runner.
------------------------------------------
[...truncated 287.93 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T08:07:59.926Z Generating 100000 events in batch mode
Apr 29, 2019 8:07:59 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 8:07:59 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 8:08:00 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T08:08:00.219Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T08:08:00.220Z skipping since configuration is not implemented
2019-04-29T08:08:00.220Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T08:07:41.001Z and ran for PT19.218S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.8 14675.7 92000
0001 3.5 28336.6 92000
0002 1.6 63371.4 351
0003 2.8 35323.2 580
Exception in thread "main" 0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0012 *** not run ***
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0013 *** not run ***
0014 *** not run ***
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
==========================================================================================
2019-04-29T08:08:00.221Z Generating 100000 events in batch mode
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 8:08:00 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 8:08:00 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 8:08:00 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T08:09:12.807Z Waiting for main pipeline to 'finish'
2019-04-29T08:09:12.807Z DONE SqlQuery7
2019-04-29T08:09:12.808Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T08:09:12.808Z skipping since configuration is not implemented
2019-04-29T08:09:12.808Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T08:09:12.809Z skipping since configuration is not implemented
2019-04-29T08:09:12.809Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T08:09:12.809Z skipping since configuration is not implemented
2019-04-29T08:09:12.809Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T08:09:12.809Z skipping since configuration is not implemented
2019-04-29T08:09:12.809Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T08:09:12.809Z skipping since configuration is not implemented
2019-04-29T08:09:12.809Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T08:09:12.811Z Generating 100000 events in batch mode
Apr 29, 2019 8:09:12 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 8:09:12 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 8:09:12 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T08:09:23.526Z Waiting for main pipeline to 'finish'
2019-04-29T08:09:23.527Z DONE SqlBoundedSideInputJoin
2019-04-29T08:09:23.527Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T08:09:23.527Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 46s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/g6rqt42bzmxjq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2631
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2631/display/redirect>
------------------------------------------
[...truncated 286.37 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T06:12:29.143Z Generating 100000 events in batch mode
Apr 29, 2019 6:12:29 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 6:12:29 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 6:12:29 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T06:12:29.444Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T06:12:29.444Z skipping since configuration is not implemented
2019-04-29T06:12:29.445Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T06:12:10.028Z and ran for PT19.416S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
Exception in thread "main" 0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0000 6.9 14473.9 92000
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0001 3.5 28901.7 92000
0002 1.6 61199.5 351
0003 3.1 31918.3 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
2019-04-29T06:12:29.447Z Generating 100000 events in batch mode at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 6:12:29 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 6:12:29 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 6:12:29 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T06:13:37.417Z Waiting for main pipeline to 'finish'
2019-04-29T06:13:37.418Z DONE SqlQuery7
2019-04-29T06:13:37.418Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T06:13:37.418Z skipping since configuration is not implemented
2019-04-29T06:13:37.419Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T06:13:37.419Z skipping since configuration is not implemented
2019-04-29T06:13:37.419Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T06:13:37.419Z skipping since configuration is not implemented
2019-04-29T06:13:37.419Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T06:13:37.419Z skipping since configuration is not implemented
2019-04-29T06:13:37.419Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T06:13:37.419Z skipping since configuration is not implemented
2019-04-29T06:13:37.420Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T06:13:37.422Z Generating 100000 events in batch mode
Apr 29, 2019 6:13:37 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 6:13:37 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 6:13:37 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T06:13:48.071Z Waiting for main pipeline to 'finish'
2019-04-29T06:13:48.071Z DONE SqlBoundedSideInputJoin
2019-04-29T06:13:48.071Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T06:13:48.072Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 41s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/wi2adrgerzw4c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2630
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2630/display/redirect>
------------------------------------------
[...truncated 286.54 KB...]
0014 *** not run ***
==========================================================================================
2019-04-29T00:12:26.228Z Generating 100000 events in batch mode
Apr 29, 2019 12:12:26 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 29, 2019 12:12:26 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 12:12:26 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T00:12:26.516Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T00:12:26.517Z skipping since configuration is not implemented
2019-04-29T00:12:26.517Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-29T00:12:07.308Z and ran for PT19.209S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.7 14874.3 92000
0001 3.7 27352.3 92000
0002 1.6 63694.3 351
0003 2.9 34025.2 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
Exception in thread "main" 0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
2019-04-29T00:12:26.519Z Generating 100000 events in batch mode
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2019 12:12:26 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 29, 2019 12:12:26 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 29, 2019 12:12:26 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-29T00:13:41.221Z Waiting for main pipeline to 'finish'
2019-04-29T00:13:41.222Z DONE SqlQuery7
2019-04-29T00:13:41.222Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-29T00:13:41.222Z skipping since configuration is not implemented
2019-04-29T00:13:41.222Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-29T00:13:41.223Z skipping since configuration is not implemented
2019-04-29T00:13:41.223Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-29T00:13:41.223Z skipping since configuration is not implemented
2019-04-29T00:13:41.223Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-29T00:13:41.223Z skipping since configuration is not implemented
2019-04-29T00:13:41.223Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-29T00:13:41.223Z skipping since configuration is not implemented
2019-04-29T00:13:41.223Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T00:13:41.225Z Generating 100000 events in batch mode
Apr 29, 2019 12:13:41 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 29, 2019 12:13:41 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 29, 2019 12:13:41 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-29T00:13:52.075Z Waiting for main pipeline to 'finish'
2019-04-29T00:13:52.076Z DONE SqlBoundedSideInputJoin
2019-04-29T00:13:52.076Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-29T00:13:52.077Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 48s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/glsjfzxj7udh6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2629
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2629/display/redirect>
------------------------------------------
[...truncated 286.37 KB...]
0014 *** not run ***
==========================================================================================
2019-04-28T18:12:26.383Z Generating 100000 events in batch mode
Apr 28, 2019 6:12:26 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 28, 2019 6:12:26 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 28, 2019 6:12:26 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-28T18:12:26.695Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-28T18:12:26.696Z skipping since configuration is not implemented
2019-04-28T18:12:26.696Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-28T18:12:07.515Z and ran for PT19.181S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:trException in thread "main" ue
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.7 14896.5 92000
0001 3.6 27472.5 92000
0002 1.6 62111.8 351
0003 3.0 33512.1 580
0004 *** not run ***
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0005 *** not run ***
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0006 *** not run ***
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
2019-04-28T18:12:26.698Z at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
Generating 100000 events in batch mode
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 28, 2019 6:12:26 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 28, 2019 6:12:26 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 28, 2019 6:12:26 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-28T18:13:41.960Z Waiting for main pipeline to 'finish'
2019-04-28T18:13:41.960Z DONE SqlQuery7
2019-04-28T18:13:41.960Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-28T18:13:41.961Z skipping since configuration is not implemented
2019-04-28T18:13:41.961Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-28T18:13:41.961Z skipping since configuration is not implemented
2019-04-28T18:13:41.961Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-28T18:13:41.963Z skipping since configuration is not implemented
2019-04-28T18:13:41.963Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-28T18:13:41.963Z skipping since configuration is not implemented
2019-04-28T18:13:41.963Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-28T18:13:41.963Z skipping since configuration is not implemented
2019-04-28T18:13:41.963Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-28T18:13:41.965Z Generating 100000 events in batch mode
Apr 28, 2019 6:13:42 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 28, 2019 6:13:42 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 28, 2019 6:13:42 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-28T18:13:52.591Z Waiting for main pipeline to 'finish'
2019-04-28T18:13:52.591Z DONE SqlBoundedSideInputJoin
2019-04-28T18:13:52.591Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-28T18:13:52.592Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 48s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/sygpgltw2eyxq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2628
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2628/display/redirect>
------------------------------------------
[...truncated 286.54 KB...]
0014 *** not run ***
==========================================================================================
2019-04-28T12:12:28.057Z Generating 100000 events in batch mode
Apr 28, 2019 12:12:28 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 28, 2019 12:12:28 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 28, 2019 12:12:28 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-28T12:12:28.332Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-28T12:12:28.332Z skipping since configuration is not implemented
2019-04-28T12:12:28.332Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-28T12:12:09.488Z and ran for PT18.844S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Exception in thread "main"
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
Performance:
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0000 6.5 15318.6 92000
0001 3.9 25933.6 92000
0002 1.6 61614.3 351
0003 2.8 36049.0 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2019-04-28T12:12:28.334Z Generating 100000 events in batch mode
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 28, 2019 12:12:28 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 28, 2019 12:12:28 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 28, 2019 12:12:28 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-28T12:13:37.015Z Waiting for main pipeline to 'finish'
2019-04-28T12:13:37.016Z DONE SqlQuery7
2019-04-28T12:13:37.016Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-28T12:13:37.017Z skipping since configuration is not implemented
2019-04-28T12:13:37.017Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-28T12:13:37.017Z skipping since configuration is not implemented
2019-04-28T12:13:37.017Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-28T12:13:37.018Z skipping since configuration is not implemented
2019-04-28T12:13:37.018Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-28T12:13:37.018Z skipping since configuration is not implemented
2019-04-28T12:13:37.018Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-28T12:13:37.019Z skipping since configuration is not implemented
2019-04-28T12:13:37.019Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-28T12:13:37.021Z Generating 100000 events in batch mode
Apr 28, 2019 12:13:37 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 28, 2019 12:13:37 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 28, 2019 12:13:37 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-28T12:13:47.870Z Waiting for main pipeline to 'finish'
2019-04-28T12:13:47.871Z DONE SqlBoundedSideInputJoin
2019-04-28T12:13:47.871Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-28T12:13:47.871Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 41s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/dkkt37mkccolu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2627
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2627/display/redirect>
------------------------------------------
[...truncated 289.74 KB...]
0014 *** not run ***
==========================================================================================
2019-04-28T06:15:10.036Z Generating 100000 events in batch mode
Apr 28, 2019 6:15:10 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 28, 2019 6:15:10 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 28, 2019 6:15:10 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-28T06:15:10.347Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-28T06:15:10.348Z skipping since configuration is not implemented
2019-04-28T06:15:10.348Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-28T06:15:10.349Z Generating 100000 events in batch mode
==========================================================================================
Run started 2019-04-28T06:14:50.900Z and ran for PT19.447S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Exception in thread "main" Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.6 15119.4 92000
0001 3.6 27870.7 92000
0002 1.6 62034.7 351
0003 3.3 30057.1 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 28, 2019 6:15:10 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 28, 2019 6:15:10 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 28, 2019 6:15:10 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-28T06:16:20.996Z Waiting for main pipeline to 'finish'
2019-04-28T06:16:20.997Z DONE SqlQuery7
2019-04-28T06:16:20.997Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-28T06:16:20.998Z skipping since configuration is not implemented
2019-04-28T06:16:20.998Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-28T06:16:20.998Z skipping since configuration is not implemented
2019-04-28T06:16:20.998Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-28T06:16:20.999Z skipping since configuration is not implemented
2019-04-28T06:16:20.999Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-28T06:16:20.999Z skipping since configuration is not implemented
2019-04-28T06:16:20.999Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-28T06:16:20.999Z skipping since configuration is not implemented
2019-04-28T06:16:20.999Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-28T06:16:21.002Z Generating 100000 events in batch mode
Apr 28, 2019 6:16:21 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 28, 2019 6:16:21 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 28, 2019 6:16:21 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-28T06:16:31.465Z Waiting for main pipeline to 'finish'
2019-04-28T06:16:31.465Z DONE SqlBoundedSideInputJoin
2019-04-28T06:16:31.465Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-28T06:16:31.466Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 44s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/fbahy7dcrid3c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2626
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2626/display/redirect>
------------------------------------------
[...truncated 287.20 KB...]
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Apr 28, 2019 12:25:08 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 28, 2019 12:25:08 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 28, 2019 12:25:08 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-28T00:24:33.517Z and ran for PT35.580S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 11.9 8411.8 92000
0001 6.8 14604.9 92000
0002 3.1 31969.3 351
0003 4.8 21026.1 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-04-28T00:25:09.135Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-28T00:25:09.138Z skipping since configuration is not implemented
2019-04-28T00:25:09.138Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-28T00:25:09.143Z Generating 100000 events in batch mode
Apr 28, 2019 12:25:09 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 28, 2019 12:25:09 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 28, 2019 12:25:09 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-28T00:26:34.204Z Waiting for main pipeline to 'finish'
2019-04-28T00:26:34.205Z DONE SqlQuery7
2019-04-28T00:26:34.206Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-28T00:26:34.207Z skipping since configuration is not implemented
2019-04-28T00:26:34.207Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-28T00:26:34.207Z skipping since configuration is not implemented
2019-04-28T00:26:34.208Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-28T00:26:34.208Z skipping since configuration is not implemented
2019-04-28T00:26:34.208Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-28T00:26:34.208Z skipping since configuration is not implemented
2019-04-28T00:26:34.208Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-28T00:26:34.209Z skipping since configuration is not implemented
2019-04-28T00:26:34.209Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-28T00:26:34.226Z Generating 100000 events in batch mode
Apr 28, 2019 12:26:34 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 28, 2019 12:26:34 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 28, 2019 12:26:34 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-28T00:26:48.409Z Waiting for main pipeline to 'finish'
2019-04-28T00:26:48.409Z DONE SqlBoundedSideInputJoin
2019-04-28T00:26:48.409Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-28T00:26:48.410Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 32s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/yrlxmval5ymfu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2625
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2625/display/redirect>
------------------------------------------
[...truncated 286.98 KB...]
0014 *** not run ***
==========================================================================================
2019-04-27T18:25:19.521Z Generating 100000 events in batch mode
Apr 27, 2019 6:25:19 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 27, 2019 6:25:19 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 27, 2019 6:25:19 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-27T18:25:19.946Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-27T18:25:19.946Z skipping since configuration is not implemented
2019-04-27T18:25:19.946Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-27T18:24:46.156Z and ran for PT33.790S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
Exception in thread "main" 0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 10.0 10041.2 92000
0001 6.2 16048.8 92000
0002 2.6 39154.3 351
0003 4.1 24177.9 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
2019-04-27T18:25:19.948Z Generating 100000 events in batch mode
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 27, 2019 6:25:19 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 27, 2019 6:25:20 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 27, 2019 6:25:20 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-27T18:26:48.420Z Waiting for main pipeline to 'finish'
2019-04-27T18:26:48.421Z DONE SqlQuery7
2019-04-27T18:26:48.422Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-27T18:26:48.422Z skipping since configuration is not implemented
2019-04-27T18:26:48.422Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-27T18:26:48.423Z skipping since configuration is not implemented
2019-04-27T18:26:48.423Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-27T18:26:48.423Z skipping since configuration is not implemented
2019-04-27T18:26:48.423Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-27T18:26:48.423Z skipping since configuration is not implemented
2019-04-27T18:26:48.423Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-27T18:26:48.423Z skipping since configuration is not implemented
2019-04-27T18:26:48.424Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-27T18:26:48.427Z Generating 100000 events in batch mode
Apr 27, 2019 6:26:48 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 27, 2019 6:26:48 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 27, 2019 6:26:48 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-27T18:27:01.174Z Waiting for main pipeline to 'finish'
2019-04-27T18:27:01.174Z DONE SqlBoundedSideInputJoin
2019-04-27T18:27:01.175Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-27T18:27:01.175Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 32s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/32bampajmzjh6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2624
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2624/display/redirect>
------------------------------------------
[...truncated 287.16 KB...]
0014 *** not run ***
==========================================================================================
2019-04-27T12:25:16.618Z Generating 100000 events in batch mode
Apr 27, 2019 12:25:16 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 27, 2019 12:25:16 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 27, 2019 12:25:17 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-27T12:25:17.413Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
==========================================================================================
Run started 2019-04-27T12:24:42.873Z and ran for PT34.539S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 11.7 8568.2 92000
0001 6.0 16600.3 92000
0002 2.1 46948.4 351
0003 6.0 16627.9 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-04-27T12:25:17.421Z skipping since configuration is not implemented
2019-04-27T12:25:17.421Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-27T12:25:17.424Z Generating 100000 events in batch mode
Apr 27, 2019 12:25:17 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 27, 2019 12:25:17 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 27, 2019 12:25:17 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-27T12:26:25.959Z Waiting for main pipeline to 'finish'
2019-04-27T12:26:25.959Z DONE SqlQuery7
2019-04-27T12:26:25.960Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-27T12:26:25.961Z skipping since configuration is not implemented
2019-04-27T12:26:25.961Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-27T12:26:25.961Z skipping since configuration is not implemented
2019-04-27T12:26:25.961Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-27T12:26:25.962Z skipping since configuration is not implemented
2019-04-27T12:26:25.962Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-27T12:26:25.962Z skipping since configuration is not implemented
2019-04-27T12:26:25.962Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-27T12:26:25.962Z skipping since configuration is not implemented
2019-04-27T12:26:25.963Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-27T12:26:25.966Z Generating 100000 events in batch mode
Apr 27, 2019 12:26:26 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 27, 2019 12:26:26 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 27, 2019 12:26:26 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-27T12:26:37.929Z Waiting for main pipeline to 'finish'
2019-04-27T12:26:37.930Z DONE SqlBoundedSideInputJoin
2019-04-27T12:26:37.930Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-27T12:26:37.931Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 9s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/q73z2wln3xhtg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2623
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2623/display/redirect?page=changes>
Changes:
[github] Merge pull request #8311: Allow Schema field selections in DoFn using
------------------------------------------
[...truncated 290.19 KB...]
0014 *** not run ***
==========================================================================================
2019-04-27T10:21:42.204Z Generating 100000 events in batch mode
Apr 27, 2019 10:21:42 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 27, 2019 10:21:42 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 27, 2019 10:21:42 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-27T10:20:53.325Z and ran for PT50.059S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 13.7 7280.1 92000
0001 9.0 11173.2 92000
0002 5.1 19723.9 351
0003 7.6 13087.3 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-04-27T10:21:43.420Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-27T10:21:43.422Z skipping since configuration is not implemented
2019-04-27T10:21:43.423Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-27T10:21:43.428Z Generating 100000 events in batch mode
Apr 27, 2019 10:21:43 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 27, 2019 10:21:43 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 27, 2019 10:21:43 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-27T10:23:35.097Z Waiting for main pipeline to 'finish'
2019-04-27T10:23:35.097Z DONE SqlQuery7
2019-04-27T10:23:35.098Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-27T10:23:35.099Z skipping since configuration is not implemented
2019-04-27T10:23:35.099Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-27T10:23:35.099Z skipping since configuration is not implemented
2019-04-27T10:23:35.099Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-27T10:23:35.099Z skipping since configuration is not implemented
2019-04-27T10:23:35.100Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-27T10:23:35.100Z skipping since configuration is not implemented
2019-04-27T10:23:35.100Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-27T10:23:35.100Z skipping since configuration is not implemented
2019-04-27T10:23:35.100Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-27T10:23:35.118Z Generating 100000 events in batch mode
Apr 27, 2019 10:23:35 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 27, 2019 10:23:35 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 27, 2019 10:23:35 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-27T10:23:52.556Z Waiting for main pipeline to 'finish'
2019-04-27T10:23:52.556Z DONE SqlBoundedSideInputJoin
2019-04-27T10:23:52.557Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-27T10:23:52.557Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 42s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/uldpizknsxwvk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2622
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2622/display/redirect>
------------------------------------------
[...truncated 286.96 KB...]
0014 *** not run ***
==========================================================================================
2019-04-27T06:25:00.721Z Generating 100000 events in batch mode
Apr 27, 2019 6:25:00 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 27, 2019 6:25:00 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 27, 2019 6:25:01 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-27T06:24:37.203Z and ran for PT24.271S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 7.0 14208.6 92000
0001 4.9 20230.6 92000
0002 2.0 49850.4 351
0003 4.3 23369.9 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)2019-04-27T06:25:01.518Z
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-27T06:25:01.527Z skipping since configuration is not implemented
2019-04-27T06:25:01.528Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-27T06:25:01.532Z Generating 100000 events in batch mode
Apr 27, 2019 6:25:01 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 27, 2019 6:25:01 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 27, 2019 6:25:01 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-27T06:26:27.097Z Waiting for main pipeline to 'finish'
2019-04-27T06:26:27.097Z DONE SqlQuery7
2019-04-27T06:26:27.099Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-27T06:26:27.100Z skipping since configuration is not implemented
2019-04-27T06:26:27.100Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-27T06:26:27.101Z skipping since configuration is not implemented
2019-04-27T06:26:27.101Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-27T06:26:27.101Z skipping since configuration is not implemented
2019-04-27T06:26:27.101Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-27T06:26:27.102Z skipping since configuration is not implemented
2019-04-27T06:26:27.102Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-27T06:26:27.102Z skipping since configuration is not implemented
2019-04-27T06:26:27.102Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-27T06:26:27.106Z Generating 100000 events in batch mode
Apr 27, 2019 6:26:27 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 27, 2019 6:26:27 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 27, 2019 6:26:27 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-27T06:26:40.684Z Waiting for main pipeline to 'finish'
2019-04-27T06:26:40.685Z DONE SqlBoundedSideInputJoin
2019-04-27T06:26:40.685Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-27T06:26:40.686Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 19s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/ttt7sl3y6w436
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2621
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2621/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-6479] Deprecate AvroIO.RecordFormatter
------------------------------------------
[...truncated 288.54 KB...]
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Apr 27, 2019 5:46:43 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 27, 2019 5:46:44 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 27, 2019 5:46:44 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-27T05:45:56.374Z and ran for PT48.579S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0011 query:USER_SESSIONS; exportSummaryToBigQuery:tr at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
ue
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 13.5 7430.5 92000
0001 8.2 12224.9 92000
0002 4.6 21762.8 351
0003 7.7 12916.6 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)2019-04-27T05:46:44.981Z
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-27T05:46:44.985Z skipping since configuration is not implemented
2019-04-27T05:46:44.985Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-27T05:46:44.989Z Generating 100000 events in batch mode
Apr 27, 2019 5:46:45 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 27, 2019 5:46:45 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 27, 2019 5:46:45 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-27T05:48:37.542Z Waiting for main pipeline to 'finish'
2019-04-27T05:48:37.543Z DONE SqlQuery7
2019-04-27T05:48:37.543Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-27T05:48:37.544Z skipping since configuration is not implemented
2019-04-27T05:48:37.544Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-27T05:48:37.544Z skipping since configuration is not implemented
2019-04-27T05:48:37.544Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-27T05:48:37.545Z skipping since configuration is not implemented
2019-04-27T05:48:37.545Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-27T05:48:37.545Z skipping since configuration is not implemented
2019-04-27T05:48:37.545Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-27T05:48:37.545Z skipping since configuration is not implemented
2019-04-27T05:48:37.545Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-27T05:48:37.548Z Generating 100000 events in batch mode
Apr 27, 2019 5:48:37 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 27, 2019 5:48:37 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 27, 2019 5:48:37 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-27T05:48:56.615Z Waiting for main pipeline to 'finish'
2019-04-27T05:48:56.615Z DONE SqlBoundedSideInputJoin
2019-04-27T05:48:56.616Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-27T05:48:56.616Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 42s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/e3iglf627qb52
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2620
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2620/display/redirect>
------------------------------------------
[...truncated 287.15 KB...]
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Apr 27, 2019 12:25:08 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 27, 2019 12:25:08 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 27, 2019 12:25:09 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-27T00:24:42.149Z and ran for PT27.480S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.4 15535.2 92000
0001 4.7 21159.5 92000
0002 2.7 36683.8 351
0003 6.3 15992.3 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-04-27T00:25:09.675Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-27T00:25:09.677Z skipping since configuration is not implemented
2019-04-27T00:25:09.678Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-27T00:25:09.681Z Generating 100000 events in batch mode
Apr 27, 2019 12:25:09 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 27, 2019 12:25:09 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 27, 2019 12:25:09 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-27T00:26:45.663Z Waiting for main pipeline to 'finish'
2019-04-27T00:26:45.664Z DONE SqlQuery7
2019-04-27T00:26:45.665Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-27T00:26:45.665Z skipping since configuration is not implemented
2019-04-27T00:26:45.666Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-27T00:26:45.666Z skipping since configuration is not implemented
2019-04-27T00:26:45.666Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-27T00:26:45.666Z skipping since configuration is not implemented
2019-04-27T00:26:45.666Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-27T00:26:45.666Z skipping since configuration is not implemented
2019-04-27T00:26:45.666Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-27T00:26:45.666Z skipping since configuration is not implemented
2019-04-27T00:26:45.666Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-27T00:26:45.669Z Generating 100000 events in batch mode
Apr 27, 2019 12:26:45 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 27, 2019 12:26:45 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 27, 2019 12:26:45 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-27T00:26:59.018Z Waiting for main pipeline to 'finish'
2019-04-27T00:26:59.018Z DONE SqlBoundedSideInputJoin
2019-04-27T00:26:59.019Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-27T00:26:59.019Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 33s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/psfzlre7dkmyq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2619
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2619/display/redirect?page=changes>
Changes:
[pabloem] Add some pydoc for runner classes in Python
------------------------------------------
[...truncated 286.94 KB...]
0014 *** not run ***
==========================================================================================
2019-04-26T20:49:51.070Z Generating 100000 events in batch mode
Apr 26, 2019 8:49:51 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 26, 2019 8:49:51 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 8:49:51 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-26T20:49:03.288Z and ran for PT48.667S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 13.6 7337.3 92000
0001 8.7 11555.4 92000
0002 5.0 19880.7 351
0003 7.2 13817.9 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-04-26T20:49:51.987Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T20:49:51.991Z skipping since configuration is not implemented
2019-04-26T20:49:51.991Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-26T20:49:51.995Z Generating 100000 events in batch mode
Apr 26, 2019 8:49:52 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 26, 2019 8:49:52 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 8:49:52 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T20:51:39.258Z Waiting for main pipeline to 'finish'
2019-04-26T20:51:39.258Z DONE SqlQuery7
2019-04-26T20:51:39.260Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-26T20:51:39.260Z skipping since configuration is not implemented
2019-04-26T20:51:39.261Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T20:51:39.261Z skipping since configuration is not implemented
2019-04-26T20:51:39.261Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-26T20:51:39.261Z skipping since configuration is not implemented
2019-04-26T20:51:39.261Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-26T20:51:39.262Z skipping since configuration is not implemented
2019-04-26T20:51:39.262Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-26T20:51:39.262Z skipping since configuration is not implemented
2019-04-26T20:51:39.262Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T20:51:39.266Z Generating 100000 events in batch mode
Apr 26, 2019 8:51:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 26, 2019 8:51:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 26, 2019 8:51:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-26T20:51:57.749Z Waiting for main pipeline to 'finish'
2019-04-26T20:51:57.749Z DONE SqlBoundedSideInputJoin
2019-04-26T20:51:57.750Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T20:51:57.752Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 37s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/zhbe5y7dnbw2y
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2618
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2618/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-7110] Add Spark master option to SparkJobServerDriver
[kcweaver] [BEAM-7133] make Spark job server Gradle task pass along system
------------------------------------------
[...truncated 287.22 KB...]
0014 *** not run ***
==========================================================================================
2019-04-26T20:19:59.793Z Generating 100000 events in batch mode
Apr 26, 2019 8:19:59 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 26, 2019 8:19:59 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 8:19:59 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T20:20:00.112Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T20:20:00.113Z skipping since configuration is not implemented
2019-04-26T20:20:00.113Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-26T20:19:40.858Z and ran for PT19.255S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
Exception in thread "main" 0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.8 14747.1 92000
0001 3.4 29086.7 92000
0002 1.6 62461.0 351
0003 3.1 32541.5 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
2019-04-26T20:20:00.115Z Generating 100000 events in batch mode
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 26, 2019 8:20:00 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 26, 2019 8:20:00 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 8:20:00 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T20:21:13.152Z Waiting for main pipeline to 'finish'
2019-04-26T20:21:13.152Z DONE SqlQuery7
2019-04-26T20:21:13.153Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-26T20:21:13.153Z skipping since configuration is not implemented
2019-04-26T20:21:13.153Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T20:21:13.154Z skipping since configuration is not implemented
2019-04-26T20:21:13.154Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-26T20:21:13.154Z skipping since configuration is not implemented
2019-04-26T20:21:13.154Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-26T20:21:13.154Z skipping since configuration is not implemented
2019-04-26T20:21:13.154Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-26T20:21:13.155Z skipping since configuration is not implemented
2019-04-26T20:21:13.155Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T20:21:13.157Z Generating 100000 events in batch mode
Apr 26, 2019 8:21:13 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 26, 2019 8:21:13 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 26, 2019 8:21:13 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-26T20:21:23.932Z Waiting for main pipeline to 'finish'
2019-04-26T20:21:23.932Z DONE SqlBoundedSideInputJoin
2019-04-26T20:21:23.933Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T20:21:23.933Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 56s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/jdxg35cmwewhc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2617
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2617/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-7162] Add ValueProvider to CassandraIO Write
------------------------------------------
[...truncated 286.97 KB...]
0014 *** not run ***
==========================================================================================
2019-04-26T19:57:03.326Z Generating 100000 events in batch mode
Apr 26, 2019 7:57:03 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 26, 2019 7:57:03 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 7:57:03 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
==========================================================================================
Run started 2019-04-26T19:56:15.500Z and ran for PT48.505S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 14.3 6974.0 92000
0001 8.6 11602.3 92000
0002 4.7 21186.4 351
0003 7.6 13109.6 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-04-26T19:57:04.012Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T19:57:04.013Z skipping since configuration is not implemented
2019-04-26T19:57:04.014Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-26T19:57:04.017Z Generating 100000 events in batch mode
Apr 26, 2019 7:57:04 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 26, 2019 7:57:04 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 7:57:04 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T19:58:50.530Z Waiting for main pipeline to 'finish'
2019-04-26T19:58:50.530Z DONE SqlQuery7
2019-04-26T19:58:50.531Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-26T19:58:50.532Z skipping since configuration is not implemented
2019-04-26T19:58:50.532Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T19:58:50.532Z skipping since configuration is not implemented
2019-04-26T19:58:50.532Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-26T19:58:50.533Z skipping since configuration is not implemented
2019-04-26T19:58:50.533Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-26T19:58:50.533Z skipping since configuration is not implemented
2019-04-26T19:58:50.533Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-26T19:58:50.533Z skipping since configuration is not implemented
2019-04-26T19:58:50.533Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T19:58:50.536Z Generating 100000 events in batch mode
Apr 26, 2019 7:58:50 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 26, 2019 7:58:50 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 26, 2019 7:58:50 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-26T19:59:08.186Z Waiting for main pipeline to 'finish'
2019-04-26T19:59:08.187Z DONE SqlBoundedSideInputJoin
2019-04-26T19:59:08.187Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T19:59:08.188Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 32s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/z25mlgajcoqjy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2616
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2616/display/redirect>
------------------------------------------
[...truncated 286.97 KB...]
0014 *** not run ***
==========================================================================================
2019-04-26T18:25:20.721Z Generating 100000 events in batch mode
Apr 26, 2019 6:25:20 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 26, 2019 6:25:20 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 6:25:20 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T18:25:21.097Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T18:25:21.097Z skipping since configuration is not implemented
2019-04-26T18:25:21.097Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-26T18:24:49.253Z and ran for PT31.844S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Exception in thread "main" Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported! 0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 9.7 10332.7 92000
0001 5.5 18345.3 92000
0002 3.4 29120.6 351
0003 4.0 24987.5 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0010 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
0011 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
0012 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
0013 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
0014 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
==========================================================================================
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
2019-04-26T18:25:21.100Z Generating 100000 events in batch mode
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 26, 2019 6:25:21 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 26, 2019 6:25:21 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 6:25:21 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T18:26:44.314Z Waiting for main pipeline to 'finish'
2019-04-26T18:26:44.315Z DONE SqlQuery7
2019-04-26T18:26:44.315Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-26T18:26:44.316Z skipping since configuration is not implemented
2019-04-26T18:26:44.316Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T18:26:44.316Z skipping since configuration is not implemented
2019-04-26T18:26:44.317Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-26T18:26:44.317Z skipping since configuration is not implemented
2019-04-26T18:26:44.317Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-26T18:26:44.317Z skipping since configuration is not implemented
2019-04-26T18:26:44.318Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-26T18:26:44.318Z skipping since configuration is not implemented
2019-04-26T18:26:44.318Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T18:26:44.320Z Generating 100000 events in batch mode
Apr 26, 2019 6:26:44 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 26, 2019 6:26:44 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 26, 2019 6:26:44 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-26T18:26:57.391Z Waiting for main pipeline to 'finish'
2019-04-26T18:26:57.392Z DONE SqlBoundedSideInputJoin
2019-04-26T18:26:57.392Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T18:26:57.392Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 25s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/ziff6k7tdrfpw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2615
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2615/display/redirect?page=changes>
Changes:
[mxm] Increase Flink parallelism for ValidatesRunner tests
------------------------------------------
[...truncated 287.16 KB...]
0014 *** not run ***
==========================================================================================
2019-04-26T13:00:11.943Z Generating 100000 events in batch mode
Apr 26, 2019 1:00:11 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 26, 2019 1:00:12 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 1:00:12 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-26T12:59:42.690Z and ran for PT29.719S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 9.3 10705.5 92000
0001 5.5 18099.5 92000
0002 2.0 49751.2 351
0003 3.5 28579.6 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
2019-04-26T13:00:12.429Z at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T13:00:12.430Z skipping since configuration is not implemented
2019-04-26T13:00:12.431Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-26T13:00:12.433Z Generating 100000 events in batch mode
Apr 26, 2019 1:00:12 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 26, 2019 1:00:12 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 1:00:12 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T13:01:37.833Z Waiting for main pipeline to 'finish'
2019-04-26T13:01:37.834Z DONE SqlQuery7
2019-04-26T13:01:37.834Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-26T13:01:37.835Z skipping since configuration is not implemented
2019-04-26T13:01:37.835Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T13:01:37.835Z skipping since configuration is not implemented
2019-04-26T13:01:37.835Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-26T13:01:37.835Z skipping since configuration is not implemented
2019-04-26T13:01:37.835Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-26T13:01:37.836Z skipping since configuration is not implemented
2019-04-26T13:01:37.836Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-26T13:01:37.836Z skipping since configuration is not implemented
2019-04-26T13:01:37.836Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T13:01:37.839Z Generating 100000 events in batch mode
Apr 26, 2019 1:01:37 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 26, 2019 1:01:37 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 26, 2019 1:01:37 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-26T13:01:49.863Z Waiting for main pipeline to 'finish'
2019-04-26T13:01:49.864Z DONE SqlBoundedSideInputJoin
2019-04-26T13:01:49.864Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T13:01:49.864Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 28s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/au2imkv7v2vp2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2614
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2614/display/redirect>
------------------------------------------
[...truncated 287.15 KB...]
0014 *** not run ***
==========================================================================================
2019-04-26T12:25:33.508Z Generating 100000 events in batch mode
Apr 26, 2019 12:25:33 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 26, 2019 12:25:33 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 12:25:34 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-26T12:25:03.985Z and ran for PT30.304S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.8 14658.5 92000
0001 6.7 15035.3 92000
0002 3.6 27909.6 351
0003 5.9 17015.5 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-04-26T12:25:34.327Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T12:25:34.329Z skipping since configuration is not implemented
2019-04-26T12:25:34.329Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-26T12:25:34.332Z Generating 100000 events in batch mode
Apr 26, 2019 12:25:34 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 26, 2019 12:25:34 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 12:25:34 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T12:27:08.005Z Waiting for main pipeline to 'finish'
2019-04-26T12:27:08.005Z DONE SqlQuery7
2019-04-26T12:27:08.006Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-26T12:27:08.006Z skipping since configuration is not implemented
2019-04-26T12:27:08.007Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T12:27:08.007Z skipping since configuration is not implemented
2019-04-26T12:27:08.007Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-26T12:27:08.007Z skipping since configuration is not implemented
2019-04-26T12:27:08.007Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-26T12:27:08.008Z skipping since configuration is not implemented
2019-04-26T12:27:08.008Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-26T12:27:08.008Z skipping since configuration is not implemented
2019-04-26T12:27:08.008Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T12:27:08.019Z Generating 100000 events in batch mode
Apr 26, 2019 12:27:08 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 26, 2019 12:27:08 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 26, 2019 12:27:08 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-26T12:27:23.598Z Waiting for main pipeline to 'finish'
2019-04-26T12:27:23.598Z DONE SqlBoundedSideInputJoin
2019-04-26T12:27:23.598Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T12:27:23.599Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 36s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/eqiiqmsuwflww
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2613
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2613/display/redirect?page=changes>
Changes:
[mxm] [BEAM-7029] Add KafkaIO.Write as an external transform
------------------------------------------
[...truncated 287.70 KB...]
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Apr 26, 2019 10:28:02 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 26, 2019 10:28:02 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 10:28:02 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-26T10:27:34.127Z and ran for PT28.886S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.2 16129.0 92000
0001 6.1 16337.2 92000
0002 3.8 26525.2 351
0003 6.0 16644.5 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-04-26T10:28:03.019Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T10:28:03.019Z skipping since configuration is not implemented
2019-04-26T10:28:03.019Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-26T10:28:03.021Z Generating 100000 events in batch mode
Apr 26, 2019 10:28:03 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 26, 2019 10:28:03 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 10:28:03 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T10:29:20.517Z Waiting for main pipeline to 'finish'
2019-04-26T10:29:20.517Z DONE SqlQuery7
2019-04-26T10:29:20.517Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-26T10:29:20.518Z skipping since configuration is not implemented
2019-04-26T10:29:20.518Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T10:29:20.518Z skipping since configuration is not implemented
2019-04-26T10:29:20.518Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-26T10:29:20.518Z skipping since configuration is not implemented
2019-04-26T10:29:20.518Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-26T10:29:20.519Z skipping since configuration is not implemented
2019-04-26T10:29:20.519Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-26T10:29:20.519Z skipping since configuration is not implemented
2019-04-26T10:29:20.519Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T10:29:20.522Z Generating 100000 events in batch mode
Apr 26, 2019 10:29:20 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 26, 2019 10:29:20 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 26, 2019 10:29:20 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-26T10:29:34.538Z Waiting for main pipeline to 'finish'
2019-04-26T10:29:34.538Z DONE SqlBoundedSideInputJoin
2019-04-26T10:29:34.538Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T10:29:34.539Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 15s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/k6hlcbd2piuh6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2612
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2612/display/redirect>
------------------------------------------
[...truncated 286.94 KB...]
0014 *** not run ***
==========================================================================================
2019-04-26T06:25:11.774Z Generating 100000 events in batch mode
Apr 26, 2019 6:25:11 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 26, 2019 6:25:11 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 6:25:11 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-26T06:24:44.013Z and ran for PT28.133S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 8.9 11249.9 92000
0001 4.8 20691.1 92000
0002 2.0 49236.8 351
0003 3.3 30349.0 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-04-26T06:25:12.164Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T06:25:12.165Z skipping since configuration is not implemented
2019-04-26T06:25:12.165Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-26T06:25:12.167Z Generating 100000 events in batch mode
Apr 26, 2019 6:25:12 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 26, 2019 6:25:12 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 6:25:12 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T06:26:36.919Z Waiting for main pipeline to 'finish'
2019-04-26T06:26:36.920Z DONE SqlQuery7
2019-04-26T06:26:36.920Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-26T06:26:36.921Z skipping since configuration is not implemented
2019-04-26T06:26:36.921Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T06:26:36.921Z skipping since configuration is not implemented
2019-04-26T06:26:36.921Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-26T06:26:36.921Z skipping since configuration is not implemented
2019-04-26T06:26:36.921Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-26T06:26:36.921Z skipping since configuration is not implemented
2019-04-26T06:26:36.921Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-26T06:26:36.921Z skipping since configuration is not implemented
2019-04-26T06:26:36.922Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T06:26:36.924Z Generating 100000 events in batch mode
Apr 26, 2019 6:26:36 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 26, 2019 6:26:36 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 26, 2019 6:26:36 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-26T06:26:48.074Z Waiting for main pipeline to 'finish'
2019-04-26T06:26:48.074Z DONE SqlBoundedSideInputJoin
2019-04-26T06:26:48.075Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T06:26:48.075Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 24s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/oy2skafwvv33g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2611
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2611/display/redirect?page=changes>
Changes:
[pabloem] [BEAM-7139] Blogpost for Kotlin Samples (#8391)
------------------------------------------
[...truncated 286.99 KB...]
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Apr 26, 2019 3:14:12 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 26, 2019 3:14:12 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 3:14:13 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-26T03:13:46.207Z and ran for PT27.342S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.1 16504.4 92000
0001 5.1 19466.6 92000
0002 2.9 34566.2 351
0003 6.5 15278.8 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-04-26T03:14:13.609Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T03:14:13.611Z skipping since configuration is not implemented
2019-04-26T03:14:13.611Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-26T03:14:13.615Z Generating 100000 events in batch mode
Apr 26, 2019 3:14:13 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 26, 2019 3:14:13 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 3:14:13 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T03:15:33.609Z Waiting for main pipeline to 'finish'
2019-04-26T03:15:33.609Z DONE SqlQuery7
2019-04-26T03:15:33.609Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-26T03:15:33.610Z skipping since configuration is not implemented
2019-04-26T03:15:33.610Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T03:15:33.610Z skipping since configuration is not implemented
2019-04-26T03:15:33.610Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-26T03:15:33.610Z skipping since configuration is not implemented
2019-04-26T03:15:33.610Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-26T03:15:33.610Z skipping since configuration is not implemented
2019-04-26T03:15:33.610Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-26T03:15:33.611Z skipping since configuration is not implemented
2019-04-26T03:15:33.611Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T03:15:33.613Z Generating 100000 events in batch mode
Apr 26, 2019 3:15:33 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 26, 2019 3:15:33 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 26, 2019 3:15:33 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-26T03:15:46.245Z Waiting for main pipeline to 'finish'
2019-04-26T03:15:46.246Z DONE SqlBoundedSideInputJoin
2019-04-26T03:15:46.247Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T03:15:46.247Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 15s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/ln5e5ccmkg7y2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2610
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2610/display/redirect?page=changes>
Changes:
[daniel.o.programmer] [BEAM-5709] Changing sleeps to CountdownLatch in test
------------------------------------------
[...truncated 287.12 KB...]
0014 *** not run ***
==========================================================================================
2019-04-26T00:45:34.388Z Generating 100000 events in batch mode
Apr 26, 2019 12:45:34 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 26, 2019 12:45:34 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 12:45:34 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-26T00:44:46.136Z and ran for PT49.127S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 14.0 7140.8 92000
0001 8.4 11943.2 92000
0002 4.2 24096.4 351
0003 8.2 12238.4 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-04-26T00:45:35.294Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T00:45:35.312Z skipping since configuration is not implemented
2019-04-26T00:45:35.313Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-26T00:45:35.316Z Generating 100000 events in batch mode
Apr 26, 2019 12:45:35 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 26, 2019 12:45:35 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 12:45:35 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T00:47:23.656Z Waiting for main pipeline to 'finish'
2019-04-26T00:47:23.657Z DONE SqlQuery7
2019-04-26T00:47:23.657Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-26T00:47:23.658Z skipping since configuration is not implemented
2019-04-26T00:47:23.658Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T00:47:23.658Z skipping since configuration is not implemented
2019-04-26T00:47:23.658Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-26T00:47:23.659Z skipping since configuration is not implemented
2019-04-26T00:47:23.659Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-26T00:47:23.659Z skipping since configuration is not implemented
2019-04-26T00:47:23.659Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-26T00:47:23.659Z skipping since configuration is not implemented
2019-04-26T00:47:23.659Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T00:47:23.662Z Generating 100000 events in batch mode
Apr 26, 2019 12:47:23 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 26, 2019 12:47:23 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 26, 2019 12:47:23 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-26T00:47:39.776Z Waiting for main pipeline to 'finish'
2019-04-26T00:47:39.777Z DONE SqlBoundedSideInputJoin
2019-04-26T00:47:39.777Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T00:47:39.778Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 36s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/h2ybg44fvfa24
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2609
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2609/display/redirect>
------------------------------------------
[...truncated 287.16 KB...]
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Apr 26, 2019 12:24:44 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 26, 2019 12:24:44 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 12:24:44 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
==========================================================================================
Run started 2019-04-26T00:24:16.659Z and ran for PT28.011S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 6.7 14961.1 92000
0001 5.0 20145.0 92000
0002 3.0 33145.5 351
0003 6.6 15218.4 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-04-26T00:24:44.686Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T00:24:44.687Z skipping since configuration is not implemented
2019-04-26T00:24:44.688Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
2019-04-26T00:24:44.690Z Generating 100000 events in batch mode
Apr 26, 2019 12:24:44 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 26, 2019 12:24:44 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 26, 2019 12:24:44 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-26T00:26:05.740Z Waiting for main pipeline to 'finish'
2019-04-26T00:26:05.740Z DONE SqlQuery7
2019-04-26T00:26:05.741Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-26T00:26:05.741Z skipping since configuration is not implemented
2019-04-26T00:26:05.741Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-26T00:26:05.742Z skipping since configuration is not implemented
2019-04-26T00:26:05.742Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-26T00:26:05.742Z skipping since configuration is not implemented
2019-04-26T00:26:05.742Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-26T00:26:05.742Z skipping since configuration is not implemented
2019-04-26T00:26:05.743Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-26T00:26:05.743Z skipping since configuration is not implemented
2019-04-26T00:26:05.743Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T00:26:05.745Z Generating 100000 events in batch mode
Apr 26, 2019 12:26:05 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 26, 2019 12:26:05 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 26, 2019 12:26:05 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-26T00:26:20.870Z Waiting for main pipeline to 'finish'
2019-04-26T00:26:20.871Z DONE SqlBoundedSideInputJoin
2019-04-26T00:26:20.871Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-26T00:26:20.872Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 20s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/7tbmr35zbpa4k
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2608
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2608/display/redirect?page=changes>
Changes:
[apilloud] Add 2.12.0 Blog post.
------------------------------------------
[...truncated 287.14 KB...]
0014 *** not run ***
==========================================================================================
2019-04-25T23:44:10.726Z Generating 100000 events in batch mode
Apr 25, 2019 11:44:10 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 25, 2019 11:44:10 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 25, 2019 11:44:10 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-25T23:44:11.100Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-25T23:44:11.101Z skipping since configuration is not implemented
2019-04-25T23:44:11.101Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-25T23:43:36.989Z and ran for PT34.112S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Exception in thread "main" 0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 11.5 8733.6 92000
0001 5.8 17132.1 92000
0002 2.6 39016.8 351
0003 5.2 19357.3 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported! 0013 *** not run ***
0014 *** not run ***
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
==========================================================================================
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
2019-04-25T23:44:11.103Z Generating 100000 events in batch mode
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 25, 2019 11:44:11 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 25, 2019 11:44:11 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 25, 2019 11:44:11 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-25T23:45:38.892Z Waiting for main pipeline to 'finish'
2019-04-25T23:45:38.892Z DONE SqlQuery7
2019-04-25T23:45:38.893Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-25T23:45:38.893Z skipping since configuration is not implemented
2019-04-25T23:45:38.894Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-25T23:45:38.894Z skipping since configuration is not implemented
2019-04-25T23:45:38.894Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-25T23:45:38.894Z skipping since configuration is not implemented
2019-04-25T23:45:38.894Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-25T23:45:38.894Z skipping since configuration is not implemented
2019-04-25T23:45:38.894Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-25T23:45:38.894Z skipping since configuration is not implemented
2019-04-25T23:45:38.894Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-25T23:45:38.897Z Generating 100000 events in batch mode
Apr 25, 2019 11:45:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 25, 2019 11:45:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 25, 2019 11:45:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-25T23:45:53.779Z Waiting for main pipeline to 'finish'
2019-04-25T23:45:53.779Z DONE SqlBoundedSideInputJoin
2019-04-25T23:45:53.779Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-25T23:45:53.780Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 33s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/utx42nx26wwy6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2607
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2607/display/redirect?page=changes>
Changes:
[apilloud] Update release guide with feedback from 2.12.0 release
------------------------------------------
[...truncated 286.98 KB...]
2019-04-25T22:11:12.247Z skipping since configuration is not implemented
2019-04-25T22:11:12.248Z Running query:HOT_ITEMS; exportSummaryToBigQuery:true
2019-04-25T22:11:12.250Z Generating 100000 events in batch mode
Apr 25, 2019 10:11:12 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 25, 2019 10:11:12 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 25, 2019 10:11:12 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-25T22:11:12.661Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-25T22:11:12.661Z skipping since configuration is not implemented
2019-04-25T22:11:12.661Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-25T22:10:39.330Z and ran for PT33.331S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
Exception in thread "main" 0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 12.5 7980.8 92000
0001 5.3 18811.1 92000
0002 2.8 36258.2 351
0003 5.5 18224.9 580
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0004 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
0005 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
0006 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
0007 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
0008 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
0009 *** not run ***
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
0010 *** not run ***
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
0011 *** not run ***
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
0012 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
0013 *** not run ***
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
0014 *** not run ***
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
==========================================================================================
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
2019-04-25T22:11:12.663Z Generating 100000 events in batch mode
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 25, 2019 10:11:12 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 25, 2019 10:11:12 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 25, 2019 10:11:12 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-25T22:12:39.591Z Waiting for main pipeline to 'finish'
2019-04-25T22:12:39.591Z DONE SqlQuery7
2019-04-25T22:12:39.591Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-25T22:12:39.592Z skipping since configuration is not implemented
2019-04-25T22:12:39.592Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-25T22:12:39.592Z skipping since configuration is not implemented
2019-04-25T22:12:39.592Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-25T22:12:39.592Z skipping since configuration is not implemented
2019-04-25T22:12:39.592Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-25T22:12:39.593Z skipping since configuration is not implemented
2019-04-25T22:12:39.593Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-25T22:12:39.593Z skipping since configuration is not implemented
2019-04-25T22:12:39.593Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-25T22:12:39.595Z Generating 100000 events in batch mode
Apr 25, 2019 10:12:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 25, 2019 10:12:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 25, 2019 10:12:39 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-25T22:12:50.839Z Waiting for main pipeline to 'finish'
2019-04-25T22:12:50.839Z DONE SqlBoundedSideInputJoin
2019-04-25T22:12:50.839Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-25T22:12:50.840Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 28s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/paycu5bwadlus
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2606
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2606/display/redirect?page=changes>
Changes:
[apilloud] Website changes for 2.12.0
------------------------------------------
[...truncated 287.20 KB...]
0014 *** not run ***
==========================================================================================
2019-04-25T21:30:23.956Z Generating 100000 events in batch mode
Apr 25, 2019 9:30:24 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 25, 2019 9:30:24 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 25, 2019 9:30:24 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-25T21:30:24.265Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-25T21:30:24.265Z skipping since configuration is not implemented
2019-04-25T21:30:24.265Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-25T21:30:04.487Z and ran for PT19.778S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIException in thread "main" N; exportSummaryToBigQuery:true
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
Performance:
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0000 6.6 15069.3 92000
0001 4.1 24325.0 92000
0002 1.7 60313.6 351
0003 3.1 32268.5 580
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
2019-04-25T21:30:24.267Z Generating 100000 events in batch mode
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 25, 2019 9:30:24 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 25, 2019 9:30:24 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 25, 2019 9:30:24 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-25T21:31:35.041Z Waiting for main pipeline to 'finish'
2019-04-25T21:31:35.042Z DONE SqlQuery7
2019-04-25T21:31:35.042Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-25T21:31:35.043Z skipping since configuration is not implemented
2019-04-25T21:31:35.043Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-25T21:31:35.043Z skipping since configuration is not implemented
2019-04-25T21:31:35.043Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-25T21:31:35.043Z skipping since configuration is not implemented
2019-04-25T21:31:35.043Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-25T21:31:35.043Z skipping since configuration is not implemented
2019-04-25T21:31:35.043Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-25T21:31:35.044Z skipping since configuration is not implemented
2019-04-25T21:31:35.044Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-25T21:31:35.045Z Generating 100000 events in batch mode
Apr 25, 2019 9:31:35 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 25, 2019 9:31:35 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 25, 2019 9:31:35 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-25T21:31:45.681Z Waiting for main pipeline to 'finish'
2019-04-25T21:31:45.681Z DONE SqlBoundedSideInputJoin
2019-04-25T21:31:45.681Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-25T21:31:45.681Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 54s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/mdsk77qh4giu4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #2605
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/2605/display/redirect?page=changes>
Changes:
[github] Merge pull request #8385: Fix broken parsing of uuid
------------------------------------------
[...truncated 287.16 KB...]
0014 *** not run ***
==========================================================================================
2019-04-25T19:35:49.586Z Generating 100000 events in batch mode
Apr 25, 2019 7:35:49 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `AuctionBids`.`auction`, `AuctionBids`.`num`
FROM (SELECT `B1`.`auction`, COUNT(*) AS `num`, HOP_START(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY `B1`.`auction`, HOP(`B1`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `AuctionBids`
INNER JOIN (SELECT MAX(`CountBids`.`num`) AS `maxnum`, `CountBids`.`starttime`
FROM (SELECT COUNT(*) AS `num`, HOP_START(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B2`
GROUP BY `B2`.`auction`, HOP(`B2`.`dateTime`, INTERVAL '5' SECOND, INTERVAL '10' SECOND)) AS `CountBids`
GROUP BY `CountBids`.`starttime`) AS `MaxBids` ON `AuctionBids`.`starttime` = `MaxBids`.`starttime` AND `AuctionBids`.`num` >= `MaxBids`.`maxnum`
Apr 25, 2019 7:35:49 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], num=[$1])
LogicalJoin(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
LogicalProject(auction=[$0], num=[$2], starttime=[$1])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxnum=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxnum=[MAX($1)])
LogicalProject(starttime=[$1], num=[$2])
LogicalAggregate(group=[{0, 1}], num=[COUNT()])
LogicalProject(auction=[$0], $f1=[HOP($3, 5000:INTERVAL SECOND, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 25, 2019 7:35:49 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], starttime=[$t1])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxnum=[MAX($1)])
BeamCalcRel(expr#0..2=[{inputs}], starttime=[$t1], num=[$t2])
BeamAggregationRel(group=[{0, 1}], num=[COUNT()], window=[SlidingWindows($1, PT5S, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], $f1=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-25T19:35:49.954Z Running query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
2019-04-25T19:35:49.954Z skipping since configuration is not implemented
2019-04-25T19:35:49.954Z Running query:HIGHEST_BID; exportSummaryToBigQuery:true
==========================================================================================
Run started 2019-04-25T19:35:17.471Z and ran for PT32.483S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
Exception in thread "main" 0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
Caused by: java.lang.UnsupportedOperationException: Non equi-join is not supported!
0000 10.0 10036.1 92000
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinPairOfRexNodes(BeamJoinRel.java:569)
0001 6.1 16270.7 92000
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.extractJoinRexNodes(BeamJoinRel.java:554)
0002 2.6 38255.5 351
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel.access$900(BeamJoinRel.java:101)
0003 5.0 19996.0 580
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:257)
0004 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$ExtractJoinKeys.expand(BeamJoinRel.java:240)
0005 *** not run ***
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
0006 *** not run ***
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
0007 *** not run ***
at org.apache.beam.sdk.values.PCollectionList.apply(PCollectionList.java:176)
0008 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:324)
0009 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel$StandardJoin.expand(BeamJoinRel.java:317)
0010 *** not run ***
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
0011 *** not run ***
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
0012 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
0013 *** not run ***
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
0014 *** not run ***
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
==========================================================================================
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
2019-04-25T19:35:49.956Z Generating 100000 events in batch mode
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:248)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:106)
at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery5.expand(SqlQuery5.java:55)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1161)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 25, 2019 7:35:49 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Apr 25, 2019 7:35:49 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000:INTERVAL SECOND)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000:INTERVAL SECOND)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Apr 25, 2019 7:35:50 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}], window=[FixedWindows($5, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t3])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)], window=[FixedWindows($0, PT10S, PT0S)])
BeamCalcRel(expr#0..4=[{inputs}], $f0=[$t3], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2019-04-25T19:37:13.996Z Waiting for main pipeline to 'finish'
2019-04-25T19:37:13.996Z DONE SqlQuery7
2019-04-25T19:37:13.997Z Running query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
2019-04-25T19:37:13.998Z skipping since configuration is not implemented
2019-04-25T19:37:13.998Z Running query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
2019-04-25T19:37:13.998Z skipping since configuration is not implemented
2019-04-25T19:37:13.998Z Running query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
2019-04-25T19:37:13.998Z skipping since configuration is not implemented
2019-04-25T19:37:13.998Z Running query:USER_SESSIONS; exportSummaryToBigQuery:true
2019-04-25T19:37:13.999Z skipping since configuration is not implemented
2019-04-25T19:37:13.999Z Running query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
2019-04-25T19:37:13.999Z skipping since configuration is not implemented
2019-04-25T19:37:13.999Z Running query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-25T19:37:14.001Z Generating 100000 events in batch mode
Apr 25, 2019 7:37:14 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
WITH `bid_with_side` (`auction`, `bidder`, `price`, `dateTime`, `extra`, `side_id`) AS (SELECT `bid`.`auction`, `bid`.`bidder`, `bid`.`price`, `bid`.`dateTime`, `bid`.`extra`, CAST(MOD(`bid`.`bidder`, 500) AS BIGINT) AS `side_id`
FROM `beam`.`bid` AS `bid`) (SELECT `bid_with_side`.`auction`, `bid_with_side`.`bidder`, `bid_with_side`.`price`, `bid_with_side`.`dateTime`, `side`.`extra`
FROM `bid_with_side` AS `bid_with_side`,
`beam`.`side` AS `side`
WHERE `bid_with_side`.`side_id` = `side`.`id`)
Apr 25, 2019 7:37:14 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$7])
LogicalFilter(condition=[=($5, $6)])
LogicalJoin(condition=[true], joinType=[inner])
LogicalProject(auction=[$0], bidder=[$1], price=[$2], dateTime=[$3], extra=[$4], side_id=[CAST(MOD($1, 500)):BIGINT NOT NULL])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
Apr 25, 2019 7:37:14 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..3=[{exprs}], extra=[$t7])
BeamJoinRel(condition=[=($5, $6)], joinType=[inner])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[500], expr#6=[MOD($t1, $t5)], expr#7=[CAST($t6):BIGINT NOT NULL], proj#0..4=[{exprs}], side_id=[$t7])
BeamIOSourceRel(table=[[beam, bid]])
BeamIOSourceRel(table=[[beam, side]])
2019-04-25T19:37:26.965Z Waiting for main pipeline to 'finish'
2019-04-25T19:37:26.965Z DONE SqlBoundedSideInputJoin
2019-04-25T19:37:26.966Z Running query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
2019-04-25T19:37:26.967Z skipping since configuration is not implemented
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 25s
65 actionable tasks: 2 executed, 63 up-to-date
Publishing build scan...
https://gradle.com/s/ydcji2g6ugx7y
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org