You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Brock Noland (JIRA)" <ji...@apache.org> on 2014/10/05 01:01:33 UTC
[jira] [Updated] (HIVE-8352) Enable windowing.q for spark
[ https://issues.apache.org/jira/browse/HIVE-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated HIVE-8352:
-------------------------------
Description:
We should enable windowing.q for basic windowing coverage. After checking out the spark branch, we would build:
{noformat}
$ mvn clean install -DskipTests -Phadoop-2
$ cd itests/
$ mvn clean install -DskipTests -Phadoop-2
{noformat}
Then generate the windowing.q.out file:
{noformat}
$ cd qtest-spark/
$ mvn test -Dtest=TestSparkCliDriver -Dqfile=windowing.q -Phadoop-2 -Dtest.output.overwrite=true
{noformat}
Compare the output against MapReduce:
{noformat}
$ diff -y -W 150 ../../ql/src/test/results/clientpositive/spark/windowing.q.out ../../ql/src/test/results/clientpositive/windowing.q.out| less
{noformat}
And if everything looks good, add it to {{spark.query.files}} in {{./itests/src/test/resources/testconfiguration.properties}}
then submit the patch including the .q file
was:
We should enable windowing.q for basic windowing coverage. After checking out the spark branch, we would build:
{noformat}
$ mvn clean install -DskipTests -Phadoop-2
$ cd itests/
$ mvn clean install -DskipTests -Phadoop-2
Then generate the windowing.q.out file:
{noformat}
$ cd qtest-spark/
$ mvn test -Dtest=TestSparkCliDriver -Dqfile=windowing.q -Phadoop-2 -Dtest.output.overwrite=true
{noformat}
Compare the output against MapReduce:
{noformat}
$ diff -y -W 150 ../../ql/src/test/results/clientpositive/spark/windowing.q.out ../../ql/src/test/results/clientpositive/windowing.q.out| less
{noformat}
And if everything looks good, add it to {{spark.query.files}} in {{./itests/src/test/resources/testconfiguration.properties}}
then submit the patch including the .q file
> Enable windowing.q for spark
> ----------------------------
>
> Key: HIVE-8352
> URL: https://issues.apache.org/jira/browse/HIVE-8352
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Brock Noland
> Priority: Minor
>
> We should enable windowing.q for basic windowing coverage. After checking out the spark branch, we would build:
> {noformat}
> $ mvn clean install -DskipTests -Phadoop-2
> $ cd itests/
> $ mvn clean install -DskipTests -Phadoop-2
> {noformat}
> Then generate the windowing.q.out file:
> {noformat}
> $ cd qtest-spark/
> $ mvn test -Dtest=TestSparkCliDriver -Dqfile=windowing.q -Phadoop-2 -Dtest.output.overwrite=true
> {noformat}
> Compare the output against MapReduce:
> {noformat}
> $ diff -y -W 150 ../../ql/src/test/results/clientpositive/spark/windowing.q.out ../../ql/src/test/results/clientpositive/windowing.q.out| less
> {noformat}
> And if everything looks good, add it to {{spark.query.files}} in {{./itests/src/test/resources/testconfiguration.properties}}
> then submit the patch including the .q file
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)