You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by ta...@apache.org on 2017/08/25 16:01:12 UTC
[1/2] incubator-impala git commit: IMPALA-5827: add test for failure
to repartition in hash join
Repository: incubator-impala
Updated Branches:
refs/heads/master 2a7c8b901 -> b524a5ce6
IMPALA-5827: add test for failure to repartition in hash join
Testing:
Ran the test locally.
Change-Id: If6d601f9d15bed4667b50576f07f6216c34ed9c4
Reviewed-on: http://gerrit.cloudera.org:8080/7811
Reviewed-by: Dan Hecht <dh...@cloudera.com>
Tested-by: Impala Public Jenkins
Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/51c7fcd5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/51c7fcd5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/51c7fcd5
Branch: refs/heads/master
Commit: 51c7fcd5dc4a7fe1d8788be534809c4969ffe8a7
Parents: 2a7c8b9
Author: Tim Armstrong <ta...@cloudera.com>
Authored: Thu Aug 24 13:58:11 2017 -0700
Committer: Impala Public Jenkins <im...@gerrit.cloudera.org>
Committed: Fri Aug 25 01:17:46 2017 +0000
----------------------------------------------------------------------
.../functional-query/queries/QueryTest/spilling.test | 9 +++++++++
1 file changed, 9 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/51c7fcd5/testdata/workloads/functional-query/queries/QueryTest/spilling.test
----------------------------------------------------------------------
diff --git a/testdata/workloads/functional-query/queries/QueryTest/spilling.test b/testdata/workloads/functional-query/queries/QueryTest/spilling.test
index 3868e4f..f8226f1 100644
--- a/testdata/workloads/functional-query/queries/QueryTest/spilling.test
+++ b/testdata/workloads/functional-query/queries/QueryTest/spilling.test
@@ -356,3 +356,12 @@ row_regex: .*Query State: FINISHED.*
row_regex: .*Query Status: OK.*
row_regex: .*SpilledPartitions: .* \([1-9][0-9]*\)
====
+---- QUERY
+# Test spilling join with many duplicates in join key. We don't expect this to succeed
+# with a memory constraint: see IMPALA-4857.
+set buffer_pool_limit=20m;
+select *
+from lineitem l1 join lineitem l2 on l1.l_linenumber = l2.l_linenumber
+---- CATCH
+Repartitioning did not reduce the size of a spilled partition
+====
[2/2] incubator-impala git commit: IMPALA-5836: Improvements to
Eclipse frontend configuration.
Posted by ta...@apache.org.
IMPALA-5836: Improvements to Eclipse frontend configuration.
Having recently gone through setting up Eclipse for frontend debugging,
I'm amending the process a tad to make it smoother for new developers.
* Adds the JDK sources package to the developer bootstrap
process. These are useful if you need to point Eclipse to the source
of Java libraries and are harmless to everyone else.
* Defaults to "downloadSource" for the "mvn eclipse:eclipse" plugin.
This was being set explicitly in the Wiki instructions, but
there's no reason not to set it on by default for everyone.
* Adds a "launcher" file for debugging the FE. Instead of creating
a debug configuration manually, it's possible to share these across
developers by checking them in. I checked it in via a mechanism
that "mvn eclipse:eclipse" has to create additional files, to
avoid literring the directory for non-Eclipse users.
* Changes the output directory used by Eclipse's compiler to
a different one than Maven uses. Because we start Impala
from the commandline, it's easy to start Impala and then
change a line of Java, which forces Eclipse to kick in and
replace the .class files. When this happens, the existing Impala
starts complaining about class not found, because the underlying
files have been overwritten. It's preferable to separate the
directories and rely on Maven for the compiled Java.
Change-Id: Ia723fbf706cf409a8fb6b5ff0297c2b1ff7c9590
Reviewed-on: http://gerrit.cloudera.org:8080/7803
Reviewed-by: Dimitris Tsirogiannis <dt...@cloudera.com>
Tested-by: Impala Public Jenkins
Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/b524a5ce
Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/b524a5ce
Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/b524a5ce
Branch: refs/heads/master
Commit: b524a5ce69945fb2c7a7667bd2a66f371161bc2a
Parents: 51c7fcd
Author: Philip Zeyliger <ph...@cloudera.com>
Authored: Thu Aug 24 09:44:04 2017 -0700
Committer: Impala Public Jenkins <im...@gerrit.cloudera.org>
Committed: Fri Aug 25 02:58:16 2017 +0000
----------------------------------------------------------------------
bin/bootstrap_development.sh | 2 +-
fe/pom.xml | 40 +++++++++++++++++++++++++++++++++++++++
2 files changed, 41 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/b524a5ce/bin/bootstrap_development.sh
----------------------------------------------------------------------
diff --git a/bin/bootstrap_development.sh b/bin/bootstrap_development.sh
index 12ce4f3..612d765 100755
--- a/bin/bootstrap_development.sh
+++ b/bin/bootstrap_development.sh
@@ -122,7 +122,7 @@ if [[ $DISTRIB_RELEASE = 14.04 ]]
then
JDK_VERSION=7
fi
-apt-get --yes install openjdk-${JDK_VERSION}-jdk
+apt-get --yes install openjdk-${JDK_VERSION}-jdk openjdk-${JDK_VERSION}-source
SET_JAVA_HOME="export JAVA_HOME=/usr/lib/jvm/java-${JDK_VERSION}-openjdk-amd64"
echo "$SET_JAVA_HOME" >> "${IMPALA_HOME}/bin/impala-config-local.sh"
eval "$SET_JAVA_HOME"
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/b524a5ce/fe/pom.xml
----------------------------------------------------------------------
diff --git a/fe/pom.xml b/fe/pom.xml
index 238f118..0f7ae8c 100644
--- a/fe/pom.xml
+++ b/fe/pom.xml
@@ -47,6 +47,7 @@ under the License.
<impala.extdatasrc.api.version>1.0-SNAPSHOT</impala.extdatasrc.api.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<kudu.version>${env.KUDU_JAVA_VERSION}</kudu.version>
+ <eclipse.output.directory>eclipse-classes</eclipse.output.directory>
</properties>
<dependencies>
@@ -626,6 +627,45 @@ under the License.
</lifecycleMappingMetadata>
</configuration>
</plugin>
+ <!-- mvn eclipse:eclipse generates Eclipse .project and .classpath files -->
+ <plugin>
+ <groupId>org.apache.maven.plugins </groupId>
+ <artifactId>maven-eclipse-plugin</artifactId>
+ <version>2.10</version>
+ <configuration>
+ <!-- By default, we separate Eclipse-built files from Maven-built
+ files. Otherwise, they are both in target/classes, and Eclipse
+ and Maven may clobber each other, complicating attaching to
+ a running process. -->
+ <buildOutputDirectory>${eclipse.output.directory}</buildOutputDirectory>
+ <downloadSources>true</downloadSources>
+ <downloadJavadocs>false</downloadJavadocs>
+ <additionalConfig>
+ <file>
+ <!-- Saved "launch configuration" for attaching Eclipse debugger to port 30000 -->
+ <name>impala-fe-30000.launch</name>
+ <content><![CDATA[<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<launchConfiguration type="org.eclipse.jdt.launching.remoteJavaApplication">
+<listAttribute key="org.eclipse.debug.core.MAPPED_RESOURCE_PATHS">
+<listEntry value="/impala-frontend"/>
+</listAttribute>
+<listAttribute key="org.eclipse.debug.core.MAPPED_RESOURCE_TYPES">
+<listEntry value="4"/>
+</listAttribute>
+<booleanAttribute key="org.eclipse.jdt.launching.ALLOW_TERMINATE" value="false"/>
+<mapAttribute key="org.eclipse.jdt.launching.CONNECT_MAP">
+<mapEntry key="hostname" value="localhost"/>
+<mapEntry key="port" value="30000"/>
+</mapAttribute>
+<stringAttribute key="org.eclipse.jdt.launching.PROJECT_ATTR" value="impala-frontend"/>
+<stringAttribute key="org.eclipse.jdt.launching.VM_CONNECTOR_ID" value="org.eclipse.jdt.launching.socketAttachConnector"/>
+</launchConfiguration>
+]]>
+ </content>
+ </file>
+ </additionalConfig>
+ </configuration>
+ </plugin>
</plugins>
</pluginManagement>
</build>