You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-commits@hadoop.apache.org by to...@apache.org on 2009/12/15 16:02:42 UTC
svn commit: r890815 - in /hadoop/mapreduce/trunk: CHANGES.txt
src/contrib/sqoop/src/java/org/apache/hadoop/sqoop/manager/SqlManager.java
Author: tomwhite
Date: Tue Dec 15 15:02:42 2009
New Revision: 890815
URL: http://svn.apache.org/viewvc?rev=890815&view=rev
Log:
MAPREDUCE-1224. Calling "SELECT t.* from <table> AS t" to get meta information is too expensive for big tables. Contributed by Spencer Ho.
Modified:
hadoop/mapreduce/trunk/CHANGES.txt
hadoop/mapreduce/trunk/src/contrib/sqoop/src/java/org/apache/hadoop/sqoop/manager/SqlManager.java
Modified: hadoop/mapreduce/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/CHANGES.txt?rev=890815&r1=890814&r2=890815&view=diff
==============================================================================
--- hadoop/mapreduce/trunk/CHANGES.txt (original)
+++ hadoop/mapreduce/trunk/CHANGES.txt Tue Dec 15 15:02:42 2009
@@ -81,6 +81,9 @@
Configuration changes:
add mapreduce.tasktracker.outofband.heartbeat
+ MAPREDUCE-1224. Calling "SELECT t.* from <table> AS t" to get meta
+ information is too expensive for big tables. (Spencer Ho via tomwhite)
+
BUG FIXES
MAPREDUCE-1089. Fix NPE in fair scheduler preemption when tasks are
Modified: hadoop/mapreduce/trunk/src/contrib/sqoop/src/java/org/apache/hadoop/sqoop/manager/SqlManager.java
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/src/contrib/sqoop/src/java/org/apache/hadoop/sqoop/manager/SqlManager.java?rev=890815&r1=890814&r2=890815&view=diff
==============================================================================
--- hadoop/mapreduce/trunk/src/contrib/sqoop/src/java/org/apache/hadoop/sqoop/manager/SqlManager.java (original)
+++ hadoop/mapreduce/trunk/src/contrib/sqoop/src/java/org/apache/hadoop/sqoop/manager/SqlManager.java Tue Dec 15 15:02:42 2009
@@ -67,7 +67,8 @@
* be tuned per-database, but the main extraction loop is still inheritable.
*/
protected String getColNamesQuery(String tableName) {
- return "SELECT t.* FROM " + tableName + " AS t";
+ // adding where clause to prevent loading a big table
+ return "SELECT t.* FROM " + tableName + " AS t WHERE 1=0";
}
@Override