You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by David Langer <da...@hotmail.com> on 2012/02/11 21:04:20 UTC
Sqoop 1.2 - Free Form Query Imports Not Working
Greetings All!
I'm using v1.2 of sqoop and I'm having trouble getting the --query command line parameter working. I can do the moral equivalent of the following using the --table parameter sucessfully.
Any pointers would be greatly appreciated.
Thanx,
Dave
>sqoop import --connect "jdbc:sqlserver://FooBar;database=AdventureWorks2008R2;user=foo;password=bar"
--query "SELECT * FROM AWBuildVersion WHERE $CONDITIONS" --split-by SystemInformationID --target-dir /tmp/Data/Dave1
Missing argument for option: e
usage: sqoop import [GENERIC-ARGS] [TOOL-ARGS]
Common arguments:
--connect <jdbc-uri> Specify JDBC connect
string
--connection-manager <conn-manager-class-name> Specify connection
manager class name
--driver <class-name> Manually specify JDBC
driver class to use
--hadoop-home <dir> Override $HADOOP_HOME
--help Print usage
instructions
-P Read password from
console
--password <password> Set authentication
password
--username <username> Set authentication
username
--verbose Print more
information while
working
Import control arguments:
--append Imports data in append mode
--as-sequencefile Imports data to SequenceFiles
--as-textfile Imports data as plain text (default)
--columns <col,col,col...> Columns to import from table
--direct Use direct import fast path
--direct-split-size <n> Split the input stream every 'n' bytes
when importing in direct mode
-e,--query <statement> Import results of SQL 'statement'
--inline-lob-limit <n> Set the maximum size for an inline LOB
-m,--num-mappers <n> Use 'n' map tasks to import in parallel
--split-by <column-name> Column of the table used to split work
units
--table <table-name> Table to read
--target-dir <dir> HDFS plain table destination
--warehouse-dir <dir> HDFS parent for table destination
--where <where clause> WHERE clause to use during import
-z,--compress Enable compression
Incremental import arguments:
--check-column <column> Source column to check for incremental
change
--incremental <import-type> Define an incremental import of type
'append' or 'lastmodified'
--last-value <value> Last imported value in the incremental
check column
Output line formatting arguments:
--enclosed-by <char> Sets a required field enclosing
character
--escaped-by <char> Sets the escape character
--fields-terminated-by <char> Sets the field separator character
--lines-terminated-by <char> Sets the end-of-line character
--mysql-delimiters Uses MySQL's default delimiter set:
fields: , lines: \n escaped-by: \
optionally-enclosed-by: '
--optionally-enclosed-by <char> Sets a field enclosing character
Input parsing arguments:
--input-enclosed-by <char> Sets a required field encloser
--input-escaped-by <char> Sets the input escape
character
--input-fields-terminated-by <char> Sets the input field separator
--input-lines-terminated-by <char> Sets the input end-of-line
char
--input-optionally-enclosed-by <char> Sets a field enclosing
character
Hive arguments:
--hive-home <dir> Override $HIVE_HOME
--hive-import Import tables into Hive (Uses Hive's
default delimiters if none are set.)
--hive-overwrite Overwrite existing data in the Hive table
--hive-table <table-name> Sets the table name to use when importing
to hive
HBase arguments:
--column-family <family> Sets the target column family for the
import
--hbase-create-table If specified, create missing HBase tables
--hbase-row-key <col> Specifies which input column to use as the
row key
--hbase-table <table> Import to <table> in HBase
Code generation arguments:
--bindir <dir> Output directory for
compiled objects
--class-name <name> Sets the generated
class name. This
overrides
--package-name. When
combined with
--jar-file, sets the
input class.
--input-null-non-string <input-null-non-string> Input null
non-string
representation
--input-null-string <input-null-string> Input null string
representation
--jar-file <file> Disable code
generation; use
specified jar
--null-non-string <null-non-string> Null non-string
representation
--null-string <null-string> Null string
representation
--outdir <dir> Output directory for
generated code
--package-name <name> Put auto-generated
classes in this
package
Generic Hadoop command-line arguments:
(must preceed any tool-specific arguments)
Generic options supported are
-conf <configuration file> specify an application configuration file
-D <property=value> use value for given property
-fs <local|namenode:port> specify a namenode
-jt <local|jobtracker:port> specify a job tracker
-files <comma separated list of files> specify comma separated files to be copied to the map reduce cluster
-libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.
-archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.
The general command line syntax is
bin/hadoop command [genericOptions] [commandOptions]
At minimum, you must specify --connect and --table
Arguments to mysqldump and other subprograms may be supplied
after a '--' on the command line.
>
Re: Sqoop 1.2 - Free Form Query Imports Not Working
Posted by Scott Foster <sc...@gmail.com>.
If you haven't got it working by now... try single quotes around the
query. the $CONDITIONS might be getting interpreted by your shell
On Sat, Feb 11, 2012 at 12:04 PM, David Langer <da...@hotmail.com> wrote:
> Greetings All!
>
> I'm using v1.2 of sqoop and I'm having trouble getting the --query command
> line parameter working. I can do the moral equivalent of the following using
> the --table parameter sucessfully.
>
> Any pointers would be greatly appreciated.
>
> Thanx,
>
> Dave
>
>
>>sqoop import --connect
>> "jdbc:sqlserver://FooBar;database=AdventureWorks2008R2;user=foo;password=bar"
> --query "SELECT * FROM AWBuildVersion WHERE $CONDITIONS" --split-by
> SystemInformationID --target-dir /tmp/Data/Dave1
> Missing argument for option: e
> usage: sqoop import [GENERIC-ARGS] [TOOL-ARGS]
>
> Common arguments:
> --connect <jdbc-uri> Specify JDBC connect
> string
> --connection-manager <conn-manager-class-name> Specify connection
> manager class name
> --driver <class-name> Manually specify JDBC
> driver class to use
> --hadoop-home <dir> Override $HADOOP_HOME
> --help Print usage
> instructions
> -P Read password from
> console
> --password <password> Set authentication
> password
> --username <username> Set authentication
> username
> --verbose Print more
> information while
> working
> Import control arguments:
> --append Imports data in append mode
> --as-sequencefile Imports data to SequenceFiles
> --as-textfile Imports data as plain text (default)
> --columns <col,col,col...> Columns to import from table
> --direct Use direct import fast path
> --direct-split-size <n> Split the input stream every 'n' bytes
> when importing in direct mode
> -e,--query <statement> Import results of SQL 'statement'
> --inline-lob-limit <n> Set the maximum size for an inline LOB
> -m,--num-mappers <n> Use 'n' map tasks to import in parallel
> --split-by <column-name> Column of the table used to split work
> units
> --table <table-name> Table to read
> --target-dir <dir> HDFS plain table destination
> --warehouse-dir <dir> HDFS parent for table destination
> --where <where clause> WHERE clause to use during import
> -z,--compress Enable compression
> Incremental import arguments:
> --check-column <column> Source column to check for incremental
> change
> --incremental <import-type> Define an incremental import of type
> 'append' or 'lastmodified'
> --last-value <value> Last imported value in the incremental
> check column
> Output line formatting arguments:
> --enclosed-by <char> Sets a required field enclosing
> character
> --escaped-by <char> Sets the escape character
> --fields-terminated-by <char> Sets the field separator character
> --lines-terminated-by <char> Sets the end-of-line character
> --mysql-delimiters Uses MySQL's default delimiter set:
> fields: , lines: \n escaped-by: \
> optionally-enclosed-by: '
> --optionally-enclosed-by <char> Sets a field enclosing character
> Input parsing arguments:
> --input-enclosed-by <char> Sets a required field encloser
> --input-escaped-by <char> Sets the input escape
> character
> --input-fields-terminated-by <char> Sets the input field separator
> --input-lines-terminated-by <char> Sets the input end-of-line
> char
> --input-optionally-enclosed-by <char> Sets a field enclosing
> character
> Hive arguments:
> --hive-home <dir> Override $HIVE_HOME
> --hive-import Import tables into Hive (Uses Hive's
> default delimiters if none are set.)
> --hive-overwrite Overwrite existing data in the Hive table
> --hive-table <table-name> Sets the table name to use when importing
> to hive
> HBase arguments:
> --column-family <family> Sets the target column family for the
> import
> --hbase-create-table If specified, create missing HBase tables
> --hbase-row-key <col> Specifies which input column to use as the
> row key
> --hbase-table <table> Import to <table> in HBase
> Code generation arguments:
> --bindir <dir> Output directory for
> compiled objects
> --class-name <name> Sets the generated
> class name. This
> overrides
> --package-name. When
> combined with
> --jar-file, sets the
> input class.
> --input-null-non-string <input-null-non-string> Input null
> non-string
> representation
> --input-null-string <input-null-string> Input null string
> representation
> --jar-file <file> Disable code
> generation; use
> specified jar
> --null-non-string <null-non-string> Null non-string
> representation
> --null-string <null-string> Null string
> representation
> --outdir <dir> Output directory for
> generated code
> --package-name <name> Put auto-generated
> classes in this
> package
> Generic Hadoop command-line arguments:
> (must preceed any tool-specific arguments)
> Generic options supported are
> -conf <configuration file> specify an application configuration file
> -D <property=value> use value for given property
> -fs <local|namenode:port> specify a namenode
> -jt <local|jobtracker:port> specify a job tracker
> -files <comma separated list of files> specify comma separated files to
> be copied to the map reduce cluster
> -libjars <comma separated list of jars> specify comma separated jar files
> to include in the classpath.
> -archives <comma separated list of archives> specify comma separated
> archives to be unarchived on the compute machines.
> The general command line syntax is
> bin/hadoop command [genericOptions] [commandOptions]
>
> At minimum, you must specify --connect and --table
> Arguments to mysqldump and other subprograms may be supplied
> after a '--' on the command line.
>>
>