You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Fatih Haltas <fa...@nyu.edu> on 2013/07/16 12:44:10 UTC
java.io.IOException: error=2, No such file or directory
Hi everyone,
I am trying to import data from postgresql to hdfs. But I am having some
problems, Here is the problem details:
Sqoop Version: 1.4.3
Hadoop Version:1.0.4
*1) When I use this command:*
*
*
*./sqoop import-all-tables --connect jdbc:postgresql://
192.168.194.158:5432/IMS --username pgsql -P*
Here is the exact output
======================================================
Warning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: $HADOOP_HOME is deprecated.
Enter password:
13/07/16 13:52:28 INFO manager.SqlManager: Using default fetchSize of 1000
13/07/16 13:52:29 INFO tool.CodeGenTool: Beginning code generation
13/07/16 13:52:29 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM "publicinici" AS t LIMIT 1
13/07/16 13:52:29 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
/home/hadoop/project/hadoop-1.0.4
Note:
/tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publicinici.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
13/07/16 13:52:30 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publicinici.jar
13/07/16 13:52:30 WARN manager.PostgresqlManager: It looks like you are
importing from postgresql.
13/07/16 13:52:30 WARN manager.PostgresqlManager: This transfer can be
faster! Use the --direct
13/07/16 13:52:30 WARN manager.PostgresqlManager: option to exercise a
postgresql-specific fast path.
13/07/16 13:52:30 INFO mapreduce.ImportJobBase: Beginning import of
publicinici
13/07/16 13:52:31 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
SELECT MIN("budapablik"), MAX("budapablik") FROM "publicinici"
13/07/16 13:52:31 INFO mapred.JobClient: Running job: job_201302261146_0525
13/07/16 13:52:32 INFO mapred.JobClient: map 0% reduce 0%
^X[hadoop@ADUAE042-LAP-V bin]./sqoop version
Warning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: $HADOOP_HOME is deprecated.
===================================================
*I have only one table which is "publicinici",that holds 2 entity in one row
* and it read but at mapreduce phase it is getting stucked.
*2) When I use this command in which --direct option is added, eveything is
going well, but getting one error, which is *
ERROR tool.ImportAllTablesTool: Encountered IOException running import job:
java.io.IOException: Cannot run program "psql": java.io.IOException:
error=2, No such file or directory
*Also, in the hdfs publicinici file has been created as empty. *
Here is the exact output.
========================
13/07/16 13:58:42 INFO manager.SqlManager: Using default fetchSize of 1000
13/07/16 13:58:42 INFO tool.CodeGenTool: Beginning code generation
13/07/16 13:58:43 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM "publicinici" AS t LIMIT 1
13/07/16 13:58:43 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
/home/hadoop/project/hadoop-1.0.4
Note:
/tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publicinici.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
13/07/16 13:58:44 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publicinici.jar
13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Beginning psql fast
path import
13/07/16 13:58:44 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM "publicinici" AS t LIMIT 1
13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Copy command is
COPY (SELECT "budapablik" FROM "publicinici" WHERE 1=1) TO STDOUT WITH
DELIMITER E'\54' CSV ;
13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Performing import
of table publicinici from database IMS
13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transfer loop
complete.
13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transferred 0 bytes
in 17,361,268.2044 seconds (0 bytes/sec)
13/07/16 13:58:45 ERROR tool.ImportAllTablesTool: Encountered IOException
running import job: java.io.IOException: Cannot run program "psql":
java.io.IOException: error=2, No such file or directory
What should I do to be able to fix the problem.
Any help is gonna be appreciated. Thank you very much.
Re: java.io.IOException: error=2, No such file or directory
Posted by Fatih Haltas <fa...@nyu.edu>.
Sure, there were some problems with postgresql installation. (Actually,
this was the solution that you said in your mail but I did not notice.)
On Tue, Jul 16, 2013 at 6:37 PM, Shahab Yunus <sh...@gmail.com>wrote:
> Great. Can you please share, if possible, what was the problem and how you
> solved it? Thanks.
>
> Regards,
> Shahab
>
>
> On Tue, Jul 16, 2013 at 9:58 AM, Fatih Haltas <fa...@nyu.edu>wrote:
>
>> Thanks Shahab, I solved my problem, in anyother way,
>>
>
>
Re: java.io.IOException: error=2, No such file or directory
Posted by Fatih Haltas <fa...@nyu.edu>.
Sure, there were some problems with postgresql installation. (Actually,
this was the solution that you said in your mail but I did not notice.)
On Tue, Jul 16, 2013 at 6:37 PM, Shahab Yunus <sh...@gmail.com>wrote:
> Great. Can you please share, if possible, what was the problem and how you
> solved it? Thanks.
>
> Regards,
> Shahab
>
>
> On Tue, Jul 16, 2013 at 9:58 AM, Fatih Haltas <fa...@nyu.edu>wrote:
>
>> Thanks Shahab, I solved my problem, in anyother way,
>>
>
>
Re: java.io.IOException: error=2, No such file or directory
Posted by Fatih Haltas <fa...@nyu.edu>.
Sure, there were some problems with postgresql installation. (Actually,
this was the solution that you said in your mail but I did not notice.)
On Tue, Jul 16, 2013 at 6:37 PM, Shahab Yunus <sh...@gmail.com>wrote:
> Great. Can you please share, if possible, what was the problem and how you
> solved it? Thanks.
>
> Regards,
> Shahab
>
>
> On Tue, Jul 16, 2013 at 9:58 AM, Fatih Haltas <fa...@nyu.edu>wrote:
>
>> Thanks Shahab, I solved my problem, in anyother way,
>>
>
>
Re: java.io.IOException: error=2, No such file or directory
Posted by Fatih Haltas <fa...@nyu.edu>.
Sure, there were some problems with postgresql installation. (Actually,
this was the solution that you said in your mail but I did not notice.)
On Tue, Jul 16, 2013 at 6:37 PM, Shahab Yunus <sh...@gmail.com>wrote:
> Great. Can you please share, if possible, what was the problem and how you
> solved it? Thanks.
>
> Regards,
> Shahab
>
>
> On Tue, Jul 16, 2013 at 9:58 AM, Fatih Haltas <fa...@nyu.edu>wrote:
>
>> Thanks Shahab, I solved my problem, in anyother way,
>>
>
>
Re: java.io.IOException: error=2, No such file or directory
Posted by Shahab Yunus <sh...@gmail.com>.
Great. Can you please share, if possible, what was the problem and how you
solved it? Thanks.
Regards,
Shahab
On Tue, Jul 16, 2013 at 9:58 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Thanks Shahab, I solved my problem, in anyother way,
>
Re: java.io.IOException: error=2, No such file or directory
Posted by Shahab Yunus <sh...@gmail.com>.
Great. Can you please share, if possible, what was the problem and how you
solved it? Thanks.
Regards,
Shahab
On Tue, Jul 16, 2013 at 9:58 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Thanks Shahab, I solved my problem, in anyother way,
>
Re: java.io.IOException: error=2, No such file or directory
Posted by Shahab Yunus <sh...@gmail.com>.
Great. Can you please share, if possible, what was the problem and how you
solved it? Thanks.
Regards,
Shahab
On Tue, Jul 16, 2013 at 9:58 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Thanks Shahab, I solved my problem, in anyother way,
>
Re: java.io.IOException: error=2, No such file or directory
Posted by Shahab Yunus <sh...@gmail.com>.
Great. Can you please share, if possible, what was the problem and how you
solved it? Thanks.
Regards,
Shahab
On Tue, Jul 16, 2013 at 9:58 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Thanks Shahab, I solved my problem, in anyother way,
>
Re: java.io.IOException: error=2, No such file or directory
Posted by Fatih Haltas <fa...@nyu.edu>.
Thanks Shahab, I solved my problem, in anyother way,
Re: java.io.IOException: error=2, No such file or directory
Posted by Fatih Haltas <fa...@nyu.edu>.
Thanks Shahab, I solved my problem, in anyother way,
Re: java.io.IOException: error=2, No such file or directory
Posted by Fatih Haltas <fa...@nyu.edu>.
Thanks Shahab, I solved my problem, in anyother way,
Re: java.io.IOException: error=2, No such file or directory
Posted by Fatih Haltas <fa...@nyu.edu>.
Thanks Shahab, I solved my problem, in anyother way,
Re: java.io.IOException: error=2, No such file or directory
Posted by Shahab Yunus <sh...@gmail.com>.
The error is:
*Please set $HBASE_HOME to the root of your HBase installation.*
*
*
Have you checked whether it is set or not? Have you verified your HBase or
Hadoop installation?
Similarly, the following:
*Cannot run program "psql": java.io.IOException: error=2, No such file or
directory *
Also seem to indicate that PostGres is not on the path. What and how are
your environment variables set? Can you access/run this independently from
this directory?
Regards,
Shahab
On Tue, Jul 16, 2013 at 6:44 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi everyone,
>
> I am trying to import data from postgresql to hdfs. But I am having some
> problems, Here is the problem details:
>
> Sqoop Version: 1.4.3
> Hadoop Version:1.0.4
>
>
> *1) When I use this command:*
> *
> *
> *./sqoop import-all-tables --connect jdbc:postgresql://
> 192.168.194.158:5432/IMS --username pgsql -P*
>
>
> Here is the exact output
> ======================================================
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
>
> Enter password:
> 13/07/16 13:52:28 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/07/16 13:52:29 INFO tool.CodeGenTool: Beginning code generation
> 13/07/16 13:52:29 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM "publicinici" AS t LIMIT 1
> 13/07/16 13:52:29 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /home/hadoop/project/hadoop-1.0.4
> Note:
> /tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publicinici.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/07/16 13:52:30 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publicinici.jar
> 13/07/16 13:52:30 WARN manager.PostgresqlManager: It looks like you are
> importing from postgresql.
> 13/07/16 13:52:30 WARN manager.PostgresqlManager: This transfer can be
> faster! Use the --direct
> 13/07/16 13:52:30 WARN manager.PostgresqlManager: option to exercise a
> postgresql-specific fast path.
> 13/07/16 13:52:30 INFO mapreduce.ImportJobBase: Beginning import of
> publicinici
> 13/07/16 13:52:31 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
> SELECT MIN("budapablik"), MAX("budapablik") FROM "publicinici"
> 13/07/16 13:52:31 INFO mapred.JobClient: Running job: job_201302261146_0525
> 13/07/16 13:52:32 INFO mapred.JobClient: map 0% reduce 0%
> ^X[hadoop@ADUAE042-LAP-V bin]./sqoop version
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
> ===================================================
>
> *I have only one table which is "publicinici",that holds 2 entity in one
> row* and it read but at mapreduce phase it is getting stucked.
>
>
> *2) When I use this command in which --direct option is added, eveything
> is going well, but getting one error, which is *
>
> ERROR tool.ImportAllTablesTool: Encountered IOException running import
> job: java.io.IOException: Cannot run program "psql": java.io.IOException:
> error=2, No such file or directory
>
> *Also, in the hdfs publicinici file has been created as empty. *
>
> Here is the exact output.
> ========================
> 13/07/16 13:58:42 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/07/16 13:58:42 INFO tool.CodeGenTool: Beginning code generation
> 13/07/16 13:58:43 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM "publicinici" AS t LIMIT 1
> 13/07/16 13:58:43 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /home/hadoop/project/hadoop-1.0.4
> Note:
> /tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publicinici.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/07/16 13:58:44 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publicinici.jar
> 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Beginning psql
> fast path import
> 13/07/16 13:58:44 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM "publicinici" AS t LIMIT 1
> 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Copy command is
> COPY (SELECT "budapablik" FROM "publicinici" WHERE 1=1) TO STDOUT WITH
> DELIMITER E'\54' CSV ;
> 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Performing import
> of table publicinici from database IMS
> 13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transfer loop
> complete.
> 13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transferred 0
> bytes in 17,361,268.2044 seconds (0 bytes/sec)
> 13/07/16 13:58:45 ERROR tool.ImportAllTablesTool: Encountered IOException
> running import job: java.io.IOException: Cannot run program "psql":
> java.io.IOException: error=2, No such file or directory
>
>
> What should I do to be able to fix the problem.
> Any help is gonna be appreciated. Thank you very much.
>
>
>
Re: java.io.IOException: error=2, No such file or directory
Posted by Shahab Yunus <sh...@gmail.com>.
The error is:
*Please set $HBASE_HOME to the root of your HBase installation.*
*
*
Have you checked whether it is set or not? Have you verified your HBase or
Hadoop installation?
Similarly, the following:
*Cannot run program "psql": java.io.IOException: error=2, No such file or
directory *
Also seem to indicate that PostGres is not on the path. What and how are
your environment variables set? Can you access/run this independently from
this directory?
Regards,
Shahab
On Tue, Jul 16, 2013 at 6:44 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi everyone,
>
> I am trying to import data from postgresql to hdfs. But I am having some
> problems, Here is the problem details:
>
> Sqoop Version: 1.4.3
> Hadoop Version:1.0.4
>
>
> *1) When I use this command:*
> *
> *
> *./sqoop import-all-tables --connect jdbc:postgresql://
> 192.168.194.158:5432/IMS --username pgsql -P*
>
>
> Here is the exact output
> ======================================================
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
>
> Enter password:
> 13/07/16 13:52:28 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/07/16 13:52:29 INFO tool.CodeGenTool: Beginning code generation
> 13/07/16 13:52:29 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM "publicinici" AS t LIMIT 1
> 13/07/16 13:52:29 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /home/hadoop/project/hadoop-1.0.4
> Note:
> /tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publicinici.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/07/16 13:52:30 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publicinici.jar
> 13/07/16 13:52:30 WARN manager.PostgresqlManager: It looks like you are
> importing from postgresql.
> 13/07/16 13:52:30 WARN manager.PostgresqlManager: This transfer can be
> faster! Use the --direct
> 13/07/16 13:52:30 WARN manager.PostgresqlManager: option to exercise a
> postgresql-specific fast path.
> 13/07/16 13:52:30 INFO mapreduce.ImportJobBase: Beginning import of
> publicinici
> 13/07/16 13:52:31 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
> SELECT MIN("budapablik"), MAX("budapablik") FROM "publicinici"
> 13/07/16 13:52:31 INFO mapred.JobClient: Running job: job_201302261146_0525
> 13/07/16 13:52:32 INFO mapred.JobClient: map 0% reduce 0%
> ^X[hadoop@ADUAE042-LAP-V bin]./sqoop version
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
> ===================================================
>
> *I have only one table which is "publicinici",that holds 2 entity in one
> row* and it read but at mapreduce phase it is getting stucked.
>
>
> *2) When I use this command in which --direct option is added, eveything
> is going well, but getting one error, which is *
>
> ERROR tool.ImportAllTablesTool: Encountered IOException running import
> job: java.io.IOException: Cannot run program "psql": java.io.IOException:
> error=2, No such file or directory
>
> *Also, in the hdfs publicinici file has been created as empty. *
>
> Here is the exact output.
> ========================
> 13/07/16 13:58:42 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/07/16 13:58:42 INFO tool.CodeGenTool: Beginning code generation
> 13/07/16 13:58:43 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM "publicinici" AS t LIMIT 1
> 13/07/16 13:58:43 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /home/hadoop/project/hadoop-1.0.4
> Note:
> /tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publicinici.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/07/16 13:58:44 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publicinici.jar
> 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Beginning psql
> fast path import
> 13/07/16 13:58:44 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM "publicinici" AS t LIMIT 1
> 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Copy command is
> COPY (SELECT "budapablik" FROM "publicinici" WHERE 1=1) TO STDOUT WITH
> DELIMITER E'\54' CSV ;
> 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Performing import
> of table publicinici from database IMS
> 13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transfer loop
> complete.
> 13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transferred 0
> bytes in 17,361,268.2044 seconds (0 bytes/sec)
> 13/07/16 13:58:45 ERROR tool.ImportAllTablesTool: Encountered IOException
> running import job: java.io.IOException: Cannot run program "psql":
> java.io.IOException: error=2, No such file or directory
>
>
> What should I do to be able to fix the problem.
> Any help is gonna be appreciated. Thank you very much.
>
>
>
Re: java.io.IOException: error=2, No such file or directory
Posted by Shahab Yunus <sh...@gmail.com>.
The error is:
*Please set $HBASE_HOME to the root of your HBase installation.*
*
*
Have you checked whether it is set or not? Have you verified your HBase or
Hadoop installation?
Similarly, the following:
*Cannot run program "psql": java.io.IOException: error=2, No such file or
directory *
Also seem to indicate that PostGres is not on the path. What and how are
your environment variables set? Can you access/run this independently from
this directory?
Regards,
Shahab
On Tue, Jul 16, 2013 at 6:44 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi everyone,
>
> I am trying to import data from postgresql to hdfs. But I am having some
> problems, Here is the problem details:
>
> Sqoop Version: 1.4.3
> Hadoop Version:1.0.4
>
>
> *1) When I use this command:*
> *
> *
> *./sqoop import-all-tables --connect jdbc:postgresql://
> 192.168.194.158:5432/IMS --username pgsql -P*
>
>
> Here is the exact output
> ======================================================
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
>
> Enter password:
> 13/07/16 13:52:28 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/07/16 13:52:29 INFO tool.CodeGenTool: Beginning code generation
> 13/07/16 13:52:29 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM "publicinici" AS t LIMIT 1
> 13/07/16 13:52:29 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /home/hadoop/project/hadoop-1.0.4
> Note:
> /tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publicinici.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/07/16 13:52:30 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publicinici.jar
> 13/07/16 13:52:30 WARN manager.PostgresqlManager: It looks like you are
> importing from postgresql.
> 13/07/16 13:52:30 WARN manager.PostgresqlManager: This transfer can be
> faster! Use the --direct
> 13/07/16 13:52:30 WARN manager.PostgresqlManager: option to exercise a
> postgresql-specific fast path.
> 13/07/16 13:52:30 INFO mapreduce.ImportJobBase: Beginning import of
> publicinici
> 13/07/16 13:52:31 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
> SELECT MIN("budapablik"), MAX("budapablik") FROM "publicinici"
> 13/07/16 13:52:31 INFO mapred.JobClient: Running job: job_201302261146_0525
> 13/07/16 13:52:32 INFO mapred.JobClient: map 0% reduce 0%
> ^X[hadoop@ADUAE042-LAP-V bin]./sqoop version
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
> ===================================================
>
> *I have only one table which is "publicinici",that holds 2 entity in one
> row* and it read but at mapreduce phase it is getting stucked.
>
>
> *2) When I use this command in which --direct option is added, eveything
> is going well, but getting one error, which is *
>
> ERROR tool.ImportAllTablesTool: Encountered IOException running import
> job: java.io.IOException: Cannot run program "psql": java.io.IOException:
> error=2, No such file or directory
>
> *Also, in the hdfs publicinici file has been created as empty. *
>
> Here is the exact output.
> ========================
> 13/07/16 13:58:42 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/07/16 13:58:42 INFO tool.CodeGenTool: Beginning code generation
> 13/07/16 13:58:43 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM "publicinici" AS t LIMIT 1
> 13/07/16 13:58:43 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /home/hadoop/project/hadoop-1.0.4
> Note:
> /tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publicinici.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/07/16 13:58:44 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publicinici.jar
> 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Beginning psql
> fast path import
> 13/07/16 13:58:44 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM "publicinici" AS t LIMIT 1
> 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Copy command is
> COPY (SELECT "budapablik" FROM "publicinici" WHERE 1=1) TO STDOUT WITH
> DELIMITER E'\54' CSV ;
> 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Performing import
> of table publicinici from database IMS
> 13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transfer loop
> complete.
> 13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transferred 0
> bytes in 17,361,268.2044 seconds (0 bytes/sec)
> 13/07/16 13:58:45 ERROR tool.ImportAllTablesTool: Encountered IOException
> running import job: java.io.IOException: Cannot run program "psql":
> java.io.IOException: error=2, No such file or directory
>
>
> What should I do to be able to fix the problem.
> Any help is gonna be appreciated. Thank you very much.
>
>
>
Re: java.io.IOException: error=2, No such file or directory
Posted by Shahab Yunus <sh...@gmail.com>.
The error is:
*Please set $HBASE_HOME to the root of your HBase installation.*
*
*
Have you checked whether it is set or not? Have you verified your HBase or
Hadoop installation?
Similarly, the following:
*Cannot run program "psql": java.io.IOException: error=2, No such file or
directory *
Also seem to indicate that PostGres is not on the path. What and how are
your environment variables set? Can you access/run this independently from
this directory?
Regards,
Shahab
On Tue, Jul 16, 2013 at 6:44 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi everyone,
>
> I am trying to import data from postgresql to hdfs. But I am having some
> problems, Here is the problem details:
>
> Sqoop Version: 1.4.3
> Hadoop Version:1.0.4
>
>
> *1) When I use this command:*
> *
> *
> *./sqoop import-all-tables --connect jdbc:postgresql://
> 192.168.194.158:5432/IMS --username pgsql -P*
>
>
> Here is the exact output
> ======================================================
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
>
> Enter password:
> 13/07/16 13:52:28 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/07/16 13:52:29 INFO tool.CodeGenTool: Beginning code generation
> 13/07/16 13:52:29 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM "publicinici" AS t LIMIT 1
> 13/07/16 13:52:29 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /home/hadoop/project/hadoop-1.0.4
> Note:
> /tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publicinici.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/07/16 13:52:30 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publicinici.jar
> 13/07/16 13:52:30 WARN manager.PostgresqlManager: It looks like you are
> importing from postgresql.
> 13/07/16 13:52:30 WARN manager.PostgresqlManager: This transfer can be
> faster! Use the --direct
> 13/07/16 13:52:30 WARN manager.PostgresqlManager: option to exercise a
> postgresql-specific fast path.
> 13/07/16 13:52:30 INFO mapreduce.ImportJobBase: Beginning import of
> publicinici
> 13/07/16 13:52:31 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
> SELECT MIN("budapablik"), MAX("budapablik") FROM "publicinici"
> 13/07/16 13:52:31 INFO mapred.JobClient: Running job: job_201302261146_0525
> 13/07/16 13:52:32 INFO mapred.JobClient: map 0% reduce 0%
> ^X[hadoop@ADUAE042-LAP-V bin]./sqoop version
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
> ===================================================
>
> *I have only one table which is "publicinici",that holds 2 entity in one
> row* and it read but at mapreduce phase it is getting stucked.
>
>
> *2) When I use this command in which --direct option is added, eveything
> is going well, but getting one error, which is *
>
> ERROR tool.ImportAllTablesTool: Encountered IOException running import
> job: java.io.IOException: Cannot run program "psql": java.io.IOException:
> error=2, No such file or directory
>
> *Also, in the hdfs publicinici file has been created as empty. *
>
> Here is the exact output.
> ========================
> 13/07/16 13:58:42 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/07/16 13:58:42 INFO tool.CodeGenTool: Beginning code generation
> 13/07/16 13:58:43 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM "publicinici" AS t LIMIT 1
> 13/07/16 13:58:43 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /home/hadoop/project/hadoop-1.0.4
> Note:
> /tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publicinici.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/07/16 13:58:44 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publicinici.jar
> 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Beginning psql
> fast path import
> 13/07/16 13:58:44 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM "publicinici" AS t LIMIT 1
> 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Copy command is
> COPY (SELECT "budapablik" FROM "publicinici" WHERE 1=1) TO STDOUT WITH
> DELIMITER E'\54' CSV ;
> 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Performing import
> of table publicinici from database IMS
> 13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transfer loop
> complete.
> 13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transferred 0
> bytes in 17,361,268.2044 seconds (0 bytes/sec)
> 13/07/16 13:58:45 ERROR tool.ImportAllTablesTool: Encountered IOException
> running import job: java.io.IOException: Cannot run program "psql":
> java.io.IOException: error=2, No such file or directory
>
>
> What should I do to be able to fix the problem.
> Any help is gonna be appreciated. Thank you very much.
>
>
>