You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Divya Gehlot <di...@gmail.com> on 2016/01/21 07:45:21 UTC
Unable to read the input file : Pig DBStorage to MySQL
am trying to store data to MySQL using DBStorage in pig script. when I run
the the script ,I am getting error unable to read the input file. Whereas
when I try to dump to store the same data file its working fine. Sample
pigscript :
%default DATABASE_HOST 'localhost';%default DATABASE_NAME
'Test';%default DATABASE_USER 'username';%default DATABASE_PASS
'password';%default DATABASE_DRIVER 'com.mysql.jdbc.Driver';%default
DATABASE_TYPE 'mysql';
A = LOAD '/tmp/TestDivya/Pig/PigSQLTest.txt' using PigStorage() as
(name: chararray);
STORE A into 'test' using
org.apache.pig.piggybank.storage.DBStorage('$DATABASE_DRIVER',
'jdbc:$DATABASE_TYPE://$DATABASE_HOST/$DATABASE_NAME',
'$DATABASE_USER', '$DATABASE_PASS', 'INSERT INTO Test(name) VALUES
(?)');
HadoopVersion PigVersion UserId StartedAt FinishedAt
Features2.7.1.2.3.4.0-3485 0.15.0.2.3.4.0-3485 hdfs
2016-01-21 01:34:552016-01-21 01:35:07 UNKNOWN
Failed!
Failed Jobs:
JobId Alias Feature Message Outputs
job_1453263223178_0042 A MAP_ONLY Message: Job failed!
hdfs://ip-172-31-29-201.ap-southeast-1.compute.internal:8020/user/hdfs/test,
Input(s):
Failed to read data from "/tmp/TestDivya/Pig/PigSQLTest.txt"
Output(s):
Failed to produce result in
"hdfs://ip-xxx-xx-xx-xxx.ap-xxxxxx-1.compute.internal:8020/user/hdfs/test"
Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_1453263223178_0042
My cluster is set up on EC2 having Hortonworks HDP 2.3.4 and using the HDP
MYSQL database for storing the pig output.
Would really appreciate the pointers.
Thanks,
Re: Unable to read the input file : Pig DBStorage to MySQL
Posted by Divya Gehlot <di...@gmail.com>.
Found the resolution
Refer the link below
http://stackoverflow.com/a/34920051/4981746
On 21 January 2016 at 14:45, Divya Gehlot <di...@gmail.com> wrote:
> am trying to store data to MySQL using DBStorage in pig script. when I
> run the the script ,I am getting error unable to read the input file.
> Whereas when I try to dump to store the same data file its working fine.
> Sample pigscript :
>
> %default DATABASE_HOST 'localhost';%default DATABASE_NAME 'Test';%default DATABASE_USER 'username';%default DATABASE_PASS 'password';%default DATABASE_DRIVER 'com.mysql.jdbc.Driver';%default DATABASE_TYPE 'mysql';
>
>
>
> A = LOAD '/tmp/TestDivya/Pig/PigSQLTest.txt' using PigStorage() as (name: chararray);
> STORE A into 'test' using org.apache.pig.piggybank.storage.DBStorage('$DATABASE_DRIVER', 'jdbc:$DATABASE_TYPE://$DATABASE_HOST/$DATABASE_NAME', '$DATABASE_USER', '$DATABASE_PASS', 'INSERT INTO Test(name) VALUES (?)');
>
> HadoopVersion PigVersion UserId StartedAt FinishedAt Features2.7.1.2.3.4.0-3485 0.15.0.2.3.4.0-3485 hdfs 2016-01-21 01:34:552016-01-21 01:35:07 UNKNOWN
>
> Failed!
>
> Failed Jobs:
> JobId Alias Feature Message Outputs
> job_1453263223178_0042 A MAP_ONLY Message: Job failed! hdfs://ip-172-31-29-201.ap-southeast-1.compute.internal:8020/user/hdfs/test,
>
> Input(s):
> Failed to read data from "/tmp/TestDivya/Pig/PigSQLTest.txt"
>
> Output(s):
> Failed to produce result in "hdfs://ip-xxx-xx-xx-xxx.ap-xxxxxx-1.compute.internal:8020/user/hdfs/test"
>
> Counters:
> Total records written : 0
> Total bytes written : 0
> Spillable Memory Manager spill count : 0
> Total bags proactively spilled: 0
> Total records proactively spilled: 0
>
> Job DAG:
> job_1453263223178_0042
>
> My cluster is set up on EC2 having Hortonworks HDP 2.3.4 and using the HDP
> MYSQL database for storing the pig output.
>
> Would really appreciate the pointers.
>
> Thanks,
>