You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by 李运田 <cu...@163.com> on 2014/12/23 08:51:32 UTC
hive-hcatalog-pig
I use hcatalog as metadata Storage .so I can use hive to create table and manipulate data .I can also use pig to manipulate the table which Hive create.
now, I can use
"a = load 'tableHIVE' using org.apache.hcatalog.pig.HCatLoader();"
2014-12-23 15:31:45,791 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive
2014-12-23 15:31:45,791 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
2014-12-23 15:31:45,791 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
2014-12-23 15:31:45,791 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack
2014-12-23 15:31:45,791 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node
2014-12-23 15:31:45,791 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
2014-12-23 15:31:45,791 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
2014-12-23 15:31:46,318 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://10.210.90.*:9083
2014-12-23 15:31:46,407 [main] INFO hive.metastore - Connected to metastore.
2014-12-23 15:31:46,702 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
but when I store the data . I always get error：
"store a into 'tablePIG' using org.apache.hcatalog.pig.HCatStorer(); "
2014-12-23 15:36:54,051 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-12-23 15:36:54,164 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://10.210.90.*:9083
2014-12-23 15:36:54,166 [main] INFO hive.metastore - Connected to metastore.
2014-12-23 15:36:54,216 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2014-12-23 15:36:54,218 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, DuplicateForEachColumnRewrite, GroupByConstParallelSetter, ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, NewPartitionFilterOptimizer, PartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter], RULES_DISABLED=[FilterLogicExpressionSimplifier]}
2014-12-23 15:36:54,219 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-12-23 15:36:54,225 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2014-12-23 15:36:54,227 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2014-12-23 15:36:54,227 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2014-12-23 15:36:54,255 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at pdmdev00/10.210.90.101:8050
2014-12-23 15:36:54,266 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2014-12-23 15:36:54,272 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2014-12-23 15:36:54,373 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-12-23 15:36:54,380 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job8750450960344518471.jar
2014-12-23 15:36:59,084 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job8750450960344518471.jar created
2014-12-23 15:36:59,096 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
2014-12-23 15:36:59,145 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
2014-12-23 15:36:59,150 [JobControl] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at pdmdev00/10.210.90.101:8050
2014-12-23 15:37:00,123 [JobControl] INFO org.apache.hadoop.mapred.FileInputFormat - Total input paths to process : 1
2014-12-23 15:37:00,123 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2014-12-23 15:37:00,376 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - number of splits:1
2014-12-23 15:37:00,527 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: job_1418807862295_0055
2014-12-23 15:37:00,772 [JobControl] INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1418807862295_0055
2014-12-23 15:37:00,782 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The url to track the job: http://pdmdev00:8088/proxy/application_1418807862295_0055/
2014-12-23 15:37:00,783 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1418807862295_0055
2014-12-23 15:37:00,783 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases a
2014-12-23 15:37:00,783 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: a[1,4] C: R:
2014-12-23 15:37:00,827 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2014-12-23 15:37:15,910 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2014-12-23 15:37:15,910 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_1418807862295_0055 has failed! Stop running all dependent jobs
2014-12-23 15:37:15,910 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2014-12-23 15:37:15,971 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
2014-12-23 15:37:15,971 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
HadoopVersionPigVersionUserIdStartedAtFinishedAtFeatures
2.3.0-bc1.0.00.12.0-cdh5.0.0root2014-12-23 15:36:542014-12-23 15:37:15UNKNOWN
Failed!
Failed Jobs:
JobIdAliasFeatureMessageOutputs
job_1418807862295_0055aMAP_ONLYMessage: Job failed!hdfs://pdmdev00:8020/tmp/temp418476715/tmp-1157967161,
Input(s):
Failed to read data from "table1418909729552"
Output(s):
Failed to produce result in "hdfs://pdmdev00:8020/tmp/temp418476715/tmp-1157967161"
Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_1418807862295_0055
2014-12-23 15:37:15,972 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
2014-12-23 15:37:15,979 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias a
Details at logfile: /root/pig_1419302861874.log
Re: hive-hcatalog-pig
Posted by Daniel Dai <da...@hortonworks.com>.
You shall find detail log in Hadoop Node Manager GUI. The console message
does not tell much.
On Mon, Dec 22, 2014 at 11:51 PM, 李运田 <cu...@163.com> wrote:
> I use hcatalog as metadata Storage .so I can use hive to create table and
> manipulate data .I can also use pig to manipulate the table which Hive
> create.
> now, I can use
> "a = load 'tableHIVE' using org.apache.hcatalog.pig.HCatLoader();"
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.input.dir.recursive is deprecated. Instead, use
> mapreduce.input.fileinputformat.input.dir.recursive
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.max.split.size is
> deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.min.split.size is
> deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.min.split.size.per.rack is deprecated. Instead, use
> mapreduce.input.fileinputformat.split.minsize.per.rack
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.min.split.size.per.node is deprecated. Instead, use
> mapreduce.input.fileinputformat.split.minsize.per.node
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.reduce.speculative
> 2014-12-23 15:31:46,318 [main] INFO hive.metastore - Trying to connect to
> metastore with URI thrift://10.210.90.*:9083
> 2014-12-23 15:31:46,407 [main] INFO hive.metastore - Connected to
> metastore.
> 2014-12-23 15:31:46,702 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> but when I store the data . I always get error：
> "store a into 'tablePIG' using org.apache.hcatalog.pig.HCatStorer(); "
> 2014-12-23 15:36:54,051 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 2014-12-23 15:36:54,164 [main] INFO hive.metastore - Trying to connect to
> metastore with URI thrift://10.210.90.*:9083
> 2014-12-23 15:36:54,166 [main] INFO hive.metastore - Connected to
> metastore.
> 2014-12-23 15:36:54,216 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> script: UNKNOWN
> 2014-12-23 15:36:54,218 [main] INFO
> org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer -
> {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune,
> DuplicateForEachColumnRewrite, GroupByConstParallelSetter,
> ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter, MergeFilter,
> MergeForEach, NewPartitionFilterOptimizer, PartitionFilterOptimizer,
> PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter],
> RULES_DISABLED=[FilterLogicExpressionSimplifier]}
> 2014-12-23 15:36:54,219 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 2014-12-23 15:36:54,225 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
> File concatenation threshold: 100 optimistic? false
> 2014-12-23 15:36:54,227 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size before optimization: 1
> 2014-12-23 15:36:54,227 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size after optimization: 1
> 2014-12-23 15:36:54,255 [main] INFO org.apache.hadoop.yarn.client.RMProxy
> - Connecting to ResourceManager at pdmdev00/10.210.90.101:8050
> 2014-12-23 15:36:54,266 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added
> to the job
> 2014-12-23 15:36:54,272 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2014-12-23 15:36:54,373 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 2014-12-23 15:36:54,380 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - creating jar file Job8750450960344518471.jar
> 2014-12-23 15:36:59,084 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - jar file Job8750450960344518471.jar created
> 2014-12-23 15:36:59,096 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting up single store job
> 2014-12-23 15:36:59,145 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 1 map-reduce job(s) waiting for submission.
> 2014-12-23 15:36:59,150 [JobControl] INFO
> org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at
> pdmdev00/10.210.90.101:8050
> 2014-12-23 15:37:00,123 [JobControl] INFO
> org.apache.hadoop.mapred.FileInputFormat - Total input paths to process : 1
> 2014-12-23 15:37:00,123 [JobControl] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths (combined) to process : 1
> 2014-12-23 15:37:00,376 [JobControl] INFO
> org.apache.hadoop.mapreduce.JobSubmitter - number of splits:1
> 2014-12-23 15:37:00,527 [JobControl] INFO
> org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job:
> job_1418807862295_0055
> 2014-12-23 15:37:00,772 [JobControl] INFO
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted
> application application_1418807862295_0055
> 2014-12-23 15:37:00,782 [JobControl] INFO org.apache.hadoop.mapreduce.Job
> - The url to track the job:
> http://pdmdev00:8088/proxy/application_1418807862295_0055/
> 2014-12-23 15:37:00,783 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - HadoopJobId: job_1418807862295_0055
> 2014-12-23 15:37:00,783 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Processing aliases a
> 2014-12-23 15:37:00,783 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - detailed locations: M: a[1,4] C: R:
> 2014-12-23 15:37:00,827 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 0% complete
> 2014-12-23 15:37:15,910 [main] WARN
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to
> stop immediately on failure.
> 2014-12-23 15:37:15,910 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - job job_1418807862295_0055 has failed! Stop running all dependent jobs
> 2014-12-23 15:37:15,910 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 2014-12-23 15:37:15,971 [main] ERROR
> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
> 2014-12-23 15:37:15,971 [main] INFO
> org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
>
>
> HadoopVersionPigVersionUserIdStartedAtFinishedAtFeatures
> 2.3.0-bc1.0.00.12.0-cdh5.0.0root2014-12-23 15:36:542014-12-23
> 15:37:15UNKNOWN
> Failed!
> Failed Jobs:
> JobIdAliasFeatureMessageOutputs
> job_1418807862295_0055aMAP_ONLYMessage: Job
> failed!hdfs://pdmdev00:8020/tmp/temp418476715/tmp-1157967161,
> Input(s):
> Failed to read data from "table1418909729552"
> Output(s):
> Failed to produce result in
> "hdfs://pdmdev00:8020/tmp/temp418476715/tmp-1157967161"
> Counters:
> Total records written : 0
> Total bytes written : 0
> Spillable Memory Manager spill count : 0
> Total bags proactively spilled: 0
> Total records proactively spilled: 0
> Job DAG:
> job_1418807862295_0055
> 2014-12-23 15:37:15,972 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Failed!
> 2014-12-23 15:37:15,979 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1066: Unable to open iterator for alias a
> Details at logfile: /root/pig_1419302861874.log
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: hive-hcatalog-pig
Posted by Daniel Dai <da...@hortonworks.com>.
You shall find detail log in Hadoop Node Manager GUI. The console message
does not tell much.
On Mon, Dec 22, 2014 at 11:51 PM, 李运田 <cu...@163.com> wrote:
> I use hcatalog as metadata Storage .so I can use hive to create table and
> manipulate data .I can also use pig to manipulate the table which Hive
> create.
> now, I can use
> "a = load 'tableHIVE' using org.apache.hcatalog.pig.HCatLoader();"
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.input.dir.recursive is deprecated. Instead, use
> mapreduce.input.fileinputformat.input.dir.recursive
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.max.split.size is
> deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.min.split.size is
> deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.min.split.size.per.rack is deprecated. Instead, use
> mapreduce.input.fileinputformat.split.minsize.per.rack
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.min.split.size.per.node is deprecated. Instead, use
> mapreduce.input.fileinputformat.split.minsize.per.node
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 2014-12-23 15:31:45,791 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.reduce.speculative
> 2014-12-23 15:31:46,318 [main] INFO hive.metastore - Trying to connect to
> metastore with URI thrift://10.210.90.*:9083
> 2014-12-23 15:31:46,407 [main] INFO hive.metastore - Connected to
> metastore.
> 2014-12-23 15:31:46,702 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> but when I store the data . I always get error：
> "store a into 'tablePIG' using org.apache.hcatalog.pig.HCatStorer(); "
> 2014-12-23 15:36:54,051 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 2014-12-23 15:36:54,164 [main] INFO hive.metastore - Trying to connect to
> metastore with URI thrift://10.210.90.*:9083
> 2014-12-23 15:36:54,166 [main] INFO hive.metastore - Connected to
> metastore.
> 2014-12-23 15:36:54,216 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> script: UNKNOWN
> 2014-12-23 15:36:54,218 [main] INFO
> org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer -
> {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune,
> DuplicateForEachColumnRewrite, GroupByConstParallelSetter,
> ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter, MergeFilter,
> MergeForEach, NewPartitionFilterOptimizer, PartitionFilterOptimizer,
> PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter],
> RULES_DISABLED=[FilterLogicExpressionSimplifier]}
> 2014-12-23 15:36:54,219 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 2014-12-23 15:36:54,225 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
> File concatenation threshold: 100 optimistic? false
> 2014-12-23 15:36:54,227 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size before optimization: 1
> 2014-12-23 15:36:54,227 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size after optimization: 1
> 2014-12-23 15:36:54,255 [main] INFO org.apache.hadoop.yarn.client.RMProxy
> - Connecting to ResourceManager at pdmdev00/10.210.90.101:8050
> 2014-12-23 15:36:54,266 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added
> to the job
> 2014-12-23 15:36:54,272 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2014-12-23 15:36:54,373 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 2014-12-23 15:36:54,380 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - creating jar file Job8750450960344518471.jar
> 2014-12-23 15:36:59,084 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - jar file Job8750450960344518471.jar created
> 2014-12-23 15:36:59,096 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting up single store job
> 2014-12-23 15:36:59,145 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 1 map-reduce job(s) waiting for submission.
> 2014-12-23 15:36:59,150 [JobControl] INFO
> org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at
> pdmdev00/10.210.90.101:8050
> 2014-12-23 15:37:00,123 [JobControl] INFO
> org.apache.hadoop.mapred.FileInputFormat - Total input paths to process : 1
> 2014-12-23 15:37:00,123 [JobControl] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths (combined) to process : 1
> 2014-12-23 15:37:00,376 [JobControl] INFO
> org.apache.hadoop.mapreduce.JobSubmitter - number of splits:1
> 2014-12-23 15:37:00,527 [JobControl] INFO
> org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job:
> job_1418807862295_0055
> 2014-12-23 15:37:00,772 [JobControl] INFO
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted
> application application_1418807862295_0055
> 2014-12-23 15:37:00,782 [JobControl] INFO org.apache.hadoop.mapreduce.Job
> - The url to track the job:
> http://pdmdev00:8088/proxy/application_1418807862295_0055/
> 2014-12-23 15:37:00,783 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - HadoopJobId: job_1418807862295_0055
> 2014-12-23 15:37:00,783 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Processing aliases a
> 2014-12-23 15:37:00,783 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - detailed locations: M: a[1,4] C: R:
> 2014-12-23 15:37:00,827 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 0% complete
> 2014-12-23 15:37:15,910 [main] WARN
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to
> stop immediately on failure.
> 2014-12-23 15:37:15,910 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - job job_1418807862295_0055 has failed! Stop running all dependent jobs
> 2014-12-23 15:37:15,910 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 2014-12-23 15:37:15,971 [main] ERROR
> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
> 2014-12-23 15:37:15,971 [main] INFO
> org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
>
>
> HadoopVersionPigVersionUserIdStartedAtFinishedAtFeatures
> 2.3.0-bc1.0.00.12.0-cdh5.0.0root2014-12-23 15:36:542014-12-23
> 15:37:15UNKNOWN
> Failed!
> Failed Jobs:
> JobIdAliasFeatureMessageOutputs
> job_1418807862295_0055aMAP_ONLYMessage: Job
> failed!hdfs://pdmdev00:8020/tmp/temp418476715/tmp-1157967161,
> Input(s):
> Failed to read data from "table1418909729552"
> Output(s):
> Failed to produce result in
> "hdfs://pdmdev00:8020/tmp/temp418476715/tmp-1157967161"
> Counters:
> Total records written : 0
> Total bytes written : 0
> Spillable Memory Manager spill count : 0
> Total bags proactively spilled: 0
> Total records proactively spilled: 0
> Job DAG:
> job_1418807862295_0055
> 2014-12-23 15:37:15,972 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Failed!
> 2014-12-23 15:37:15,979 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1066: Unable to open iterator for alias a
> Details at logfile: /root/pig_1419302861874.log
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.