You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@gobblin.apache.org by Moyo Oyegunle <mo...@gmail.com> on 2018/09/04 14:00:57 UTC
Gobblin Class Not Found Exception
Hello
I am trying to run gobblin in mapreduce method first(plan to move to native
yarn after that). It seems to keep failing due to a class not found
error.This is the error i see on one of the nodes that runs the application:
2018-09-03 23:15:26,875 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for
application appattempt_1536024741699_0007_000002
2018-09-03 23:15:27,142 WARN [main]
org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
2018-09-03 23:15:27,236 INFO [main]
org.apache.hadoop.security.SecurityUtil: Updating Configuration
2018-09-03 23:15:27,236 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2018-09-03 23:15:27,236 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN,
Service: , Ident: (appAttemptId { application_id { id: 7 cluster_timestamp:
1536024741699 } attemptId: 2 } keyId: 1668433044)
2018-09-03 23:15:27,513 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred
newApiCommitter.
2018-09-03 23:15:27,515 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in
config null
2018-09-03 23:15:27,561 FATAL [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.NoClassDefFoundError: org/apache/gobblin/configuration/State
at
org.apache.gobblin.runtime.mapreduce.GobblinOutputFormat.getOutputCommitter(GobblinOutputFormat.java:33)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:519)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:501)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1640)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:501)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:287)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1598)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1595)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1526)
Caused by: java.lang.ClassNotFoundException:
org.apache.gobblin.configuration.State
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 13 more
2018-09-03 23:15:27,566 INFO [main] org.apache.hadoop.util.ExitUtil:
Exiting with status 1
I get this error even when running the default job here:
https://gobblin.readthedocs.io/en/latest/case-studies/Kafka-HDFS-Ingestion/
My conf:
job.name=GobblinKafkaQuickStart
job.group=GobblinKafka
job.description=Gobblin quick start job for Kafka
job.lock.enabled=false
kafka.brokers=confluent1:9092
source.class=org.apache.gobblin.source.extractor.extract.kafka.KafkaSimpleSource
extract.namespace=org.apache.gobblin.extract.kafka
writer.builder.class=org.apache.gobblin.writer.SimpleDataWriterBuilder
writer.file.path.type=tablename
writer.destination.type=HDFS
writer.output.format=txt
data.publisher.type=org.apache.gobblin.publisher.BaseDataPublisher
mr.job.max.mappers=1
metrics.reporting.file.enabled=true
metrics.log.dir=/gobblin-kafka/metrics
metrics.reporting.file.suffix=txt
bootstrap.with.offset=earliest
fs.uri=hdfs://ambari-1.mytest.lab.local:8020
writer.fs.uri=hdfs://ambari-1.mytest.lab.local:8020
state.store.fs.uri=hdfs://ambari-1.mytest.lab.local:8020
mr.job.root.dir=/gobblin-kafka/working
state.store.dir=/gobblin-kafka/state-store
task.data.root.dir=/jobs/kafkaetl/gobblin/gobblin-kafka/task-data
data.publisher.final.dir=/gobblintest/job-output
this is the error i see in gobblin log:
2018-09-03 23:15:28 EDT INFO [main] org.apache.hadoop.mapreduce.Job 1660
- Job job_1536024741699_0007 failed with state FAILED due to: Application
application_1536024741699_0007 failed 2 times due to AM Container for
appattempt_1536024741699_0007_000002 exited with exitCode: 1
For more detailed output, check the application tracking page:
http://ambari-2.mytest.lab.local:8088/cluster/app/application_1536024741699_0007
Then click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_e06_1536024741699_0007_02_000001
Exit code: 1
Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
Failing this attempt. Failing the application.
Would appreciate any help?
--
Best Regards
Moyo
Re: Gobblin Class Not Found Exception
Posted by Moyo Oyegunle <mo...@gmail.com>.
Hello All
Figured out what my problems were and am documenting them for posterity
sake.
1 The 1st problem seems to have been a permissions issue while the job jars
files looked like they were copied to the containers they actually were
not. Using the "job.hdfs.files" option and copying the files into that hdfs
folder and setting rwx on that folder seems to have fixed this part.
2 The 2nd problem seems to have been I was building gobblin with the open
source hadoop version and it didn't work correctly cause I was using the
hortonworks distribution. Adding the hortonworks distribution repo(Can be
found on the maven repository or hortonworks site) to the gobblin
build.gradle file and then building gobblin with the -phadoop option that
matches the hortonworks hadoop version fixed this problem.
Thanks.
On Tue, Sep 4, 2018 at 10:00 AM Moyo Oyegunle <mo...@gmail.com>
wrote:
> Hello
> I am trying to run gobblin in mapreduce method first(plan to move to
> native yarn after that). It seems to keep failing due to a class not found
> error.This is the error i see on one of the nodes that runs the application:
>
> 2018-09-03 23:15:26,875 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for
> application appattempt_1536024741699_0007_000002
> 2018-09-03 23:15:27,142 WARN [main]
> org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 2018-09-03 23:15:27,236 INFO [main]
> org.apache.hadoop.security.SecurityUtil: Updating Configuration
> 2018-09-03 23:15:27,236 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
> 2018-09-03 23:15:27,236 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN,
> Service: , Ident: (appAttemptId { application_id { id: 7 cluster_timestamp:
> 1536024741699 } attemptId: 2 } keyId: 1668433044)
> 2018-09-03 23:15:27,513 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred
> newApiCommitter.
> 2018-09-03 23:15:27,515 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in
> config null
> 2018-09-03 23:15:27,561 FATAL [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
> java.lang.NoClassDefFoundError: org/apache/gobblin/configuration/State
> at
> org.apache.gobblin.runtime.mapreduce.GobblinOutputFormat.getOutputCommitter(GobblinOutputFormat.java:33)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:519)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:501)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1640)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:501)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:287)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1598)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1595)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1526)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.gobblin.configuration.State
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 13 more
> 2018-09-03 23:15:27,566 INFO [main] org.apache.hadoop.util.ExitUtil:
> Exiting with status 1
> I get this error even when running the default job here:
> https://gobblin.readthedocs.io/en/latest/case-studies/Kafka-HDFS-Ingestion/
> My conf:
> job.name=GobblinKafkaQuickStart
> job.group=GobblinKafka
> job.description=Gobblin quick start job for Kafka
> job.lock.enabled=false
>
> kafka.brokers=confluent1:9092
>
>
> source.class=org.apache.gobblin.source.extractor.extract.kafka.KafkaSimpleSource
> extract.namespace=org.apache.gobblin.extract.kafka
>
> writer.builder.class=org.apache.gobblin.writer.SimpleDataWriterBuilder
> writer.file.path.type=tablename
> writer.destination.type=HDFS
> writer.output.format=txt
>
> data.publisher.type=org.apache.gobblin.publisher.BaseDataPublisher
>
> mr.job.max.mappers=1
>
> metrics.reporting.file.enabled=true
> metrics.log.dir=/gobblin-kafka/metrics
> metrics.reporting.file.suffix=txt
>
> bootstrap.with.offset=earliest
>
> fs.uri=hdfs://ambari-1.mytest.lab.local:8020
> writer.fs.uri=hdfs://ambari-1.mytest.lab.local:8020
> state.store.fs.uri=hdfs://ambari-1.mytest.lab.local:8020
>
> mr.job.root.dir=/gobblin-kafka/working
> state.store.dir=/gobblin-kafka/state-store
> task.data.root.dir=/jobs/kafkaetl/gobblin/gobblin-kafka/task-data
> data.publisher.final.dir=/gobblintest/job-output
>
> this is the error i see in gobblin log:
> 2018-09-03 23:15:28 EDT INFO [main] org.apache.hadoop.mapreduce.Job 1660
> - Job job_1536024741699_0007 failed with state FAILED due to: Application
> application_1536024741699_0007 failed 2 times due to AM Container for
> appattempt_1536024741699_0007_000002 exited with exitCode: 1
> For more detailed output, check the application tracking page:
> http://ambari-2.mytest.lab.local:8088/cluster/app/application_1536024741699_0007
> Then click on links to logs of each attempt.
> Diagnostics: Exception from container-launch.
> Container id: container_e06_1536024741699_0007_02_000001
> Exit code: 1
>
> Container exited with a non-zero exit code 1. Error file: prelaunch.err.
> Last 4096 bytes of prelaunch.err :
> Last 4096 bytes of stderr :
> log4j:WARN No appenders could be found for logger
> (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> more info.
>
> Failing this attempt. Failing the application.
>
>
> Would appreciate any help?
> --
> Best Regards
> Moyo
>
--
Best Regards
Moyo Oyegunle