You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "Chris Li (Jira)" <ji...@apache.org> on 2020/02/19 03:51:00 UTC
[jira] [Commented] (GOBBLIN-1055) Gaas jobs not able to write to
state store
[ https://issues.apache.org/jira/browse/GOBBLIN-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039683#comment-17039683 ]
Chris Li commented on GOBBLIN-1055:
-----------------------------------
{quote}[Loaded org.apache.log4j.DefaultThrowableRenderer from file:/export/apps/azkaban/azkaban-exec-server/azkaban-exec-server-1102356.01216_00358/executions/22966290/lib/gobblin-azkaban-0.0.13-all.jar]
[Loaded java.io.LineNumberReader from /export/apps/jdk/JDK-1_8_0_172/jre/lib/rt.jar]
WARN Serialization class not found:
java.lang.ClassNotFoundException: Class *org.apache.gobblin.util.WritableShimSerialization* not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2124)
at org.apache.hadoop.io.serializer.SerializationFactory.add(SerializationFactory.java:71)
at org.apache.hadoop.io.serializer.SerializationFactory.<init>(SerializationFactory.java:63)
at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1238)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:1156)
at org.apache.hadoop.io.SequenceFile$BlockCompressWriter.<init>(SequenceFile.java:1511)
at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:277)
at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:372)
at org.apache.gobblin.metastore.FsStateStore.put(FsStateStore.java:167)
at org.apache.gobblin.runtime.FsDatasetStateStore.persistDatasetState(FsDatasetStateStore.java:390)
at org.apache.gobblin.runtime.FsDatasetStateStore.persistDatasetState(FsDatasetStateStore.java:90)
at org.apache.gobblin.runtime.SafeDatasetCommit.persistDatasetState(SafeDatasetCommit.java:422)
at org.apache.gobblin.runtime.SafeDatasetCommit.call(SafeDatasetCommit.java:192)
at org.apache.gobblin.runtime.SafeDatasetCommit.call(SafeDatasetCommit.java:65)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.gobblin.util.executors.MDCPropagatingRunnable.run(MDCPropagatingRunnable.java:39)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[Loaded org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer from file:/export/apps/hadoop/hadoop-bin_274273/share/hadoop/common/hadoop-common-2.7.4.273.jar]
[Loaded org.apache.hadoop.io.serializer.avro.AvroReflectSerializable from file:/export/apps/hadoop/hadoop-bin_274273/share/hadoop/common/hadoop-common-2.7.4.273.jar]
[Loaded com.google.common.base.Throwables from file:/export/apps/azkaban/azkaban-exec-server/azkaban-exec-server-1102356.01216_00358/executions/22966290/lib/gobblin-azkaban-0.0.13-all.jar]
ERROR Failed to persist dataset state for dataset [watermark.datetime.1546329600000, watermark.unit.{}] of job job_zendeskTest_zendeskTicketMetricsChrisTest7_PullTicketMetricsFromZendesk_zendesk_holdem_httpserver-pull_-2010674809_1581127818658
java.io.IOException: Could not find a serializer for the Value class: 'org.apache.gobblin.runtime.JobState.DatasetState'. Please ensure that the configuration 'io.serializations' is properly configured, if you're usingcustom serialization.
at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1254)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:1156)
at org.apache.hadoop.io.SequenceFile$BlockCompressWriter.<init>(SequenceFile.java:1511)
at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:277)
at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:372)
at org.apache.gobblin.metastore.FsStateStore.put(FsStateStore.java:167)
at org.apache.gobblin.runtime.FsDatasetStateStore.persistDatasetState(FsDatasetStateStore.java:390)
at org.apache.gobblin.runtime.FsDatasetStateStore.persistDatasetState(FsDatasetStateStore.java:90)
at org.apache.gobblin.runtime.SafeDatasetCommit.persistDatasetState(SafeDatasetCommit.java:422)
at org.apache.gobblin.runtime.SafeDatasetCommit.call(SafeDatasetCommit.java:192)
at org.apache.gobblin.runtime.SafeDatasetCommit.call(SafeDatasetCommit.java:65)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.gobblin.util.executors.MDCPropagatingRunnable.run(MDCPropagatingRunnable.java:39)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
INFO Attempting to shutdown ExecutorService: com.google.common.util.concurrent.MoreExecutors$ListeningDecorator@5b9c78
{quote}
> Gaas jobs not able to write to state store
> ------------------------------------------
>
> Key: GOBBLIN-1055
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1055
> Project: Apache Gobblin
> Issue Type: Bug
> Components: state-management
> Affects Versions: 0.14.0
> Reporter: Chris Li
> Assignee: Hung Tran
> Priority: Major
>
> When state.store.enabled=true and state.store target is on HDFS, Gaas flows run into error when writing job state. The reason is that Gaas used a particularly GobblinClassLoader to load classes, and without giving that information to Hadoop Configuration, the Hadoop hook tries to load the custom serializer using default Class Loader.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)