You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/07/06 16:06:13 UTC
[GitHub] [hudi] tooptoop4 opened a new issue #1801: [SUPPORT] org.apache.avro.AvroTypeException: Found com.uber.hoodie.avro.model.HoodieCleanMetadata, expecting org.apache.hudi.avro.model.HoodieCleanerPlan, missing required field policy
tooptoop4 opened a new issue #1801:
URL: https://github.com/apache/hudi/issues/1801
COW data was written with hoodie 0.4.6, now when doing writes with 0.5.3 get this warning (job still completes)
```
2020-07-06 08:39:05,801 [main] INFO org.apache.hudi.common.table.HoodieTableConfig - Loading table properties from s3a://redact/.hoodie/hoodie.properties
2020-07-06 08:39:05,830 [main] INFO org.apache.hudi.common.table.HoodieTableMetaClient - Finished Loading Table of type COPY_ON_WRITE(version=1) from s3a://redact/
2020-07-06 08:39:05,830 [main] INFO org.apache.hudi.common.table.HoodieTableMetaClient - Loading Active commit timeline for s3a://redact/
2020-07-06 08:39:05,907 [main] INFO org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants [[==>20200520013324__commit__INFLIGHT], [==>20200520013643__commit__INFLIGHT], [==>20200528042521__commit__INFLIGHT], [==>20200528042827__commit__INFLIGHT], [==>20200528043104__commit__INFLIGHT], [==>20200528043932__commit__INFLIGHT], [==>20200528073521__commit__INFLIGHT], [==>20200603095037__commit__INFLIGHT], [==>20200624045351__clean__INFLIGHT], [==>20200701042622__commit__INFLIGHT], [==>20200701044730__clean__INFLIGHT], [20200706081301__clean__COMPLETED], [20200706081301__commit__COMPLETED], [20200706083709__clean__COMPLETED], [20200706083709__commit__COMPLETED], [20200706083850__commit__COMPLETED]]
2020-07-06 08:39:05,910 [main] INFO org.apache.hudi.client.HoodieCleanClient - There were previously unfinished cleaner operations. Finishing Instant=[==>20200624045351__clean__INFLIGHT]
2020-07-06 08:39:05,968 [main] WARN org.apache.hudi.client.HoodieCleanClient - Failed to perform previous clean operation, instant: [==>20200624045351__clean__INFLIGHT]
org.apache.avro.AvroTypeException: Found com.uber.hoodie.avro.model.HoodieCleanMetadata, expecting org.apache.hudi.avro.model.HoodieCleanerPlan, missing required field policy
at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:292)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
at org.apache.avro.io.ResolvingDecoder.readFieldOrder(ResolvingDecoder.java:130)
at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:215)
at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145)
at org.apache.avro.file.DataFileStream.next(DataFileStream.java:233)
at org.apache.avro.file.DataFileStream.next(DataFileStream.java:220)
at org.apache.hudi.common.util.AvroUtils.deserializeAvroMetadata(AvroUtils.java:148)
at org.apache.hudi.common.util.CleanerUtils.getCleanerPlan(CleanerUtils.java:87)
at org.apache.hudi.client.HoodieCleanClient.runClean(HoodieCleanClient.java:145)
at org.apache.hudi.client.HoodieCleanClient.lambda$clean$32(HoodieCleanClient.java:89)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647)
at org.apache.hudi.client.HoodieCleanClient.clean(HoodieCleanClient.java:86)
at org.apache.hudi.client.HoodieWriteClient.clean(HoodieWriteClient.java:836)
at org.apache.hudi.client.HoodieWriteClient.postCommit(HoodieWriteClient.java:513)
at org.apache.hudi.client.AbstractHoodieWriteClient.commit(AbstractHoodieWriteClient.java:157)
at org.apache.hudi.client.AbstractHoodieWriteClient.commit(AbstractHoodieWriteClient.java:101)
at org.apache.hudi.client.AbstractHoodieWriteClient.commit(AbstractHoodieWriteClient.java:92)
2020-07-06 08:39:05,971 [main] INFO org.apache.hudi.client.HoodieCleanClient - There were previously unfinished cleaner operations. Finishing Instant=[==>20200701044730__clean__INFLIGHT]
2020-07-06 08:39:06,015 [main] WARN org.apache.hudi.client.HoodieCleanClient - Failed to perform previous clean operation, instant: [==>20200701044730__clean__INFLIGHT]
org.apache.avro.AvroTypeException: Found com.uber.hoodie.avro.model.HoodieCleanMetadata, expecting org.apache.hudi.avro.model.HoodieCleanerPlan, missing required field policy
at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:292)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
at org.apache.avro.io.ResolvingDecoder.readFieldOrder(ResolvingDecoder.java:130)
at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:215)
at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145)
at org.apache.avro.file.DataFileStream.next(DataFileStream.java:233)
at org.apache.avro.file.DataFileStream.next(DataFileStream.java:220)
at org.apache.hudi.common.util.AvroUtils.deserializeAvroMetadata(AvroUtils.java:148)
at org.apache.hudi.common.util.CleanerUtils.getCleanerPlan(CleanerUtils.java:87)
at org.apache.hudi.client.HoodieCleanClient.runClean(HoodieCleanClient.java:145)
at org.apache.hudi.client.HoodieCleanClient.lambda$clean$32(HoodieCleanClient.java:89)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647)
at org.apache.hudi.client.HoodieCleanClient.clean(HoodieCleanClient.java:86)
at org.apache.hudi.client.HoodieWriteClient.clean(HoodieWriteClient.java:836)
at org.apache.hudi.client.HoodieWriteClient.postCommit(HoodieWriteClient.java:513)
at org.apache.hudi.client.AbstractHoodieWriteClient.commit(AbstractHoodieWriteClient.java:157)
at org.apache.hudi.client.AbstractHoodieWriteClient.commit(AbstractHoodieWriteClient.java:101)
at org.apache.hudi.client.AbstractHoodieWriteClient.commit(AbstractHoodieWriteClient.java:92)
2020-07-06 08:39:06,022 [main] INFO org.apache.hudi.common.table.HoodieTableMetaClient - Loading HoodieTableMetaClient from
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] tooptoop4 commented on issue #1801: [SUPPORT] org.apache.avro.AvroTypeException: Found com.uber.hoodie.avro.model.HoodieCleanMetadata, expecting org.apache.hudi.avro.model.HoodieCleanerPlan, missing required field policy
Posted by GitBox <gi...@apache.org>.
tooptoop4 commented on issue #1801:
URL: https://github.com/apache/hudi/issues/1801#issuecomment-654534604
just warning, closing
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] tooptoop4 closed issue #1801: [SUPPORT] org.apache.avro.AvroTypeException: Found com.uber.hoodie.avro.model.HoodieCleanMetadata, expecting org.apache.hudi.avro.model.HoodieCleanerPlan, missing required field policy
Posted by GitBox <gi...@apache.org>.
tooptoop4 closed issue #1801:
URL: https://github.com/apache/hudi/issues/1801
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org