You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Ryan Skraba (Jira)" <ji...@apache.org> on 2022/07/01 17:42:00 UTC
[jira] [Updated] (AVRO-3266) Output stream incompatible with MagicS3GuardCommitter
[ https://issues.apache.org/jira/browse/AVRO-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan Skraba updated AVRO-3266:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks so much for the two implementations! I'll make sure that the Jira for dropping Hadoop2 support includes using the non-reflection change instead.
Cherry-picked to [branch-1.11|https://github.com/apache/avro/commit/8caab7bd4a47e2c8e048f2c42478bdb22b18c993].
> Output stream incompatible with MagicS3GuardCommitter
> -----------------------------------------------------
>
> Key: AVRO-3266
> URL: https://issues.apache.org/jira/browse/AVRO-3266
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Reporter: Michiel de Jong
> Assignee: Emil Ejbyfeldt
> Priority: Minor
> Labels: pull-request-available
> Fix For: 1.11.1
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Avro's output stream can not be used in combination with the MagicS3GuardCommitter
> {code:java}
> Error: java.lang.ClassCastException: class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter cannot be cast to class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter{code}
> The reason for this problem is that AvroOutputFormatBase.getAvroFileOutputStream tries to cast the outputcommiter to a FileOutputCommitter.
> It can be solved by casting to a PathOutputCommitter instead (which is a superclass of both the FileOutputCommitter and the MagicS3GuardCommitter)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)