You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by TrevniUser <di...@cerner.com> on 2013/07/02 19:05:03 UTC
Re: Type cast exception
I am facing this same issue while using SortedKeyValueFile. Could you please
provide some pointers why in the first place is GenericRecord being picked
up and not SpecificRecord?
@Test
public void testSortedKeyValueFile() throws IOException {
Path filePath = new Path(test.getMethodName());
Schema keySchema = ReflectData.get().getSchema(Long.class);
Schema valSchema =
ReflectData.get().getSchema(DataSinkChangeLogContent.class);
SortedKeyValueFile.Writer.Options opts = new Options();
opts.withKeySchema(keySchema);
opts.withValueSchema(valSchema);
opts.withConfiguration(conf);
opts.withPath(filePath);
SortedKeyValueFile.Writer<Long, DataSinkChangeLogContent> writer =
new SortedKeyValueFile.Writer<Long, DataSinkChangeLogContent>(
opts);
for (long i = 0; i < 10; i++) {
writer.append(i, new DataSinkChangeLogContent(eType.toString(),
eKey.toString(), i, 15L));
}
writer.close();
SortedKeyValueFile.Reader.Options readerOpts = new
SortedKeyValueFile.Reader.Options();
readerOpts.withKeySchema(keySchema);
readerOpts.withValueSchema(valSchema);
readerOpts.withConfiguration(conf);
readerOpts.withPath(filePath);
SortedKeyValueFile.Reader<Long, DataSinkChangeLogContent> reader =
new SortedKeyValueFile.Reader<Long, DataSinkChangeLogContent>(
readerOpts);
Iterator<AvroKeyValue<Long, DataSinkChangeLogContent>> i = null;
if (reader.get(5L) != null) {
i = reader.iterator();
}
while (i != null && i.hasNext()) {
AvroKeyValue<Long, DataSinkChangeLogContent> keyValue =
i.next();
System.out.println("---------------------------- Clock: " +
keyValue.getValue());
}
* DataSinkChangeLogContent e = reader.get(5L);* // FAILS AT THIS LINE
reader.close();
}
EXCEPTION:
java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record
cannot be cast to com.abc.kepler.datasink.DataSinkChangeLogContent
at
com.abc.kepler.datasink.hdfs.HDFSDataSinkChangeLogScannerTest.testSortedKeyValueFile(HDFSDataSinkChangeLogScannerTest.java:926)
--
View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027727.html
Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by TrevniUser <di...@cerner.com>.
For now, as a work-around, we are thinking of switching to directly using the
schema instead of the generated class file. So, something like this:
Schema
schema=Schema.parse(getClass().getResourceAsStream("ChangeLogContent.avsc"));
Its still unresolved why avro 1.7.4 couldn't fix this issue though.
--
View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027743.html
Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by TrevniUser <di...@cerner.com>.
Thanks, Doug. I will track that issue for further updates.
--
View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027747.html
Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by Doug Cutting <cu...@apache.org>.
On Tue, Jul 2, 2013 at 10:45 AM, TrevniUser <di...@cerner.com> wrote:
> Updated but still facing the same issue. Any ideas why?
I just noticed that SortedKeyValueFile uses GenericDatumReader and
GenericDatumWriter, when it needs to use SpecificDatumReader and
SpecificDatumWriter to be able to return specific records. I filed
AVRO-1351 to address this.
https://issues.apache.org/jira/browse/AVRO-1351
Doug
Re: Type cast exception
Posted by TrevniUser <di...@cerner.com>.
Updated but still facing the same issue. Any ideas why?
Updated project dependencies. Now pulling in:
[INFO] +- org.apache.avro:avro:jar:1.7.4:compile
[INFO] +- org.apache.avro:avro-mapred:jar:1.7.4:compile
[INFO] | +- org.apache.avro:avro-ipc:jar:1.7.4:compile
[INFO] | \- org.apache.avro:avro-ipc:jar:tests:1.7.4:compile
EXCEPTION:
java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record
cannot be cast to com.abc.kepler.datasink.DataSinkChangeLogContent
--
View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027732.html
Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by Doug Cutting <cu...@apache.org>.
This was fixed in Avro 1.7.4. Please try upgrading and see if that helps.
Doug
On Tue, Jul 2, 2013 at 10:33 AM, TrevniUser <di...@cerner.com> wrote:
> Seems like it could be. Please confirm. I am using 'avro 1.7.3'. This is what
> my project dependencies look like:
>
> [INFO] +- org.apache.avro:avro:jar:1.7.3:compile
> [INFO] +- org.apache.avro:avro-mapred:jar:1.7.3:compile
> [INFO] | +- org.apache.avro:avro-ipc:jar:1.7.3:compile
> [INFO] | \- org.apache.avro:avro-ipc:jar:tests:1.7.3:compile
>
>
>
> --
> View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027730.html
> Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by TrevniUser <di...@cerner.com>.
Seems like it could be. Please confirm. I am using 'avro 1.7.3'. This is what
my project dependencies look like:
[INFO] +- org.apache.avro:avro:jar:1.7.3:compile
[INFO] +- org.apache.avro:avro-mapred:jar:1.7.3:compile
[INFO] | +- org.apache.avro:avro-ipc:jar:1.7.3:compile
[INFO] | \- org.apache.avro:avro-ipc:jar:tests:1.7.3:compile
--
View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027730.html
Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by Doug Cutting <cu...@apache.org>.
Could this be AVRO-1240? What version of Avro are you using?
https://issues.apache.org/jira/browse/AVRO-1240
Doug
On Tue, Jul 2, 2013 at 10:05 AM, TrevniUser <di...@cerner.com> wrote:
> I am facing this same issue while using SortedKeyValueFile. Could you please
> provide some pointers why in the first place is GenericRecord being picked
> up and not SpecificRecord?
>
> @Test
> public void testSortedKeyValueFile() throws IOException {
> Path filePath = new Path(test.getMethodName());
>
> Schema keySchema = ReflectData.get().getSchema(Long.class);
> Schema valSchema =
> ReflectData.get().getSchema(DataSinkChangeLogContent.class);
>
> SortedKeyValueFile.Writer.Options opts = new Options();
> opts.withKeySchema(keySchema);
> opts.withValueSchema(valSchema);
> opts.withConfiguration(conf);
> opts.withPath(filePath);
> SortedKeyValueFile.Writer<Long, DataSinkChangeLogContent> writer =
> new SortedKeyValueFile.Writer<Long, DataSinkChangeLogContent>(
> opts);
> for (long i = 0; i < 10; i++) {
> writer.append(i, new DataSinkChangeLogContent(eType.toString(),
> eKey.toString(), i, 15L));
> }
> writer.close();
>
> SortedKeyValueFile.Reader.Options readerOpts = new
> SortedKeyValueFile.Reader.Options();
> readerOpts.withKeySchema(keySchema);
> readerOpts.withValueSchema(valSchema);
> readerOpts.withConfiguration(conf);
> readerOpts.withPath(filePath);
> SortedKeyValueFile.Reader<Long, DataSinkChangeLogContent> reader =
> new SortedKeyValueFile.Reader<Long, DataSinkChangeLogContent>(
> readerOpts);
> Iterator<AvroKeyValue<Long, DataSinkChangeLogContent>> i = null;
> if (reader.get(5L) != null) {
> i = reader.iterator();
> }
> while (i != null && i.hasNext()) {
> AvroKeyValue<Long, DataSinkChangeLogContent> keyValue =
> i.next();
> System.out.println("---------------------------- Clock: " +
> keyValue.getValue());
> }
> * DataSinkChangeLogContent e = reader.get(5L);* // FAILS AT THIS LINE
> reader.close();
> }
>
> EXCEPTION:
>
> java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record
> cannot be cast to com.abc.kepler.datasink.DataSinkChangeLogContent
> at
> com.abc.kepler.datasink.hdfs.HDFSDataSinkChangeLogScannerTest.testSortedKeyValueFile(HDFSDataSinkChangeLogScannerTest.java:926)
>
>
>
> --
> View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027727.html
> Sent from the Avro - Users mailing list archive at Nabble.com.