You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by Jaydeep Ayachit <ja...@persistent.co.in> on 2010/04/26 16:59:33 UTC
Type cast exception
Hello,
I am trying to read from avro data file and want to use specific object (exa sample.LogMessage)
The following code gives exception mentioned below. Reader.next() when collected in Object/Record, no exception is thrown which is expected.
Any help is appreciated.
public void testMethod5() throws Exception {
DatumWriter<LogMessage> writer = new GenericDatumWriter<LogMessage>(LogMessage.SCHEMA$);
DataFileWriter<LogMessage> out = new DataFileWriter<LogMessage>(writer);
out.create(LogMessage.SCHEMA$, <File>);
for (int i = 0; i < 100; i++) {
LogMessage msg = new LogMessage();
msg.applicationName = new Utf8("app" + i);
msg.userid = new Utf8("user" + i);
msg.message = new Utf8("this is test log message " + i);
out.append(msg);
}
out.close();
DataFileReader<LogMessage> reader;
GenericDatumReader<LogMessage> din = new GenericDatumReader<LogMessage>(LogMessage.SCHEMA$);
reader = new DataFileReader<LogMessage>(new File(<>), din);
while (reader.hasNext()) {
LogMessage msg = reader.next();
System.out.println(msg.toString());
}
reader.close();
}
Exception
java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record cannot be cast to sample.LogMessage
at com.ibm.clas.ReaderWriter.testMethod5(ReaderWriter.java:313)
at com.ibm.clas.ReaderWriter.main(ReaderWriter.java:322)
Thanks
Jaydeep
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
Re: Type cast exception
Posted by TrevniUser <di...@cerner.com>.
For now, as a work-around, we are thinking of switching to directly using the
schema instead of the generated class file. So, something like this:
Schema
schema=Schema.parse(getClass().getResourceAsStream("ChangeLogContent.avsc"));
Its still unresolved why avro 1.7.4 couldn't fix this issue though.
--
View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027743.html
Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by TrevniUser <di...@cerner.com>.
Thanks, Doug. I will track that issue for further updates.
--
View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027747.html
Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by Doug Cutting <cu...@apache.org>.
On Tue, Jul 2, 2013 at 10:45 AM, TrevniUser <di...@cerner.com> wrote:
> Updated but still facing the same issue. Any ideas why?
I just noticed that SortedKeyValueFile uses GenericDatumReader and
GenericDatumWriter, when it needs to use SpecificDatumReader and
SpecificDatumWriter to be able to return specific records. I filed
AVRO-1351 to address this.
https://issues.apache.org/jira/browse/AVRO-1351
Doug
Re: Type cast exception
Posted by TrevniUser <di...@cerner.com>.
Updated but still facing the same issue. Any ideas why?
Updated project dependencies. Now pulling in:
[INFO] +- org.apache.avro:avro:jar:1.7.4:compile
[INFO] +- org.apache.avro:avro-mapred:jar:1.7.4:compile
[INFO] | +- org.apache.avro:avro-ipc:jar:1.7.4:compile
[INFO] | \- org.apache.avro:avro-ipc:jar:tests:1.7.4:compile
EXCEPTION:
java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record
cannot be cast to com.abc.kepler.datasink.DataSinkChangeLogContent
--
View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027732.html
Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by Doug Cutting <cu...@apache.org>.
This was fixed in Avro 1.7.4. Please try upgrading and see if that helps.
Doug
On Tue, Jul 2, 2013 at 10:33 AM, TrevniUser <di...@cerner.com> wrote:
> Seems like it could be. Please confirm. I am using 'avro 1.7.3'. This is what
> my project dependencies look like:
>
> [INFO] +- org.apache.avro:avro:jar:1.7.3:compile
> [INFO] +- org.apache.avro:avro-mapred:jar:1.7.3:compile
> [INFO] | +- org.apache.avro:avro-ipc:jar:1.7.3:compile
> [INFO] | \- org.apache.avro:avro-ipc:jar:tests:1.7.3:compile
>
>
>
> --
> View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027730.html
> Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by TrevniUser <di...@cerner.com>.
Seems like it could be. Please confirm. I am using 'avro 1.7.3'. This is what
my project dependencies look like:
[INFO] +- org.apache.avro:avro:jar:1.7.3:compile
[INFO] +- org.apache.avro:avro-mapred:jar:1.7.3:compile
[INFO] | +- org.apache.avro:avro-ipc:jar:1.7.3:compile
[INFO] | \- org.apache.avro:avro-ipc:jar:tests:1.7.3:compile
--
View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027730.html
Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by Doug Cutting <cu...@apache.org>.
Could this be AVRO-1240? What version of Avro are you using?
https://issues.apache.org/jira/browse/AVRO-1240
Doug
On Tue, Jul 2, 2013 at 10:05 AM, TrevniUser <di...@cerner.com> wrote:
> I am facing this same issue while using SortedKeyValueFile. Could you please
> provide some pointers why in the first place is GenericRecord being picked
> up and not SpecificRecord?
>
> @Test
> public void testSortedKeyValueFile() throws IOException {
> Path filePath = new Path(test.getMethodName());
>
> Schema keySchema = ReflectData.get().getSchema(Long.class);
> Schema valSchema =
> ReflectData.get().getSchema(DataSinkChangeLogContent.class);
>
> SortedKeyValueFile.Writer.Options opts = new Options();
> opts.withKeySchema(keySchema);
> opts.withValueSchema(valSchema);
> opts.withConfiguration(conf);
> opts.withPath(filePath);
> SortedKeyValueFile.Writer<Long, DataSinkChangeLogContent> writer =
> new SortedKeyValueFile.Writer<Long, DataSinkChangeLogContent>(
> opts);
> for (long i = 0; i < 10; i++) {
> writer.append(i, new DataSinkChangeLogContent(eType.toString(),
> eKey.toString(), i, 15L));
> }
> writer.close();
>
> SortedKeyValueFile.Reader.Options readerOpts = new
> SortedKeyValueFile.Reader.Options();
> readerOpts.withKeySchema(keySchema);
> readerOpts.withValueSchema(valSchema);
> readerOpts.withConfiguration(conf);
> readerOpts.withPath(filePath);
> SortedKeyValueFile.Reader<Long, DataSinkChangeLogContent> reader =
> new SortedKeyValueFile.Reader<Long, DataSinkChangeLogContent>(
> readerOpts);
> Iterator<AvroKeyValue<Long, DataSinkChangeLogContent>> i = null;
> if (reader.get(5L) != null) {
> i = reader.iterator();
> }
> while (i != null && i.hasNext()) {
> AvroKeyValue<Long, DataSinkChangeLogContent> keyValue =
> i.next();
> System.out.println("---------------------------- Clock: " +
> keyValue.getValue());
> }
> * DataSinkChangeLogContent e = reader.get(5L);* // FAILS AT THIS LINE
> reader.close();
> }
>
> EXCEPTION:
>
> java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record
> cannot be cast to com.abc.kepler.datasink.DataSinkChangeLogContent
> at
> com.abc.kepler.datasink.hdfs.HDFSDataSinkChangeLogScannerTest.testSortedKeyValueFile(HDFSDataSinkChangeLogScannerTest.java:926)
>
>
>
> --
> View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027727.html
> Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by TrevniUser <di...@cerner.com>.
I am facing this same issue while using SortedKeyValueFile. Could you please
provide some pointers why in the first place is GenericRecord being picked
up and not SpecificRecord?
@Test
public void testSortedKeyValueFile() throws IOException {
Path filePath = new Path(test.getMethodName());
Schema keySchema = ReflectData.get().getSchema(Long.class);
Schema valSchema =
ReflectData.get().getSchema(DataSinkChangeLogContent.class);
SortedKeyValueFile.Writer.Options opts = new Options();
opts.withKeySchema(keySchema);
opts.withValueSchema(valSchema);
opts.withConfiguration(conf);
opts.withPath(filePath);
SortedKeyValueFile.Writer<Long, DataSinkChangeLogContent> writer =
new SortedKeyValueFile.Writer<Long, DataSinkChangeLogContent>(
opts);
for (long i = 0; i < 10; i++) {
writer.append(i, new DataSinkChangeLogContent(eType.toString(),
eKey.toString(), i, 15L));
}
writer.close();
SortedKeyValueFile.Reader.Options readerOpts = new
SortedKeyValueFile.Reader.Options();
readerOpts.withKeySchema(keySchema);
readerOpts.withValueSchema(valSchema);
readerOpts.withConfiguration(conf);
readerOpts.withPath(filePath);
SortedKeyValueFile.Reader<Long, DataSinkChangeLogContent> reader =
new SortedKeyValueFile.Reader<Long, DataSinkChangeLogContent>(
readerOpts);
Iterator<AvroKeyValue<Long, DataSinkChangeLogContent>> i = null;
if (reader.get(5L) != null) {
i = reader.iterator();
}
while (i != null && i.hasNext()) {
AvroKeyValue<Long, DataSinkChangeLogContent> keyValue =
i.next();
System.out.println("---------------------------- Clock: " +
keyValue.getValue());
}
* DataSinkChangeLogContent e = reader.get(5L);* // FAILS AT THIS LINE
reader.close();
}
EXCEPTION:
java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record
cannot be cast to com.abc.kepler.datasink.DataSinkChangeLogContent
at
com.abc.kepler.datasink.hdfs.HDFSDataSinkChangeLogScannerTest.testSortedKeyValueFile(HDFSDataSinkChangeLogScannerTest.java:926)
--
View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027727.html
Sent from the Avro - Users mailing list archive at Nabble.com.
Re: Type cast exception
Posted by Scott Carey <sc...@richrelevance.com>.
Try
SpecificDatumReader<LogMessage> din = new SpecificDatumReader<LogMessage>(LogMessage.class);
instead of the GenericDatumReader below.
On Apr 26, 2010, at 7:59 AM, Jaydeep Ayachit wrote:
Hello,
I am trying to read from avro data file and want to use specific object (exa sample.LogMessage)
The following code gives exception mentioned below. Reader.next() when collected in Object/Record, no exception is thrown which is expected.
Any help is appreciated.
public void testMethod5() throws Exception {
DatumWriter<LogMessage> writer = new GenericDatumWriter<LogMessage>(LogMessage.SCHEMA$);
DataFileWriter<LogMessage> out = new DataFileWriter<LogMessage>(writer);
out.create(LogMessage.SCHEMA$, <File>);
for (int i = 0; i < 100; i++) {
LogMessage msg = new LogMessage();
msg.applicationName = new Utf8("app" + i);
msg.userid = new Utf8("user" + i);
msg.message = new Utf8("this is test log message " + i);
out.append(msg);
}
out.close();
DataFileReader<LogMessage> reader;
GenericDatumReader<LogMessage> din = new GenericDatumReader<LogMessage>(LogMessage.SCHEMA$);
reader = new DataFileReader<LogMessage>(new File(<>), din);
while (reader.hasNext()) {
LogMessage msg = reader.next();
System.out.println(msg.toString());
}
reader.close();
}
Exception
java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record cannot be cast to sample.LogMessage
at com.ibm.clas.ReaderWriter.testMethod5(ReaderWriter.java:313)
at com.ibm.clas.ReaderWriter.main(ReaderWriter.java:322)
Thanks
Jaydeep
DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.