You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by "W.P. McNeill" <bi...@gmail.com> on 2011/09/02 23:41:25 UTC
How should I migrate 1.4 code to avro 1.5?
I'm new to Avro. Since I'm having trouble finding simple examples online I'm
writing one of my own that I'm putting on github.
https://github.com/wpm/AvroExample
Hopefully, this will be of help to people like me who are also having
trouble finding simple code examples.
I want to get this compiling without of hitch in Maven. I had it running
with a 1.4 version of Avro, but when I changed that to 1.5, some of the code
no longer works. Specifically, BinaryEncoder can no longer be instantiated
directly because it is now an abstract class (AvroExample.java: line 33) and
DecoderFactory.defaultFactory is deprecated (AvroExample.java: line 41).
How should I modify this code so that it works with the latest and greatest
version of Avro? I looked through the Release Notes, but the answers
weren't obvious.
Thanks.
Re: How should I migrate 1.4 code to avro 1.5?
Posted by Scott Carey <sc...@apache.org>.
The maven repositories required are in the pom.xml of the avro project. If
you are using Netty for a socket protocol, you will need the JBoss
repository.
Otherwise, you can exclude that dependency in the pom.
see the repositories section at the bottom of the Avro pom section here:
http://svn.apache.org/viewvc/avro/tags/release-1.5.3/lang/java/pom.xml?view=
markup
On 9/2/11 3:48 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
> Also, could someone who has built an Avro client using Maven post a list of
> all the Maven repositories that contain the needed dependencies. It seems like
> the default repository is not enough.
>
> On Fri, Sep 2, 2011 at 2:41 PM, W.P. McNeill <bi...@gmail.com> wrote:
>> I'm new to Avro. Since I'm having trouble finding simple examples online I'm
>> writing one of my own that I'm putting on github.
>>
>> https://github.com/wpm/AvroExample
>>
>> Hopefully, this will be of help to people like me who are also having trouble
>> finding simple code examples.
>>
>> I want to get this compiling without of hitch in Maven. I had it running with
>> a 1.4 version of Avro, but when I changed that to 1.5, some of the code no
>> longer works. Specifically, BinaryEncoder can no longer be instantiated
>> directly because it is now an abstract class (AvroExample.java: line 33) and
>> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line 41).
>>
>> How should I modify this code so that it works with the latest and greatest
>> version of Avro? I looked through the Release Notes, but the answers weren't
>> obvious.
>>
>> Thanks.
>>
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by "W.P. McNeill" <bi...@gmail.com>.
Also, could someone who has built an Avro client using Maven post a list of
all the Maven repositories that contain the needed dependencies. It seems
like the default repository is not enough.
On Fri, Sep 2, 2011 at 2:41 PM, W.P. McNeill <bi...@gmail.com> wrote:
> I'm new to Avro. Since I'm having trouble finding simple examples online
> I'm writing one of my own that I'm putting on github.
>
> https://github.com/wpm/AvroExample
>
> Hopefully, this will be of help to people like me who are also having
> trouble finding simple code examples.
>
> I want to get this compiling without of hitch in Maven. I had it running
> with a 1.4 version of Avro, but when I changed that to 1.5, some of the code
> no longer works. Specifically, BinaryEncoder can no longer be instantiated
> directly because it is now an abstract class (AvroExample.java: line 33) and
> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line 41).
>
> How should I modify this code so that it works with the latest and greatest
> version of Avro? I looked through the Release Notes, but the answers
> weren't obvious.
>
> Thanks.
>
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by "W.P. McNeill" <bi...@gmail.com>.
I opened https://issues.apache.org/jira/browse/AVRO-883. Let me know if you
need anything else to incorporate this example.
On Fri, Sep 2, 2011 at 6:09 PM, Scott Carey <sc...@apache.org> wrote:
> Start with a JIRA ticket and we can discuss and refine there.
>
> What we accept into the project must be attached as a patch to the JIRA
> ticket with the sign-off to Apache and proper license headers on the
> content.
>
> Thanks!
>
> -Scott
>
> On 9/2/11 5:53 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
>
> I've got a building version with Chris Wilkes' changes. I'd be happy to
> include this in an Avro distribution. Should I just open a JIRA to that
> effect and point to this github project?
>
> On Fri, Sep 2, 2011 at 5:28 PM, Chris Wilkes <cw...@gmail.com> wrote:
>
>> Oh and I'm the one that did the pull request. I changed the name of
>> the avro class to "MyPair" as I was confused when reading it with
>> avro's own "Pair" class.
>>
>> What I usually do is put all of my avro schemas into a separate
>> project with nothing else in it. Then I have all my other projects
>> depend on that one, in this case AvroExample.java would be a in a
>> separate project from MyPair.avsc. This gets around weirdness with
>> mvn install vs Eclipse seeing the updated files, etc.
>>
>> On Fri, Sep 2, 2011 at 5:20 PM, W.P. McNeill <bi...@gmail.com> wrote:
>> > Still having trouble with this.
>> > The name change was part of merging the pull request on github. My last
>> > email details where I'm at right now. The pull request code looks
>> correct;
>> > I'm just trying to get it to build in my Maven environment.
>> >
>> > On Fri, Sep 2, 2011 at 5:19 PM, Scott Carey <sc...@apache.org>
>> wrote:
>> >>
>> >> Are you still having trouble with this? I noticed that the code has
>> >> changed and you are using MyPair instead of Pair. Was there a naming
>> >> conflict bug with Avro's Pair.java?
>> >> -Scott
>> >> On 9/2/11 3:46 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
>> >>
>> >> I made changes that got rid of all the deprecated calls. I think I am
>> >> using the 1.5 interface correctly. However, I get a runtime error when
>> I
>> >> try to deserialize into a class using a SpecificDataumReader. The
>> problem
>> >> starts at line 62 of AvroExample.java. The code looks like this:
>> >> DatumReader<Pair> reader = new
>> >> SpecificDatumReader<Pair>(Pair.class);
>> >> BinaryDecoder decoder =
>> >> DecoderFactory.get().binaryDecoder(out.toByteArray(), null);
>> >> Pair result = reader.read(null, decoder);
>> >> System.out.printf("Left: %s, Right: %s\n", result.left,
>> >> result.right);
>> >> Where Pair is an object I have SpecificRecord that I have in this
>> project.
>> >> When I deserialize with reader.read() I get the following runtime
>> error:
>> >> Exception in thread "main" java.lang.ClassCastException:
>> >> org.apache.avro.generic.GenericData$Record cannot be cast to wpmcn.Pair
>> >> at wpmcn.AvroExample.serializeSpecific(AvroExample.java:64)
>> >> at wpmcn.AvroExample.main(AvroExample.java:73)
>> >> When I step into the debugger I see that the GenericDatumReader.read()
>> >> function has type D as GenericData.
>> >> Presumably I'm calling something wrong but I can't figure out what.
>> >> On Fri, Sep 2, 2011 at 3:02 PM, Philip Zeyliger <ph...@cloudera.com>
>> >> wrote:
>> >>>
>> >>> EncoderFactory.get().binaryEncoder(...).
>> >>> I encourage you to file a JIRA and submit a patch to AVRO. Having
>> >>> example code in the code base seems like a win to me.
>> >>> -- Philip
>> >>>
>> >>> On Fri, Sep 2, 2011 at 2:41 PM, W.P. McNeill <bi...@gmail.com>
>> wrote:
>> >>>>
>> >>>> I'm new to Avro. Since I'm having trouble finding simple examples
>> online
>> >>>> I'm writing one of my own that I'm putting on github.
>> >>>> https://github.com/wpm/AvroExample
>> >>>> Hopefully, this will be of help to people like me who are also having
>> >>>> trouble finding simple code examples.
>> >>>> I want to get this compiling without of hitch in Maven. I had it
>> running
>> >>>> with a 1.4 version of Avro, but when I changed that to 1.5, some of
>> the code
>> >>>> no longer works. Specifically, BinaryEncoder can no longer be
>> instantiated
>> >>>> directly because it is now an abstract class (AvroExample.java: line
>> 33) and
>> >>>> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line
>> 41).
>> >>>> How should I modify this code so that it works with the latest and
>> >>>> greatest version of Avro? I looked through the Release Notes, but
>> the
>> >>>> answers weren't obvious.
>> >>>> Thanks.
>> >>>
>> >>
>> >
>> >
>>
>
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by Scott Carey <sc...@apache.org>.
Start with a JIRA ticket and we can discuss and refine there.
What we accept into the project must be attached as a patch to the JIRA
ticket with the sign-off to Apache and proper license headers on the
content.
Thanks!
-Scott
On 9/2/11 5:53 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
> I've got a building version with Chris Wilkes' changes. I'd be happy to
> include this in an Avro distribution. Should I just open a JIRA to that effect
> and point to this github project?
>
> On Fri, Sep 2, 2011 at 5:28 PM, Chris Wilkes <cw...@gmail.com> wrote:
>> Oh and I'm the one that did the pull request. I changed the name of
>> the avro class to "MyPair" as I was confused when reading it with
>> avro's own "Pair" class.
>>
>> What I usually do is put all of my avro schemas into a separate
>> project with nothing else in it. Then I have all my other projects
>> depend on that one, in this case AvroExample.java would be a in a
>> separate project from MyPair.avsc. This gets around weirdness with
>> mvn install vs Eclipse seeing the updated files, etc.
>>
>> On Fri, Sep 2, 2011 at 5:20 PM, W.P. McNeill <bi...@gmail.com> wrote:
>>> > Still having trouble with this.
>>> > The name change was part of merging the pull request on github. My last
>>> > email details where I'm at right now. The pull request code looks correct;
>>> > I'm just trying to get it to build in my Maven environment.
>>> >
>>> > On Fri, Sep 2, 2011 at 5:19 PM, Scott Carey <sc...@apache.org> wrote:
>>>> >>
>>>> >> Are you still having trouble with this? I noticed that the code has
>>>> >> changed and you are using MyPair instead of Pair. Was there a naming
>>>> >> conflict bug with Avro's Pair.java?
>>>> >> -Scott
>>>> >> On 9/2/11 3:46 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
>>>> >>
>>>> >> I made changes that got rid of all the deprecated calls. I think I am
>>>> >> using the 1.5 interface correctly. However, I get a runtime error when
I
>>>> >> try to deserialize into a class using a SpecificDataumReader. The
>>>> problem
>>>> >> starts at line 62 of AvroExample.java. The code looks like this:
>>>> >> DatumReader<Pair> reader = new
>>>> >> SpecificDatumReader<Pair>(Pair.class);
>>>> >> BinaryDecoder decoder =
>>>> >> DecoderFactory.get().binaryDecoder(out.toByteArray(), null);
>>>> >> Pair result = reader.read(null, decoder);
>>>> >> System.out.printf("Left: %s, Right: %s\n", result.left,
>>>> >> result.right);
>>>> >> Where Pair is an object I have SpecificRecord that I have in this
>>>> project.
>>>> >> When I deserialize with reader.read() I get the following runtime error:
>>>> >> Exception in thread "main" java.lang.ClassCastException:
>>>> >> org.apache.avro.generic.GenericData$Record cannot be cast to wpmcn.Pair
>>>> >> at wpmcn.AvroExample.serializeSpecific(AvroExample.java:64)
>>>> >> at wpmcn.AvroExample.main(AvroExample.java:73)
>>>> >> When I step into the debugger I see that the GenericDatumReader.read()
>>>> >> function has type D as GenericData.
>>>> >> Presumably I'm calling something wrong but I can't figure out what.
>>>> >> On Fri, Sep 2, 2011 at 3:02 PM, Philip Zeyliger <ph...@cloudera.com>
>>>> >> wrote:
>>>>> >>>
>>>>> >>> EncoderFactory.get().binaryEncoder(...).
>>>>> >>> I encourage you to file a JIRA and submit a patch to AVRO. Having
>>>>> >>> example code in the code base seems like a win to me.
>>>>> >>> -- Philip
>>>>> >>>
>>>>> >>> On Fri, Sep 2, 2011 at 2:41 PM, W.P. McNeill <bi...@gmail.com>
>>>>> wrote:
>>>>>> >>>>
>>>>>> >>>> I'm new to Avro. Since I'm having trouble finding simple examples
>>>>>> online
>>>>>> >>>> I'm writing one of my own that I'm putting on github.
>>>>>> >>>> https://github.com/wpm/AvroExample
>>>>>> >>>> Hopefully, this will be of help to people like me who are also
>>>>>> having
>>>>>> >>>> trouble finding simple code examples.
>>>>>> >>>> I want to get this compiling without of hitch in Maven. I had it
>>>>>> running
>>>>>> >>>> with a 1.4 version of Avro, but when I changed that to 1.5, some of
>>>>>> the code
>>>>>> >>>> no longer works. Specifically, BinaryEncoder can no longer be
>>>>>> instantiated
>>>>>> >>>> directly because it is now an abstract class (AvroExample.java: line
>>>>>> 33) and
>>>>>> >>>> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line
41).
>>>>>> >>>> How should I modify this code so that it works with the latest and
>>>>>> >>>> greatest version of Avro? I looked through the Release Notes, but
the
>>>>>> >>>> answers weren't obvious.
>>>>>> >>>> Thanks.
>>>>> >>>
>>>> >>
>>> >
>>> >
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by "W.P. McNeill" <bi...@gmail.com>.
I've got a building version with Chris Wilkes' changes. I'd be happy to
include this in an Avro distribution. Should I just open a JIRA to that
effect and point to this github project?
On Fri, Sep 2, 2011 at 5:28 PM, Chris Wilkes <cw...@gmail.com> wrote:
> Oh and I'm the one that did the pull request. I changed the name of
> the avro class to "MyPair" as I was confused when reading it with
> avro's own "Pair" class.
>
> What I usually do is put all of my avro schemas into a separate
> project with nothing else in it. Then I have all my other projects
> depend on that one, in this case AvroExample.java would be a in a
> separate project from MyPair.avsc. This gets around weirdness with
> mvn install vs Eclipse seeing the updated files, etc.
>
> On Fri, Sep 2, 2011 at 5:20 PM, W.P. McNeill <bi...@gmail.com> wrote:
> > Still having trouble with this.
> > The name change was part of merging the pull request on github. My last
> > email details where I'm at right now. The pull request code looks
> correct;
> > I'm just trying to get it to build in my Maven environment.
> >
> > On Fri, Sep 2, 2011 at 5:19 PM, Scott Carey <sc...@apache.org>
> wrote:
> >>
> >> Are you still having trouble with this? I noticed that the code has
> >> changed and you are using MyPair instead of Pair. Was there a naming
> >> conflict bug with Avro's Pair.java?
> >> -Scott
> >> On 9/2/11 3:46 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
> >>
> >> I made changes that got rid of all the deprecated calls. I think I am
> >> using the 1.5 interface correctly. However, I get a runtime error when
> I
> >> try to deserialize into a class using a SpecificDataumReader. The
> problem
> >> starts at line 62 of AvroExample.java. The code looks like this:
> >> DatumReader<Pair> reader = new
> >> SpecificDatumReader<Pair>(Pair.class);
> >> BinaryDecoder decoder =
> >> DecoderFactory.get().binaryDecoder(out.toByteArray(), null);
> >> Pair result = reader.read(null, decoder);
> >> System.out.printf("Left: %s, Right: %s\n", result.left,
> >> result.right);
> >> Where Pair is an object I have SpecificRecord that I have in this
> project.
> >> When I deserialize with reader.read() I get the following runtime error:
> >> Exception in thread "main" java.lang.ClassCastException:
> >> org.apache.avro.generic.GenericData$Record cannot be cast to wpmcn.Pair
> >> at wpmcn.AvroExample.serializeSpecific(AvroExample.java:64)
> >> at wpmcn.AvroExample.main(AvroExample.java:73)
> >> When I step into the debugger I see that the GenericDatumReader.read()
> >> function has type D as GenericData.
> >> Presumably I'm calling something wrong but I can't figure out what.
> >> On Fri, Sep 2, 2011 at 3:02 PM, Philip Zeyliger <ph...@cloudera.com>
> >> wrote:
> >>>
> >>> EncoderFactory.get().binaryEncoder(...).
> >>> I encourage you to file a JIRA and submit a patch to AVRO. Having
> >>> example code in the code base seems like a win to me.
> >>> -- Philip
> >>>
> >>> On Fri, Sep 2, 2011 at 2:41 PM, W.P. McNeill <bi...@gmail.com>
> wrote:
> >>>>
> >>>> I'm new to Avro. Since I'm having trouble finding simple examples
> online
> >>>> I'm writing one of my own that I'm putting on github.
> >>>> https://github.com/wpm/AvroExample
> >>>> Hopefully, this will be of help to people like me who are also having
> >>>> trouble finding simple code examples.
> >>>> I want to get this compiling without of hitch in Maven. I had it
> running
> >>>> with a 1.4 version of Avro, but when I changed that to 1.5, some of
> the code
> >>>> no longer works. Specifically, BinaryEncoder can no longer be
> instantiated
> >>>> directly because it is now an abstract class (AvroExample.java: line
> 33) and
> >>>> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line
> 41).
> >>>> How should I modify this code so that it works with the latest and
> >>>> greatest version of Avro? I looked through the Release Notes, but the
> >>>> answers weren't obvious.
> >>>> Thanks.
> >>>
> >>
> >
> >
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by Chris Wilkes <cw...@gmail.com>.
Oh and I'm the one that did the pull request. I changed the name of
the avro class to "MyPair" as I was confused when reading it with
avro's own "Pair" class.
What I usually do is put all of my avro schemas into a separate
project with nothing else in it. Then I have all my other projects
depend on that one, in this case AvroExample.java would be a in a
separate project from MyPair.avsc. This gets around weirdness with
mvn install vs Eclipse seeing the updated files, etc.
On Fri, Sep 2, 2011 at 5:20 PM, W.P. McNeill <bi...@gmail.com> wrote:
> Still having trouble with this.
> The name change was part of merging the pull request on github. My last
> email details where I'm at right now. The pull request code looks correct;
> I'm just trying to get it to build in my Maven environment.
>
> On Fri, Sep 2, 2011 at 5:19 PM, Scott Carey <sc...@apache.org> wrote:
>>
>> Are you still having trouble with this? I noticed that the code has
>> changed and you are using MyPair instead of Pair. Was there a naming
>> conflict bug with Avro's Pair.java?
>> -Scott
>> On 9/2/11 3:46 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
>>
>> I made changes that got rid of all the deprecated calls. I think I am
>> using the 1.5 interface correctly. However, I get a runtime error when I
>> try to deserialize into a class using a SpecificDataumReader. The problem
>> starts at line 62 of AvroExample.java. The code looks like this:
>> DatumReader<Pair> reader = new
>> SpecificDatumReader<Pair>(Pair.class);
>> BinaryDecoder decoder =
>> DecoderFactory.get().binaryDecoder(out.toByteArray(), null);
>> Pair result = reader.read(null, decoder);
>> System.out.printf("Left: %s, Right: %s\n", result.left,
>> result.right);
>> Where Pair is an object I have SpecificRecord that I have in this project.
>> When I deserialize with reader.read() I get the following runtime error:
>> Exception in thread "main" java.lang.ClassCastException:
>> org.apache.avro.generic.GenericData$Record cannot be cast to wpmcn.Pair
>> at wpmcn.AvroExample.serializeSpecific(AvroExample.java:64)
>> at wpmcn.AvroExample.main(AvroExample.java:73)
>> When I step into the debugger I see that the GenericDatumReader.read()
>> function has type D as GenericData.
>> Presumably I'm calling something wrong but I can't figure out what.
>> On Fri, Sep 2, 2011 at 3:02 PM, Philip Zeyliger <ph...@cloudera.com>
>> wrote:
>>>
>>> EncoderFactory.get().binaryEncoder(...).
>>> I encourage you to file a JIRA and submit a patch to AVRO. Having
>>> example code in the code base seems like a win to me.
>>> -- Philip
>>>
>>> On Fri, Sep 2, 2011 at 2:41 PM, W.P. McNeill <bi...@gmail.com> wrote:
>>>>
>>>> I'm new to Avro. Since I'm having trouble finding simple examples online
>>>> I'm writing one of my own that I'm putting on github.
>>>> https://github.com/wpm/AvroExample
>>>> Hopefully, this will be of help to people like me who are also having
>>>> trouble finding simple code examples.
>>>> I want to get this compiling without of hitch in Maven. I had it running
>>>> with a 1.4 version of Avro, but when I changed that to 1.5, some of the code
>>>> no longer works. Specifically, BinaryEncoder can no longer be instantiated
>>>> directly because it is now an abstract class (AvroExample.java: line 33) and
>>>> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line 41).
>>>> How should I modify this code so that it works with the latest and
>>>> greatest version of Avro? I looked through the Release Notes, but the
>>>> answers weren't obvious.
>>>> Thanks.
>>>
>>
>
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by "W.P. McNeill" <bi...@gmail.com>.
Still having trouble with this.
The name change was part of merging the pull request on github. My last
email details where I'm at right now. The pull request code looks correct;
I'm just trying to get it to build in my Maven environment.
On Fri, Sep 2, 2011 at 5:19 PM, Scott Carey <sc...@apache.org> wrote:
> Are you still having trouble with this? I noticed that the code has
> changed and you are using MyPair instead of Pair. Was there a naming
> conflict bug with Avro's Pair.java?
>
> -Scott
>
> On 9/2/11 3:46 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
>
> I made changes that got rid of all the deprecated calls. I think I am
> using the 1.5 interface correctly. However, I get a runtime error when I
> try to deserialize into a class using a SpecificDataumReader. The problem
> starts at line 62 of AvroExample.java<https://github.com/wpm/AvroExample/blob/master/src/main/java/wpmcn/AvroExample.java#L62>.
> The code looks like this:
>
> DatumReader<Pair> reader = new SpecificDatumReader<Pair>(Pair.class);
> BinaryDecoder decoder =
> DecoderFactory.get().binaryDecoder(out.toByteArray(), null);
> Pair result = reader.read(null, decoder);
> System.out.printf("Left: %s, Right: %s\n", result.left,
> result.right);
>
> Where Pair is an object I have SpecificRecord that I have in this project.
> When I deserialize with reader.read() I get the following runtime error:
>
> Exception in thread "main" java.lang.ClassCastException:
> org.apache.avro.generic.GenericData$Record cannot be cast to wpmcn.Pair
> at wpmcn.AvroExample.serializeSpecific(AvroExample.java:64)
> at wpmcn.AvroExample.main(AvroExample.java:73)
>
> When I step into the debugger I see that the GenericDatumReader.read()
> function has type D as GenericData.
>
> Presumably I'm calling something wrong but I can't figure out what.
>
> On Fri, Sep 2, 2011 at 3:02 PM, Philip Zeyliger <ph...@cloudera.com>wrote:
>
>> EncoderFactory.get().binaryEncoder(...).
>>
>> I encourage you to file a JIRA and submit a patch to AVRO. Having example
>> code in the code base seems like a win to me.
>>
>> -- Philip
>>
>>
>> On Fri, Sep 2, 2011 at 2:41 PM, W.P. McNeill <bi...@gmail.com> wrote:
>>
>>> I'm new to Avro. Since I'm having trouble finding simple examples online
>>> I'm writing one of my own that I'm putting on github.
>>>
>>> https://github.com/wpm/AvroExample
>>>
>>> Hopefully, this will be of help to people like me who are also having
>>> trouble finding simple code examples.
>>>
>>> I want to get this compiling without of hitch in Maven. I had it running
>>> with a 1.4 version of Avro, but when I changed that to 1.5, some of the code
>>> no longer works. Specifically, BinaryEncoder can no longer be instantiated
>>> directly because it is now an abstract class (AvroExample.java: line 33) and
>>> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line 41).
>>>
>>> How should I modify this code so that it works with the latest and
>>> greatest version of Avro? I looked through the Release Notes, but the
>>> answers weren't obvious.
>>>
>>> Thanks.
>>>
>>>
>>
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by Scott Carey <sc...@apache.org>.
Are you still having trouble with this? I noticed that the code has changed
and you are using MyPair instead of Pair. Was there a naming conflict bug
with Avro's Pair.java?
-Scott
On 9/2/11 3:46 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
> I made changes that got rid of all the deprecated calls. I think I am using
> the 1.5 interface correctly. However, I get a runtime error when I try to
> deserialize into a class using a SpecificDataumReader. The problem starts at
> line 62 of AvroExample.java
> <https://github.com/wpm/AvroExample/blob/master/src/main/java/wpmcn/AvroExampl
> e.java#L62> . The code looks like this:
>
> DatumReader<Pair> reader = new SpecificDatumReader<Pair>(Pair.class);
> BinaryDecoder decoder =
> DecoderFactory.get().binaryDecoder(out.toByteArray(), null);
> Pair result = reader.read(null, decoder);
> System.out.printf("Left: %s, Right: %s\n", result.left, result.right);
>
> Where Pair is an object I have SpecificRecord that I have in this project.
> When I deserialize with reader.read() I get the following runtime error:
>
> Exception in thread "main" java.lang.ClassCastException:
> org.apache.avro.generic.GenericData$Record cannot be cast to wpmcn.Pair
> at wpmcn.AvroExample.serializeSpecific(AvroExample.java:64)
> at wpmcn.AvroExample.main(AvroExample.java:73)
>
> When I step into the debugger I see that the GenericDatumReader.read()
> function has type D as GenericData.
>
> Presumably I'm calling something wrong but I can't figure out what.
>
> On Fri, Sep 2, 2011 at 3:02 PM, Philip Zeyliger <ph...@cloudera.com> wrote:
>> EncoderFactory.get().binaryEncoder(...).
>>
>> I encourage you to file a JIRA and submit a patch to AVRO. Having example
>> code in the code base seems like a win to me.
>>
>> -- Philip
>>
>>
>> On Fri, Sep 2, 2011 at 2:41 PM, W.P. McNeill <bi...@gmail.com> wrote:
>>> I'm new to Avro. Since I'm having trouble finding simple examples online I'm
>>> writing one of my own that I'm putting on github.
>>>
>>> https://github.com/wpm/AvroExample
>>>
>>> Hopefully, this will be of help to people like me who are also having
>>> trouble finding simple code examples.
>>>
>>> I want to get this compiling without of hitch in Maven. I had it running
>>> with a 1.4 version of Avro, but when I changed that to 1.5, some of the code
>>> no longer works. Specifically, BinaryEncoder can no longer be instantiated
>>> directly because it is now an abstract class (AvroExample.java: line 33) and
>>> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line 41).
>>>
>>> How should I modify this code so that it works with the latest and greatest
>>> version of Avro? I looked through the Release Notes, but the answers
>>> weren't obvious.
>>>
>>> Thanks.
>>>
>>
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by "W.P. McNeill" <bi...@gmail.com>.
I made changes that got rid of all the deprecated calls. I think I am using
the 1.5 interface correctly. However, I get a runtime error when I try to
deserialize into a class using a SpecificDataumReader. The problem starts
at line 62 of AvroExample.java<https://github.com/wpm/AvroExample/blob/master/src/main/java/wpmcn/AvroExample.java#L62>.
The code looks like this:
DatumReader<Pair> reader = new SpecificDatumReader<Pair>(Pair.class);
BinaryDecoder decoder =
DecoderFactory.get().binaryDecoder(out.toByteArray(), null);
Pair result = reader.read(null, decoder);
System.out.printf("Left: %s, Right: %s\n", result.left, result.right);
Where Pair is an object I have SpecificRecord that I have in this project.
When I deserialize with reader.read() I get the following runtime error:
Exception in thread "main" java.lang.ClassCastException:
org.apache.avro.generic.GenericData$Record cannot be cast to wpmcn.Pair
at wpmcn.AvroExample.serializeSpecific(AvroExample.java:64)
at wpmcn.AvroExample.main(AvroExample.java:73)
When I step into the debugger I see that the GenericDatumReader.read()
function has type D as GenericData.
Presumably I'm calling something wrong but I can't figure out what.
On Fri, Sep 2, 2011 at 3:02 PM, Philip Zeyliger <ph...@cloudera.com> wrote:
> EncoderFactory.get().binaryEncoder(...).
>
> I encourage you to file a JIRA and submit a patch to AVRO. Having example
> code in the code base seems like a win to me.
>
> -- Philip
>
>
> On Fri, Sep 2, 2011 at 2:41 PM, W.P. McNeill <bi...@gmail.com> wrote:
>
>> I'm new to Avro. Since I'm having trouble finding simple examples online
>> I'm writing one of my own that I'm putting on github.
>>
>> https://github.com/wpm/AvroExample
>>
>> Hopefully, this will be of help to people like me who are also having
>> trouble finding simple code examples.
>>
>> I want to get this compiling without of hitch in Maven. I had it running
>> with a 1.4 version of Avro, but when I changed that to 1.5, some of the code
>> no longer works. Specifically, BinaryEncoder can no longer be instantiated
>> directly because it is now an abstract class (AvroExample.java: line 33) and
>> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line 41).
>>
>> How should I modify this code so that it works with the latest and
>> greatest version of Avro? I looked through the Release Notes, but the
>> answers weren't obvious.
>>
>> Thanks.
>>
>>
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by Philip Zeyliger <ph...@cloudera.com>.
EncoderFactory.get().binaryEncoder(...).
I encourage you to file a JIRA and submit a patch to AVRO. Having example
code in the code base seems like a win to me.
-- Philip
On Fri, Sep 2, 2011 at 2:41 PM, W.P. McNeill <bi...@gmail.com> wrote:
> I'm new to Avro. Since I'm having trouble finding simple examples online
> I'm writing one of my own that I'm putting on github.
>
> https://github.com/wpm/AvroExample
>
> Hopefully, this will be of help to people like me who are also having
> trouble finding simple code examples.
>
> I want to get this compiling without of hitch in Maven. I had it running
> with a 1.4 version of Avro, but when I changed that to 1.5, some of the code
> no longer works. Specifically, BinaryEncoder can no longer be instantiated
> directly because it is now an abstract class (AvroExample.java: line 33) and
> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line 41).
>
> How should I modify this code so that it works with the latest and greatest
> version of Avro? I looked through the Release Notes, but the answers
> weren't obvious.
>
> Thanks.
>
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by Chris Wilkes <cw...@gmail.com>.
I've never used it but try "mvn clean:clean" Usually I just blow away
my maven repo that I'm having a problem with, in this case:
rm -rf /Users/williammcneill/.m2/repository/org/codehaus/mojo/exec-maven-plugin/
try that and then do the mvn build command.
On Fri, Sep 2, 2011 at 5:18 PM, W.P. McNeill <bi...@gmail.com> wrote:
> I got a pull request on github for a change that does this. I'm looking
> through it now.
> The problem I'm having is that Maven goals fail with the following error:
> Downloading:
> http://repo1.maven.org/maven2/org/codehaus/mojo/exec-maven-plugin/1.1/exec-maven-plugin-1.1.pom
> [WARNING] Failed to create parent directories for resolution tracking file
> /Users/williammcneill/.m2/repository/org/codehaus/mojo/exec-maven-plugin/1.1/exec-maven-plugin-1.1.pom.lastUpdated
>
> Does this mean I have to add extra repositories to my settings.xml, or
> should those be picked up via the avro poms?
> On Fri, Sep 2, 2011 at 5:16 PM, Scott Carey <sc...@apache.org> wrote:
>>
>> Some comments:
>> You should check in the *.avdl or *.avsc file that describes your MyPair
>> schema instead of the generated code. Maven can be used to generate the
>> Java class using the avro-maven-plugin.
>> For example, add:
>> <plugin>
>> <groupId>org.apache.avro</groupId>
>> <artifactId>avro-maven-plugin</artifactId>
>> <version>1.5.3</version>
>> <executions>
>> <execution>
>> <id>schemas</id>
>> <phase>generate-sources</phase>
>> <goals>
>> <goal>schema</goal>
>> </goals>
>> <configuration>
>> <sourceDirectory>src/main/avro</sourceDirectory>
>>
>> <outputDirectory>target/generated-sources/avro</outputDirectory>
>> </configuration>
>> </execution>
>> </executions>
>> </plugin>
>> To your pom to compile *.avsc files from src/main/avro into
>> target/generated-sources/avro.
>> For more help on this, add the plugin above to your pom, and type:
>> 'mvn avro:help' for basic documentation, or 'mvn avro:help –Ddetail=true'
>> for full documentation on the maven plugin.
>> Some IDE's are a little picky with this (especially Eclipse, and moreso
>> with recent versions) and you may need to manually generate the source from
>> command line or in the IDE ('mvn generate-sources');
>> -Scott
>> On 9/2/11 2:41 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
>>
>> I'm new to Avro. Since I'm having trouble finding simple examples online
>> I'm writing one of my own that I'm putting on github.
>> https://github.com/wpm/AvroExample
>> Hopefully, this will be of help to people like me who are also having
>> trouble finding simple code examples.
>> I want to get this compiling without of hitch in Maven. I had it running
>> with a 1.4 version of Avro, but when I changed that to 1.5, some of the code
>> no longer works. Specifically, BinaryEncoder can no longer be instantiated
>> directly because it is now an abstract class (AvroExample.java: line 33) and
>> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line 41).
>> How should I modify this code so that it works with the latest and
>> greatest version of Avro? I looked through the Release Notes, but the
>> answers weren't obvious.
>> Thanks.
>
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by "W.P. McNeill" <bi...@gmail.com>.
I got a pull request on github for a change that does this. I'm looking
through it now.
The problem I'm having is that Maven goals fail with the following error:
Downloading:
http://repo1.maven.org/maven2/org/codehaus/mojo/exec-maven-plugin/1.1/exec-maven-plugin-1.1.pom
[WARNING] Failed to create parent directories for resolution tracking file
/Users/williammcneill/.m2/repository/org/codehaus/mojo/exec-maven-plugin/1.1/exec-maven-plugin-1.1.pom.lastUpdated
Does this mean I have to add extra repositories to my settings.xml, or
should those be picked up via the avro poms?
On Fri, Sep 2, 2011 at 5:16 PM, Scott Carey <sc...@apache.org> wrote:
> Some comments:
>
> You should check in the *.avdl or *.avsc file that describes your MyPair
> schema instead of the generated code. Maven can be used to generate the
> Java class using the avro-maven-plugin.
>
> For example, add:
> <plugin>
> <groupId>org.apache.avro</groupId>
> <artifactId>avro-maven-plugin</artifactId>
> <version>1.5.3</version>
> <executions>
> <execution>
> <id>schemas</id>
> <phase>generate-sources</phase>
> <goals>
> <goal>schema</goal>
> </goals>
> <configuration>
> <sourceDirectory>src/main/avro</sourceDirectory>
>
> <outputDirectory>target/generated-sources/avro</outputDirectory>
> </configuration>
> </execution>
> </executions>
> </plugin>
>
> To your pom to compile *.avsc files from src/main/avro into
> target/generated-sources/avro.
>
> For more help on this, add the plugin above to your pom, and type:
>
> 'mvn avro:help' for basic documentation, or 'mvn avro:help –Ddetail=true'
> for full documentation on the maven plugin.
>
> Some IDE's are a little picky with this (especially Eclipse, and moreso
> with recent versions) and you may need to manually generate the source from
> command line or in the IDE ('mvn generate-sources');
>
> -Scott
>
> On 9/2/11 2:41 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
>
> I'm new to Avro. Since I'm having trouble finding simple examples online
> I'm writing one of my own that I'm putting on github.
>
> https://github.com/wpm/AvroExample
>
> Hopefully, this will be of help to people like me who are also having
> trouble finding simple code examples.
>
> I want to get this compiling without of hitch in Maven. I had it running
> with a 1.4 version of Avro, but when I changed that to 1.5, some of the code
> no longer works. Specifically, BinaryEncoder can no longer be instantiated
> directly because it is now an abstract class (AvroExample.java: line 33) and
> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line 41).
>
> How should I modify this code so that it works with the latest and greatest
> version of Avro? I looked through the Release Notes, but the answers
> weren't obvious.
>
> Thanks.
>
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by Scott Carey <sc...@apache.org>.
The javadoc for the deprecated method directs users to the replacement.
BinaryEncoder and BinaryDecoder are well documented, with docs available via
maven for IDE's to consume easily, or via the Apache Avro website:
http://avro.apache.org/docs/1.5.3/api/java/org/apache/avro/io/BinaryEncoder.
html
http://avro.apache.org/docs/1.5.3/api/java/org/apache/avro/io/DecoderFactory
.html
defaultFactory
<http://avro.apache.org/docs/1.5.3/api/java/org/apache/avro/io/DecoderFactor
y.html#defaultFactory%28%29> ()
" Deprecated. use the equivalent get()
<http://avro.apache.org/docs/1.5.3/api/java/org/apache/avro/io/DecoderFactor
y.html#get%28%29> instead"
Generally, when using Avro you will have an easier time if you have the docs
available in your IDE or at least available for reference in a browser.
There are not a lot of blog posts and examples out there, but the javadoc is
mostly decent and we try hard to make sure all public and protected methods
and constructors have documentation. Many classes and packages have solid
documentation as well. Please report any documentation bugs or suggestions
for improvement.
Thanks!
-Scott
On 9/2/11 2:41 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
> I'm new to Avro. Since I'm having trouble finding simple examples online I'm
> writing one of my own that I'm putting on github.
>
> https://github.com/wpm/AvroExample
>
> Hopefully, this will be of help to people like me who are also having trouble
> finding simple code examples.
>
> I want to get this compiling without of hitch in Maven. I had it running with
> a 1.4 version of Avro, but when I changed that to 1.5, some of the code no
> longer works. Specifically, BinaryEncoder can no longer be instantiated
> directly because it is now an abstract class (AvroExample.java: line 33) and
> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line 41).
>
> How should I modify this code so that it works with the latest and greatest
> version of Avro? I looked through the Release Notes, but the answers weren't
> obvious.
>
> Thanks.
>
Re: How should I migrate 1.4 code to avro 1.5?
Posted by Scott Carey <sc...@apache.org>.
Some comments:
You should check in the *.avdl or *.avsc file that describes your MyPair
schema instead of the generated code. Maven can be used to generate the
Java class using the avro-maven-plugin.
For example, add:
<plugin>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>1.5.3</version>
<executions>
<execution>
<id>schemas</id>
<phase>generate-sources</phase>
<goals>
<goal>schema</goal>
</goals>
<configuration>
<sourceDirectory>src/main/avro</sourceDirectory>
<outputDirectory>target/generated-sources/avro</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
To your pom to compile *.avsc files from src/main/avro into
target/generated-sources/avro.
For more help on this, add the plugin above to your pom, and type:
'mvn avro:help' for basic documentation, or 'mvn avro:help Ddetail=true'
for full documentation on the maven plugin.
Some IDE's are a little picky with this (especially Eclipse, and moreso with
recent versions) and you may need to manually generate the source from
command line or in the IDE ('mvn generate-sources');
-Scott
On 9/2/11 2:41 PM, "W.P. McNeill" <bi...@gmail.com> wrote:
> I'm new to Avro. Since I'm having trouble finding simple examples online I'm
> writing one of my own that I'm putting on github.
>
> https://github.com/wpm/AvroExample
>
> Hopefully, this will be of help to people like me who are also having trouble
> finding simple code examples.
>
> I want to get this compiling without of hitch in Maven. I had it running with
> a 1.4 version of Avro, but when I changed that to 1.5, some of the code no
> longer works. Specifically, BinaryEncoder can no longer be instantiated
> directly because it is now an abstract class (AvroExample.java: line 33) and
> DecoderFactory.defaultFactory is deprecated (AvroExample.java: line 41).
>
> How should I modify this code so that it works with the latest and greatest
> version of Avro? I looked through the Release Notes, but the answers weren't
> obvious.
>
> Thanks.
>