You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@avro.apache.org by Rohini Uppuluri <ro...@glassdoor.com> on 2014/05/13 19:27:28 UTC

Importing Java class in Avro IDL

Hi,

I am looking to using avro serialization for our data. I see that we can import avro idl within another idl file.
I was wondering if there was a way we could import java object as I have a use case for the same.

For example:

Class B has:
{
  String element1;
  A a;

}

And A is in another project.
Class A {
    String elem1;
   String elem2;
   int id;
};

I am trying to definite avro schema for B.  A is from a totally different project that does not use avro.

I could do it after code generation but I was wondering if there was any other way to do it.

Thanks,
-Rohini

RE: Importing Java class in Avro IDL

Posted by Rohini Uppuluri <ro...@glassdoor.com>.

Thanks Martin. I will check it out.

From: Martin Kleppmann [mailto:mkleppmann@linkedin.com]
Sent: Friday, May 23, 2014 1:05 AM
To: user@avro.apache.org
Subject: Re: Importing Java class in Avro IDL

There is a mechanism for using richer, language-specific types called logical types, which was introduced recently: https://issues.apache.org/jira/browse/AVRO-1402 -- this might be able to do what you want. It's currently used for encoding a Java BigDecimal in Avro, but it can be generalised.

However, it's very new, and not yet released. If you want to try it, you'll need to check out Avro from svn and build it.

Martin

On 14 May 2014, at 21:44, Rohini Uppuluri <ro...@glassdoor.com>> wrote:
Thanks Martin.
That's true. It would break cross language compability.
I am going to create an Avro schema for my java classes.

My java classes can have additions/deletion of fields.  That's why, I was trying to see if there was a better way to handle it.

From: Martin Kleppmann [mailto:mkleppmann@linkedin.com]
Sent: Wednesday, May 14, 2014 1:35 AM
To: user@avro.apache.org<ma...@avro.apache.org>
Subject: Re: Importing Java class in Avro IDL

Hi Rohini,

Avro schemas can only refer to types defined within Avro. Referring to a Java type would break Avro's cross-language compatibility. I'm afraid you'll need to create an Avro schema that is the equivalent of Java class B.

Martin

On 13 May 2014, at 18:27, Rohini Uppuluri <ro...@glassdoor.com>> wrote:
Hi,

I am looking to using avro serialization for our data. I see that we can import avro idl within another idl file.
I was wondering if there was a way we could import java object as I have a use case for the same.

For example:

Class B has:
{
  String element1;
  A a;

}

And A is in another project.
Class A {
    String elem1;
   String elem2;
   int id;
};

I am trying to definite avro schema for B.  A is from a totally different project that does not use avro.

I could do it after code generation but I was wondering if there was any other way to do it.

Thanks,
-Rohini

Re: Importing Java class in Avro IDL

Posted by Martin Kleppmann <mk...@linkedin.com>.

There is a mechanism for using richer, language-specific types called logical types, which was introduced recently: https://issues.apache.org/jira/browse/AVRO-1402 -- this might be able to do what you want. It's currently used for encoding a Java BigDecimal in Avro, but it can be generalised.

However, it's very new, and not yet released. If you want to try it, you'll need to check out Avro from svn and build it.

Martin

On 14 May 2014, at 21:44, Rohini Uppuluri <ro...@glassdoor.com>> wrote:
Thanks Martin.
That’s true. It would break cross language compability.
I am going to create an Avro schema for my java classes.

My java classes can have additions/deletion of fields.  That’s why, I was trying to see if there was a better way to handle it.

From: Martin Kleppmann [mailto:mkleppmann@linkedin.com]
Sent: Wednesday, May 14, 2014 1:35 AM
To: user@avro.apache.org<ma...@avro.apache.org>
Subject: Re: Importing Java class in Avro IDL

Hi Rohini,

Avro schemas can only refer to types defined within Avro. Referring to a Java type would break Avro's cross-language compatibility. I'm afraid you'll need to create an Avro schema that is the equivalent of Java class B.

Martin

On 13 May 2014, at 18:27, Rohini Uppuluri <ro...@glassdoor.com>> wrote:
Hi,

I am looking to using avro serialization for our data. I see that we can import avro idl within another idl file.
I was wondering if there was a way we could import java object as I have a use case for the same.

For example:

Class B has:
{
  String element1;
  A a;

}

And A is in another project.
Class A {
    String elem1;
   String elem2;
   int id;
};

I am trying to definite avro schema for B.  A is from a totally different project that does not use avro.

I could do it after code generation but I was wondering if there was any other way to do it.

Thanks,
-Rohini

RE: Importing Java class in Avro IDL

Posted by Rohini Uppuluri <ro...@glassdoor.com>.

Thanks Martin.
That's true. It would break cross language compability.
I am going to create an Avro schema for my java classes.

My java classes can have additions/deletion of fields.  That's why, I was trying to see if there was a better way to handle it.

From: Martin Kleppmann [mailto:mkleppmann@linkedin.com]
Sent: Wednesday, May 14, 2014 1:35 AM
To: user@avro.apache.org<ma...@avro.apache.org>
Subject: Re: Importing Java class in Avro IDL

Hi Rohini,

Avro schemas can only refer to types defined within Avro. Referring to a Java type would break Avro's cross-language compatibility. I'm afraid you'll need to create an Avro schema that is the equivalent of Java class B.

Martin

On 13 May 2014, at 18:27, Rohini Uppuluri <ro...@glassdoor.com>> wrote:
Hi,

I am looking to using avro serialization for our data. I see that we can import avro idl within another idl file.
I was wondering if there was a way we could import java object as I have a use case for the same.

For example:

Class B has:
{
  String element1;
  A a;

}

And A is in another project.
Class A {
    String elem1;
   String elem2;
   int id;
};

I am trying to definite avro schema for B.  A is from a totally different project that does not use avro.

I could do it after code generation but I was wondering if there was any other way to do it.

Thanks,
-Rohini

Re: Importing Java class in Avro IDL

Posted by Martin Kleppmann <mk...@linkedin.com>.

Hi Rohini,

Avro schemas can only refer to types defined within Avro. Referring to a Java type would break Avro's cross-language compatibility. I'm afraid you'll need to create an Avro schema that is the equivalent of Java class B.

Martin

On 13 May 2014, at 18:27, Rohini Uppuluri <ro...@glassdoor.com>> wrote:
Hi,

I am looking to using avro serialization for our data. I see that we can import avro idl within another idl file.
I was wondering if there was a way we could import java object as I have a use case for the same.

For example:

Class B has:
{
  String element1;
  A a;

}

And A is in another project.
Class A {
    String elem1;
   String elem2;
   int id;
};

I am trying to definite avro schema for B.  A is from a totally different project that does not use avro.

I could do it after code generation but I was wondering if there was any other way to do it.

Thanks,
-Rohini