You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by michał <mi...@gmail.com> on 2013/10/01 13:24:08 UTC
Re: Avro Client-Server - generic vs specific
In this situation then wen we have a AVDL protocol definition. The
documentation mentions Avro tools for generation.
But I need runtime protocol changes.
Can the generation of protocol classes be done automatically at runtime?
W dniu 30.09.2013 23:08, Doug Cutting pisze:
> For RPC, specific is usually most convenient. The generated interface
> and classes can be used to create requests and make calls, while a
> service can implement the interface. Generic might be useful for
> proxy-type RPC services, that can handle RPCs made in any protocol.
>
> Doug
>
> On Mon, Sep 30, 2013 at 3:32 AM, michał <mi...@gmail.com> wrote:
>> Hello,
>>
>> We have got two ways of creating an RPC communication protocol from AVDL.
>>
>> Generic:
>> https://github.com/matteobertozzi/Hadoop/blob/master/avro-examples/java/ipc/HTServer.java
>> Specific:https://github.com/phunt/avro-rpc-quickstart/blob/master/src/main/java/example/Main.java
>>
>> Specific provide a custom implementation (it require code generation based
>> on avdl) while generic provide 'generic' implementation. What this *generic*
>> actually means in terms of a protocol specification and code use?
>>
>> 1. What is the difference between the two apart from code look?
>> 2. I can not understand what would be a good example code showing the
>> advantages of Specific vs Generic responder?
>> 3. When would be the good choice to use *generic* and when *specific (code
>> generation?)* responder?
>>
>> thank you in advance for your time replying.
>>
>> Mick
Re: Avro Client-Server - generic vs specific
Posted by Mick <mi...@gmail.com>.
Can I use the following code rather the maven plugin:
Idl parserIDL = new
Idl(Main.class.getResourceAsStream("/net/avro/protocol_man.avdl"));
Protocol protocol = parserIDL.CompilationUnit();
mediatorResponder = new MediatorResponder(protocol);
startNettyServer(mediatorResponder);
parserIDL.close();
Where mediator Responder is:
public class MediatorResponder extends GenericResponder{
public MediatorResponder(Protocol protocol) {
super(protocol);
}
@Override
public Object respond(Protocol.Message msg, Object request) throws
Exception {
String msgName = msg.getName();
switch(msgName){
case "hello": System.out.println("HELLO"); break;
case "bye": System.out.println("BYE"); break;
default:
throw new AvroRuntimeException("unexcepcted message: "
+ msgName);
}
return null;
}
}
?
W dniu 01.10.2013 20:41, Doug Cutting pisze:
> On Tue, Oct 1, 2013 at 10:57 AM, michał <mi...@gmail.com> wrote:
>> 1. Would it be possible to program client-server and generate the protocol
>> classes programmatically at compile time?
> Yes. This is supported through Avro's Maven plugins by adding
> something like the following to your pom.xml:
>
> <plugin>
> <groupId>org.apache.avro</groupId>
> <artifactId>avro-maven-plugin</artifactId>
> <version>${avro.version}</version>
> <executions>
> <execution>
> <phase>generate-sources</phase>
> <goals>
> <goal>schema</goal>
> <goal>protocol</goal>
> <goal>idl-protocol</goal>
> </goals>
> </execution>
> </executions>
> </plugin>
>
> This will compile .avpr files from src/main/avro, generating .avsc and
> .avpr files in target/generated-sources/avro, then compile these to
> create .java files in target/generated-sources/java which will be
> compiled by the normal Maven java compilation plugins.
>
> Doug
Re: Avro Client-Server - generic vs specific
Posted by Doug Cutting <cu...@apache.org>.
On Tue, Oct 1, 2013 at 10:57 AM, michał <mi...@gmail.com> wrote:
> 1. Would it be possible to program client-server and generate the protocol
> classes programmatically at compile time?
Yes. This is supported through Avro's Maven plugins by adding
something like the following to your pom.xml:
<plugin>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>${avro.version}</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>schema</goal>
<goal>protocol</goal>
<goal>idl-protocol</goal>
</goals>
</execution>
</executions>
</plugin>
This will compile .avpr files from src/main/avro, generating .avsc and
.avpr files in target/generated-sources/avro, then compile these to
create .java files in target/generated-sources/java which will be
compiled by the normal Maven java compilation plugins.
Doug
Re: Avro Client-Server - generic vs specific
Posted by michał <mi...@gmail.com>.
I see. In that case I have to cut on this requirement.
It would be enough if I could generate programmatically all the classes
at compile-time.
1. Would it be possible to program client-server and generate the
protocol classes programmatically at compile time?
(I am aware of avro tools but they require a separate call for the
specific tool. I would like to avoid any extra calls and embed this call
in my code.)
2. Would it be enough if I store the protocol AVDL/schemas files at
server and the client can read from the handshake the protocol?
P.S.1
My goal at the moment is to write a full Client-Server app using
eclipse, maven while defining protocol in AVDL without the additional
calls for Avro Tools. I want to run such project with simple maven
package (mvn package). There is no working and complete example on
internet for such use case and the doc does not seem to cover all of
the details.
P.S.2
I would be more than happy to contribute to docs and provide a full
detailed example/tutorial when I accomplish this task for the sake of
future users. That is if you have nothing against Doug.
W dniu 01.10.2013 18:32, Doug Cutting pisze:
> You would like to make runtime protocol changes and would also still
> like to use code generation? You could perhaps use
> java.lang.reflect.Proxy to implement a generated protocol interface.
> If you need to process new protocols, then you could perhaps compile
> an AVDL file to Java at runtime, compile that Java to .class files
> (with javax.tools.JavaCompiler) then load those class files (using
> URLClassLoader), then use Java reflection to create instances, make
> calls and implement services. But this seems like a lot more work
> than just using generic. More information about what you need to
> accomplish would be useful.
>
> Doug
>
> On Tue, Oct 1, 2013 at 4:24 AM, michał <mi...@gmail.com> wrote:
>> In this situation then wen we have a AVDL protocol definition. The
>> documentation mentions Avro tools for generation.
>> But I need runtime protocol changes.
>>
>> Can the generation of protocol classes be done automatically at runtime?
>>
>> W dniu 30.09.2013 23:08, Doug Cutting pisze:
>>
>>> For RPC, specific is usually most convenient. The generated interface
>>> and classes can be used to create requests and make calls, while a
>>> service can implement the interface. Generic might be useful for
>>> proxy-type RPC services, that can handle RPCs made in any protocol.
>>>
>>> Doug
>>>
>>> On Mon, Sep 30, 2013 at 3:32 AM, michał <mi...@gmail.com> wrote:
>>>> Hello,
>>>>
>>>> We have got two ways of creating an RPC communication protocol from AVDL.
>>>>
>>>> Generic:
>>>>
>>>> https://github.com/matteobertozzi/Hadoop/blob/master/avro-examples/java/ipc/HTServer.java
>>>>
>>>> Specific:https://github.com/phunt/avro-rpc-quickstart/blob/master/src/main/java/example/Main.java
>>>>
>>>> Specific provide a custom implementation (it require code generation
>>>> based
>>>> on avdl) while generic provide 'generic' implementation. What this
>>>> *generic*
>>>> actually means in terms of a protocol specification and code use?
>>>>
>>>> 1. What is the difference between the two apart from code look?
>>>> 2. I can not understand what would be a good example code showing the
>>>> advantages of Specific vs Generic responder?
>>>> 3. When would be the good choice to use *generic* and when *specific
>>>> (code
>>>> generation?)* responder?
>>>>
>>>> thank you in advance for your time replying.
>>>>
>>>> Mick
>>
Re: Avro Client-Server - generic vs specific
Posted by Doug Cutting <cu...@apache.org>.
You would like to make runtime protocol changes and would also still
like to use code generation? You could perhaps use
java.lang.reflect.Proxy to implement a generated protocol interface.
If you need to process new protocols, then you could perhaps compile
an AVDL file to Java at runtime, compile that Java to .class files
(with javax.tools.JavaCompiler) then load those class files (using
URLClassLoader), then use Java reflection to create instances, make
calls and implement services. But this seems like a lot more work
than just using generic. More information about what you need to
accomplish would be useful.
Doug
On Tue, Oct 1, 2013 at 4:24 AM, michał <mi...@gmail.com> wrote:
> In this situation then wen we have a AVDL protocol definition. The
> documentation mentions Avro tools for generation.
> But I need runtime protocol changes.
>
> Can the generation of protocol classes be done automatically at runtime?
>
> W dniu 30.09.2013 23:08, Doug Cutting pisze:
>
>> For RPC, specific is usually most convenient. The generated interface
>> and classes can be used to create requests and make calls, while a
>> service can implement the interface. Generic might be useful for
>> proxy-type RPC services, that can handle RPCs made in any protocol.
>>
>> Doug
>>
>> On Mon, Sep 30, 2013 at 3:32 AM, michał <mi...@gmail.com> wrote:
>>>
>>> Hello,
>>>
>>> We have got two ways of creating an RPC communication protocol from AVDL.
>>>
>>> Generic:
>>>
>>> https://github.com/matteobertozzi/Hadoop/blob/master/avro-examples/java/ipc/HTServer.java
>>>
>>> Specific:https://github.com/phunt/avro-rpc-quickstart/blob/master/src/main/java/example/Main.java
>>>
>>> Specific provide a custom implementation (it require code generation
>>> based
>>> on avdl) while generic provide 'generic' implementation. What this
>>> *generic*
>>> actually means in terms of a protocol specification and code use?
>>>
>>> 1. What is the difference between the two apart from code look?
>>> 2. I can not understand what would be a good example code showing the
>>> advantages of Specific vs Generic responder?
>>> 3. When would be the good choice to use *generic* and when *specific
>>> (code
>>> generation?)* responder?
>>>
>>> thank you in advance for your time replying.
>>>
>>> Mick
>
>