You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by terreyshih <te...@gmail.com> on 2014/09/15 01:48:04 UTC
starting agent from eclipse ?
Hi,
I have written a customized sink which calls some POJO API and does some processing. The problem is I would like to debug the whole process via Eclipse.
How can I start an agent which uses this customized sink in Eclipse so I can trace/debut it ?
I can probably do remote debugging but I was wondering if anyone has done it in Eclipse.
thanks,
-Gary
Re: starting agent from eclipse ?
Posted by terreyshih <te...@gmail.com>.
thanks.
On Sep 15, 2014, at 1:33 AM, Ahmed Vila <av...@devlogic.eu> wrote:
> Hi Gary,
>
> I'm not quite sure how it works on windows, but it might be that there is a need for backward slash after the port, just like there is one on a first line.
>
> Anyway, I see that there is @JAVA_OPTS variable.
> Try to search across startup script and I can bet it's set to something like:
> JAVA_OPTS="-Xmx20m"
>
> In that case, just add the line I've gave you into this var and revert the startup lines you've modified, so java_opts looks something like this:
> JAVA_OPTS="-Xmx20m -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=10100"
>
>
>
>
> On Mon, Sep 15, 2014 at 10:22 AM, terreyshih <te...@gmail.com> wrote:
> Hi,
>
> Do you mind sharing with me the modified startup java option ?
>
> This is what I have (adding the agent lib line)
>
> $EXEC $JAVA_HOME/bin/java $JAVA_OPTS $FLUME_JAVA_OPTS "${arr_java_props[@]}" -cp "$FLUME_CLASSPATH" \
> -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=10100
> -Djava.library.path=$FLUME_JAVA_LIBRARY_PATH "$FLUME_APPLICATION_CLASS" $
>
> I can’t seem to get it attached correctly
>
> thx
> On Sep 15, 2014, at 12:55 AM, Ahmed Vila <av...@devlogic.eu> wrote:
>
>> Hi Gary,
>>
>> Since the Flume is Java app and your custom Sink code is running in it's VM space, you can do it either way you normally debug Java apps.
>>
>> I've changed Flume startup Java options and added in order to enable remote debugging:
>> -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=10100
>>
>> Change opts accordingly: suspend=y/n to define if JVM will wait for the debugger to attach before it starts the Flume; address=10100 is a port debugger socket will listen to.
>> I find it difficult to debug without suspension because sinks are configured during startup, so having a breakpoint in configure method might be worthwhile.
>>
>> In Eclipse, open any file from your Sink so it can pick up correct project and go to menu Run > Debug Configurations. In that window choose Remote Java Application and click in new icon. Everything will be offered by default and the only thing that needs to be changed is port. From that point on, debug configuration with your project name or wharever the name you gave to debug configuration will be available for attaching.
>>
>>
>>
>>
>> On Mon, Sep 15, 2014 at 1:48 AM, terreyshih <te...@gmail.com> wrote:
>> Hi,
>>
>> I have written a customized sink which calls some POJO API and does some processing. The problem is I would like to debug the whole process via Eclipse.
>>
>> How can I start an agent which uses this customized sink in Eclipse so I can trace/debut it ?
>>
>> I can probably do remote debugging but I was wondering if anyone has done it in Eclipse.
>>
>> thanks,
>> -Gary
>>
>>
>>
>> --
>> Best regards,
>>
>> Ahmed Vila
>>
>> ---------------------------------------------------------------------
>> This e-mail and any attachment is for authorised use by the intended recipient(s) only. This email contains confidential information. It should not be copied, disclosed to, retained or used by, any party other than the intended recipient. Any unauthorised distribution, dissemination or copying of this E-mail or its attachments, and/or any use of any information contained in them, is strictly prohibited and may be illegal. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender directly via email. Any emails that you send to us may be monitored by systems or persons other than the named communicant for the purposes of ascertaining whether the communication complies with the law and company policies.
>
>
>
>
> --
> Best regards,
>
> Ahmed Vila
>
> ---------------------------------------------------------------------
> This e-mail and any attachment is for authorised use by the intended recipient(s) only. This email contains confidential information. It should not be copied, disclosed to, retained or used by, any party other than the intended recipient. Any unauthorised distribution, dissemination or copying of this E-mail or its attachments, and/or any use of any information contained in them, is strictly prohibited and may be illegal. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender directly via email. Any emails that you send to us may be monitored by systems or persons other than the named communicant for the purposes of ascertaining whether the communication complies with the law and company policies.
Re: starting agent from eclipse ?
Posted by Ahmed Vila <av...@devlogic.eu>.
Hi Gary,
I'm not quite sure how it works on windows, but it might be that there is a
need for backward slash after the port, just like there is one on a first
line.
Anyway, I see that there is @JAVA_OPTS variable.
Try to search across startup script and I can bet it's set to something
like:
JAVA_OPTS="-Xmx20m"
In that case, just add the line I've gave you into this var and revert the
startup lines you've modified, so java_opts looks something like this:
JAVA_OPTS="-Xmx20m
-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=10100"
On Mon, Sep 15, 2014 at 10:22 AM, terreyshih <te...@gmail.com> wrote:
> Hi,
>
> Do you mind sharing with me the modified startup java option ?
>
> This is what I have (adding the agent lib line)
>
> $EXEC $JAVA_HOME/bin/java $JAVA_OPTS $FLUME_JAVA_OPTS "${arr_java_props[@
> ]}" -cp "$FLUME_CLASSPATH" \
> -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=
> 10100
> -Djava.library.path=$FLUME_JAVA_LIBRARY_PATH "
> $FLUME_APPLICATION_CLASS" $
>
> I can’t seem to get it attached correctly
>
> thx
> On Sep 15, 2014, at 12:55 AM, Ahmed Vila <av...@devlogic.eu> wrote:
>
> Hi Gary,
>
> Since the Flume is Java app and your custom Sink code is running in it's
> VM space, you can do it either way you normally debug Java apps.
>
> I've changed Flume startup Java options and added in order to enable
> remote debugging:
> -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=10100
>
> Change opts accordingly: suspend=y/n to define if JVM will wait for the
> debugger to attach before it starts the Flume; address=10100 is a port
> debugger socket will listen to.
> I find it difficult to debug without suspension because sinks are
> configured during startup, so having a breakpoint in configure method might
> be worthwhile.
>
> In Eclipse, open any file from your Sink so it can pick up correct project
> and go to menu Run > Debug Configurations. In that window choose Remote
> Java Application and click in new icon. Everything will be offered by
> default and the only thing that needs to be changed is port. From that
> point on, debug configuration with your project name or wharever the name
> you gave to debug configuration will be available for attaching.
>
>
>
>
> On Mon, Sep 15, 2014 at 1:48 AM, terreyshih <te...@gmail.com> wrote:
>
>> Hi,
>>
>> I have written a customized sink which calls some POJO API and does some
>> processing. The problem is I would like to debug the whole process via
>> Eclipse.
>>
>> How can I start an agent which uses this customized sink in Eclipse so I
>> can trace/debut it ?
>>
>> I can probably do remote debugging but I was wondering if anyone has done
>> it in Eclipse.
>>
>> thanks,
>> -Gary
>
>
>
>
> --
>
> Best regards,
> Ahmed Vila
>
> ---------------------------------------------------------------------
> This e-mail and any attachment is for authorised use by the intended
> recipient(s) only. This email contains confidential information. It should
> not be copied, disclosed to, retained or used by, any party other than the
> intended recipient. Any unauthorised distribution, dissemination or copying
> of this E-mail or its attachments, and/or any use of any information
> contained in them, is strictly prohibited and may be illegal. If you are
> not an intended recipient then please promptly delete this e-mail and any
> attachment and all copies and inform the sender directly via email. Any
> emails that you send to us may be monitored by systems or persons other
> than the named communicant for the purposes of ascertaining whether the
> communication complies with the law and company policies.
>
>
>
--
Best regards,
Ahmed Vila
--
---------------------------------------------------------------------
This e-mail and any attachment is for authorised use by the intended
recipient(s) only. This email contains confidential information. It should
not be copied, disclosed to, retained or used by, any party other than the
intended recipient. Any unauthorised distribution, dissemination or copying
of this E-mail or its attachments, and/or any use of any information
contained in them, is strictly prohibited and may be illegal. If you are
not an intended recipient then please promptly delete this e-mail and any
attachment and all copies and inform the sender directly via email. Any
emails that you send to us may be monitored by systems or persons other
than the named communicant for the purposes of ascertaining whether the
communication complies with the law and company policies.
Re: starting agent from eclipse ?
Posted by terreyshih <te...@gmail.com>.
Hi,
Do you mind sharing with me the modified startup java option ?
This is what I have (adding the agent lib line)
$EXEC $JAVA_HOME/bin/java $JAVA_OPTS $FLUME_JAVA_OPTS "${arr_java_props[@]}" -cp "$FLUME_CLASSPATH" \
-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=10100
-Djava.library.path=$FLUME_JAVA_LIBRARY_PATH "$FLUME_APPLICATION_CLASS" $
I can’t seem to get it attached correctly
thx
On Sep 15, 2014, at 12:55 AM, Ahmed Vila <av...@devlogic.eu> wrote:
> Hi Gary,
>
> Since the Flume is Java app and your custom Sink code is running in it's VM space, you can do it either way you normally debug Java apps.
>
> I've changed Flume startup Java options and added in order to enable remote debugging:
> -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=10100
>
> Change opts accordingly: suspend=y/n to define if JVM will wait for the debugger to attach before it starts the Flume; address=10100 is a port debugger socket will listen to.
> I find it difficult to debug without suspension because sinks are configured during startup, so having a breakpoint in configure method might be worthwhile.
>
> In Eclipse, open any file from your Sink so it can pick up correct project and go to menu Run > Debug Configurations. In that window choose Remote Java Application and click in new icon. Everything will be offered by default and the only thing that needs to be changed is port. From that point on, debug configuration with your project name or wharever the name you gave to debug configuration will be available for attaching.
>
>
>
>
> On Mon, Sep 15, 2014 at 1:48 AM, terreyshih <te...@gmail.com> wrote:
> Hi,
>
> I have written a customized sink which calls some POJO API and does some processing. The problem is I would like to debug the whole process via Eclipse.
>
> How can I start an agent which uses this customized sink in Eclipse so I can trace/debut it ?
>
> I can probably do remote debugging but I was wondering if anyone has done it in Eclipse.
>
> thanks,
> -Gary
>
>
>
> --
> Best regards,
>
> Ahmed Vila
>
> ---------------------------------------------------------------------
> This e-mail and any attachment is for authorised use by the intended recipient(s) only. This email contains confidential information. It should not be copied, disclosed to, retained or used by, any party other than the intended recipient. Any unauthorised distribution, dissemination or copying of this E-mail or its attachments, and/or any use of any information contained in them, is strictly prohibited and may be illegal. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender directly via email. Any emails that you send to us may be monitored by systems or persons other than the named communicant for the purposes of ascertaining whether the communication complies with the law and company policies.
Re: starting agent from eclipse ?
Posted by Ahmed Vila <av...@devlogic.eu>.
Hi Gary,
Since the Flume is Java app and your custom Sink code is running in it's VM
space, you can do it either way you normally debug Java apps.
I've changed Flume startup Java options and added in order to enable remote
debugging:
-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=10100
Change opts accordingly: suspend=y/n to define if JVM will wait for the
debugger to attach before it starts the Flume; address=10100 is a port
debugger socket will listen to.
I find it difficult to debug without suspension because sinks are
configured during startup, so having a breakpoint in configure method might
be worthwhile.
In Eclipse, open any file from your Sink so it can pick up correct project
and go to menu Run > Debug Configurations. In that window choose Remote
Java Application and click in new icon. Everything will be offered by
default and the only thing that needs to be changed is port. From that
point on, debug configuration with your project name or wharever the name
you gave to debug configuration will be available for attaching.
On Mon, Sep 15, 2014 at 1:48 AM, terreyshih <te...@gmail.com> wrote:
> Hi,
>
> I have written a customized sink which calls some POJO API and does some
> processing. The problem is I would like to debug the whole process via
> Eclipse.
>
> How can I start an agent which uses this customized sink in Eclipse so I
> can trace/debut it ?
>
> I can probably do remote debugging but I was wondering if anyone has done
> it in Eclipse.
>
> thanks,
> -Gary
--
Best regards,
Ahmed Vila
--
---------------------------------------------------------------------
This e-mail and any attachment is for authorised use by the intended
recipient(s) only. This email contains confidential information. It should
not be copied, disclosed to, retained or used by, any party other than the
intended recipient. Any unauthorised distribution, dissemination or copying
of this E-mail or its attachments, and/or any use of any information
contained in them, is strictly prohibited and may be illegal. If you are
not an intended recipient then please promptly delete this e-mail and any
attachment and all copies and inform the sender directly via email. Any
emails that you send to us may be monitored by systems or persons other
than the named communicant for the purposes of ascertaining whether the
communication complies with the law and company policies.
Re: starting agent from eclipse ?
Posted by Santiago Mola <sm...@stratio.com>.
Hi Gary,
2014-09-15 1:48 GMT+02:00 terreyshih <te...@gmail.com>:
>
> I have written a customized sink which calls some POJO API and does some
> processing. The problem is I would like to debug the whole process via
> Eclipse.
>
> How can I start an agent which uses this customized sink in Eclipse so I
> can trace/debut it ?
>
> I can probably do remote debugging but I was wondering if anyone has done
> it in Eclipse.
>
>
I usually set up some tests and then debug them in my IDE. You can see some
sink tests at the flume-ng-sinks package:
https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchSink.java
https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-irc-sink/src/test/java/org/apache/flume/sink/irc/TestIRCSink.java
These tests use a channel and a sink and feed test events directly to the
channel. The general idea is:
// Set up context
Context context = new Context();
context.put("foo", "bar");
// Instantiate sink and configure it
Sink sink = new MySink();
Configurables.configure(sink, context);
// Instantiate memoryChannel and configure it
Channel memoryChannel = new MemoryChannel();
Configurables.configure(memoryChannel, context);
sink.setChannel(memoryChannel);
sink.start();
// Start a transaction and put some events
Transaction txn = memoryChannel.getTransaction();
txn.begin();
Event event = // ... generate your test event ...
memoryChannel.put(event);
txn.commit();
txn.close();
// Process events with sink
Sink.Status status = sink.process();
// ... check status, sink effects, etc ...
You can add a source to the mix if that's really needed or even set up a
full agent.
Best,
--
Santiago M. Mola
Avenida de Europa, 26. Ática 5. 3ª Planta
28224 Pozuelo de Alarcón, Madrid
Tel: 91 352 59 42 // @stratiobd