You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Jarek Jarcec Cecho <ja...@apache.org> on 2015/09/29 17:00:12 UTC
Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
I wanted to gave heads up to Sqoop dev community - Colin is working on SQOOP-910 that will switch Sqoop from Tomcat to Jetty. The patch is almost ready to go and I’m expecting that I’ll commit it in a day or two. The “user visible” commands had remained, so I’m not anticipating huge impact, but nevertheless :)
Jarcec
Re: Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
Posted by Abraham Fine <ab...@cloudera.com>.
David-
I just tried debugging outside of integration test, this command works for me:
mvnDebug exec:java -Dexec.mainClass="org.apache.sqoop.server.SqoopJettyServer" -Dsqoop.config.dir=$SQOOP_CONF_DIR -pl server
Thanks,
Abe
> On Oct 14, 2015, at 11:55 AM, Abraham Fine <ab...@cloudera.com> wrote:
>
> David-
>
> Yep. Maybe we should consider adding a profile for the mvn exec plugin to automatically launch sqoop with the debugging port open. I am not super familiar with maven but I will try to set aside some time in the next week to see if I can get that set up.
>
> Thanks,
> Abe
>
> What do you think
>> On Oct 13, 2015, at 10:09 PM, David Robson <Da...@software.dell.com> wrote:
>>
>> Hi Abe,
>>
>> That would probably work - so do you basically run the maven build, then launch it and connect the remote debugger?
>>
>> I guess the problem with that is if you change 1 file, it can be a fair bit of time between changing the file and actually debugging the code. I'm running in a VM so it would be a minute or so. On tomcat I could use WTP which had the new code running within a second or so which made it a bit quicker to change some code, try it etc.
>>
>> I guess the other way to fix this would be to update the logic for finding the Sqoop jar. In Sqoop1 it had the following code:
>>
>> String sqoopJar = Jars.getSqoopJarPath();
>> if (null != sqoopJar) {
>> sqoopJar = File.pathSeparator + sqoopJar;
>> } else {
>> LOG.warn("Could not find sqoop jar; child compilation may fail");
>> sqoopJar = "";
>> }
>>
>> So basically if you were running in standalone mode in the one JVM it skipped the jar stuff and produced a warning which was handy for debugging. Looks like Sqoop2 just fails in this scenario.
>>
>> David
>>
>> -----Original Message-----
>> From: Abraham Fine [mailto:abefine@cloudera.com]
>> Sent: Wednesday, 14 October 2015 3:11 PM
>> To: dev@sqoop.apache.org
>> Subject: Re: Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
>>
>> Hi David,
>>
>> While I do not use Eclipse (I use Intellij Idea), I am pretty confident that this would apply to eclipse as well.
>>
>> I have used remote debugging with the integration tests (via -Dmaven.surefire.debug) and I am pretty confident that it would work well with your use case. It seems to be a recommended way debugging Jetty based applications.
>> http://www.eclipse.org/jetty/documentation/current/debugging-with-eclipse.html
>>
>> Abe
>>
>>> On Oct 13, 2015, at 8:52 PM, David Robson <Da...@software.dell.com> wrote:
>>>
>>> Hi Jarcec/Colin,
>>>
>>> I had Eclipse setup with Tomcat before where I could debug the code easily. So I ran Tomcat in debug mode from Eclipse, then when I changed a file in Eclipse it would automatically redeploy the app, and I could continue debugging.
>>>
>>> I can't work out how to do this with Jetty yet - do you have a document on how to setup Eclipse to debug Sqoop?
>>>
>>> I tried to run "org.apache.sqoop.server.SqoopJettyServer" from Eclipse - after much messing around with classpaths I got it to run but then it won't run the actual job. It says "failed to find jar for class: org.apache.sqoop.common.MapContext". I guess because Eclipse is running it off the class files rather than jar files.
>>>
>>> Can you please advise the best way to set this up?
>>>
>>> Thanks,
>>>
>>> David
>>>
>>> -----Original Message-----
>>> From: Jarek Jarcec Cecho [mailto:jarcec@gmail.com] On Behalf Of Jarek Jarcec Cecho
>>> Sent: Thursday, 1 October 2015 2:09 AM
>>> To: dev@sqoop.apache.org
>>> Subject: Re: Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
>>>
>>> SQOOP-910 was just committed. Please don’t hesitate and let me or Colin now if you see any issues.
>>>
>>> Jarcec
>>>
>>>> On Sep 29, 2015, at 8:00 AM, Jarek Jarcec Cecho <ja...@apache.org> wrote:
>>>>
>>>> I wanted to gave heads up to Sqoop dev community - Colin is working on SQOOP-910 that will switch Sqoop from Tomcat to Jetty. The patch is almost ready to go and I’m expecting that I’ll commit it in a day or two. The “user visible” commands had remained, so I’m not anticipating huge impact, but nevertheless :)
>>>>
>>>> Jarcec
>>>
>>
>
Re: Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
Posted by Abraham Fine <ab...@cloudera.com>.
David-
Yep. Maybe we should consider adding a profile for the mvn exec plugin to automatically launch sqoop with the debugging port open. I am not super familiar with maven but I will try to set aside some time in the next week to see if I can get that set up.
Thanks,
Abe
What do you think
> On Oct 13, 2015, at 10:09 PM, David Robson <Da...@software.dell.com> wrote:
>
> Hi Abe,
>
> That would probably work - so do you basically run the maven build, then launch it and connect the remote debugger?
>
> I guess the problem with that is if you change 1 file, it can be a fair bit of time between changing the file and actually debugging the code. I'm running in a VM so it would be a minute or so. On tomcat I could use WTP which had the new code running within a second or so which made it a bit quicker to change some code, try it etc.
>
> I guess the other way to fix this would be to update the logic for finding the Sqoop jar. In Sqoop1 it had the following code:
>
> String sqoopJar = Jars.getSqoopJarPath();
> if (null != sqoopJar) {
> sqoopJar = File.pathSeparator + sqoopJar;
> } else {
> LOG.warn("Could not find sqoop jar; child compilation may fail");
> sqoopJar = "";
> }
>
> So basically if you were running in standalone mode in the one JVM it skipped the jar stuff and produced a warning which was handy for debugging. Looks like Sqoop2 just fails in this scenario.
>
> David
>
> -----Original Message-----
> From: Abraham Fine [mailto:abefine@cloudera.com]
> Sent: Wednesday, 14 October 2015 3:11 PM
> To: dev@sqoop.apache.org
> Subject: Re: Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
>
> Hi David,
>
> While I do not use Eclipse (I use Intellij Idea), I am pretty confident that this would apply to eclipse as well.
>
> I have used remote debugging with the integration tests (via -Dmaven.surefire.debug) and I am pretty confident that it would work well with your use case. It seems to be a recommended way debugging Jetty based applications.
> http://www.eclipse.org/jetty/documentation/current/debugging-with-eclipse.html
>
> Abe
>
>> On Oct 13, 2015, at 8:52 PM, David Robson <Da...@software.dell.com> wrote:
>>
>> Hi Jarcec/Colin,
>>
>> I had Eclipse setup with Tomcat before where I could debug the code easily. So I ran Tomcat in debug mode from Eclipse, then when I changed a file in Eclipse it would automatically redeploy the app, and I could continue debugging.
>>
>> I can't work out how to do this with Jetty yet - do you have a document on how to setup Eclipse to debug Sqoop?
>>
>> I tried to run "org.apache.sqoop.server.SqoopJettyServer" from Eclipse - after much messing around with classpaths I got it to run but then it won't run the actual job. It says "failed to find jar for class: org.apache.sqoop.common.MapContext". I guess because Eclipse is running it off the class files rather than jar files.
>>
>> Can you please advise the best way to set this up?
>>
>> Thanks,
>>
>> David
>>
>> -----Original Message-----
>> From: Jarek Jarcec Cecho [mailto:jarcec@gmail.com] On Behalf Of Jarek Jarcec Cecho
>> Sent: Thursday, 1 October 2015 2:09 AM
>> To: dev@sqoop.apache.org
>> Subject: Re: Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
>>
>> SQOOP-910 was just committed. Please don’t hesitate and let me or Colin now if you see any issues.
>>
>> Jarcec
>>
>>> On Sep 29, 2015, at 8:00 AM, Jarek Jarcec Cecho <ja...@apache.org> wrote:
>>>
>>> I wanted to gave heads up to Sqoop dev community - Colin is working on SQOOP-910 that will switch Sqoop from Tomcat to Jetty. The patch is almost ready to go and I’m expecting that I’ll commit it in a day or two. The “user visible” commands had remained, so I’m not anticipating huge impact, but nevertheless :)
>>>
>>> Jarcec
>>
>
RE: Heads up: Migration from Jetty to Tomcat will happen soon
(SQOOP-910)
Posted by David Robson <Da...@software.dell.com>.
Hi Abe,
That would probably work - so do you basically run the maven build, then launch it and connect the remote debugger?
I guess the problem with that is if you change 1 file, it can be a fair bit of time between changing the file and actually debugging the code. I'm running in a VM so it would be a minute or so. On tomcat I could use WTP which had the new code running within a second or so which made it a bit quicker to change some code, try it etc.
I guess the other way to fix this would be to update the logic for finding the Sqoop jar. In Sqoop1 it had the following code:
String sqoopJar = Jars.getSqoopJarPath();
if (null != sqoopJar) {
sqoopJar = File.pathSeparator + sqoopJar;
} else {
LOG.warn("Could not find sqoop jar; child compilation may fail");
sqoopJar = "";
}
So basically if you were running in standalone mode in the one JVM it skipped the jar stuff and produced a warning which was handy for debugging. Looks like Sqoop2 just fails in this scenario.
David
-----Original Message-----
From: Abraham Fine [mailto:abefine@cloudera.com]
Sent: Wednesday, 14 October 2015 3:11 PM
To: dev@sqoop.apache.org
Subject: Re: Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
Hi David,
While I do not use Eclipse (I use Intellij Idea), I am pretty confident that this would apply to eclipse as well.
I have used remote debugging with the integration tests (via -Dmaven.surefire.debug) and I am pretty confident that it would work well with your use case. It seems to be a recommended way debugging Jetty based applications.
http://www.eclipse.org/jetty/documentation/current/debugging-with-eclipse.html
Abe
> On Oct 13, 2015, at 8:52 PM, David Robson <Da...@software.dell.com> wrote:
>
> Hi Jarcec/Colin,
>
> I had Eclipse setup with Tomcat before where I could debug the code easily. So I ran Tomcat in debug mode from Eclipse, then when I changed a file in Eclipse it would automatically redeploy the app, and I could continue debugging.
>
> I can't work out how to do this with Jetty yet - do you have a document on how to setup Eclipse to debug Sqoop?
>
> I tried to run "org.apache.sqoop.server.SqoopJettyServer" from Eclipse - after much messing around with classpaths I got it to run but then it won't run the actual job. It says "failed to find jar for class: org.apache.sqoop.common.MapContext". I guess because Eclipse is running it off the class files rather than jar files.
>
> Can you please advise the best way to set this up?
>
> Thanks,
>
> David
>
> -----Original Message-----
> From: Jarek Jarcec Cecho [mailto:jarcec@gmail.com] On Behalf Of Jarek Jarcec Cecho
> Sent: Thursday, 1 October 2015 2:09 AM
> To: dev@sqoop.apache.org
> Subject: Re: Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
>
> SQOOP-910 was just committed. Please don’t hesitate and let me or Colin now if you see any issues.
>
> Jarcec
>
>> On Sep 29, 2015, at 8:00 AM, Jarek Jarcec Cecho <ja...@apache.org> wrote:
>>
>> I wanted to gave heads up to Sqoop dev community - Colin is working on SQOOP-910 that will switch Sqoop from Tomcat to Jetty. The patch is almost ready to go and I’m expecting that I’ll commit it in a day or two. The “user visible” commands had remained, so I’m not anticipating huge impact, but nevertheless :)
>>
>> Jarcec
>
Re: Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
Posted by Abraham Fine <ab...@cloudera.com>.
Hi David,
While I do not use Eclipse (I use Intellij Idea), I am pretty confident that this would apply to eclipse as well.
I have used remote debugging with the integration tests (via -Dmaven.surefire.debug) and I am pretty confident that it would work well with your use case. It seems to be a recommended way debugging Jetty based applications.
http://www.eclipse.org/jetty/documentation/current/debugging-with-eclipse.html
Abe
> On Oct 13, 2015, at 8:52 PM, David Robson <Da...@software.dell.com> wrote:
>
> Hi Jarcec/Colin,
>
> I had Eclipse setup with Tomcat before where I could debug the code easily. So I ran Tomcat in debug mode from Eclipse, then when I changed a file in Eclipse it would automatically redeploy the app, and I could continue debugging.
>
> I can't work out how to do this with Jetty yet - do you have a document on how to setup Eclipse to debug Sqoop?
>
> I tried to run "org.apache.sqoop.server.SqoopJettyServer" from Eclipse - after much messing around with classpaths I got it to run but then it won't run the actual job. It says "failed to find jar for class: org.apache.sqoop.common.MapContext". I guess because Eclipse is running it off the class files rather than jar files.
>
> Can you please advise the best way to set this up?
>
> Thanks,
>
> David
>
> -----Original Message-----
> From: Jarek Jarcec Cecho [mailto:jarcec@gmail.com] On Behalf Of Jarek Jarcec Cecho
> Sent: Thursday, 1 October 2015 2:09 AM
> To: dev@sqoop.apache.org
> Subject: Re: Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
>
> SQOOP-910 was just committed. Please don’t hesitate and let me or Colin now if you see any issues.
>
> Jarcec
>
>> On Sep 29, 2015, at 8:00 AM, Jarek Jarcec Cecho <ja...@apache.org> wrote:
>>
>> I wanted to gave heads up to Sqoop dev community - Colin is working on SQOOP-910 that will switch Sqoop from Tomcat to Jetty. The patch is almost ready to go and I’m expecting that I’ll commit it in a day or two. The “user visible” commands had remained, so I’m not anticipating huge impact, but nevertheless :)
>>
>> Jarcec
>
RE: Heads up: Migration from Jetty to Tomcat will happen soon
(SQOOP-910)
Posted by David Robson <Da...@software.dell.com>.
Hi Jarcec/Colin,
I had Eclipse setup with Tomcat before where I could debug the code easily. So I ran Tomcat in debug mode from Eclipse, then when I changed a file in Eclipse it would automatically redeploy the app, and I could continue debugging.
I can't work out how to do this with Jetty yet - do you have a document on how to setup Eclipse to debug Sqoop?
I tried to run "org.apache.sqoop.server.SqoopJettyServer" from Eclipse - after much messing around with classpaths I got it to run but then it won't run the actual job. It says "failed to find jar for class: org.apache.sqoop.common.MapContext". I guess because Eclipse is running it off the class files rather than jar files.
Can you please advise the best way to set this up?
Thanks,
David
-----Original Message-----
From: Jarek Jarcec Cecho [mailto:jarcec@gmail.com] On Behalf Of Jarek Jarcec Cecho
Sent: Thursday, 1 October 2015 2:09 AM
To: dev@sqoop.apache.org
Subject: Re: Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
SQOOP-910 was just committed. Please don’t hesitate and let me or Colin now if you see any issues.
Jarcec
> On Sep 29, 2015, at 8:00 AM, Jarek Jarcec Cecho <ja...@apache.org> wrote:
>
> I wanted to gave heads up to Sqoop dev community - Colin is working on SQOOP-910 that will switch Sqoop from Tomcat to Jetty. The patch is almost ready to go and I’m expecting that I’ll commit it in a day or two. The “user visible” commands had remained, so I’m not anticipating huge impact, but nevertheless :)
>
> Jarcec
Re: Heads up: Migration from Jetty to Tomcat will happen soon (SQOOP-910)
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
SQOOP-910 was just committed. Please don’t hesitate and let me or Colin now if you see any issues.
Jarcec
> On Sep 29, 2015, at 8:00 AM, Jarek Jarcec Cecho <ja...@apache.org> wrote:
>
> I wanted to gave heads up to Sqoop dev community - Colin is working on SQOOP-910 that will switch Sqoop from Tomcat to Jetty. The patch is almost ready to go and I’m expecting that I’ll commit it in a day or two. The “user visible” commands had remained, so I’m not anticipating huge impact, but nevertheless :)
>
> Jarcec