You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Yu Qiao <yq...@columbia.edu> on 2013/04/10 08:28:13 UTC
Core dump caused by org.apache.http.client.protocol.RequestAddCookies.process
Hi Httpclient-users,
We are using HttpAsyncClient library to do web crawling. Basically we are just using the method provided by the sample code (http://hc.apache.org/httpcomponents-asyncclient-dev/httpasyncclient/examples/org/apache/http/examples/nio/client/AsyncClientHttpExchangeFutureCallback.java), but has a much larger request size (~9000).
While our program runs perfectly fine on my laptop (MacOSX 10.8.2, Intel i5), often times it core dumps on our cluster. Furthermore, even the program aborted for one time, when we re-started it, it became normal. The JVM log indicates that it is caused by org.apache.http.client.protocol.RequestAddCookies.process. We found a relevant issue at http://mail-archives.apache.org/mod_mbox/hc-httpclient-users/201208.mbox/%3CCAOGo0VaG+W8Rv54d2dYBk1kv3Rb0tO0eVKvS-REq_yFTk31a4Q@mail.gmail.com%3E, but there was no solution. Could you help to look into this issue? Also, We have the core dump file. Do you have a FTP server so we can drop the file?
Thanks a lot !
---------------------------------------------------------
JVM Log File (only the very beginning):
---------------------------------------------------------
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f284d1d9967, pid=1985, tid=139810931042048
#
# JRE version: 7.0_17-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (23.7-b01 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# J org.apache.http.client.protocol.RequestAddCookies.process(Lorg/apache/http/HttpRequest;Lorg/apache/http/protocol/HttpContext;)V
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
# http://bugreport.sun.com/bugreport/crash.jsp
#
--------------------------------------------------
Java Runtime arguments
-------------------------------------------------
-Xmx4g
----------------------------------------------------
System Information:
----------------------------------------------------
$ uname -a
Linux ip-10-212-71-144 3.2.0-36-virtual #57-Ubuntu SMP Tue Jan 8 22:04:49 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
$ java -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
$ cat /proc/meminfo | head - n 1
MemTotal: 7629484 kB
$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5430 @ 2.66GHz
stepping : 10
microcode : 0xa07
cpu MHz : 2666.760
cache size : 6144 KB
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu tsc msr pae cx8 cmov pat pse36 clflush dts mmx fxsr sse sse2 ss ht pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm dtherm tpr_shadow vnmi flexpriority
bogomips : 5333.52
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5430 @ 2.66GHz
stepping : 10
microcode : 0xa07
cpu MHz : 2666.760
cache size : 6144 KB
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu tsc msr pae cx8 cmov pat pse36 clflush dts mmx fxsr sse sse2 ss ht pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm dtherm tpr_shadow vnmi flexpriority
bogomips : 5333.52
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
Re: [twitter-semantic-search] Core dump caused by org.apache.http.client.protocol.RequestAddCookies.process
Posted by Yu Qiao <yq...@columbia.edu>.
Hi Oleg,
Thanks for your advice. After degrading from JVM 1.7 to JVM 1.6.0_43, the issue seems to be resolved, at least for now.
Thanks,
Yu
On Apr 10, 2013, at 3:45 AM, Oleg Kalnichevski <ol...@apache.org> wrote:
> On Wed, 2013-04-10 at 02:35 -0400, Yu Qiao wrote:
>> Sorry I forgot to mention that we are using HttpAsyncClient 4.0-beta3.
>>
>> Thanks,
>> Yu
>>
>>
>> On Apr 10, 2013, at 2:28 AM, Yu Qiao <yq...@columbia.edu> wrote:
>>
>>> Hi Httpclient-users,
>>>
>>> We are using HttpAsyncClient library to do web crawling. Basically we are just using the method provided by the sample code (http://hc.apache.org/httpcomponents-asyncclient-dev/httpasyncclient/examples/org/apache/http/examples/nio/client/AsyncClientHttpExchangeFutureCallback.java), but has a much larger request size (~9000).
>>>
>>> While our program runs perfectly fine on my laptop (MacOSX 10.8.2, Intel i5), often times it core dumps on our cluster. Furthermore, even the program aborted for one time, when we re-started it, it became normal. The JVM log indicates that it is caused by org.apache.http.client.protocol.RequestAddCookies.process. We found a relevant issue at http://mail-archives.apache.org/mod_mbox/hc-httpclient-users/201208.mbox/%3CCAOGo0VaG+W8Rv54d2dYBk1kv3Rb0tO0eVKvS-REq_yFTk31a4Q@mail.gmail.com%3E, but there was no solution. Could you help to look into this issue? Also, We have the core dump file. Do you have a FTP server so we can drop the file?
>>>
>>> Thanks a lot !
>>>
>>> ---------------------------------------------------------
>>> JVM Log File (only the very beginning):
>>> ---------------------------------------------------------
>>> #
>>> # A fatal error has been detected by the Java Runtime Environment:
>>> #
>>> # SIGSEGV (0xb) at pc=0x00007f284d1d9967, pid=1985, tid=139810931042048
>>> #
>>> # JRE version: 7.0_17-b02
>>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (23.7-b01 mixed mode linux-amd64 compressed oops)
>>> # Problematic frame:
>>> # J org.apache.http.client.protocol.RequestAddCookies.process(Lorg/apache/http/HttpRequest;Lorg/apache/http/protocol/HttpContext;)V
>>> #
>>> # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
>>> #
>>> # If you would like to submit a bug report, please visit:
>>> # http://bugreport.sun.com/bugreport/crash.jsp
>>> #
>>>
>
> Anything that results in segmentation fault inside JRE and a core dump
> is most likely to be a JVM or JIT compile bug. I do not think there is
> anything I could to help you other than suggesting to upgrade to the
> latest JRE 1.7 or downgrade to the latest JRE 1.6 and see if that makes
> any difference or reporting the problem to Oracle.
>
> Oleg
>
>
> _______________________________________________
> twitter-semantic-search mailing list
> twitter-semantic-search@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/cucslists/listinfo/twitter-semantic-search
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Core dump caused by
org.apache.http.client.protocol.RequestAddCookies.process
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2013-04-10 at 02:35 -0400, Yu Qiao wrote:
> Sorry I forgot to mention that we are using HttpAsyncClient 4.0-beta3.
>
> Thanks,
> Yu
>
>
> On Apr 10, 2013, at 2:28 AM, Yu Qiao <yq...@columbia.edu> wrote:
>
> > Hi Httpclient-users,
> >
> > We are using HttpAsyncClient library to do web crawling. Basically we are just using the method provided by the sample code (http://hc.apache.org/httpcomponents-asyncclient-dev/httpasyncclient/examples/org/apache/http/examples/nio/client/AsyncClientHttpExchangeFutureCallback.java), but has a much larger request size (~9000).
> >
> > While our program runs perfectly fine on my laptop (MacOSX 10.8.2, Intel i5), often times it core dumps on our cluster. Furthermore, even the program aborted for one time, when we re-started it, it became normal. The JVM log indicates that it is caused by org.apache.http.client.protocol.RequestAddCookies.process. We found a relevant issue at http://mail-archives.apache.org/mod_mbox/hc-httpclient-users/201208.mbox/%3CCAOGo0VaG+W8Rv54d2dYBk1kv3Rb0tO0eVKvS-REq_yFTk31a4Q@mail.gmail.com%3E, but there was no solution. Could you help to look into this issue? Also, We have the core dump file. Do you have a FTP server so we can drop the file?
> >
> > Thanks a lot !
> >
> > ---------------------------------------------------------
> > JVM Log File (only the very beginning):
> > ---------------------------------------------------------
> > #
> > # A fatal error has been detected by the Java Runtime Environment:
> > #
> > # SIGSEGV (0xb) at pc=0x00007f284d1d9967, pid=1985, tid=139810931042048
> > #
> > # JRE version: 7.0_17-b02
> > # Java VM: Java HotSpot(TM) 64-Bit Server VM (23.7-b01 mixed mode linux-amd64 compressed oops)
> > # Problematic frame:
> > # J org.apache.http.client.protocol.RequestAddCookies.process(Lorg/apache/http/HttpRequest;Lorg/apache/http/protocol/HttpContext;)V
> > #
> > # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> > #
> > # If you would like to submit a bug report, please visit:
> > # http://bugreport.sun.com/bugreport/crash.jsp
> > #
> >
Anything that results in segmentation fault inside JRE and a core dump
is most likely to be a JVM or JIT compile bug. I do not think there is
anything I could to help you other than suggesting to upgrade to the
latest JRE 1.7 or downgrade to the latest JRE 1.6 and see if that makes
any difference or reporting the problem to Oracle.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Core dump caused by org.apache.http.client.protocol.RequestAddCookies.process
Posted by Yu Qiao <yq...@columbia.edu>.
Sorry I forgot to mention that we are using HttpAsyncClient 4.0-beta3.
Thanks,
Yu
On Apr 10, 2013, at 2:28 AM, Yu Qiao <yq...@columbia.edu> wrote:
> Hi Httpclient-users,
>
> We are using HttpAsyncClient library to do web crawling. Basically we are just using the method provided by the sample code (http://hc.apache.org/httpcomponents-asyncclient-dev/httpasyncclient/examples/org/apache/http/examples/nio/client/AsyncClientHttpExchangeFutureCallback.java), but has a much larger request size (~9000).
>
> While our program runs perfectly fine on my laptop (MacOSX 10.8.2, Intel i5), often times it core dumps on our cluster. Furthermore, even the program aborted for one time, when we re-started it, it became normal. The JVM log indicates that it is caused by org.apache.http.client.protocol.RequestAddCookies.process. We found a relevant issue at http://mail-archives.apache.org/mod_mbox/hc-httpclient-users/201208.mbox/%3CCAOGo0VaG+W8Rv54d2dYBk1kv3Rb0tO0eVKvS-REq_yFTk31a4Q@mail.gmail.com%3E, but there was no solution. Could you help to look into this issue? Also, We have the core dump file. Do you have a FTP server so we can drop the file?
>
> Thanks a lot !
>
> ---------------------------------------------------------
> JVM Log File (only the very beginning):
> ---------------------------------------------------------
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x00007f284d1d9967, pid=1985, tid=139810931042048
> #
> # JRE version: 7.0_17-b02
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (23.7-b01 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # J org.apache.http.client.protocol.RequestAddCookies.process(Lorg/apache/http/HttpRequest;Lorg/apache/http/protocol/HttpContext;)V
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.sun.com/bugreport/crash.jsp
> #
>
> --------------------------------------------------
> Java Runtime arguments
> -------------------------------------------------
> -Xmx4g
>
> ----------------------------------------------------
> System Information:
> ----------------------------------------------------
> $ uname -a
> Linux ip-10-212-71-144 3.2.0-36-virtual #57-Ubuntu SMP Tue Jan 8 22:04:49 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>
> $ java -version
> java version "1.7.0_17"
> Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
> Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
>
> $ cat /proc/meminfo | head - n 1
> MemTotal: 7629484 kB
>
> $ cat /proc/cpuinfo
> processor : 0
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU E5430 @ 2.66GHz
> stepping : 10
> microcode : 0xa07
> cpu MHz : 2666.760
> cache size : 6144 KB
> fpu : yes
> fpu_exception : yes
> cpuid level : 13
> wp : yes
> flags : fpu tsc msr pae cx8 cmov pat pse36 clflush dts mmx fxsr sse sse2 ss ht pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm dtherm tpr_shadow vnmi flexpriority
> bogomips : 5333.52
> clflush size : 64
> cache_alignment : 64
> address sizes : 38 bits physical, 48 bits virtual
> power management:
>
> processor : 1
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU E5430 @ 2.66GHz
> stepping : 10
> microcode : 0xa07
> cpu MHz : 2666.760
> cache size : 6144 KB
> fpu : yes
> fpu_exception : yes
> cpuid level : 13
> wp : yes
> flags : fpu tsc msr pae cx8 cmov pat pse36 clflush dts mmx fxsr sse sse2 ss ht pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm dtherm tpr_shadow vnmi flexpriority
> bogomips : 5333.52
> clflush size : 64
> cache_alignment : 64
> address sizes : 38 bits physical, 48 bits virtual
> power management:
>
>
>
>
>