You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Niklas Nielsen <ni...@qni.dk> on 2015/03/07 01:19:48 UTC
Review Request 31818: Fixed race in getFieldID helper
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31818/
-----------------------------------------------------------
Review request for mesos, Ben Mahler, Dario Rexin, and switched to 'mcypark'.
Bugs: MESOS-2414
https://issues.apache.org/jira/browse/MESOS-2414
Repository: mesos
Description
-------
The getFieldID helper looks up the java/lang/NoSuchFieldError class and stores it in a static. It has turned out to provoke a racy behavior with Java 8 when multiple drivers are created (and the class object may have been created by another thread).
This patch reverts the 'static' optimization and looks up the class object when exceptions are thrown.
Diffs
-----
src/java/jni/convert.cpp 9f99031bb54b0720d5454ef8795145b4b92da8d4
Diff: https://reviews.apache.org/r/31818/diff/
Testing
-------
make check and tested marathon's stopDriver() with 0.21.0 mesos jar.
Before:
[2015-03-06 22:03:02,508] INFO Started SelectChannelConnector@0.0.0.0:8080 (org.eclipse.jetty.server.AbstractConnector:338)
[2015-03-06 22:03:11,687] INFO Abdicating (mesosphere.marathon.MarathonSchedulerService:297)
[2015-03-06 22:03:11,687] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
[2015-03-06 22:03:11,689] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
[INFO] [03/06/2015 22:03:11.688] [marathon-akka.actor.default-dispatcher-2] [akka://marathon/user/MarathonScheduler/$a] Suspending scheduler actor
I0306 22:03:11.690840 32401 sched.cpp:1589] Asked to stop the driver
I0306 22:03:11.692275 32272 sched.cpp:831] Stopping framework '20150304-234928-16777343-5050-8149-0000'
[2015-03-06 22:03:11,693] INFO Driver future completed. Executing optional abdication command. (mesosphere.marathon.MarathonSchedulerService:191)
[2015-03-06 22:03:11,697] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
# A fatal error has been detected by the Java Runtime Environment: [1890/1934]
#
# SIGSEGV (0xb) at pc=0x00007f65f0eb0b07, pid=32230, tid=140074863777536
#
# JRE version: Java(TM) SE Runtime Environment (8.0_31-b13) (build 1.8.0_31-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V [libjvm.so+0x6a1b07][2015-03-06 22:03:11,698] INFO Offering leadership (mesosphere.marathon.MarathonSchedulerService:304)
[2015-03-06 22:03:11,699] INFO Defeated (Leader Interface) (mesosphere.marathon.MarathonSchedulerService:245)
[2015-03-06 22:03:11,699] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
[2015-03-06 22:03:11,699] INFO Using HA and therefore offering leadership (mesosphere.marathon.MarathonSchedulerService:311)
jni_IsInstanceOf+0xd7
#
# Core dump written. Default location: /home/vagrant/marathon/core or core.32230
#
# An error report file with more information is saved as:
# /home/vagrant/marathon/hs_err_pid32230.log
[2015-03-06 22:03:11,701] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
I0306 22:03:11.701335 32260 sched.cpp:1589] Asked to stop the driver
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
Aborted (core dumped)
After:
[2015-03-07 00:05:37,553] INFO Started SelectChannelConnector@0.0.0.0:8080 (org.eclipse.jetty.server.AbstractConnector:338)
^C[2015-03-07 00:05:40,850] INFO Shutting down services (mesosphere.marathon.Main$:42)
[2015-03-07 00:05:40,852] INFO Shutdown triggered (mesosphere.marathon.MarathonSchedulerService:165)
[2015-03-07 00:05:40,854] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
I0307 00:05:40.854619 14555 sched.cpp:1589] Asked to stop the driver
I0307 00:05:40.859328 14525 sched.cpp:831] Stopping framework '20150304-234928-16777343-5050-8149-0000'
[2015-03-07 00:05:40,864] INFO Driver future completed. Executing optional abdication command. (mesosphere.marathon.MarathonSchedulerService:191)
[2015-03-07 00:05:40,865] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
[2015-03-07 00:05:40,878] INFO Defeated (Leader Interface) (mesosphere.marathon.MarathonSchedulerService:245)
[2015-03-07 00:05:40,879] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
[2015-03-07 00:05:40,879] INFO Cancelling reconciliation timer (mesosphere.marathon.MarathonSchedulerService:171)
[2015-03-07 00:05:40,879] INFO Removing the blocking of run() (mesosphere.marathon.MarathonSchedulerService:174)
[2015-03-07 00:05:40,879] INFO Waiting for services to shut down (mesosphere.marathon.Main$:48)
[2015-03-07 00:05:40,880] INFO Completed run (mesosphere.marathon.MarathonSchedulerService:161)
[2015-03-07 00:05:40,882] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
[INFO] [03/07/2015 00:05:40.881] [marathon-akka.actor.default-dispatcher-2] [akka://marathon/user/MarathonScheduler/$a] Suspending scheduler actorI0307 00:05:40.884512 14516 sched.cpp:1589] Asked to stop
the driver
[2015-03-07 00:05:40,891] INFO stopped o.e.j.s.ServletContextHandler{/,null} (org.eclipse.jetty.server.handler.ContextHandler:843)
[2015-03-07 00:05:40,894] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
Thanks,
Niklas Nielsen
Re: Review Request 31818: Fixed race in getFieldID helper
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31818/#review75587
-----------------------------------------------------------
Ship it!
Good job getting to the bottom of this!
src/java/jni/convert.cpp
<https://reviews.apache.org/r/31818/#comment122782>
I don't think you need this NOTE, since we should already be avoiding non-POD statics.
If you want to leave a trace:
```
// NOTE: This was previously a static variable in order to only
// do the lookup once. But this is not thread-safe and led to
// crashes on Java 8!
```
src/java/jni/convert.cpp
<https://reviews.apache.org/r/31818/#comment122783>
s/NO_SUCH_FIELD_ERROR/noSuchFieldError/
- Ben Mahler
On March 7, 2015, 12:19 a.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31818/
> -----------------------------------------------------------
>
> (Updated March 7, 2015, 12:19 a.m.)
>
>
> Review request for mesos, Ben Mahler, Dario Rexin, and switched to 'mcypark'.
>
>
> Bugs: MESOS-2414
> https://issues.apache.org/jira/browse/MESOS-2414
>
>
> Repository: mesos
>
>
> Description
> -------
>
> The getFieldID helper looks up the java/lang/NoSuchFieldError class and stores it in a static. It has turned out to provoke a racy behavior with Java 8 when multiple drivers are created (and the class object may have been created by another thread).
> This patch reverts the 'static' optimization and looks up the class object when exceptions are thrown.
>
>
> Diffs
> -----
>
> src/java/jni/convert.cpp 9f99031bb54b0720d5454ef8795145b4b92da8d4
>
> Diff: https://reviews.apache.org/r/31818/diff/
>
>
> Testing
> -------
>
> make check and tested marathon's stopDriver() with 0.21.0 mesos jar.
>
> Before:
>
> [2015-03-06 22:03:02,508] INFO Started SelectChannelConnector@0.0.0.0:8080 (org.eclipse.jetty.server.AbstractConnector:338)
> [2015-03-06 22:03:11,687] INFO Abdicating (mesosphere.marathon.MarathonSchedulerService:297)
> [2015-03-06 22:03:11,687] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
> [2015-03-06 22:03:11,689] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> [INFO] [03/06/2015 22:03:11.688] [marathon-akka.actor.default-dispatcher-2] [akka://marathon/user/MarathonScheduler/$a] Suspending scheduler actor
> I0306 22:03:11.690840 32401 sched.cpp:1589] Asked to stop the driver
> I0306 22:03:11.692275 32272 sched.cpp:831] Stopping framework '20150304-234928-16777343-5050-8149-0000'
> [2015-03-06 22:03:11,693] INFO Driver future completed. Executing optional abdication command. (mesosphere.marathon.MarathonSchedulerService:191)
> [2015-03-06 22:03:11,697] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
> # A fatal error has been detected by the Java Runtime Environment: [1890/1934]
> #
> # SIGSEGV (0xb) at pc=0x00007f65f0eb0b07, pid=32230, tid=140074863777536
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_31-b13) (build 1.8.0_31-b13)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # V [libjvm.so+0x6a1b07][2015-03-06 22:03:11,698] INFO Offering leadership (mesosphere.marathon.MarathonSchedulerService:304)
> [2015-03-06 22:03:11,699] INFO Defeated (Leader Interface) (mesosphere.marathon.MarathonSchedulerService:245)
> [2015-03-06 22:03:11,699] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
> [2015-03-06 22:03:11,699] INFO Using HA and therefore offering leadership (mesosphere.marathon.MarathonSchedulerService:311)
> jni_IsInstanceOf+0xd7
> #
> # Core dump written. Default location: /home/vagrant/marathon/core or core.32230
> #
> # An error report file with more information is saved as:
> # /home/vagrant/marathon/hs_err_pid32230.log
> [2015-03-06 22:03:11,701] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> I0306 22:03:11.701335 32260 sched.cpp:1589] Asked to stop the driver
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.java.com/bugreport/crash.jsp
> #
> Aborted (core dumped)
>
> After:
>
> [2015-03-07 00:05:37,553] INFO Started SelectChannelConnector@0.0.0.0:8080 (org.eclipse.jetty.server.AbstractConnector:338)
> ^C[2015-03-07 00:05:40,850] INFO Shutting down services (mesosphere.marathon.Main$:42)
> [2015-03-07 00:05:40,852] INFO Shutdown triggered (mesosphere.marathon.MarathonSchedulerService:165)
> [2015-03-07 00:05:40,854] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> I0307 00:05:40.854619 14555 sched.cpp:1589] Asked to stop the driver
> I0307 00:05:40.859328 14525 sched.cpp:831] Stopping framework '20150304-234928-16777343-5050-8149-0000'
> [2015-03-07 00:05:40,864] INFO Driver future completed. Executing optional abdication command. (mesosphere.marathon.MarathonSchedulerService:191)
> [2015-03-07 00:05:40,865] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
> [2015-03-07 00:05:40,878] INFO Defeated (Leader Interface) (mesosphere.marathon.MarathonSchedulerService:245)
> [2015-03-07 00:05:40,879] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
> [2015-03-07 00:05:40,879] INFO Cancelling reconciliation timer (mesosphere.marathon.MarathonSchedulerService:171)
> [2015-03-07 00:05:40,879] INFO Removing the blocking of run() (mesosphere.marathon.MarathonSchedulerService:174)
> [2015-03-07 00:05:40,879] INFO Waiting for services to shut down (mesosphere.marathon.Main$:48)
> [2015-03-07 00:05:40,880] INFO Completed run (mesosphere.marathon.MarathonSchedulerService:161)
> [2015-03-07 00:05:40,882] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> [INFO] [03/07/2015 00:05:40.881] [marathon-akka.actor.default-dispatcher-2] [akka://marathon/user/MarathonScheduler/$a] Suspending scheduler actorI0307 00:05:40.884512 14516 sched.cpp:1589] Asked to stop
> the driver
>
> [2015-03-07 00:05:40,891] INFO stopped o.e.j.s.ServletContextHandler{/,null} (org.eclipse.jetty.server.handler.ContextHandler:843)
> [2015-03-07 00:05:40,894] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 31818: Fixed race in getFieldID helper
Posted by Niklas Nielsen <ni...@qni.dk>.
> On March 6, 2015, 4:29 p.m., Connor Doyle wrote:
> > src/java/jni/convert.cpp, line 530
> > <https://reviews.apache.org/r/31818/diff/1/?file=888034#file888034line530>
> >
> > s/have/has
Took BenM's comment instead :) So my danlish is not in the landed patch
- Niklas
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31818/#review75581
-----------------------------------------------------------
On March 6, 2015, 5:08 p.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31818/
> -----------------------------------------------------------
>
> (Updated March 6, 2015, 5:08 p.m.)
>
>
> Review request for mesos, Ben Mahler, Dario Rexin, and switched to 'mcypark'.
>
>
> Bugs: MESOS-2414
> https://issues.apache.org/jira/browse/MESOS-2414
>
>
> Repository: mesos
>
>
> Description
> -------
>
> The getFieldID helper looks up the java/lang/NoSuchFieldError class and stores it in a static. It has turned out to provoke a racy behavior with Java 8 when multiple drivers are created (and the class object may have been created by another thread).
> This patch reverts the 'static' optimization and looks up the class object when exceptions are thrown.
>
>
> Diffs
> -----
>
> src/java/jni/convert.cpp 9f99031bb54b0720d5454ef8795145b4b92da8d4
>
> Diff: https://reviews.apache.org/r/31818/diff/
>
>
> Testing
> -------
>
> make check and tested marathon's stopDriver() with 0.21.0 mesos jar.
>
> Before:
>
> [2015-03-06 22:03:02,508] INFO Started SelectChannelConnector@0.0.0.0:8080 (org.eclipse.jetty.server.AbstractConnector:338)
> [2015-03-06 22:03:11,687] INFO Abdicating (mesosphere.marathon.MarathonSchedulerService:297)
> [2015-03-06 22:03:11,687] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
> [2015-03-06 22:03:11,689] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> [INFO] [03/06/2015 22:03:11.688] [marathon-akka.actor.default-dispatcher-2] [akka://marathon/user/MarathonScheduler/$a] Suspending scheduler actor
> I0306 22:03:11.690840 32401 sched.cpp:1589] Asked to stop the driver
> I0306 22:03:11.692275 32272 sched.cpp:831] Stopping framework '20150304-234928-16777343-5050-8149-0000'
> [2015-03-06 22:03:11,693] INFO Driver future completed. Executing optional abdication command. (mesosphere.marathon.MarathonSchedulerService:191)
> [2015-03-06 22:03:11,697] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
> # A fatal error has been detected by the Java Runtime Environment: [1890/1934]
> #
> # SIGSEGV (0xb) at pc=0x00007f65f0eb0b07, pid=32230, tid=140074863777536
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_31-b13) (build 1.8.0_31-b13)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # V [libjvm.so+0x6a1b07][2015-03-06 22:03:11,698] INFO Offering leadership (mesosphere.marathon.MarathonSchedulerService:304)
> [2015-03-06 22:03:11,699] INFO Defeated (Leader Interface) (mesosphere.marathon.MarathonSchedulerService:245)
> [2015-03-06 22:03:11,699] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
> [2015-03-06 22:03:11,699] INFO Using HA and therefore offering leadership (mesosphere.marathon.MarathonSchedulerService:311)
> jni_IsInstanceOf+0xd7
> #
> # Core dump written. Default location: /home/vagrant/marathon/core or core.32230
> #
> # An error report file with more information is saved as:
> # /home/vagrant/marathon/hs_err_pid32230.log
> [2015-03-06 22:03:11,701] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> I0306 22:03:11.701335 32260 sched.cpp:1589] Asked to stop the driver
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.java.com/bugreport/crash.jsp
> #
> Aborted (core dumped)
>
> After:
>
> [2015-03-07 00:05:37,553] INFO Started SelectChannelConnector@0.0.0.0:8080 (org.eclipse.jetty.server.AbstractConnector:338)
> ^C[2015-03-07 00:05:40,850] INFO Shutting down services (mesosphere.marathon.Main$:42)
> [2015-03-07 00:05:40,852] INFO Shutdown triggered (mesosphere.marathon.MarathonSchedulerService:165)
> [2015-03-07 00:05:40,854] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> I0307 00:05:40.854619 14555 sched.cpp:1589] Asked to stop the driver
> I0307 00:05:40.859328 14525 sched.cpp:831] Stopping framework '20150304-234928-16777343-5050-8149-0000'
> [2015-03-07 00:05:40,864] INFO Driver future completed. Executing optional abdication command. (mesosphere.marathon.MarathonSchedulerService:191)
> [2015-03-07 00:05:40,865] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
> [2015-03-07 00:05:40,878] INFO Defeated (Leader Interface) (mesosphere.marathon.MarathonSchedulerService:245)
> [2015-03-07 00:05:40,879] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
> [2015-03-07 00:05:40,879] INFO Cancelling reconciliation timer (mesosphere.marathon.MarathonSchedulerService:171)
> [2015-03-07 00:05:40,879] INFO Removing the blocking of run() (mesosphere.marathon.MarathonSchedulerService:174)
> [2015-03-07 00:05:40,879] INFO Waiting for services to shut down (mesosphere.marathon.Main$:48)
> [2015-03-07 00:05:40,880] INFO Completed run (mesosphere.marathon.MarathonSchedulerService:161)
> [2015-03-07 00:05:40,882] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> [INFO] [03/07/2015 00:05:40.881] [marathon-akka.actor.default-dispatcher-2] [akka://marathon/user/MarathonScheduler/$a] Suspending scheduler actorI0307 00:05:40.884512 14516 sched.cpp:1589] Asked to stop
> the driver
>
> [2015-03-07 00:05:40,891] INFO stopped o.e.j.s.ServletContextHandler{/,null} (org.eclipse.jetty.server.handler.ContextHandler:843)
> [2015-03-07 00:05:40,894] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 31818: Fixed race in getFieldID helper
Posted by Connor Doyle <co...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31818/#review75581
-----------------------------------------------------------
src/java/jni/convert.cpp
<https://reviews.apache.org/r/31818/#comment122769>
s/have/has
- Connor Doyle
On March 7, 2015, 12:19 a.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31818/
> -----------------------------------------------------------
>
> (Updated March 7, 2015, 12:19 a.m.)
>
>
> Review request for mesos, Ben Mahler, Dario Rexin, and switched to 'mcypark'.
>
>
> Bugs: MESOS-2414
> https://issues.apache.org/jira/browse/MESOS-2414
>
>
> Repository: mesos
>
>
> Description
> -------
>
> The getFieldID helper looks up the java/lang/NoSuchFieldError class and stores it in a static. It has turned out to provoke a racy behavior with Java 8 when multiple drivers are created (and the class object may have been created by another thread).
> This patch reverts the 'static' optimization and looks up the class object when exceptions are thrown.
>
>
> Diffs
> -----
>
> src/java/jni/convert.cpp 9f99031bb54b0720d5454ef8795145b4b92da8d4
>
> Diff: https://reviews.apache.org/r/31818/diff/
>
>
> Testing
> -------
>
> make check and tested marathon's stopDriver() with 0.21.0 mesos jar.
>
> Before:
>
> [2015-03-06 22:03:02,508] INFO Started SelectChannelConnector@0.0.0.0:8080 (org.eclipse.jetty.server.AbstractConnector:338)
> [2015-03-06 22:03:11,687] INFO Abdicating (mesosphere.marathon.MarathonSchedulerService:297)
> [2015-03-06 22:03:11,687] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
> [2015-03-06 22:03:11,689] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> [INFO] [03/06/2015 22:03:11.688] [marathon-akka.actor.default-dispatcher-2] [akka://marathon/user/MarathonScheduler/$a] Suspending scheduler actor
> I0306 22:03:11.690840 32401 sched.cpp:1589] Asked to stop the driver
> I0306 22:03:11.692275 32272 sched.cpp:831] Stopping framework '20150304-234928-16777343-5050-8149-0000'
> [2015-03-06 22:03:11,693] INFO Driver future completed. Executing optional abdication command. (mesosphere.marathon.MarathonSchedulerService:191)
> [2015-03-06 22:03:11,697] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
> # A fatal error has been detected by the Java Runtime Environment: [1890/1934]
> #
> # SIGSEGV (0xb) at pc=0x00007f65f0eb0b07, pid=32230, tid=140074863777536
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_31-b13) (build 1.8.0_31-b13)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # V [libjvm.so+0x6a1b07][2015-03-06 22:03:11,698] INFO Offering leadership (mesosphere.marathon.MarathonSchedulerService:304)
> [2015-03-06 22:03:11,699] INFO Defeated (Leader Interface) (mesosphere.marathon.MarathonSchedulerService:245)
> [2015-03-06 22:03:11,699] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
> [2015-03-06 22:03:11,699] INFO Using HA and therefore offering leadership (mesosphere.marathon.MarathonSchedulerService:311)
> jni_IsInstanceOf+0xd7
> #
> # Core dump written. Default location: /home/vagrant/marathon/core or core.32230
> #
> # An error report file with more information is saved as:
> # /home/vagrant/marathon/hs_err_pid32230.log
> [2015-03-06 22:03:11,701] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> I0306 22:03:11.701335 32260 sched.cpp:1589] Asked to stop the driver
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.java.com/bugreport/crash.jsp
> #
> Aborted (core dumped)
>
> After:
>
> [2015-03-07 00:05:37,553] INFO Started SelectChannelConnector@0.0.0.0:8080 (org.eclipse.jetty.server.AbstractConnector:338)
> ^C[2015-03-07 00:05:40,850] INFO Shutting down services (mesosphere.marathon.Main$:42)
> [2015-03-07 00:05:40,852] INFO Shutdown triggered (mesosphere.marathon.MarathonSchedulerService:165)
> [2015-03-07 00:05:40,854] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> I0307 00:05:40.854619 14555 sched.cpp:1589] Asked to stop the driver
> I0307 00:05:40.859328 14525 sched.cpp:831] Stopping framework '20150304-234928-16777343-5050-8149-0000'
> [2015-03-07 00:05:40,864] INFO Driver future completed. Executing optional abdication command. (mesosphere.marathon.MarathonSchedulerService:191)
> [2015-03-07 00:05:40,865] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
> [2015-03-07 00:05:40,878] INFO Defeated (Leader Interface) (mesosphere.marathon.MarathonSchedulerService:245)
> [2015-03-07 00:05:40,879] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
> [2015-03-07 00:05:40,879] INFO Cancelling reconciliation timer (mesosphere.marathon.MarathonSchedulerService:171)
> [2015-03-07 00:05:40,879] INFO Removing the blocking of run() (mesosphere.marathon.MarathonSchedulerService:174)
> [2015-03-07 00:05:40,879] INFO Waiting for services to shut down (mesosphere.marathon.Main$:48)
> [2015-03-07 00:05:40,880] INFO Completed run (mesosphere.marathon.MarathonSchedulerService:161)
> [2015-03-07 00:05:40,882] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> [INFO] [03/07/2015 00:05:40.881] [marathon-akka.actor.default-dispatcher-2] [akka://marathon/user/MarathonScheduler/$a] Suspending scheduler actorI0307 00:05:40.884512 14516 sched.cpp:1589] Asked to stop
> the driver
>
> [2015-03-07 00:05:40,891] INFO stopped o.e.j.s.ServletContextHandler{/,null} (org.eclipse.jetty.server.handler.ContextHandler:843)
> [2015-03-07 00:05:40,894] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 31818: Fixed race in getFieldID helper
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31818/#review75606
-----------------------------------------------------------
Patch looks great!
Reviews applied: [31818]
All tests passed.
- Mesos ReviewBot
On March 7, 2015, 1:08 a.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31818/
> -----------------------------------------------------------
>
> (Updated March 7, 2015, 1:08 a.m.)
>
>
> Review request for mesos, Ben Mahler, Dario Rexin, and switched to 'mcypark'.
>
>
> Bugs: MESOS-2414
> https://issues.apache.org/jira/browse/MESOS-2414
>
>
> Repository: mesos
>
>
> Description
> -------
>
> The getFieldID helper looks up the java/lang/NoSuchFieldError class and stores it in a static. It has turned out to provoke a racy behavior with Java 8 when multiple drivers are created (and the class object may have been created by another thread).
> This patch reverts the 'static' optimization and looks up the class object when exceptions are thrown.
>
>
> Diffs
> -----
>
> src/java/jni/convert.cpp 9f99031bb54b0720d5454ef8795145b4b92da8d4
>
> Diff: https://reviews.apache.org/r/31818/diff/
>
>
> Testing
> -------
>
> make check and tested marathon's stopDriver() with 0.21.0 mesos jar.
>
> Before:
>
> [2015-03-06 22:03:02,508] INFO Started SelectChannelConnector@0.0.0.0:8080 (org.eclipse.jetty.server.AbstractConnector:338)
> [2015-03-06 22:03:11,687] INFO Abdicating (mesosphere.marathon.MarathonSchedulerService:297)
> [2015-03-06 22:03:11,687] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
> [2015-03-06 22:03:11,689] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> [INFO] [03/06/2015 22:03:11.688] [marathon-akka.actor.default-dispatcher-2] [akka://marathon/user/MarathonScheduler/$a] Suspending scheduler actor
> I0306 22:03:11.690840 32401 sched.cpp:1589] Asked to stop the driver
> I0306 22:03:11.692275 32272 sched.cpp:831] Stopping framework '20150304-234928-16777343-5050-8149-0000'
> [2015-03-06 22:03:11,693] INFO Driver future completed. Executing optional abdication command. (mesosphere.marathon.MarathonSchedulerService:191)
> [2015-03-06 22:03:11,697] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
> # A fatal error has been detected by the Java Runtime Environment: [1890/1934]
> #
> # SIGSEGV (0xb) at pc=0x00007f65f0eb0b07, pid=32230, tid=140074863777536
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_31-b13) (build 1.8.0_31-b13)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # V [libjvm.so+0x6a1b07][2015-03-06 22:03:11,698] INFO Offering leadership (mesosphere.marathon.MarathonSchedulerService:304)
> [2015-03-06 22:03:11,699] INFO Defeated (Leader Interface) (mesosphere.marathon.MarathonSchedulerService:245)
> [2015-03-06 22:03:11,699] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
> [2015-03-06 22:03:11,699] INFO Using HA and therefore offering leadership (mesosphere.marathon.MarathonSchedulerService:311)
> jni_IsInstanceOf+0xd7
> #
> # Core dump written. Default location: /home/vagrant/marathon/core or core.32230
> #
> # An error report file with more information is saved as:
> # /home/vagrant/marathon/hs_err_pid32230.log
> [2015-03-06 22:03:11,701] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> I0306 22:03:11.701335 32260 sched.cpp:1589] Asked to stop the driver
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.java.com/bugreport/crash.jsp
> #
> Aborted (core dumped)
>
> After:
>
> [2015-03-07 00:05:37,553] INFO Started SelectChannelConnector@0.0.0.0:8080 (org.eclipse.jetty.server.AbstractConnector:338)
> ^C[2015-03-07 00:05:40,850] INFO Shutting down services (mesosphere.marathon.Main$:42)
> [2015-03-07 00:05:40,852] INFO Shutdown triggered (mesosphere.marathon.MarathonSchedulerService:165)
> [2015-03-07 00:05:40,854] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> I0307 00:05:40.854619 14555 sched.cpp:1589] Asked to stop the driver
> I0307 00:05:40.859328 14525 sched.cpp:831] Stopping framework '20150304-234928-16777343-5050-8149-0000'
> [2015-03-07 00:05:40,864] INFO Driver future completed. Executing optional abdication command. (mesosphere.marathon.MarathonSchedulerService:191)
> [2015-03-07 00:05:40,865] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
> [2015-03-07 00:05:40,878] INFO Defeated (Leader Interface) (mesosphere.marathon.MarathonSchedulerService:245)
> [2015-03-07 00:05:40,879] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
> [2015-03-07 00:05:40,879] INFO Cancelling reconciliation timer (mesosphere.marathon.MarathonSchedulerService:171)
> [2015-03-07 00:05:40,879] INFO Removing the blocking of run() (mesosphere.marathon.MarathonSchedulerService:174)
> [2015-03-07 00:05:40,879] INFO Waiting for services to shut down (mesosphere.marathon.Main$:48)
> [2015-03-07 00:05:40,880] INFO Completed run (mesosphere.marathon.MarathonSchedulerService:161)
> [2015-03-07 00:05:40,882] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
> [INFO] [03/07/2015 00:05:40.881] [marathon-akka.actor.default-dispatcher-2] [akka://marathon/user/MarathonScheduler/$a] Suspending scheduler actorI0307 00:05:40.884512 14516 sched.cpp:1589] Asked to stop
> the driver
>
> [2015-03-07 00:05:40,891] INFO stopped o.e.j.s.ServletContextHandler{/,null} (org.eclipse.jetty.server.handler.ContextHandler:843)
> [2015-03-07 00:05:40,894] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 31818: Fixed race in getFieldID helper
Posted by Niklas Nielsen <ni...@qni.dk>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31818/
-----------------------------------------------------------
(Updated March 6, 2015, 5:08 p.m.)
Review request for mesos, Ben Mahler, Dario Rexin, and switched to 'mcypark'.
Changes
-------
Addressed Ben Mahler's comments
Bugs: MESOS-2414
https://issues.apache.org/jira/browse/MESOS-2414
Repository: mesos
Description
-------
The getFieldID helper looks up the java/lang/NoSuchFieldError class and stores it in a static. It has turned out to provoke a racy behavior with Java 8 when multiple drivers are created (and the class object may have been created by another thread).
This patch reverts the 'static' optimization and looks up the class object when exceptions are thrown.
Diffs (updated)
-----
src/java/jni/convert.cpp 9f99031bb54b0720d5454ef8795145b4b92da8d4
Diff: https://reviews.apache.org/r/31818/diff/
Testing
-------
make check and tested marathon's stopDriver() with 0.21.0 mesos jar.
Before:
[2015-03-06 22:03:02,508] INFO Started SelectChannelConnector@0.0.0.0:8080 (org.eclipse.jetty.server.AbstractConnector:338)
[2015-03-06 22:03:11,687] INFO Abdicating (mesosphere.marathon.MarathonSchedulerService:297)
[2015-03-06 22:03:11,687] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
[2015-03-06 22:03:11,689] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
[INFO] [03/06/2015 22:03:11.688] [marathon-akka.actor.default-dispatcher-2] [akka://marathon/user/MarathonScheduler/$a] Suspending scheduler actor
I0306 22:03:11.690840 32401 sched.cpp:1589] Asked to stop the driver
I0306 22:03:11.692275 32272 sched.cpp:831] Stopping framework '20150304-234928-16777343-5050-8149-0000'
[2015-03-06 22:03:11,693] INFO Driver future completed. Executing optional abdication command. (mesosphere.marathon.MarathonSchedulerService:191)
[2015-03-06 22:03:11,697] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
# A fatal error has been detected by the Java Runtime Environment: [1890/1934]
#
# SIGSEGV (0xb) at pc=0x00007f65f0eb0b07, pid=32230, tid=140074863777536
#
# JRE version: Java(TM) SE Runtime Environment (8.0_31-b13) (build 1.8.0_31-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V [libjvm.so+0x6a1b07][2015-03-06 22:03:11,698] INFO Offering leadership (mesosphere.marathon.MarathonSchedulerService:304)
[2015-03-06 22:03:11,699] INFO Defeated (Leader Interface) (mesosphere.marathon.MarathonSchedulerService:245)
[2015-03-06 22:03:11,699] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
[2015-03-06 22:03:11,699] INFO Using HA and therefore offering leadership (mesosphere.marathon.MarathonSchedulerService:311)
jni_IsInstanceOf+0xd7
#
# Core dump written. Default location: /home/vagrant/marathon/core or core.32230
#
# An error report file with more information is saved as:
# /home/vagrant/marathon/hs_err_pid32230.log
[2015-03-06 22:03:11,701] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
I0306 22:03:11.701335 32260 sched.cpp:1589] Asked to stop the driver
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
Aborted (core dumped)
After:
[2015-03-07 00:05:37,553] INFO Started SelectChannelConnector@0.0.0.0:8080 (org.eclipse.jetty.server.AbstractConnector:338)
^C[2015-03-07 00:05:40,850] INFO Shutting down services (mesosphere.marathon.Main$:42)
[2015-03-07 00:05:40,852] INFO Shutdown triggered (mesosphere.marathon.MarathonSchedulerService:165)
[2015-03-07 00:05:40,854] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
I0307 00:05:40.854619 14555 sched.cpp:1589] Asked to stop the driver
I0307 00:05:40.859328 14525 sched.cpp:831] Stopping framework '20150304-234928-16777343-5050-8149-0000'
[2015-03-07 00:05:40,864] INFO Driver future completed. Executing optional abdication command. (mesosphere.marathon.MarathonSchedulerService:191)
[2015-03-07 00:05:40,865] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
[2015-03-07 00:05:40,878] INFO Defeated (Leader Interface) (mesosphere.marathon.MarathonSchedulerService:245)
[2015-03-07 00:05:40,879] INFO Defeat leadership (mesosphere.marathon.MarathonSchedulerService:270)
[2015-03-07 00:05:40,879] INFO Cancelling reconciliation timer (mesosphere.marathon.MarathonSchedulerService:171)
[2015-03-07 00:05:40,879] INFO Removing the blocking of run() (mesosphere.marathon.MarathonSchedulerService:174)
[2015-03-07 00:05:40,879] INFO Waiting for services to shut down (mesosphere.marathon.Main$:48)
[2015-03-07 00:05:40,880] INFO Completed run (mesosphere.marathon.MarathonSchedulerService:161)
[2015-03-07 00:05:40,882] INFO Stopping driver (mesosphere.marathon.MarathonSchedulerService:220)
[INFO] [03/07/2015 00:05:40.881] [marathon-akka.actor.default-dispatcher-2] [akka://marathon/user/MarathonScheduler/$a] Suspending scheduler actorI0307 00:05:40.884512 14516 sched.cpp:1589] Asked to stop
the driver
[2015-03-07 00:05:40,891] INFO stopped o.e.j.s.ServletContextHandler{/,null} (org.eclipse.jetty.server.handler.ContextHandler:843)
[2015-03-07 00:05:40,894] INFO Setting framework ID to 20150304-234928-16777343-5050-8149-0000 (mesosphere.marathon.MarathonSchedulerService:72)
Thanks,
Niklas Nielsen