You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Tony Wei <to...@gmail.com> on 2018/11/05 05:38:22 UTC
sys.exist(1) led to standalonesession daemon closed
Hi,
I used a scala library called scallop[1] to parse my job’s arguments. When
the argument didn’t
exist in the config setting, the default behavior of scallop would call
sys.exit(1).
It is not a problem when I’m using flink cli to submit job. However, when I
used rest api to submit
job, it seems that sys.exit(1) will leads to standalonesession daemon
closed. Maybe the reason is
that rest server is also in the same process as standalonesession daemon.
Am I correct?
If this is the root cause, is this an expected behavior and users should be
aware of not using
sys.exit(1) in their jobs?
I tested this on 1.6.0 standalone session cluster with flip-6 mode. And
here are my testing job
and logs before and after the submission.
package com.appier.rt.rt_match
> import org.apache.flink.api.scala.createTypeInformation
> import org.apache.flink.streaming.api.scala.StreamExecutionEnvironment
> import org.rogach.scallop.{ScallopConf, ScallopOption}
> object TestMain {
> def main(args: Array[String]): Unit = {
> object Args extends ScallopConf(args) {
> val mode: ScallopOption[String] = opt[String](default =
> Some("development"))
> verify
> }
> val env = StreamExecutionEnvironment.getExecutionEnvironment
> env.fromElements(Args.mode()).map(a => a)
> env.execute()
> }
> }
Submit by flink-cli
> $ ./bin/flink run -c com.appier.rt.rt_match.TestMain -p 2 -d
> rt-match-assembly-4.5.1-SNAPSHOT.jar --mo xyz
> Starting execution of program
> [scallop] Error: Unknown option 'mo'
Submit by rest-api
> 2018-11-05 13:27:58,800 TRACE
> org.apache.flink.runtime.webmonitor.handlers.JarListHandler - Received
> request /jars/.
> 2018-11-05 13:27:59,679 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler - Received
> request. URL:/jobs/overview Method:GET
> 2018-11-05 13:27:59,680 TRACE
> org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler - Received
> request /jobs/overview.
> 2018-11-05 13:28:01,752 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler - Received
> request. URL:/jars/ Method:GET
> 2018-11-05 13:28:01,753 TRACE
> org.apache.flink.runtime.webmonitor.handlers.JarListHandler - Received
> request /jars/.
> 2018-11-05 13:28:02,682 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler - Received
> request. URL:/jobs/overview Method:GET
> 2018-11-05 13:28:02,683 TRACE
> org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler - Received
> request /jobs/overview.
> 2018-11-05 13:28:03,899 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler - Received
> request.
> URL:/jars/7413f82a-d650-4729-873e-a94150ffe9d0_rt-match-assembly-4.5.1-SNAPSHOT.jar/run?entry
> class=com.appier.rt.rt_match.TestMain¶llelism=2&program-args=--mo+xyz
> Method:POST
> 2018-11-05 13:28:03,902 TRACE
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler - Received
> request
> /jars/7413f82a-d650-4729-873e-a94150ffe9d0_rt-match-assembly-4.5.1-SNAPSHOT.jar/run?entry
> class=com.appier.rt.rt_match.TestMain¶llelism=2&program-args=--mo+xyz.
> 2018-11-05 13:28:04,751 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler - Received
> request. URL:/jars/ Method:GET
> 2018-11-05 13:28:04,752 TRACE
> org.apache.flink.runtime.webmonitor.handlers.JarListHandler - Received
> request /jars/.
> 2018-11-05 13:28:04,760 INFO
> org.apache.flink.runtime.blob.TransientBlobCache - Shutting
> down BLOB cache
> 2018-11-05 13:28:04,761 INFO org.apache.flink.runtime.blob.BlobServer
> - Stopped BLOB server at 0.0.0.0:42075
Best,
Tony Wei.
[1] https://github.com/scallop/scallop
Re: sys.exist(1) led to standalonesession daemon closed
Posted by Dawid Wysakowicz <dw...@apache.org>.
Hi Tony,
I think your reasoning is correct that this is because of the fact that
rest server runs in the same process as standalonesession.
It's hard to say if it is an expected behavior or not, but there is
really not much we can do about it. User code can actually call
System.exit from any
place in the code, which will result in the process executing this code
to terminate. In general I would say calling System.exit is rather
discouraged.
Best,
Dawid
On 05/11/2018 06:38, Tony Wei wrote:
> Hi,
>
> I used a scala library called scallop[1] to parse my job’s arguments.
> When the argument didn’t
> exist in the config setting, the default behavior of scallop would
> call sys.exit(1).
>
> It is not a problem when I’m using flink cli to submit job. However,
> when I used rest api to submit
> job, it seems that sys.exit(1) will leads to standalonesession daemon
> closed. Maybe the reason is
> that rest server is also in the same process as standalonesession
> daemon. Am I correct?
>
> If this is the root cause, is this an expected behavior and users
> should be aware of not using
> sys.exit(1) in their jobs?
>
> I tested this on 1.6.0 standalone session cluster with flip-6 mode.
> And here are my testing job
> and logs before and after the submission.
>
> package com.appier.rt.rt_match
> import org.apache.flink.api.scala.createTypeInformation
> import org.apache.flink.streaming.api.scala.StreamExecutionEnvironment
> import org.rogach.scallop.{ScallopConf, ScallopOption}
> object TestMain {
> def main(args: Array[String]): Unit = {
> object Args extends ScallopConf(args) {
> val mode: ScallopOption[String] = opt[String](default =
> Some("development"))
> verify
> }
> val env = StreamExecutionEnvironment.getExecutionEnvironment
> env.fromElements(Args.mode()).map(a => a)
> env.execute()
> }
> }
>
>
> Submit by flink-cli
>
> $ ./bin/flink run -c com.appier.rt.rt_match.TestMain -p 2 -d
> rt-match-assembly-4.5.1-SNAPSHOT.jar --mo xyz
> Starting execution of program
> [scallop] Error: Unknown option 'mo'
>
>
> Submit by rest-api
>
> 2018-11-05 13:27:58,800 TRACE
> org.apache.flink.runtime.webmonitor.handlers.JarListHandler -
> Received request /jars/.
> 2018-11-05 13:27:59,679 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler -
> Received request. URL:/jobs/overview Method:GET
> 2018-11-05 13:27:59,680 TRACE
> org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler -
> Received request /jobs/overview.
> 2018-11-05 13:28:01,752 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler -
> Received request. URL:/jars/ Method:GET
> 2018-11-05 13:28:01,753 TRACE
> org.apache.flink.runtime.webmonitor.handlers.JarListHandler -
> Received request /jars/.
> 2018-11-05 13:28:02,682 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler -
> Received request. URL:/jobs/overview Method:GET
> 2018-11-05 13:28:02,683 TRACE
> org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler -
> Received request /jobs/overview.
> 2018-11-05 13:28:03,899 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler -
> Received request.
> URL:/jars/7413f82a-d650-4729-873e-a94150ffe9d0_rt-match-assembly-4.5.1-SNAPSHOT.jar/run?entry
> class=com.appier.rt.rt_match.TestMain¶llelism=2&program-args=--mo+xyz
> Method:POST
> 2018-11-05 13:28:03,902 TRACE
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler -
> Received request
> /jars/7413f82a-d650-4729-873e-a94150ffe9d0_rt-match-assembly-4.5.1-SNAPSHOT.jar/run?entry
> class=com.appier.rt.rt_match.TestMain¶llelism=2&program-args=--mo+xyz.
> 2018-11-05 13:28:04,751 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler -
> Received request. URL:/jars/ Method:GET
> 2018-11-05 13:28:04,752 TRACE
> org.apache.flink.runtime.webmonitor.handlers.JarListHandler -
> Received request /jars/.
> 2018-11-05 13:28:04,760 INFO
> org.apache.flink.runtime.blob.TransientBlobCache -
> Shutting down BLOB cache
> 2018-11-05 13:28:04,761 INFO
> org.apache.flink.runtime.blob.BlobServer -
> Stopped BLOB server at 0.0.0.0:42075 <http://0.0.0.0:42075>
>
>
> Best,
> Tony Wei.
>
> [1] https://github.com/scallop/scallop