You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "John Lin (Jira)" <ji...@apache.org> on 2020/06/01 01:08:00 UTC
[jira] [Commented] (SPARK-28861) Jetty property handling:
java.lang.NumberFormatException: For input string: "unknown".
[ https://issues.apache.org/jira/browse/SPARK-28861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120669#comment-17120669 ]
John Lin commented on SPARK-28861:
----------------------------------
Hi,
I can reproduce this problem. The full stack trace is:
{code:java}
java.lang.NumberFormatException: For input string: "unknown"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:1.8.0_181]
at java.lang.Long.parseLong(Long.java:589) ~[?:1.8.0_181]
at java.lang.Long.valueOf(Long.java:803) ~[?:1.8.0_181]
at org.spark_project.jetty.util.Jetty.formatTimestamp(Jetty.java:89) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.spark_project.jetty.util.Jetty.<clinit>(Jetty.java:61) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.spark_project.jetty.server.Server.getVersion(Server.java:159) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.spark_project.jetty.server.handler.ContextHandler.<clinit>(ContextHandler.java:128) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:143) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:130) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.apache.spark.ui.WebUI.attachPage(WebUI.scala:89) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.apache.spark.ui.WebUI$$anonfun$attachTab$1.apply(WebUI.scala:71) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.apache.spark.ui.WebUI$$anonfun$attachTab$1.apply(WebUI.scala:71) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) ~[scala-library-2.11.12.jar:?]
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) ~[scala-library-2.11.12.jar:?]
at org.apache.spark.ui.WebUI.attachTab(WebUI.scala:71) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.apache.spark.ui.SparkUI.initialize(SparkUI.scala:62) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:80) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:178) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.apache.spark.SparkContext.<init>(SparkContext.scala:444) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520) ~[spark-core_2.11-2.4.5.jar:2.4.5]
at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:935) ~[spark-sql_2.11-2.4.5.jar:2.4.5]
at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:926) ~[spark-sql_2.11-2.4.5.jar:2.4.5]
at scala.Option.getOrElse(Option.scala:121) ~[scala-library-2.11.12.jar:?]
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:926) ~[spark-sql_2.11-2.4.5.jar:2.4.5]
at io.github.johnlinp.spark.Config.spark(Config.java:19) ~[classes/:?]
at io.github.johnlinp.spark.Config$$EnhancerBySpringCGLIB$$79e68d23.CGLIB$spark$0(<generated>) ~[classes/:?]
at io.github.johnlinp.spark.Config$$EnhancerBySpringCGLIB$$79e68d23$$FastClassBySpringCGLIB$$f482d305.invoke(<generated>) ~[classes/:?]
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363) ~[spring-context-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at io.github.johnlinp.spark.Config$$EnhancerBySpringCGLIB$$79e68d23.spark(<generated>) ~[classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_181]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_181]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:640) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:475) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:557) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1287) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1207) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:874) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:778) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:226) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1358) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1204) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:557) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) [spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) [spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) [spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879) [spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878) [spring-context-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550) [spring-context-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) [spring-boot-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) [spring-boot-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:140) [spring-boot-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at io.github.johnlinp.spark.Main.main(Main.java:25) [classes/:?]
{code}
A full demo is at [https://github.com/johnlinp/misc-demo/tree/master/spark-startup-number-format-exception]. Thanks.
> Jetty property handling: java.lang.NumberFormatException: For input string: "unknown".
> --------------------------------------------------------------------------------------
>
> Key: SPARK-28861
> URL: https://issues.apache.org/jira/browse/SPARK-28861
> Project: Spark
> Issue Type: Wish
> Components: Spark Submit
> Affects Versions: 2.4.3
> Reporter: Ketan
> Priority: Minor
>
> While processing data from certain files a {{NumberFormatExceltion}} was seen in the logs. The processing was fine but the following stacktrace was observed:
> {code}
> {"time":"2019-08-16 08:21:36,733","level":"DEBUG","class":"o.s.j.u.Jetty","message":"","thread":"Driver","appName":"app-name","appVersion":"APPLICATION_VERSION","type":"APPLICATION","errorCode":"ERROR_CODE","errorId":""}
> java.lang.NumberFormatException: For input string: "unknown".
> {code}
> On investigation it is found that in the class Jetty there is the following:
> {code}
> BUILD_TIMESTAMP = formatTimestamp(__buildProperties.getProperty("timestamp", "unknown"));
> {code}
> which indicates that the config should have the 'timestamp' property. If the property is not there then the default value is set as 'unknown' and this value causes the stacktrace to show up in the logs in our application. It has no detrimental effect on the application as such but could be addressed.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org