You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by Rainie Li <ra...@pinterest.com.INVALID> on 2020/07/16 07:46:46 UTC
Flink yarn session exception
大佬们好，我是flink新手，正在用flink 1.9.1
Flink APP cannot run, APP log error, 想求教一下会是什么原因造成的，多谢🙏
2020-06-16 17:06:21,921 WARN org.apache.flink.client.cli.CliFrontend
- Could not load CLI class
org.apache.flink.yarn.cli.FlinkYarnSessionCli.
java.lang.NoClassDefFoundError:
org/apache/hadoop/yarn/exceptions/YarnException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.flink.client.cli.CliFrontend.loadCustomCommandLine(CliFrontend.java:1185)
at
org.apache.flink.client.cli.CliFrontend.loadCustomCommandLines(CliFrontend.java:1145)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1070)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.exceptions.YarnException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 5 more
2020-06-16 17:06:21,980 INFO org.apache.flink.core.fs.FileSystem
- Hadoop is not in the classpath/dependencies. The extended
set of supported File Systems via Hadoop is not available.
Re: Flink yarn session exception
Posted by Rainie Li <ra...@pinterest.com.INVALID>.
多谢，我set了这些envs:
export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export PATH=$JAVA_HOME/bin:$PATH
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_CLASSPATH=`hadoop classpath`
export FLINK_CONF_DIR=/etc/flink-1.9.1/conf
export FLINK_LOG_DIR=/home/rainieli/
有什么问题吗？🙏
On Thu, Jul 16, 2020 at 1:12 AM Paul Lam <pa...@gmail.com> wrote:
> 日志里说得比较清楚了，classpath 里没有 Hadoop 的 lib。可以参考这个文档 [1] 来配置你的环境。
>
> 1.
> https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html
> <
> https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html
> >
> Best,
> Paul Lam
>
> > 2020年7月16日 15:46，Rainie Li <ra...@pinterest.com.INVALID> 写道：
> >
> > 大佬们好，我是flink新手，正在用flink 1.9.1
> > Flink APP cannot run, APP log error, 想求教一下会是什么原因造成的，多谢🙏
> >
> > 2020-06-16 17:06:21,921 WARN org.apache.flink.client.cli.CliFrontend
> > - Could not load CLI class
> > org.apache.flink.yarn.cli.FlinkYarnSessionCli.
> > java.lang.NoClassDefFoundError:
> > org/apache/hadoop/yarn/exceptions/YarnException
> > at java.lang.Class.forName0(Native Method)
> > at java.lang.Class.forName(Class.java:264)
> > at
> >
> org.apache.flink.client.cli.CliFrontend.loadCustomCommandLine(CliFrontend.java:1185)
> > at
> >
> org.apache.flink.client.cli.CliFrontend.loadCustomCommandLines(CliFrontend.java:1145)
> > at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1070)
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.hadoop.yarn.exceptions.YarnException
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> > ... 5 more
> > 2020-06-16 17:06:21,980 INFO org.apache.flink.core.fs.FileSystem
> > - Hadoop is not in the classpath/dependencies. The
> extended
> > set of supported File Systems via Hadoop is not available.
>
>
Re: Flink yarn session exception
Posted by Paul Lam <pa...@gmail.com>.
日志里说得比较清楚了，classpath 里没有 Hadoop 的 lib。可以参考这个文档 [1] 来配置你的环境。
1. https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html <https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html>
Best,
Paul Lam
> 2020年7月16日 15:46，Rainie Li <ra...@pinterest.com.INVALID> 写道：
>
> 大佬们好，我是flink新手，正在用flink 1.9.1
> Flink APP cannot run, APP log error, 想求教一下会是什么原因造成的，多谢🙏
>
> 2020-06-16 17:06:21,921 WARN org.apache.flink.client.cli.CliFrontend
> - Could not load CLI class
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.
> java.lang.NoClassDefFoundError:
> org/apache/hadoop/yarn/exceptions/YarnException
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.flink.client.cli.CliFrontend.loadCustomCommandLine(CliFrontend.java:1185)
> at
> org.apache.flink.client.cli.CliFrontend.loadCustomCommandLines(CliFrontend.java:1145)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1070)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.exceptions.YarnException
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 5 more
> 2020-06-16 17:06:21,980 INFO org.apache.flink.core.fs.FileSystem
> - Hadoop is not in the classpath/dependencies. The extended
> set of supported File Systems via Hadoop is not available.