You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by Jason_H <hy...@163.com> on 2023/04/19 07:56:17 UTC
flink命令行提交作业读取不到properties配置文件
hi，大家好
我在使用命令行提交任务时，发现任务刚起来就会报错，根据错误发现没有读去到jar包中resource目录下的properties配置文件，导致在使用redis时，初始化报错
提交命令如下：
flink run -c com.test..etl.OdsChangeApplication /opt/dobrain/app/etl/test-etl-0.0.2-SNAPSHOT.jar \
-p 4 \
-job-name test-etl \
此处没有添加redis配置参数，但是配置文件中已经有默认的，提交运行后报错：
java.lang.IllegalArgumentException: template not initialized; call afterPropertiesSet() before using it
at org.springframework.util.Assert.isTrue(Assert.java:121) ~[spring-core-5.2.14.RELEASE.jar:5.2.14.RELEASE]
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:204) ~[spring-data-redis-2.3.9.RELEASE.jar:2.3.9.RELEASE]
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:188) ~[spring-data-redis-2.3.9.RELEASE.jar:2.3.9.RELEASE]
at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:96) ~[spring-data-redis-2.3.9.RELEASE.jar:2.3.9.RELEASE]
at org.springframework.data.redis.core.DefaultValueOperations.get(DefaultValueOperations.java:53) ~[spring-data-redis-2.3.9.RELEASE.jar:2.3.9.RELEASE]
at com.test.etl.client.RedisService.getStringValue(RedisService.java:30) ~[classes/:?]
at com.test.etl.manager.impl.RedisChangeManager.getCustId(RedisChangeManager.java:53) ~[classes/:?]
at com.test.etl.transformation.process.MsgHandleProcess.processElement(MsgHandleProcess.java:46) ~[classes/:?]
at com.test.etl.transformation.process.MsgHandleProcess.processElement(MsgHandleProcess.java:21) ~[classes/:?]
at org.apache.flink.streaming.api.operators.KeyedProcessOperator.processElement(KeyedProcessOperator.java:83) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:519) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:804) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:753) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:948) ~[flink-runtime-1.15.2.jar:1.15.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927) ~[flink-runtime-1.15.2.jar:1.15.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741) ~[flink-runtime-1.15.2.jar:1.15.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) ~[flink-runtime-1.15.2.jar:1.15.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_201]
我尝试在命令行添加了redis的参数，启动任务测试发现也会报如下的错误
请问大佬们，这个怎么解决，就是在命令行提交任务，怎么可以读取到jar包中定义的properties配置文件呢
| |
Jason_H
|
|
hyb_hello@163.com
|
回复：flink命令行提交作业读取不到properties配置文件
Posted by 小昌同学 <cc...@163.com>.
我这边的做法是将配置文件也当作一条流进行读取，程序会自动读取，不需要再任务启动的时候指定；希望对你有帮助呀
| |
小昌同学
|
|
ccc0606fighting@163.com
|
---- 回复的原邮件 ----
| 发件人 | Jason_H<hy...@163.com> |
| 发送日期 | 2023年4月19日 15:57 |
| 收件人 | flink中文邮件组<us...@flink.apache.org> ,
user-zh-subscribe<us...@flink.apache.org> |
| 主题 | flink命令行提交作业读取不到properties配置文件 |
hi，大家好
我在使用命令行提交任务时，发现任务刚起来就会报错，根据错误发现没有读去到jar包中resource目录下的properties配置文件，导致在使用redis时，初始化报错
提交命令如下：
flink run -c com.test..etl.OdsChangeApplication /opt/dobrain/app/etl/test-etl-0.0.2-SNAPSHOT.jar \
-p 4 \
-job-name test-etl \
此处没有添加redis配置参数，但是配置文件中已经有默认的，提交运行后报错：
java.lang.IllegalArgumentException: template not initialized; call afterPropertiesSet() before using it
at org.springframework.util.Assert.isTrue(Assert.java:121) ~[spring-core-5.2.14.RELEASE.jar:5.2.14.RELEASE]
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:204) ~[spring-data-redis-2.3.9.RELEASE.jar:2.3.9.RELEASE]
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:188) ~[spring-data-redis-2.3.9.RELEASE.jar:2.3.9.RELEASE]
at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:96) ~[spring-data-redis-2.3.9.RELEASE.jar:2.3.9.RELEASE]
at org.springframework.data.redis.core.DefaultValueOperations.get(DefaultValueOperations.java:53) ~[spring-data-redis-2.3.9.RELEASE.jar:2.3.9.RELEASE]
at com.test.etl.client.RedisService.getStringValue(RedisService.java:30) ~[classes/:?]
at com.test.etl.manager.impl.RedisChangeManager.getCustId(RedisChangeManager.java:53) ~[classes/:?]
at com.test.etl.transformation.process.MsgHandleProcess.processElement(MsgHandleProcess.java:46) ~[classes/:?]
at com.test.etl.transformation.process.MsgHandleProcess.processElement(MsgHandleProcess.java:21) ~[classes/:?]
at org.apache.flink.streaming.api.operators.KeyedProcessOperator.processElement(KeyedProcessOperator.java:83) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:519) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:804) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:753) ~[flink-streaming-java-1.15.2.jar:1.15.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:948) ~[flink-runtime-1.15.2.jar:1.15.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927) ~[flink-runtime-1.15.2.jar:1.15.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741) ~[flink-runtime-1.15.2.jar:1.15.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) ~[flink-runtime-1.15.2.jar:1.15.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_201]
我尝试在命令行添加了redis的参数，启动任务测试发现也会报如下的错误
请问大佬们，这个怎么解决，就是在命令行提交任务，怎么可以读取到jar包中定义的properties配置文件呢
| |
Jason_H
|
|
hyb_hello@163.com
|