You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Dan Hill <qu...@gmail.com> on 2020/09/11 03:24:38 UTC
I hit a bad jobmanager address when trying to use Flink SQL Client
I just tried using the Flink SQL Client. A simple job is not running
because it cannot hit jobmanager. I'm not sure why Flink SQL Client is
hitting "flink-jobmanager/10.98.253.58:8081". I'd expect either
"flink-jobmanager:8081" or "10.98.253.58:8081" (which should work with my
kubernetes setup).
I'm using riskfocus's Flink helm chart
<https://github.com/riskfocus/helm-charts-public/tree/master/flink>.
The last SELECT errors out. I have an environment file that indicates this
"execution.type: batch". My setup works when using DataSet and
DataStream. The jobmanager and taskmanager logs look fine. This seems
like a weird configuration with SQL Client that is either broken with that
Flink helm chart or with SQL Client.
Flink SQL> DROP TABLE `default_catalog.mydb.user`;
[INFO] Table has been removed.
Flink SQL> CREATE TABLE `default_catalog.mydb.user` (`platformId` BIGINT,
`userId` STRING) WITH ('connector' = 'filesystem', 'path' =
's3://mys3bucket/users.csv','format' = 'csv');
[INFO] Table has been created.
Flink SQL> SELECT * FROM `default_catalog.mydb.user` LIMIT 10;
*[ERROR] Could not execute SQL statement. Reason:*
*org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException:
connection timed out: flink-jobmanager/10.98.253.58:8081
<http://10.98.253.58:8081>*
My config logs from jobmanager.
2020-09-11 02:33:07,962 INFO
org.apache.flink.contrib.streaming.state.RocksDBStateBackend
[] - Using predefined options: DEFAULT.
2020-09-11 02:33:07,962 INFO
org.apache.flink.contrib.streaming.state.RocksDBStateBackend
[] - Using default options factory:
DefaultConfigurableOptionsFactory{configuredOptions={}}.
2020-09-11 02:33:08,100 INFO
org.apache.flink.table.client.gateway.local.ProgramDeployer [] -
Submitting job org.apache.flink.streaming.api.graph.StreamGraph@3bd08435
for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10`
2020-09-11 02:33:08,112 INFO org.apache.flink.configuration.Configuration
[] - Config uses fallback configuration key
'jobmanager.rpc.address' instead of key 'rest.address'
tail: log/flink--sql-client-flink-jobmanager-0.log: file truncated
2020-09-11 02:34:35,848 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
taskmanager.numberOfTaskSlots, 2
2020-09-11 02:34:35,884 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: blob.server.port, 6124
2020-09-11 02:34:35,884 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: taskmanager.rpc.port, 6122
2020-09-11 02:34:35,885 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: jobmanager.heap.size, 1g
2020-09-11 02:34:35,886 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
taskmanager.memory.process.size, 1g
2020-09-11 02:34:35,886 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: state.backend, rocksdb
2020-09-11 02:34:35,886 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: state.checkpoints.dir,
file:///flink_state/checkpoints
2020-09-11 02:34:35,887 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: state.savepoints.dir,
file:///flink_state/savepoints
2020-09-11 02:34:35,887 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: state.backend.async, true
2020-09-11 02:34:35,887 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.fs.memory-threshold, 1024
2020-09-11 02:34:35,888 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.fs.write-buffer-size, 4096
2020-09-11 02:34:35,888 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: state.backend.incremental,
true
2020-09-11 02:34:35,888 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.local-recovery, true
2020-09-11 02:34:35,889 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.checkpoints.num-retained, 1
2020-09-11 02:34:35,889 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
taskmanager.state.local.root-dirs, file:///flink_state/local-recovery
2020-09-11 02:34:35,889 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.rocksdb.checkpoint.transfer.thread.num, 1
2020-09-11 02:34:35,890 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.rocksdb.localdir, /flink_state/rocksdb
2020-09-11 02:34:35,890 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.rocksdb.options-factory,
org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory
2020-09-11 02:34:35,890 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.rocksdb.predefined-options, DEFAULT
2020-09-11 02:34:35,891 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.rocksdb.timer-service.factory, HEAP
2020-09-11 02:34:35,891 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.rocksdb.ttl.compaction.filter.enabled, false
2020-09-11 02:34:35,892 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: jobmanager.rpc.address,
flink-jobmanager
2020-09-11 02:34:35,892 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: jobmanager.rpc.port, 6123
2020-09-11 02:34:35,892 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
taskmanager.memory.jvm-metaspace.size, 256mb
2020-09-11 02:34:35,893 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: s3a.endpoint,
http://minio:9000
2020-09-11 02:34:35,893 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: s3a.path.style.access, true
2020-09-11 02:34:35,894 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: s3a.access-key, YOURACCESSKEY
2020-09-11 02:34:35,894 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: s3a.secret-key, ******
2020-09-11 02:34:35,895 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
s3a.aws.credentials.provider,
org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
2020-09-11 02:34:36,492 INFO org.apache.flink.client.cli.CliFrontend
[] - Loading FallbackYarnSessionCli
2020-09-11 02:34:36,498 INFO
org.apache.flink.table.client.gateway.local.LocalExecutor [] - Using
default environment file: file:/opt/flink/conf/sql-client-defaults.yaml
2020-09-11 02:34:37,786 INFO org.apache.flink.table.client.SqlClient
[] - Using session environment file:
file:/opt/flink/sql-client-defaults.yaml
2020-09-11 02:34:37,931 INFO
org.apache.flink.table.client.config.entries.ExecutionEntry [] - Property
'execution.restart-strategy.type' not specified. Using default value:
fallback
2020-09-11 02:34:43,269 INFO
org.apache.flink.table.client.gateway.local.ExecutionContext
[] - Executor config: {blob.server.port=6124, s3a.secret-key=YOURSECRETKEY,
state.checkpoints.num-retained=1, s3a.access-key=YOURACCESSKEY,
state.backend.rocksdb.options-factory=org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory,
jobmanager.rpc.address=flink-jobmanager,
taskmanager.memory.jvm-metaspace.size=256mb,
state.backend.rocksdb.predefined-options=DEFAULT,
state.savepoints.dir=file:///flink_state/savepoints,
state.backend.fs.memory-threshold=1024,
state.backend.rocksdb.timer-service.factory=HEAP,
execution.savepoint.ignore-unclaimed-state=false,
taskmanager.numberOfTaskSlots=2, state.backend.async=true,
pipeline.classpaths=[],
state.backend.rocksdb.ttl.compaction.filter.enabled=false,
jobmanager.heap.size=1g, taskmanager.memory.process.size=1g, s3a.endpoint=
http://minio:9000, state.backend.rocksdb.checkpoint.transfer.thread.num=1,
state.backend.local-recovery=true,
state.backend.rocksdb.localdir=/flink_state/rocksdb,
state.backend.incremental=true, execution.target=remote,
taskmanager.rpc.port=6122, jobmanager.rpc.port=6123,
s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider,
state.backend.fs.write-buffer-size=4096, execution.attached=true,
execution.shutdown-on-attached-exit=false,
pipeline.jars=[file:/opt/flink/opt/flink-sql-client_2.12-1.11.1.jar],
state.backend=rocksdb, s3a.path.style.access=true,
taskmanager.state.local.root-dirs=file:///flink_state/local-recovery,
state.checkpoints.dir=file:///flink_state/checkpoints}
2020-09-11 02:34:43,286 INFO
org.apache.flink.client.deployment.DefaultClusterClientServiceLoader
[] - Could not load factory due to missing dependencies.
2020-09-11 02:34:44,079 INFO org.apache.flink.table.client.cli.CliClient
[] - Command history file path: /root/.flink-sql-history
2020-09-11 02:35:15,396 INFO
org.apache.flink.contrib.streaming.state.RocksDBStateBackend
[] - Using predefined options: DEFAULT.
2020-09-11 02:35:15,397 INFO
org.apache.flink.contrib.streaming.state.RocksDBStateBackend
[] - Using default options factory:
DefaultConfigurableOptionsFactory{configuredOptions={}}.
2020-09-11 02:35:17,529 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
taskmanager.numberOfTaskSlots, 2
2020-09-11 02:35:17,530 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: blob.server.port, 6124
2020-09-11 02:35:17,530 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: taskmanager.rpc.port, 6122
2020-09-11 02:35:17,530 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: jobmanager.heap.size, 1g
2020-09-11 02:35:17,530 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
taskmanager.memory.process.size, 1g
2020-09-11 02:35:17,530 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: state.backend, rocksdb
2020-09-11 02:35:17,530 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: state.checkpoints.dir,
file:///flink_state/checkpoints
2020-09-11 02:35:17,531 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: state.savepoints.dir,
file:///flink_state/savepoints
2020-09-11 02:35:17,531 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: state.backend.async, true
2020-09-11 02:35:17,531 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.fs.memory-threshold, 1024
2020-09-11 02:35:17,531 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.fs.write-buffer-size, 4096
2020-09-11 02:35:17,531 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: state.backend.incremental,
true
2020-09-11 02:35:17,531 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.local-recovery, true
2020-09-11 02:35:17,532 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.checkpoints.num-retained, 1
2020-09-11 02:35:17,532 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
taskmanager.state.local.root-dirs, file:///flink_state/local-recovery
2020-09-11 02:35:17,532 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.rocksdb.checkpoint.transfer.thread.num, 1
2020-09-11 02:35:17,532 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.rocksdb.localdir, /flink_state/rocksdb
2020-09-11 02:35:17,532 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.rocksdb.options-factory,
org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory
2020-09-11 02:35:17,532 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.rocksdb.predefined-options, DEFAULT
2020-09-11 02:35:17,532 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.rocksdb.timer-service.factory, HEAP
2020-09-11 02:35:17,533 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
state.backend.rocksdb.ttl.compaction.filter.enabled, false
2020-09-11 02:35:17,533 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: jobmanager.rpc.address,
flink-jobmanager
2020-09-11 02:35:17,533 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: jobmanager.rpc.port, 6123
2020-09-11 02:35:17,533 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
taskmanager.memory.jvm-metaspace.size, 256mb
2020-09-11 02:35:17,533 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: s3a.endpoint,
http://minio:9000
2020-09-11 02:35:17,534 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: s3a.path.style.access, true
2020-09-11 02:35:17,534 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: s3a.access-key, YOURACCESSKEY
2020-09-11 02:35:17,534 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: s3a.secret-key, ******
2020-09-11 02:35:17,535 INFO
org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
s3a.aws.credentials.provider,
org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
2020-09-11 02:35:18,047 INFO
org.apache.flink.table.client.gateway.local.ProgramDeployer [] -
Submitting job org.apache.flink.streaming.api.graph.StreamGraph@2f95653f
for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10`
2020-09-11 02:35:18,382 INFO org.apache.flink.configuration.Configuration
[] - Config uses fallback configuration key
'jobmanager.rpc.address' instead of key 'rest.address'
2020-09-11 02:41:39,539 WARN org.apache.flink.runtime.rest.RestClient
[] - Rest endpoint shutdown failed.
java.util.concurrent.TimeoutException: null
Re: I hit a bad jobmanager address when trying to use Flink SQL Client
Posted by Dan Hill <qu...@gmail.com>.
Hi Robert! Sorry for the delay. This worked! Thanks! I used slightly
different deployment parameters.
deployment:
gateway-address: flink-jobmanager
gateway-port: 8081
On Mon, Sep 14, 2020 at 6:21 AM Robert Metzger <rm...@apache.org> wrote:
> Hi Dan,
>
> I don't think the SQL Client officially supports running against
> Kubernetes.
> What you could try is using an undocumented, untested feature:
> Put something like
> jobmanager: kubernetes
> into the "deployment:" section of the Sql Client configuration
> <https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sqlClient.html#environment-files>
> .
> Proper support for Kubernetes, YARN etc. is (probably stalled) work in
> progress
> <https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sqlClient.html#limitations--future>
> .
>
>
> On Mon, Sep 14, 2020 at 2:18 PM Arvid Heise <ar...@ververica.com> wrote:
>
>> Hi Dan,
>>
>> Can you verify from the pod that jobmanager and *10.98.253.58:8081
>> <http://10.98.253.58:8081> *is actually accessible (e.g., with curl)?
>> I'd probably also try out localhost:8081 as you are connecting to the
>> respective pod directly.
>>
>> On Fri, Sep 11, 2020 at 9:59 PM Dan Hill <qu...@gmail.com> wrote:
>>
>>> Hi Robert!
>>>
>>> I have Flink running locally on minikube. I'm running SQL client using
>>> exec on the jobmanager.
>>>
>>> kubectl exec pod/flink-jobmanager-0 -i -t --
>>> /opt/flink/bin/sql-client.sh embedded -e /opt/flink/sql-client-defaults.yaml
>>>
>>>
>>> Here's the sql-client-defaults.yaml. I didn't specify a session.
>>> execution:
>>> type: batch
>>> result-mode: table
>>> max-table-result-rows: 1000000
>>>
>>> I'm prototyping the Table SQL interface. I got blocked using the Table
>>> SQL interface and figured I'd try the SQL Client to see if I could get
>>> unblocked.
>>>
>>>
>>> On Fri, Sep 11, 2020 at 11:18 AM Robert Metzger <rm...@apache.org>
>>> wrote:
>>>
>>>> Hi Dan,
>>>>
>>>> the notation of "flink-jobmanager/10.98.253.58:8081" is not a problem.
>>>> It is how java.net.InetAddress stringifies a resolved address (with
>>>> both hostname and IP).
>>>>
>>>> How did you configure the SQL client to work with a Kubernetes Session?
>>>> Afaik this is not a documented, tested and officially supported feature
>>>> (this doesn't mean we should not support it -- apparently it is something
>>>> we should do rather soon ;) ).
>>>>
>>>> Best,
>>>> Robert
>>>>
>>>> On Fri, Sep 11, 2020 at 5:25 AM Dan Hill <qu...@gmail.com> wrote:
>>>>
>>>>> I just tried using the Flink SQL Client. A simple job is not running
>>>>> because it cannot hit jobmanager. I'm not sure why Flink SQL Client is
>>>>> hitting "flink-jobmanager/10.98.253.58:8081". I'd expect either
>>>>> "flink-jobmanager:8081" or "10.98.253.58:8081" (which should work
>>>>> with my kubernetes setup).
>>>>>
>>>>> I'm using riskfocus's Flink helm chart
>>>>> <https://github.com/riskfocus/helm-charts-public/tree/master/flink>.
>>>>>
>>>>> The last SELECT errors out. I have an environment file that indicates
>>>>> this "execution.type: batch". My setup works when using DataSet and
>>>>> DataStream. The jobmanager and taskmanager logs look fine. This seems
>>>>> like a weird configuration with SQL Client that is either broken with that
>>>>> Flink helm chart or with SQL Client.
>>>>>
>>>>>
>>>>> Flink SQL> DROP TABLE `default_catalog.mydb.user`;
>>>>>
>>>>> [INFO] Table has been removed.
>>>>>
>>>>>
>>>>> Flink SQL> CREATE TABLE `default_catalog.mydb.user` (`platformId`
>>>>> BIGINT, `userId` STRING) WITH ('connector' = 'filesystem', 'path' =
>>>>> 's3://mys3bucket/users.csv','format' = 'csv');
>>>>>
>>>>> [INFO] Table has been created.
>>>>>
>>>>>
>>>>> Flink SQL> SELECT * FROM `default_catalog.mydb.user` LIMIT 10;
>>>>>
>>>>> *[ERROR] Could not execute SQL statement. Reason:*
>>>>>
>>>>> *org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException:
>>>>> connection timed out: flink-jobmanager/10.98.253.58:8081
>>>>> <http://10.98.253.58:8081>*
>>>>>
>>>>> My config logs from jobmanager.
>>>>>
>>>>> 2020-09-11 02:33:07,962 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>>>>> [] - Using predefined options: DEFAULT.
>>>>>
>>>>> 2020-09-11 02:33:07,962 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>>>>> [] - Using default options factory:
>>>>> DefaultConfigurableOptionsFactory{configuredOptions={}}.
>>>>>
>>>>> 2020-09-11 02:33:08,100 INFO
>>>>> org.apache.flink.table.client.gateway.local.ProgramDeployer [] -
>>>>> Submitting job org.apache.flink.streaming.api.graph.StreamGraph@3bd08435
>>>>> for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10`
>>>>>
>>>>> 2020-09-11 02:33:08,112 INFO org.apache.flink.configuration.Configuration
>>>>> [] - Config uses fallback configuration key
>>>>> 'jobmanager.rpc.address' instead of key 'rest.address'
>>>>>
>>>>> tail: log/flink--sql-client-flink-jobmanager-0.log: file truncated
>>>>>
>>>>> 2020-09-11 02:34:35,848 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> taskmanager.numberOfTaskSlots, 2
>>>>>
>>>>> 2020-09-11 02:34:35,884 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: blob.server.port, 6124
>>>>>
>>>>> 2020-09-11 02:34:35,884 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: taskmanager.rpc.port,
>>>>> 6122
>>>>>
>>>>> 2020-09-11 02:34:35,885 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: jobmanager.heap.size,
>>>>> 1g
>>>>>
>>>>> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> taskmanager.memory.process.size, 1g
>>>>>
>>>>> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: state.backend, rocksdb
>>>>>
>>>>> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: state.checkpoints.dir,
>>>>> file:///flink_state/checkpoints
>>>>>
>>>>> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: state.savepoints.dir,
>>>>> file:///flink_state/savepoints
>>>>>
>>>>> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: state.backend.async,
>>>>> true
>>>>>
>>>>> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.fs.memory-threshold, 1024
>>>>>
>>>>> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.fs.write-buffer-size, 4096
>>>>>
>>>>> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.incremental, true
>>>>>
>>>>> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.local-recovery, true
>>>>>
>>>>> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.checkpoints.num-retained, 1
>>>>>
>>>>> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> taskmanager.state.local.root-dirs, file:///flink_state/local-recovery
>>>>>
>>>>> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.rocksdb.checkpoint.transfer.thread.num, 1
>>>>>
>>>>> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.rocksdb.localdir, /flink_state/rocksdb
>>>>>
>>>>> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.rocksdb.options-factory,
>>>>> org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory
>>>>>
>>>>> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.rocksdb.predefined-options, DEFAULT
>>>>>
>>>>> 2020-09-11 02:34:35,891 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.rocksdb.timer-service.factory, HEAP
>>>>>
>>>>> 2020-09-11 02:34:35,891 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.rocksdb.ttl.compaction.filter.enabled, false
>>>>>
>>>>> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> jobmanager.rpc.address, flink-jobmanager
>>>>>
>>>>> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: jobmanager.rpc.port,
>>>>> 6123
>>>>>
>>>>> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> taskmanager.memory.jvm-metaspace.size, 256mb
>>>>>
>>>>> 2020-09-11 02:34:35,893 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: s3a.endpoint,
>>>>> http://minio:9000
>>>>>
>>>>> 2020-09-11 02:34:35,893 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: s3a.path.style.access,
>>>>> true
>>>>>
>>>>> 2020-09-11 02:34:35,894 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: s3a.access-key,
>>>>> YOURACCESSKEY
>>>>>
>>>>> 2020-09-11 02:34:35,894 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: s3a.secret-key, ******
>>>>>
>>>>> 2020-09-11 02:34:35,895 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> s3a.aws.credentials.provider,
>>>>> org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
>>>>>
>>>>> 2020-09-11 02:34:36,492 INFO org.apache.flink.client.cli.CliFrontend
>>>>> [] - Loading FallbackYarnSessionCli
>>>>>
>>>>> 2020-09-11 02:34:36,498 INFO
>>>>> org.apache.flink.table.client.gateway.local.LocalExecutor [] -
>>>>> Using default environment file:
>>>>> file:/opt/flink/conf/sql-client-defaults.yaml
>>>>>
>>>>> 2020-09-11 02:34:37,786 INFO org.apache.flink.table.client.SqlClient
>>>>> [] - Using session environment file:
>>>>> file:/opt/flink/sql-client-defaults.yaml
>>>>>
>>>>> 2020-09-11 02:34:37,931 INFO
>>>>> org.apache.flink.table.client.config.entries.ExecutionEntry [] -
>>>>> Property 'execution.restart-strategy.type' not specified. Using default
>>>>> value: fallback
>>>>>
>>>>> 2020-09-11 02:34:43,269 INFO org.apache.flink.table.client.gateway.local.ExecutionContext
>>>>> [] - Executor config: {blob.server.port=6124, s3a.secret-key=YOURSECRETKEY,
>>>>> state.checkpoints.num-retained=1, s3a.access-key=YOURACCESSKEY,
>>>>> state.backend.rocksdb.options-factory=org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory,
>>>>> jobmanager.rpc.address=flink-jobmanager,
>>>>> taskmanager.memory.jvm-metaspace.size=256mb,
>>>>> state.backend.rocksdb.predefined-options=DEFAULT,
>>>>> state.savepoints.dir=file:///flink_state/savepoints,
>>>>> state.backend.fs.memory-threshold=1024,
>>>>> state.backend.rocksdb.timer-service.factory=HEAP,
>>>>> execution.savepoint.ignore-unclaimed-state=false,
>>>>> taskmanager.numberOfTaskSlots=2, state.backend.async=true,
>>>>> pipeline.classpaths=[],
>>>>> state.backend.rocksdb.ttl.compaction.filter.enabled=false,
>>>>> jobmanager.heap.size=1g, taskmanager.memory.process.size=1g, s3a.endpoint=
>>>>> http://minio:9000,
>>>>> state.backend.rocksdb.checkpoint.transfer.thread.num=1,
>>>>> state.backend.local-recovery=true,
>>>>> state.backend.rocksdb.localdir=/flink_state/rocksdb,
>>>>> state.backend.incremental=true, execution.target=remote,
>>>>> taskmanager.rpc.port=6122, jobmanager.rpc.port=6123,
>>>>> s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider,
>>>>> state.backend.fs.write-buffer-size=4096, execution.attached=true,
>>>>> execution.shutdown-on-attached-exit=false,
>>>>> pipeline.jars=[file:/opt/flink/opt/flink-sql-client_2.12-1.11.1.jar],
>>>>> state.backend=rocksdb, s3a.path.style.access=true,
>>>>> taskmanager.state.local.root-dirs=file:///flink_state/local-recovery,
>>>>> state.checkpoints.dir=file:///flink_state/checkpoints}
>>>>>
>>>>> 2020-09-11 02:34:43,286 INFO org.apache.flink.client.deployment.DefaultClusterClientServiceLoader
>>>>> [] - Could not load factory due to missing dependencies.
>>>>>
>>>>> 2020-09-11 02:34:44,079 INFO
>>>>> org.apache.flink.table.client.cli.CliClient [] -
>>>>> Command history file path: /root/.flink-sql-history
>>>>>
>>>>> 2020-09-11 02:35:15,396 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>>>>> [] - Using predefined options: DEFAULT.
>>>>>
>>>>> 2020-09-11 02:35:15,397 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>>>>> [] - Using default options factory:
>>>>> DefaultConfigurableOptionsFactory{configuredOptions={}}.
>>>>>
>>>>> 2020-09-11 02:35:17,529 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> taskmanager.numberOfTaskSlots, 2
>>>>>
>>>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: blob.server.port, 6124
>>>>>
>>>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: taskmanager.rpc.port,
>>>>> 6122
>>>>>
>>>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: jobmanager.heap.size,
>>>>> 1g
>>>>>
>>>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> taskmanager.memory.process.size, 1g
>>>>>
>>>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: state.backend, rocksdb
>>>>>
>>>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: state.checkpoints.dir,
>>>>> file:///flink_state/checkpoints
>>>>>
>>>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: state.savepoints.dir,
>>>>> file:///flink_state/savepoints
>>>>>
>>>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: state.backend.async,
>>>>> true
>>>>>
>>>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.fs.memory-threshold, 1024
>>>>>
>>>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.fs.write-buffer-size, 4096
>>>>>
>>>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.incremental, true
>>>>>
>>>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.local-recovery, true
>>>>>
>>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.checkpoints.num-retained, 1
>>>>>
>>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> taskmanager.state.local.root-dirs, file:///flink_state/local-recovery
>>>>>
>>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.rocksdb.checkpoint.transfer.thread.num, 1
>>>>>
>>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.rocksdb.localdir, /flink_state/rocksdb
>>>>>
>>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.rocksdb.options-factory,
>>>>> org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory
>>>>>
>>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.rocksdb.predefined-options, DEFAULT
>>>>>
>>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.rocksdb.timer-service.factory, HEAP
>>>>>
>>>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> state.backend.rocksdb.ttl.compaction.filter.enabled, false
>>>>>
>>>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> jobmanager.rpc.address, flink-jobmanager
>>>>>
>>>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: jobmanager.rpc.port,
>>>>> 6123
>>>>>
>>>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> taskmanager.memory.jvm-metaspace.size, 256mb
>>>>>
>>>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: s3a.endpoint,
>>>>> http://minio:9000
>>>>>
>>>>> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: s3a.path.style.access,
>>>>> true
>>>>>
>>>>> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: s3a.access-key,
>>>>> YOURACCESSKEY
>>>>>
>>>>> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property: s3a.secret-key, ******
>>>>>
>>>>> 2020-09-11 02:35:17,535 INFO org.apache.flink.configuration.GlobalConfiguration
>>>>> [] - Loading configuration property:
>>>>> s3a.aws.credentials.provider,
>>>>> org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
>>>>>
>>>>> 2020-09-11 02:35:18,047 INFO
>>>>> org.apache.flink.table.client.gateway.local.ProgramDeployer [] -
>>>>> Submitting job org.apache.flink.streaming.api.graph.StreamGraph@2f95653f
>>>>> for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10`
>>>>>
>>>>> 2020-09-11 02:35:18,382 INFO org.apache.flink.configuration.Configuration
>>>>> [] - Config uses fallback configuration key
>>>>> 'jobmanager.rpc.address' instead of key 'rest.address'
>>>>>
>>>>> 2020-09-11 02:41:39,539 WARN org.apache.flink.runtime.rest.RestClient
>>>>> [] - Rest endpoint shutdown failed.
>>>>>
>>>>> java.util.concurrent.TimeoutException: null
>>>>>
>>>>
>>
>> --
>>
>> Arvid Heise | Senior Java Developer
>>
>> <https://www.ververica.com/>
>>
>> Follow us @VervericaData
>>
>> --
>>
>> Join Flink Forward <https://flink-forward.org/> - The Apache Flink
>> Conference
>>
>> Stream Processing | Event Driven | Real Time
>>
>> --
>>
>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
>>
>> --
>> Ververica GmbH
>> Registered at Amtsgericht Charlottenburg: HRB 158244 B
>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
>> (Toni) Cheng
>>
>
Re: I hit a bad jobmanager address when trying to use Flink SQL Client
Posted by Robert Metzger <rm...@apache.org>.
Hi Dan,
I don't think the SQL Client officially supports running against Kubernetes.
What you could try is using an undocumented, untested feature:
Put something like
jobmanager: kubernetes
into the "deployment:" section of the Sql Client configuration
<https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sqlClient.html#environment-files>
.
Proper support for Kubernetes, YARN etc. is (probably stalled) work in
progress
<https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sqlClient.html#limitations--future>
.
On Mon, Sep 14, 2020 at 2:18 PM Arvid Heise <ar...@ververica.com> wrote:
> Hi Dan,
>
> Can you verify from the pod that jobmanager and *10.98.253.58:8081
> <http://10.98.253.58:8081> *is actually accessible (e.g., with curl)? I'd
> probably also try out localhost:8081 as you are connecting to the
> respective pod directly.
>
> On Fri, Sep 11, 2020 at 9:59 PM Dan Hill <qu...@gmail.com> wrote:
>
>> Hi Robert!
>>
>> I have Flink running locally on minikube. I'm running SQL client using
>> exec on the jobmanager.
>>
>> kubectl exec pod/flink-jobmanager-0 -i -t -- /opt/flink/bin/sql-client.sh
>> embedded -e /opt/flink/sql-client-defaults.yaml
>>
>> Here's the sql-client-defaults.yaml. I didn't specify a session.
>> execution:
>> type: batch
>> result-mode: table
>> max-table-result-rows: 1000000
>>
>> I'm prototyping the Table SQL interface. I got blocked using the Table
>> SQL interface and figured I'd try the SQL Client to see if I could get
>> unblocked.
>>
>>
>> On Fri, Sep 11, 2020 at 11:18 AM Robert Metzger <rm...@apache.org>
>> wrote:
>>
>>> Hi Dan,
>>>
>>> the notation of "flink-jobmanager/10.98.253.58:8081" is not a problem.
>>> It is how java.net.InetAddress stringifies a resolved address (with
>>> both hostname and IP).
>>>
>>> How did you configure the SQL client to work with a Kubernetes Session?
>>> Afaik this is not a documented, tested and officially supported feature
>>> (this doesn't mean we should not support it -- apparently it is something
>>> we should do rather soon ;) ).
>>>
>>> Best,
>>> Robert
>>>
>>> On Fri, Sep 11, 2020 at 5:25 AM Dan Hill <qu...@gmail.com> wrote:
>>>
>>>> I just tried using the Flink SQL Client. A simple job is not running
>>>> because it cannot hit jobmanager. I'm not sure why Flink SQL Client is
>>>> hitting "flink-jobmanager/10.98.253.58:8081". I'd expect either
>>>> "flink-jobmanager:8081" or "10.98.253.58:8081" (which should work with
>>>> my kubernetes setup).
>>>>
>>>> I'm using riskfocus's Flink helm chart
>>>> <https://github.com/riskfocus/helm-charts-public/tree/master/flink>.
>>>>
>>>> The last SELECT errors out. I have an environment file that indicates
>>>> this "execution.type: batch". My setup works when using DataSet and
>>>> DataStream. The jobmanager and taskmanager logs look fine. This seems
>>>> like a weird configuration with SQL Client that is either broken with that
>>>> Flink helm chart or with SQL Client.
>>>>
>>>>
>>>> Flink SQL> DROP TABLE `default_catalog.mydb.user`;
>>>>
>>>> [INFO] Table has been removed.
>>>>
>>>>
>>>> Flink SQL> CREATE TABLE `default_catalog.mydb.user` (`platformId`
>>>> BIGINT, `userId` STRING) WITH ('connector' = 'filesystem', 'path' =
>>>> 's3://mys3bucket/users.csv','format' = 'csv');
>>>>
>>>> [INFO] Table has been created.
>>>>
>>>>
>>>> Flink SQL> SELECT * FROM `default_catalog.mydb.user` LIMIT 10;
>>>>
>>>> *[ERROR] Could not execute SQL statement. Reason:*
>>>>
>>>> *org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException:
>>>> connection timed out: flink-jobmanager/10.98.253.58:8081
>>>> <http://10.98.253.58:8081>*
>>>>
>>>> My config logs from jobmanager.
>>>>
>>>> 2020-09-11 02:33:07,962 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>>>> [] - Using predefined options: DEFAULT.
>>>>
>>>> 2020-09-11 02:33:07,962 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>>>> [] - Using default options factory:
>>>> DefaultConfigurableOptionsFactory{configuredOptions={}}.
>>>>
>>>> 2020-09-11 02:33:08,100 INFO
>>>> org.apache.flink.table.client.gateway.local.ProgramDeployer [] -
>>>> Submitting job org.apache.flink.streaming.api.graph.StreamGraph@3bd08435
>>>> for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10`
>>>>
>>>> 2020-09-11 02:33:08,112 INFO org.apache.flink.configuration.Configuration
>>>> [] - Config uses fallback configuration key
>>>> 'jobmanager.rpc.address' instead of key 'rest.address'
>>>>
>>>> tail: log/flink--sql-client-flink-jobmanager-0.log: file truncated
>>>>
>>>> 2020-09-11 02:34:35,848 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> taskmanager.numberOfTaskSlots, 2
>>>>
>>>> 2020-09-11 02:34:35,884 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: blob.server.port, 6124
>>>>
>>>> 2020-09-11 02:34:35,884 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: taskmanager.rpc.port,
>>>> 6122
>>>>
>>>> 2020-09-11 02:34:35,885 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: jobmanager.heap.size, 1g
>>>>
>>>> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> taskmanager.memory.process.size, 1g
>>>>
>>>> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: state.backend, rocksdb
>>>>
>>>> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: state.checkpoints.dir,
>>>> file:///flink_state/checkpoints
>>>>
>>>> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: state.savepoints.dir,
>>>> file:///flink_state/savepoints
>>>>
>>>> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: state.backend.async,
>>>> true
>>>>
>>>> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.fs.memory-threshold, 1024
>>>>
>>>> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.fs.write-buffer-size, 4096
>>>>
>>>> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.incremental, true
>>>>
>>>> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.local-recovery, true
>>>>
>>>> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.checkpoints.num-retained, 1
>>>>
>>>> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> taskmanager.state.local.root-dirs, file:///flink_state/local-recovery
>>>>
>>>> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.rocksdb.checkpoint.transfer.thread.num, 1
>>>>
>>>> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.rocksdb.localdir, /flink_state/rocksdb
>>>>
>>>> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.rocksdb.options-factory,
>>>> org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory
>>>>
>>>> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.rocksdb.predefined-options, DEFAULT
>>>>
>>>> 2020-09-11 02:34:35,891 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.rocksdb.timer-service.factory, HEAP
>>>>
>>>> 2020-09-11 02:34:35,891 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.rocksdb.ttl.compaction.filter.enabled, false
>>>>
>>>> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: jobmanager.rpc.address,
>>>> flink-jobmanager
>>>>
>>>> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: jobmanager.rpc.port,
>>>> 6123
>>>>
>>>> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> taskmanager.memory.jvm-metaspace.size, 256mb
>>>>
>>>> 2020-09-11 02:34:35,893 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: s3a.endpoint,
>>>> http://minio:9000
>>>>
>>>> 2020-09-11 02:34:35,893 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: s3a.path.style.access,
>>>> true
>>>>
>>>> 2020-09-11 02:34:35,894 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: s3a.access-key,
>>>> YOURACCESSKEY
>>>>
>>>> 2020-09-11 02:34:35,894 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: s3a.secret-key, ******
>>>>
>>>> 2020-09-11 02:34:35,895 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> s3a.aws.credentials.provider,
>>>> org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
>>>>
>>>> 2020-09-11 02:34:36,492 INFO org.apache.flink.client.cli.CliFrontend
>>>> [] - Loading FallbackYarnSessionCli
>>>>
>>>> 2020-09-11 02:34:36,498 INFO
>>>> org.apache.flink.table.client.gateway.local.LocalExecutor [] -
>>>> Using default environment file:
>>>> file:/opt/flink/conf/sql-client-defaults.yaml
>>>>
>>>> 2020-09-11 02:34:37,786 INFO org.apache.flink.table.client.SqlClient
>>>> [] - Using session environment file:
>>>> file:/opt/flink/sql-client-defaults.yaml
>>>>
>>>> 2020-09-11 02:34:37,931 INFO
>>>> org.apache.flink.table.client.config.entries.ExecutionEntry [] -
>>>> Property 'execution.restart-strategy.type' not specified. Using default
>>>> value: fallback
>>>>
>>>> 2020-09-11 02:34:43,269 INFO org.apache.flink.table.client.gateway.local.ExecutionContext
>>>> [] - Executor config: {blob.server.port=6124, s3a.secret-key=YOURSECRETKEY,
>>>> state.checkpoints.num-retained=1, s3a.access-key=YOURACCESSKEY,
>>>> state.backend.rocksdb.options-factory=org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory,
>>>> jobmanager.rpc.address=flink-jobmanager,
>>>> taskmanager.memory.jvm-metaspace.size=256mb,
>>>> state.backend.rocksdb.predefined-options=DEFAULT,
>>>> state.savepoints.dir=file:///flink_state/savepoints,
>>>> state.backend.fs.memory-threshold=1024,
>>>> state.backend.rocksdb.timer-service.factory=HEAP,
>>>> execution.savepoint.ignore-unclaimed-state=false,
>>>> taskmanager.numberOfTaskSlots=2, state.backend.async=true,
>>>> pipeline.classpaths=[],
>>>> state.backend.rocksdb.ttl.compaction.filter.enabled=false,
>>>> jobmanager.heap.size=1g, taskmanager.memory.process.size=1g, s3a.endpoint=
>>>> http://minio:9000,
>>>> state.backend.rocksdb.checkpoint.transfer.thread.num=1,
>>>> state.backend.local-recovery=true,
>>>> state.backend.rocksdb.localdir=/flink_state/rocksdb,
>>>> state.backend.incremental=true, execution.target=remote,
>>>> taskmanager.rpc.port=6122, jobmanager.rpc.port=6123,
>>>> s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider,
>>>> state.backend.fs.write-buffer-size=4096, execution.attached=true,
>>>> execution.shutdown-on-attached-exit=false,
>>>> pipeline.jars=[file:/opt/flink/opt/flink-sql-client_2.12-1.11.1.jar],
>>>> state.backend=rocksdb, s3a.path.style.access=true,
>>>> taskmanager.state.local.root-dirs=file:///flink_state/local-recovery,
>>>> state.checkpoints.dir=file:///flink_state/checkpoints}
>>>>
>>>> 2020-09-11 02:34:43,286 INFO org.apache.flink.client.deployment.DefaultClusterClientServiceLoader
>>>> [] - Could not load factory due to missing dependencies.
>>>>
>>>> 2020-09-11 02:34:44,079 INFO
>>>> org.apache.flink.table.client.cli.CliClient [] -
>>>> Command history file path: /root/.flink-sql-history
>>>>
>>>> 2020-09-11 02:35:15,396 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>>>> [] - Using predefined options: DEFAULT.
>>>>
>>>> 2020-09-11 02:35:15,397 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>>>> [] - Using default options factory:
>>>> DefaultConfigurableOptionsFactory{configuredOptions={}}.
>>>>
>>>> 2020-09-11 02:35:17,529 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> taskmanager.numberOfTaskSlots, 2
>>>>
>>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: blob.server.port, 6124
>>>>
>>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: taskmanager.rpc.port,
>>>> 6122
>>>>
>>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: jobmanager.heap.size, 1g
>>>>
>>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> taskmanager.memory.process.size, 1g
>>>>
>>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: state.backend, rocksdb
>>>>
>>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: state.checkpoints.dir,
>>>> file:///flink_state/checkpoints
>>>>
>>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: state.savepoints.dir,
>>>> file:///flink_state/savepoints
>>>>
>>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: state.backend.async,
>>>> true
>>>>
>>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.fs.memory-threshold, 1024
>>>>
>>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.fs.write-buffer-size, 4096
>>>>
>>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.incremental, true
>>>>
>>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.local-recovery, true
>>>>
>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.checkpoints.num-retained, 1
>>>>
>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> taskmanager.state.local.root-dirs, file:///flink_state/local-recovery
>>>>
>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.rocksdb.checkpoint.transfer.thread.num, 1
>>>>
>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.rocksdb.localdir, /flink_state/rocksdb
>>>>
>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.rocksdb.options-factory,
>>>> org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory
>>>>
>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.rocksdb.predefined-options, DEFAULT
>>>>
>>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.rocksdb.timer-service.factory, HEAP
>>>>
>>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> state.backend.rocksdb.ttl.compaction.filter.enabled, false
>>>>
>>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: jobmanager.rpc.address,
>>>> flink-jobmanager
>>>>
>>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: jobmanager.rpc.port,
>>>> 6123
>>>>
>>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> taskmanager.memory.jvm-metaspace.size, 256mb
>>>>
>>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: s3a.endpoint,
>>>> http://minio:9000
>>>>
>>>> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: s3a.path.style.access,
>>>> true
>>>>
>>>> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: s3a.access-key,
>>>> YOURACCESSKEY
>>>>
>>>> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property: s3a.secret-key, ******
>>>>
>>>> 2020-09-11 02:35:17,535 INFO org.apache.flink.configuration.GlobalConfiguration
>>>> [] - Loading configuration property:
>>>> s3a.aws.credentials.provider,
>>>> org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
>>>>
>>>> 2020-09-11 02:35:18,047 INFO
>>>> org.apache.flink.table.client.gateway.local.ProgramDeployer [] -
>>>> Submitting job org.apache.flink.streaming.api.graph.StreamGraph@2f95653f
>>>> for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10`
>>>>
>>>> 2020-09-11 02:35:18,382 INFO org.apache.flink.configuration.Configuration
>>>> [] - Config uses fallback configuration key
>>>> 'jobmanager.rpc.address' instead of key 'rest.address'
>>>>
>>>> 2020-09-11 02:41:39,539 WARN org.apache.flink.runtime.rest.RestClient
>>>> [] - Rest endpoint shutdown failed.
>>>>
>>>> java.util.concurrent.TimeoutException: null
>>>>
>>>
>
> --
>
> Arvid Heise | Senior Java Developer
>
> <https://www.ververica.com/>
>
> Follow us @VervericaData
>
> --
>
> Join Flink Forward <https://flink-forward.org/> - The Apache Flink
> Conference
>
> Stream Processing | Event Driven | Real Time
>
> --
>
> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
>
> --
> Ververica GmbH
> Registered at Amtsgericht Charlottenburg: HRB 158244 B
> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
> (Toni) Cheng
>
Re: I hit a bad jobmanager address when trying to use Flink SQL Client
Posted by Arvid Heise <ar...@ververica.com>.
Hi Dan,
Can you verify from the pod that jobmanager and *10.98.253.58:8081
<http://10.98.253.58:8081> *is actually accessible (e.g., with curl)? I'd
probably also try out localhost:8081 as you are connecting to the
respective pod directly.
On Fri, Sep 11, 2020 at 9:59 PM Dan Hill <qu...@gmail.com> wrote:
> Hi Robert!
>
> I have Flink running locally on minikube. I'm running SQL client using
> exec on the jobmanager.
>
> kubectl exec pod/flink-jobmanager-0 -i -t -- /opt/flink/bin/sql-client.sh
> embedded -e /opt/flink/sql-client-defaults.yaml
>
> Here's the sql-client-defaults.yaml. I didn't specify a session.
> execution:
> type: batch
> result-mode: table
> max-table-result-rows: 1000000
>
> I'm prototyping the Table SQL interface. I got blocked using the Table
> SQL interface and figured I'd try the SQL Client to see if I could get
> unblocked.
>
>
> On Fri, Sep 11, 2020 at 11:18 AM Robert Metzger <rm...@apache.org>
> wrote:
>
>> Hi Dan,
>>
>> the notation of "flink-jobmanager/10.98.253.58:8081" is not a problem.
>> It is how java.net.InetAddress stringifies a resolved address (with both
>> hostname and IP).
>>
>> How did you configure the SQL client to work with a Kubernetes Session?
>> Afaik this is not a documented, tested and officially supported feature
>> (this doesn't mean we should not support it -- apparently it is something
>> we should do rather soon ;) ).
>>
>> Best,
>> Robert
>>
>> On Fri, Sep 11, 2020 at 5:25 AM Dan Hill <qu...@gmail.com> wrote:
>>
>>> I just tried using the Flink SQL Client. A simple job is not running
>>> because it cannot hit jobmanager. I'm not sure why Flink SQL Client is
>>> hitting "flink-jobmanager/10.98.253.58:8081". I'd expect either
>>> "flink-jobmanager:8081" or "10.98.253.58:8081" (which should work with
>>> my kubernetes setup).
>>>
>>> I'm using riskfocus's Flink helm chart
>>> <https://github.com/riskfocus/helm-charts-public/tree/master/flink>.
>>>
>>> The last SELECT errors out. I have an environment file that indicates
>>> this "execution.type: batch". My setup works when using DataSet and
>>> DataStream. The jobmanager and taskmanager logs look fine. This seems
>>> like a weird configuration with SQL Client that is either broken with that
>>> Flink helm chart or with SQL Client.
>>>
>>>
>>> Flink SQL> DROP TABLE `default_catalog.mydb.user`;
>>>
>>> [INFO] Table has been removed.
>>>
>>>
>>> Flink SQL> CREATE TABLE `default_catalog.mydb.user` (`platformId`
>>> BIGINT, `userId` STRING) WITH ('connector' = 'filesystem', 'path' =
>>> 's3://mys3bucket/users.csv','format' = 'csv');
>>>
>>> [INFO] Table has been created.
>>>
>>>
>>> Flink SQL> SELECT * FROM `default_catalog.mydb.user` LIMIT 10;
>>>
>>> *[ERROR] Could not execute SQL statement. Reason:*
>>>
>>> *org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException:
>>> connection timed out: flink-jobmanager/10.98.253.58:8081
>>> <http://10.98.253.58:8081>*
>>>
>>> My config logs from jobmanager.
>>>
>>> 2020-09-11 02:33:07,962 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>>> [] - Using predefined options: DEFAULT.
>>>
>>> 2020-09-11 02:33:07,962 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>>> [] - Using default options factory:
>>> DefaultConfigurableOptionsFactory{configuredOptions={}}.
>>>
>>> 2020-09-11 02:33:08,100 INFO
>>> org.apache.flink.table.client.gateway.local.ProgramDeployer [] -
>>> Submitting job org.apache.flink.streaming.api.graph.StreamGraph@3bd08435
>>> for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10`
>>>
>>> 2020-09-11 02:33:08,112 INFO org.apache.flink.configuration.Configuration
>>> [] - Config uses fallback configuration key
>>> 'jobmanager.rpc.address' instead of key 'rest.address'
>>>
>>> tail: log/flink--sql-client-flink-jobmanager-0.log: file truncated
>>>
>>> 2020-09-11 02:34:35,848 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> taskmanager.numberOfTaskSlots, 2
>>>
>>> 2020-09-11 02:34:35,884 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: blob.server.port, 6124
>>>
>>> 2020-09-11 02:34:35,884 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: taskmanager.rpc.port,
>>> 6122
>>>
>>> 2020-09-11 02:34:35,885 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: jobmanager.heap.size, 1g
>>>
>>> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> taskmanager.memory.process.size, 1g
>>>
>>> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: state.backend, rocksdb
>>>
>>> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: state.checkpoints.dir,
>>> file:///flink_state/checkpoints
>>>
>>> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: state.savepoints.dir,
>>> file:///flink_state/savepoints
>>>
>>> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: state.backend.async, true
>>>
>>> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.fs.memory-threshold, 1024
>>>
>>> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.fs.write-buffer-size, 4096
>>>
>>> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.incremental, true
>>>
>>> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.local-recovery, true
>>>
>>> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.checkpoints.num-retained, 1
>>>
>>> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> taskmanager.state.local.root-dirs, file:///flink_state/local-recovery
>>>
>>> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.rocksdb.checkpoint.transfer.thread.num, 1
>>>
>>> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.rocksdb.localdir, /flink_state/rocksdb
>>>
>>> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.rocksdb.options-factory,
>>> org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory
>>>
>>> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.rocksdb.predefined-options, DEFAULT
>>>
>>> 2020-09-11 02:34:35,891 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.rocksdb.timer-service.factory, HEAP
>>>
>>> 2020-09-11 02:34:35,891 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.rocksdb.ttl.compaction.filter.enabled, false
>>>
>>> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: jobmanager.rpc.address,
>>> flink-jobmanager
>>>
>>> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: jobmanager.rpc.port, 6123
>>>
>>> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> taskmanager.memory.jvm-metaspace.size, 256mb
>>>
>>> 2020-09-11 02:34:35,893 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: s3a.endpoint,
>>> http://minio:9000
>>>
>>> 2020-09-11 02:34:35,893 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: s3a.path.style.access,
>>> true
>>>
>>> 2020-09-11 02:34:35,894 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: s3a.access-key,
>>> YOURACCESSKEY
>>>
>>> 2020-09-11 02:34:35,894 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: s3a.secret-key, ******
>>>
>>> 2020-09-11 02:34:35,895 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> s3a.aws.credentials.provider,
>>> org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
>>>
>>> 2020-09-11 02:34:36,492 INFO org.apache.flink.client.cli.CliFrontend
>>> [] - Loading FallbackYarnSessionCli
>>>
>>> 2020-09-11 02:34:36,498 INFO
>>> org.apache.flink.table.client.gateway.local.LocalExecutor [] - Using
>>> default environment file: file:/opt/flink/conf/sql-client-defaults.yaml
>>>
>>> 2020-09-11 02:34:37,786 INFO org.apache.flink.table.client.SqlClient
>>> [] - Using session environment file:
>>> file:/opt/flink/sql-client-defaults.yaml
>>>
>>> 2020-09-11 02:34:37,931 INFO
>>> org.apache.flink.table.client.config.entries.ExecutionEntry [] -
>>> Property 'execution.restart-strategy.type' not specified. Using default
>>> value: fallback
>>>
>>> 2020-09-11 02:34:43,269 INFO org.apache.flink.table.client.gateway.local.ExecutionContext
>>> [] - Executor config: {blob.server.port=6124, s3a.secret-key=YOURSECRETKEY,
>>> state.checkpoints.num-retained=1, s3a.access-key=YOURACCESSKEY,
>>> state.backend.rocksdb.options-factory=org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory,
>>> jobmanager.rpc.address=flink-jobmanager,
>>> taskmanager.memory.jvm-metaspace.size=256mb,
>>> state.backend.rocksdb.predefined-options=DEFAULT,
>>> state.savepoints.dir=file:///flink_state/savepoints,
>>> state.backend.fs.memory-threshold=1024,
>>> state.backend.rocksdb.timer-service.factory=HEAP,
>>> execution.savepoint.ignore-unclaimed-state=false,
>>> taskmanager.numberOfTaskSlots=2, state.backend.async=true,
>>> pipeline.classpaths=[],
>>> state.backend.rocksdb.ttl.compaction.filter.enabled=false,
>>> jobmanager.heap.size=1g, taskmanager.memory.process.size=1g, s3a.endpoint=
>>> http://minio:9000,
>>> state.backend.rocksdb.checkpoint.transfer.thread.num=1,
>>> state.backend.local-recovery=true,
>>> state.backend.rocksdb.localdir=/flink_state/rocksdb,
>>> state.backend.incremental=true, execution.target=remote,
>>> taskmanager.rpc.port=6122, jobmanager.rpc.port=6123,
>>> s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider,
>>> state.backend.fs.write-buffer-size=4096, execution.attached=true,
>>> execution.shutdown-on-attached-exit=false,
>>> pipeline.jars=[file:/opt/flink/opt/flink-sql-client_2.12-1.11.1.jar],
>>> state.backend=rocksdb, s3a.path.style.access=true,
>>> taskmanager.state.local.root-dirs=file:///flink_state/local-recovery,
>>> state.checkpoints.dir=file:///flink_state/checkpoints}
>>>
>>> 2020-09-11 02:34:43,286 INFO org.apache.flink.client.deployment.DefaultClusterClientServiceLoader
>>> [] - Could not load factory due to missing dependencies.
>>>
>>> 2020-09-11 02:34:44,079 INFO
>>> org.apache.flink.table.client.cli.CliClient [] -
>>> Command history file path: /root/.flink-sql-history
>>>
>>> 2020-09-11 02:35:15,396 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>>> [] - Using predefined options: DEFAULT.
>>>
>>> 2020-09-11 02:35:15,397 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>>> [] - Using default options factory:
>>> DefaultConfigurableOptionsFactory{configuredOptions={}}.
>>>
>>> 2020-09-11 02:35:17,529 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> taskmanager.numberOfTaskSlots, 2
>>>
>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: blob.server.port, 6124
>>>
>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: taskmanager.rpc.port,
>>> 6122
>>>
>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: jobmanager.heap.size, 1g
>>>
>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> taskmanager.memory.process.size, 1g
>>>
>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: state.backend, rocksdb
>>>
>>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: state.checkpoints.dir,
>>> file:///flink_state/checkpoints
>>>
>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: state.savepoints.dir,
>>> file:///flink_state/savepoints
>>>
>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: state.backend.async, true
>>>
>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.fs.memory-threshold, 1024
>>>
>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.fs.write-buffer-size, 4096
>>>
>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.incremental, true
>>>
>>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.local-recovery, true
>>>
>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.checkpoints.num-retained, 1
>>>
>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> taskmanager.state.local.root-dirs, file:///flink_state/local-recovery
>>>
>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.rocksdb.checkpoint.transfer.thread.num, 1
>>>
>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.rocksdb.localdir, /flink_state/rocksdb
>>>
>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.rocksdb.options-factory,
>>> org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory
>>>
>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.rocksdb.predefined-options, DEFAULT
>>>
>>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.rocksdb.timer-service.factory, HEAP
>>>
>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> state.backend.rocksdb.ttl.compaction.filter.enabled, false
>>>
>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: jobmanager.rpc.address,
>>> flink-jobmanager
>>>
>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: jobmanager.rpc.port, 6123
>>>
>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> taskmanager.memory.jvm-metaspace.size, 256mb
>>>
>>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: s3a.endpoint,
>>> http://minio:9000
>>>
>>> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: s3a.path.style.access,
>>> true
>>>
>>> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: s3a.access-key,
>>> YOURACCESSKEY
>>>
>>> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property: s3a.secret-key, ******
>>>
>>> 2020-09-11 02:35:17,535 INFO org.apache.flink.configuration.GlobalConfiguration
>>> [] - Loading configuration property:
>>> s3a.aws.credentials.provider,
>>> org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
>>>
>>> 2020-09-11 02:35:18,047 INFO
>>> org.apache.flink.table.client.gateway.local.ProgramDeployer [] -
>>> Submitting job org.apache.flink.streaming.api.graph.StreamGraph@2f95653f
>>> for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10`
>>>
>>> 2020-09-11 02:35:18,382 INFO org.apache.flink.configuration.Configuration
>>> [] - Config uses fallback configuration key
>>> 'jobmanager.rpc.address' instead of key 'rest.address'
>>>
>>> 2020-09-11 02:41:39,539 WARN org.apache.flink.runtime.rest.RestClient
>>> [] - Rest endpoint shutdown failed.
>>>
>>> java.util.concurrent.TimeoutException: null
>>>
>>
--
Arvid Heise | Senior Java Developer
<https://www.ververica.com/>
Follow us @VervericaData
--
Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Toni) Cheng
Re: I hit a bad jobmanager address when trying to use Flink SQL Client
Posted by Dan Hill <qu...@gmail.com>.
Hi Robert!
I have Flink running locally on minikube. I'm running SQL client using
exec on the jobmanager.
kubectl exec pod/flink-jobmanager-0 -i -t -- /opt/flink/bin/sql-client.sh
embedded -e /opt/flink/sql-client-defaults.yaml
Here's the sql-client-defaults.yaml. I didn't specify a session.
execution:
type: batch
result-mode: table
max-table-result-rows: 1000000
I'm prototyping the Table SQL interface. I got blocked using the Table SQL
interface and figured I'd try the SQL Client to see if I could get
unblocked.
On Fri, Sep 11, 2020 at 11:18 AM Robert Metzger <rm...@apache.org> wrote:
> Hi Dan,
>
> the notation of "flink-jobmanager/10.98.253.58:8081" is not a problem. It
> is how java.net.InetAddress stringifies a resolved address (with both
> hostname and IP).
>
> How did you configure the SQL client to work with a Kubernetes Session?
> Afaik this is not a documented, tested and officially supported feature
> (this doesn't mean we should not support it -- apparently it is something
> we should do rather soon ;) ).
>
> Best,
> Robert
>
> On Fri, Sep 11, 2020 at 5:25 AM Dan Hill <qu...@gmail.com> wrote:
>
>> I just tried using the Flink SQL Client. A simple job is not running
>> because it cannot hit jobmanager. I'm not sure why Flink SQL Client is
>> hitting "flink-jobmanager/10.98.253.58:8081". I'd expect either
>> "flink-jobmanager:8081" or "10.98.253.58:8081" (which should work with
>> my kubernetes setup).
>>
>> I'm using riskfocus's Flink helm chart
>> <https://github.com/riskfocus/helm-charts-public/tree/master/flink>.
>>
>> The last SELECT errors out. I have an environment file that indicates
>> this "execution.type: batch". My setup works when using DataSet and
>> DataStream. The jobmanager and taskmanager logs look fine. This seems
>> like a weird configuration with SQL Client that is either broken with that
>> Flink helm chart or with SQL Client.
>>
>>
>> Flink SQL> DROP TABLE `default_catalog.mydb.user`;
>>
>> [INFO] Table has been removed.
>>
>>
>> Flink SQL> CREATE TABLE `default_catalog.mydb.user` (`platformId`
>> BIGINT, `userId` STRING) WITH ('connector' = 'filesystem', 'path' =
>> 's3://mys3bucket/users.csv','format' = 'csv');
>>
>> [INFO] Table has been created.
>>
>>
>> Flink SQL> SELECT * FROM `default_catalog.mydb.user` LIMIT 10;
>>
>> *[ERROR] Could not execute SQL statement. Reason:*
>>
>> *org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException:
>> connection timed out: flink-jobmanager/10.98.253.58:8081
>> <http://10.98.253.58:8081>*
>>
>> My config logs from jobmanager.
>>
>> 2020-09-11 02:33:07,962 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>> [] - Using predefined options: DEFAULT.
>>
>> 2020-09-11 02:33:07,962 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>> [] - Using default options factory:
>> DefaultConfigurableOptionsFactory{configuredOptions={}}.
>>
>> 2020-09-11 02:33:08,100 INFO
>> org.apache.flink.table.client.gateway.local.ProgramDeployer [] -
>> Submitting job org.apache.flink.streaming.api.graph.StreamGraph@3bd08435
>> for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10`
>>
>> 2020-09-11 02:33:08,112 INFO org.apache.flink.configuration.Configuration
>> [] - Config uses fallback configuration key
>> 'jobmanager.rpc.address' instead of key 'rest.address'
>>
>> tail: log/flink--sql-client-flink-jobmanager-0.log: file truncated
>>
>> 2020-09-11 02:34:35,848 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> taskmanager.numberOfTaskSlots, 2
>>
>> 2020-09-11 02:34:35,884 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: blob.server.port, 6124
>>
>> 2020-09-11 02:34:35,884 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: taskmanager.rpc.port, 6122
>>
>> 2020-09-11 02:34:35,885 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: jobmanager.heap.size, 1g
>>
>> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> taskmanager.memory.process.size, 1g
>>
>> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: state.backend, rocksdb
>>
>> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: state.checkpoints.dir,
>> file:///flink_state/checkpoints
>>
>> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: state.savepoints.dir,
>> file:///flink_state/savepoints
>>
>> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: state.backend.async, true
>>
>> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.fs.memory-threshold, 1024
>>
>> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.fs.write-buffer-size, 4096
>>
>> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.incremental, true
>>
>> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.local-recovery, true
>>
>> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.checkpoints.num-retained, 1
>>
>> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> taskmanager.state.local.root-dirs, file:///flink_state/local-recovery
>>
>> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.rocksdb.checkpoint.transfer.thread.num, 1
>>
>> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.rocksdb.localdir, /flink_state/rocksdb
>>
>> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.rocksdb.options-factory,
>> org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory
>>
>> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.rocksdb.predefined-options, DEFAULT
>>
>> 2020-09-11 02:34:35,891 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.rocksdb.timer-service.factory, HEAP
>>
>> 2020-09-11 02:34:35,891 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.rocksdb.ttl.compaction.filter.enabled, false
>>
>> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: jobmanager.rpc.address,
>> flink-jobmanager
>>
>> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: jobmanager.rpc.port, 6123
>>
>> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> taskmanager.memory.jvm-metaspace.size, 256mb
>>
>> 2020-09-11 02:34:35,893 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: s3a.endpoint,
>> http://minio:9000
>>
>> 2020-09-11 02:34:35,893 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: s3a.path.style.access,
>> true
>>
>> 2020-09-11 02:34:35,894 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: s3a.access-key,
>> YOURACCESSKEY
>>
>> 2020-09-11 02:34:35,894 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: s3a.secret-key, ******
>>
>> 2020-09-11 02:34:35,895 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> s3a.aws.credentials.provider,
>> org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
>>
>> 2020-09-11 02:34:36,492 INFO org.apache.flink.client.cli.CliFrontend
>> [] - Loading FallbackYarnSessionCli
>>
>> 2020-09-11 02:34:36,498 INFO
>> org.apache.flink.table.client.gateway.local.LocalExecutor [] - Using
>> default environment file: file:/opt/flink/conf/sql-client-defaults.yaml
>>
>> 2020-09-11 02:34:37,786 INFO org.apache.flink.table.client.SqlClient
>> [] - Using session environment file:
>> file:/opt/flink/sql-client-defaults.yaml
>>
>> 2020-09-11 02:34:37,931 INFO
>> org.apache.flink.table.client.config.entries.ExecutionEntry [] -
>> Property 'execution.restart-strategy.type' not specified. Using default
>> value: fallback
>>
>> 2020-09-11 02:34:43,269 INFO org.apache.flink.table.client.gateway.local.ExecutionContext
>> [] - Executor config: {blob.server.port=6124, s3a.secret-key=YOURSECRETKEY,
>> state.checkpoints.num-retained=1, s3a.access-key=YOURACCESSKEY,
>> state.backend.rocksdb.options-factory=org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory,
>> jobmanager.rpc.address=flink-jobmanager,
>> taskmanager.memory.jvm-metaspace.size=256mb,
>> state.backend.rocksdb.predefined-options=DEFAULT,
>> state.savepoints.dir=file:///flink_state/savepoints,
>> state.backend.fs.memory-threshold=1024,
>> state.backend.rocksdb.timer-service.factory=HEAP,
>> execution.savepoint.ignore-unclaimed-state=false,
>> taskmanager.numberOfTaskSlots=2, state.backend.async=true,
>> pipeline.classpaths=[],
>> state.backend.rocksdb.ttl.compaction.filter.enabled=false,
>> jobmanager.heap.size=1g, taskmanager.memory.process.size=1g, s3a.endpoint=
>> http://minio:9000,
>> state.backend.rocksdb.checkpoint.transfer.thread.num=1,
>> state.backend.local-recovery=true,
>> state.backend.rocksdb.localdir=/flink_state/rocksdb,
>> state.backend.incremental=true, execution.target=remote,
>> taskmanager.rpc.port=6122, jobmanager.rpc.port=6123,
>> s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider,
>> state.backend.fs.write-buffer-size=4096, execution.attached=true,
>> execution.shutdown-on-attached-exit=false,
>> pipeline.jars=[file:/opt/flink/opt/flink-sql-client_2.12-1.11.1.jar],
>> state.backend=rocksdb, s3a.path.style.access=true,
>> taskmanager.state.local.root-dirs=file:///flink_state/local-recovery,
>> state.checkpoints.dir=file:///flink_state/checkpoints}
>>
>> 2020-09-11 02:34:43,286 INFO org.apache.flink.client.deployment.DefaultClusterClientServiceLoader
>> [] - Could not load factory due to missing dependencies.
>>
>> 2020-09-11 02:34:44,079 INFO org.apache.flink.table.client.cli.CliClient
>> [] - Command history file path: /root/.flink-sql-history
>>
>> 2020-09-11 02:35:15,396 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>> [] - Using predefined options: DEFAULT.
>>
>> 2020-09-11 02:35:15,397 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
>> [] - Using default options factory:
>> DefaultConfigurableOptionsFactory{configuredOptions={}}.
>>
>> 2020-09-11 02:35:17,529 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> taskmanager.numberOfTaskSlots, 2
>>
>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: blob.server.port, 6124
>>
>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: taskmanager.rpc.port, 6122
>>
>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: jobmanager.heap.size, 1g
>>
>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> taskmanager.memory.process.size, 1g
>>
>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: state.backend, rocksdb
>>
>> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: state.checkpoints.dir,
>> file:///flink_state/checkpoints
>>
>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: state.savepoints.dir,
>> file:///flink_state/savepoints
>>
>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: state.backend.async, true
>>
>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.fs.memory-threshold, 1024
>>
>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.fs.write-buffer-size, 4096
>>
>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.incremental, true
>>
>> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.local-recovery, true
>>
>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.checkpoints.num-retained, 1
>>
>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> taskmanager.state.local.root-dirs, file:///flink_state/local-recovery
>>
>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.rocksdb.checkpoint.transfer.thread.num, 1
>>
>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.rocksdb.localdir, /flink_state/rocksdb
>>
>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.rocksdb.options-factory,
>> org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory
>>
>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.rocksdb.predefined-options, DEFAULT
>>
>> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.rocksdb.timer-service.factory, HEAP
>>
>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> state.backend.rocksdb.ttl.compaction.filter.enabled, false
>>
>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: jobmanager.rpc.address,
>> flink-jobmanager
>>
>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: jobmanager.rpc.port, 6123
>>
>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> taskmanager.memory.jvm-metaspace.size, 256mb
>>
>> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: s3a.endpoint,
>> http://minio:9000
>>
>> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: s3a.path.style.access,
>> true
>>
>> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: s3a.access-key,
>> YOURACCESSKEY
>>
>> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property: s3a.secret-key, ******
>>
>> 2020-09-11 02:35:17,535 INFO org.apache.flink.configuration.GlobalConfiguration
>> [] - Loading configuration property:
>> s3a.aws.credentials.provider,
>> org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
>>
>> 2020-09-11 02:35:18,047 INFO
>> org.apache.flink.table.client.gateway.local.ProgramDeployer [] -
>> Submitting job org.apache.flink.streaming.api.graph.StreamGraph@2f95653f
>> for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10`
>>
>> 2020-09-11 02:35:18,382 INFO org.apache.flink.configuration.Configuration
>> [] - Config uses fallback configuration key
>> 'jobmanager.rpc.address' instead of key 'rest.address'
>>
>> 2020-09-11 02:41:39,539 WARN org.apache.flink.runtime.rest.RestClient
>> [] - Rest endpoint shutdown failed.
>>
>> java.util.concurrent.TimeoutException: null
>>
>
Re: I hit a bad jobmanager address when trying to use Flink SQL Client
Posted by Robert Metzger <rm...@apache.org>.
Hi Dan,
the notation of "flink-jobmanager/10.98.253.58:8081" is not a problem. It
is how java.net.InetAddress stringifies a resolved address (with both
hostname and IP).
How did you configure the SQL client to work with a Kubernetes Session?
Afaik this is not a documented, tested and officially supported feature
(this doesn't mean we should not support it -- apparently it is something
we should do rather soon ;) ).
Best,
Robert
On Fri, Sep 11, 2020 at 5:25 AM Dan Hill <qu...@gmail.com> wrote:
> I just tried using the Flink SQL Client. A simple job is not running
> because it cannot hit jobmanager. I'm not sure why Flink SQL Client is
> hitting "flink-jobmanager/10.98.253.58:8081". I'd expect either
> "flink-jobmanager:8081" or "10.98.253.58:8081" (which should work with my
> kubernetes setup).
>
> I'm using riskfocus's Flink helm chart
> <https://github.com/riskfocus/helm-charts-public/tree/master/flink>.
>
> The last SELECT errors out. I have an environment file that indicates
> this "execution.type: batch". My setup works when using DataSet and
> DataStream. The jobmanager and taskmanager logs look fine. This seems
> like a weird configuration with SQL Client that is either broken with that
> Flink helm chart or with SQL Client.
>
>
> Flink SQL> DROP TABLE `default_catalog.mydb.user`;
>
> [INFO] Table has been removed.
>
>
> Flink SQL> CREATE TABLE `default_catalog.mydb.user` (`platformId` BIGINT,
> `userId` STRING) WITH ('connector' = 'filesystem', 'path' =
> 's3://mys3bucket/users.csv','format' = 'csv');
>
> [INFO] Table has been created.
>
>
> Flink SQL> SELECT * FROM `default_catalog.mydb.user` LIMIT 10;
>
> *[ERROR] Could not execute SQL statement. Reason:*
>
> *org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException:
> connection timed out: flink-jobmanager/10.98.253.58:8081
> <http://10.98.253.58:8081>*
>
> My config logs from jobmanager.
>
> 2020-09-11 02:33:07,962 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
> [] - Using predefined options: DEFAULT.
>
> 2020-09-11 02:33:07,962 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
> [] - Using default options factory:
> DefaultConfigurableOptionsFactory{configuredOptions={}}.
>
> 2020-09-11 02:33:08,100 INFO
> org.apache.flink.table.client.gateway.local.ProgramDeployer [] -
> Submitting job org.apache.flink.streaming.api.graph.StreamGraph@3bd08435
> for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10`
>
> 2020-09-11 02:33:08,112 INFO org.apache.flink.configuration.Configuration
> [] - Config uses fallback configuration key
> 'jobmanager.rpc.address' instead of key 'rest.address'
>
> tail: log/flink--sql-client-flink-jobmanager-0.log: file truncated
>
> 2020-09-11 02:34:35,848 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> taskmanager.numberOfTaskSlots, 2
>
> 2020-09-11 02:34:35,884 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: blob.server.port, 6124
>
> 2020-09-11 02:34:35,884 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: taskmanager.rpc.port, 6122
>
> 2020-09-11 02:34:35,885 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: jobmanager.heap.size, 1g
>
> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> taskmanager.memory.process.size, 1g
>
> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: state.backend, rocksdb
>
> 2020-09-11 02:34:35,886 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: state.checkpoints.dir,
> file:///flink_state/checkpoints
>
> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: state.savepoints.dir,
> file:///flink_state/savepoints
>
> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: state.backend.async, true
>
> 2020-09-11 02:34:35,887 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.fs.memory-threshold, 1024
>
> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.fs.write-buffer-size, 4096
>
> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: state.backend.incremental,
> true
>
> 2020-09-11 02:34:35,888 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.local-recovery, true
>
> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.checkpoints.num-retained, 1
>
> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> taskmanager.state.local.root-dirs, file:///flink_state/local-recovery
>
> 2020-09-11 02:34:35,889 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.rocksdb.checkpoint.transfer.thread.num, 1
>
> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.rocksdb.localdir, /flink_state/rocksdb
>
> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.rocksdb.options-factory,
> org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory
>
> 2020-09-11 02:34:35,890 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.rocksdb.predefined-options, DEFAULT
>
> 2020-09-11 02:34:35,891 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.rocksdb.timer-service.factory, HEAP
>
> 2020-09-11 02:34:35,891 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.rocksdb.ttl.compaction.filter.enabled, false
>
> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: jobmanager.rpc.address,
> flink-jobmanager
>
> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: jobmanager.rpc.port, 6123
>
> 2020-09-11 02:34:35,892 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> taskmanager.memory.jvm-metaspace.size, 256mb
>
> 2020-09-11 02:34:35,893 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: s3a.endpoint,
> http://minio:9000
>
> 2020-09-11 02:34:35,893 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: s3a.path.style.access, true
>
> 2020-09-11 02:34:35,894 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: s3a.access-key,
> YOURACCESSKEY
>
> 2020-09-11 02:34:35,894 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: s3a.secret-key, ******
>
> 2020-09-11 02:34:35,895 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> s3a.aws.credentials.provider,
> org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
>
> 2020-09-11 02:34:36,492 INFO org.apache.flink.client.cli.CliFrontend
> [] - Loading FallbackYarnSessionCli
>
> 2020-09-11 02:34:36,498 INFO
> org.apache.flink.table.client.gateway.local.LocalExecutor [] - Using
> default environment file: file:/opt/flink/conf/sql-client-defaults.yaml
>
> 2020-09-11 02:34:37,786 INFO org.apache.flink.table.client.SqlClient
> [] - Using session environment file:
> file:/opt/flink/sql-client-defaults.yaml
>
> 2020-09-11 02:34:37,931 INFO
> org.apache.flink.table.client.config.entries.ExecutionEntry [] -
> Property 'execution.restart-strategy.type' not specified. Using default
> value: fallback
>
> 2020-09-11 02:34:43,269 INFO org.apache.flink.table.client.gateway.local.ExecutionContext
> [] - Executor config: {blob.server.port=6124, s3a.secret-key=YOURSECRETKEY,
> state.checkpoints.num-retained=1, s3a.access-key=YOURACCESSKEY,
> state.backend.rocksdb.options-factory=org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory,
> jobmanager.rpc.address=flink-jobmanager,
> taskmanager.memory.jvm-metaspace.size=256mb,
> state.backend.rocksdb.predefined-options=DEFAULT,
> state.savepoints.dir=file:///flink_state/savepoints,
> state.backend.fs.memory-threshold=1024,
> state.backend.rocksdb.timer-service.factory=HEAP,
> execution.savepoint.ignore-unclaimed-state=false,
> taskmanager.numberOfTaskSlots=2, state.backend.async=true,
> pipeline.classpaths=[],
> state.backend.rocksdb.ttl.compaction.filter.enabled=false,
> jobmanager.heap.size=1g, taskmanager.memory.process.size=1g, s3a.endpoint=
> http://minio:9000,
> state.backend.rocksdb.checkpoint.transfer.thread.num=1,
> state.backend.local-recovery=true,
> state.backend.rocksdb.localdir=/flink_state/rocksdb,
> state.backend.incremental=true, execution.target=remote,
> taskmanager.rpc.port=6122, jobmanager.rpc.port=6123,
> s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider,
> state.backend.fs.write-buffer-size=4096, execution.attached=true,
> execution.shutdown-on-attached-exit=false,
> pipeline.jars=[file:/opt/flink/opt/flink-sql-client_2.12-1.11.1.jar],
> state.backend=rocksdb, s3a.path.style.access=true,
> taskmanager.state.local.root-dirs=file:///flink_state/local-recovery,
> state.checkpoints.dir=file:///flink_state/checkpoints}
>
> 2020-09-11 02:34:43,286 INFO org.apache.flink.client.deployment.DefaultClusterClientServiceLoader
> [] - Could not load factory due to missing dependencies.
>
> 2020-09-11 02:34:44,079 INFO org.apache.flink.table.client.cli.CliClient
> [] - Command history file path: /root/.flink-sql-history
>
> 2020-09-11 02:35:15,396 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
> [] - Using predefined options: DEFAULT.
>
> 2020-09-11 02:35:15,397 INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend
> [] - Using default options factory:
> DefaultConfigurableOptionsFactory{configuredOptions={}}.
>
> 2020-09-11 02:35:17,529 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> taskmanager.numberOfTaskSlots, 2
>
> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: blob.server.port, 6124
>
> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: taskmanager.rpc.port, 6122
>
> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: jobmanager.heap.size, 1g
>
> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> taskmanager.memory.process.size, 1g
>
> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: state.backend, rocksdb
>
> 2020-09-11 02:35:17,530 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: state.checkpoints.dir,
> file:///flink_state/checkpoints
>
> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: state.savepoints.dir,
> file:///flink_state/savepoints
>
> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: state.backend.async, true
>
> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.fs.memory-threshold, 1024
>
> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.fs.write-buffer-size, 4096
>
> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: state.backend.incremental,
> true
>
> 2020-09-11 02:35:17,531 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.local-recovery, true
>
> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.checkpoints.num-retained, 1
>
> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> taskmanager.state.local.root-dirs, file:///flink_state/local-recovery
>
> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.rocksdb.checkpoint.transfer.thread.num, 1
>
> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.rocksdb.localdir, /flink_state/rocksdb
>
> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.rocksdb.options-factory,
> org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory
>
> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.rocksdb.predefined-options, DEFAULT
>
> 2020-09-11 02:35:17,532 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.rocksdb.timer-service.factory, HEAP
>
> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> state.backend.rocksdb.ttl.compaction.filter.enabled, false
>
> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: jobmanager.rpc.address,
> flink-jobmanager
>
> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: jobmanager.rpc.port, 6123
>
> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> taskmanager.memory.jvm-metaspace.size, 256mb
>
> 2020-09-11 02:35:17,533 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: s3a.endpoint,
> http://minio:9000
>
> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: s3a.path.style.access, true
>
> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: s3a.access-key,
> YOURACCESSKEY
>
> 2020-09-11 02:35:17,534 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: s3a.secret-key, ******
>
> 2020-09-11 02:35:17,535 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> s3a.aws.credentials.provider,
> org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
>
> 2020-09-11 02:35:18,047 INFO
> org.apache.flink.table.client.gateway.local.ProgramDeployer [] -
> Submitting job org.apache.flink.streaming.api.graph.StreamGraph@2f95653f
> for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10`
>
> 2020-09-11 02:35:18,382 INFO org.apache.flink.configuration.Configuration
> [] - Config uses fallback configuration key
> 'jobmanager.rpc.address' instead of key 'rest.address'
>
> 2020-09-11 02:41:39,539 WARN org.apache.flink.runtime.rest.RestClient
> [] - Rest endpoint shutdown failed.
>
> java.util.concurrent.TimeoutException: null
>