You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Dipanjan Mazumder <ja...@yahoo.com> on 2019/09/07 05:57:14 UTC
Flink SQL client support for running in Flink cluster
Hi Guys,
I was going through the Flink sql client configuration YAML from the training example and came across a section in the configuration as below:
############################################################ # Deployment properties allow for describing the cluster to which table# programs are submitted to.
deployment: type: standalone # only the 'standalone' deployment is supported response-timeout: 5000 # general cluster communication timeout in ms gateway-address: "" # (optional) address from cluster to gateway gateway-port: 0 # (optional) port from cluster to gateway
##############################################################
The first property of the deployment section "type" says that only standalone deployment of the program is supported , does that mean that Flink SQL client is non functional in when Flink running in Cluster mode. Also i cannot find any documentation for this configuration stating any different value of "deployment.type" for example like "cluster" or something.
I want to implement an API on the SQL client which will support dynamic submission of configurations and thereby sql queries on the fly for the new configuration. If Flink sql is non-functional in Flink cluster then the effort will be not worth because in that case i will have to first see how the program can be made to work in Flink cluster first and then implement the API on top of that.
Please kindly help me on the same , as i am stuck for further changes and i am still not going in to identify if what i see is true because it will take more time to identify through code which can be probably answered faster by your team.
RegardsDipanjan
Re: Flink SQL client support for running in Flink cluster
Posted by Xingcan Cui <xi...@gmail.com>.
Hi Dipanjan,
Sorry that I didn’t make it clear. The ’standalone cluster’ could be fully distributed. It refers to a static cluster deployed without any cluster managers (or other related tools), such as YARN and Kubernetes.
For more information, please check the documentation [1].
Best,
Xingcan
[1] https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/cluster_setup.html <https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/cluster_setup.html>
> On Sep 7, 2019, at 3:30 AM, Dipanjan Mazumder <ja...@yahoo.com> wrote:
>
> Hi Xingcan,
>
> Thanks a lot for this info and this verifies my suspect , so if i want to reuse the SQL client to implement an API based framework , it will not work in Flink cluster running in cluster mode, is there any plan in near future to support that, if its planned is there any expected release event for the same ,because i need to prototype my platform and present the same.
>
> Or is there any other way i can achieve this dynamic configuration and query submission to Flink running as a full fledged cluster.
>
> Regards
> Dipanjan
>
> On Saturday, September 7, 2019, 12:52:02 PM GMT+5:30, Xingcan Cui <xi...@gmail.com> wrote:
>
>
> Hi Dipanjan,
>
> Here the ’standalone’ is short for ’standalone cluster mode’, which means you can setup a standalone Flink cluster and submit your SQL queries (via SQL client) to the cluster.
>
> As far as I know, the current SQL client can only connect to a Flink cluster deployed in standalone mode.
>
> Hope that helps.
>
> Best,
> Xingcan
>
>> On Sep 7, 2019, at 1:57 AM, Dipanjan Mazumder <javahub@yahoo.com <ma...@yahoo.com>> wrote:
>>
>> Hi Guys,
>>
>> I was going through the Flink sql client configuration YAML from the training example and came across a section in the configuration as below:
>>
>> ############################################################
>> # Deployment properties allow for describing the cluster to which table
>> # programs are submitted to.
>>
>> deployment:
>> type: standalone # only the 'standalone' deployment is supported
>> response-timeout: 5000 # general cluster communication timeout in ms
>> gateway-address: "" # (optional) address from cluster to gateway
>> gateway-port: 0 # (optional) port from cluster to gateway
>>
>> ##############################################################
>>
>> The first property of the deployment section "type" says that only standalone deployment of the program is supported , does that mean that Flink SQL client is non functional in when Flink running in Cluster mode. Also i cannot find any documentation for this configuration stating any different value of "deployment.type" for example like "cluster" or something.
>>
>> I want to implement an API on the SQL client which will support dynamic submission of configurations and thereby sql queries on the fly for the new configuration. If Flink sql is non-functional in Flink cluster then the effort will be not worth because in that case i will have to first see how the program can be made to work in Flink cluster first and then implement the API on top of that.
>>
>> Please kindly help me on the same , as i am stuck for further changes and i am still not going in to identify if what i see is true because it will take more time to identify through code which can be probably answered faster by your team.
>>
>> Regards
>> Dipanjan
>>
>
Re: Flink SQL client support for running in Flink cluster
Posted by Dipanjan Mazumder <ja...@yahoo.com>.
Hi Xingcan,
Thanks a lot for this info and this verifies my suspect , so if i want to reuse the SQL client to implement an API based framework , it will not work in Flink cluster running in cluster mode, is there any plan in near future to support that, if its planned is there any expected release event for the same ,because i need to prototype my platform and present the same.
Or is there any other way i can achieve this dynamic configuration and query submission to Flink running as a full fledged cluster.
RegardsDipanjan
On Saturday, September 7, 2019, 12:52:02 PM GMT+5:30, Xingcan Cui <xi...@gmail.com> wrote:
Hi Dipanjan,
Here the ’standalone’ is short for ’standalone cluster mode’, which means you can setup a standalone Flink cluster and submit your SQL queries (via SQL client) to the cluster.
As far as I know, the current SQL client can only connect to a Flink cluster deployed in standalone mode.
Hope that helps.
Best,Xingcan
On Sep 7, 2019, at 1:57 AM, Dipanjan Mazumder <ja...@yahoo.com> wrote:
Hi Guys,
I was going through the Flink sql client configuration YAML from the training example and came across a section in the configuration as below:
############################################################ # Deployment properties allow for describing the cluster to which table# programs are submitted to.
deployment: type: standalone # only the 'standalone' deployment is supported response-timeout: 5000 # general cluster communication timeout in ms gateway-address: "" # (optional) address from cluster to gateway gateway-port: 0 # (optional) port from cluster to gateway
##############################################################
The first property of the deployment section "type" says that only standalone deployment of the program is supported , does that mean that Flink SQL client is non functional in when Flink running in Cluster mode. Also i cannot find any documentation for this configuration stating any different value of "deployment.type" for example like "cluster" or something.
I want to implement an API on the SQL client which will support dynamic submission of configurations and thereby sql queries on the fly for the new configuration. If Flink sql is non-functional in Flink cluster then the effort will be not worth because in that case i will have to first see how the program can be made to work in Flink cluster first and then implement the API on top of that.
Please kindly help me on the same , as i am stuck for further changes and i am still not going in to identify if what i see is true because it will take more time to identify through code which can be probably answered faster by your team.
RegardsDipanjan
Re: Flink SQL client support for running in Flink cluster
Posted by Xingcan Cui <xi...@gmail.com>.
Hi Dipanjan,
Here the ’standalone’ is short for ’standalone cluster mode’, which means you can setup a standalone Flink cluster and submit your SQL queries (via SQL client) to the cluster.
As far as I know, the current SQL client can only connect to a Flink cluster deployed in standalone mode.
Hope that helps.
Best,
Xingcan
> On Sep 7, 2019, at 1:57 AM, Dipanjan Mazumder <ja...@yahoo.com> wrote:
>
> Hi Guys,
>
> I was going through the Flink sql client configuration YAML from the training example and came across a section in the configuration as below:
>
> ############################################################
> # Deployment properties allow for describing the cluster to which table
> # programs are submitted to.
>
> deployment:
> type: standalone # only the 'standalone' deployment is supported
> response-timeout: 5000 # general cluster communication timeout in ms
> gateway-address: "" # (optional) address from cluster to gateway
> gateway-port: 0 # (optional) port from cluster to gateway
>
> ##############################################################
>
> The first property of the deployment section "type" says that only standalone deployment of the program is supported , does that mean that Flink SQL client is non functional in when Flink running in Cluster mode. Also i cannot find any documentation for this configuration stating any different value of "deployment.type" for example like "cluster" or something.
>
> I want to implement an API on the SQL client which will support dynamic submission of configurations and thereby sql queries on the fly for the new configuration. If Flink sql is non-functional in Flink cluster then the effort will be not worth because in that case i will have to first see how the program can be made to work in Flink cluster first and then implement the API on top of that.
>
> Please kindly help me on the same , as i am stuck for further changes and i am still not going in to identify if what i see is true because it will take more time to identify through code which can be probably answered faster by your team.
>
> Regards
> Dipanjan
>