You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@impala.apache.org by Adam Holley <ah...@cloudera.com> on 2018/06/25 18:38:23 UTC
Request for feedback on variable substitution in comments
Hello all,
There is a Jira (https://issues.apache.org/jira/browse/IMPALA-3956) to
change Impala so that variables within comments are not replaced. However,
since Impala can use query hints in comments, fixing this as stated removes
the ability to use hints through variables. Does the community think this
should be fixed for all comments in variables, left alone, fixed in a
manner that would still allow query hints, or some other alternative?
Thanks.
--
*Adam Holley* | Engineering
t. (512) 484- <javascript:void(0);>4060
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
Re: Request for feedback on variable substitution in comments
Posted by Adam Holley <ah...@cloudera.com>.
Would it be acceptable to make this a doc Jira? Add a note in the
documentation (
https://www.cloudera.com/documentation/enterprise/latest/topics/impala_shell_options.html)
to escape the $ if this error is received and the incorrect declaration
needs to stay for whatever reason.
On Tue, Jun 26, 2018 at 10:50 AM Fredy Wijaya <fw...@cloudera.com> wrote:
> I think the confusion is why ${var_name} (without the VAR prefix) is
> considered as an error, especially if we only support ${VAR:var_name}.
> [localhost:21000] default> -- ${hello}
> select 1;
> Error: Unknown substitution syntax (HELLO). Use ${VAR:var_name}
>
> The workaround is rather trivial that is to escape it.
> [localhost:21000] default> -- \${hello}
> select 1;
> Query: -- \${hello}
> select 1
>
> I also agree that unset variables should still be treated as an error
> regardless where they're located whether in comments or non-comments.
> Escaping should be use to ignore variable substitutions.
>
> On Tue, Jun 26, 2018 at 7:36 AM Csaba Ringhofer <cs...@cloudera.com>
> wrote:
>
>> >fixed in a manner that would still allow query hints
>> I would vote for this.
>> A possible solution could be to replace variables in comments only if the
>> variable is set (so no longer treat it as an error). Referencing unset
>> variables in non-comment code should be still treated as an error.
>> This would not break any existing queries, and the only problematic side
>> effect would be that "mistyped variables that should be hints" would be
>> harder to detect, but that should be a very rare issue.
>> I checked Hive shell and replaces it variables only if they are set, even
>> in non-comment code, but I think that it is better to treat this as en
>> error.
>>
>> > fixed for all comments in variables
>> I would not go this way, as it could break some existing use cases. This
>> should be very rare, but as the original issue seems like a minor nuisance
>> to me, I think that it would be better to leave it as it is than to
>> introduce a breaking change.
>>
>> On Mon, Jun 25, 2018 at 9:01 PM, Tim Armstrong <ta...@cloudera.com>
>> wrote:
>>
>>> What is the original motivation for the JIRA? Is there a use case where
>>> the current behaviour is a problem?
>>>
>>> On Mon, Jun 25, 2018 at 11:38 AM, Adam Holley <ah...@cloudera.com>
>>> wrote:
>>>
>>>> Hello all,
>>>>
>>>> There is a Jira (https://issues.apache.org/jira/browse/IMPALA-3956) to
>>>> change Impala so that variables within comments are not replaced. However,
>>>> since Impala can use query hints in comments, fixing this as stated removes
>>>> the ability to use hints through variables. Does the community think this
>>>> should be fixed for all comments in variables, left alone, fixed in a
>>>> manner that would still allow query hints, or some other alternative?
>>>>
>>>> Thanks.
>>>>
>>>>
>>>> --
>>>> *Adam Holley* | Engineering
>>>> t. (512) 484-4060
>>>> cloudera.com <https://www.cloudera.com>
>>>>
>>>> [image: Cloudera] <https://www.cloudera.com/>
>>>>
>>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>>>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>>> ------------------------------
>>>>
>>>
>>>
>>
--
*Adam Holley* | Engineering
t. (512) 484- <javascript:void(0);>4060
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
Re: Request for feedback on variable substitution in comments
Posted by Fredy Wijaya <fw...@cloudera.com>.
I think the confusion is why ${var_name} (without the VAR prefix) is
considered as an error, especially if we only support ${VAR:var_name}.
[localhost:21000] default> -- ${hello}
select 1;
Error: Unknown substitution syntax (HELLO). Use ${VAR:var_name}
The workaround is rather trivial that is to escape it.
[localhost:21000] default> -- \${hello}
select 1;
Query: -- \${hello}
select 1
I also agree that unset variables should still be treated as an error
regardless where they're located whether in comments or non-comments.
Escaping should be use to ignore variable substitutions.
On Tue, Jun 26, 2018 at 7:36 AM Csaba Ringhofer <cs...@cloudera.com>
wrote:
> >fixed in a manner that would still allow query hints
> I would vote for this.
> A possible solution could be to replace variables in comments only if the
> variable is set (so no longer treat it as an error). Referencing unset
> variables in non-comment code should be still treated as an error.
> This would not break any existing queries, and the only problematic side
> effect would be that "mistyped variables that should be hints" would be
> harder to detect, but that should be a very rare issue.
> I checked Hive shell and replaces it variables only if they are set, even
> in non-comment code, but I think that it is better to treat this as en
> error.
>
> > fixed for all comments in variables
> I would not go this way, as it could break some existing use cases. This
> should be very rare, but as the original issue seems like a minor nuisance
> to me, I think that it would be better to leave it as it is than to
> introduce a breaking change.
>
> On Mon, Jun 25, 2018 at 9:01 PM, Tim Armstrong <ta...@cloudera.com>
> wrote:
>
>> What is the original motivation for the JIRA? Is there a use case where
>> the current behaviour is a problem?
>>
>> On Mon, Jun 25, 2018 at 11:38 AM, Adam Holley <ah...@cloudera.com>
>> wrote:
>>
>>> Hello all,
>>>
>>> There is a Jira (https://issues.apache.org/jira/browse/IMPALA-3956) to
>>> change Impala so that variables within comments are not replaced. However,
>>> since Impala can use query hints in comments, fixing this as stated removes
>>> the ability to use hints through variables. Does the community think this
>>> should be fixed for all comments in variables, left alone, fixed in a
>>> manner that would still allow query hints, or some other alternative?
>>>
>>> Thanks.
>>>
>>>
>>> --
>>> *Adam Holley* | Engineering
>>> t. (512) 484-4060
>>> cloudera.com <https://www.cloudera.com>
>>>
>>> [image: Cloudera] <https://www.cloudera.com/>
>>>
>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>> ------------------------------
>>>
>>
>>
>
Re: Request for feedback on variable substitution in comments
Posted by Csaba Ringhofer <cs...@cloudera.com>.
>fixed in a manner that would still allow query hints
I would vote for this.
A possible solution could be to replace variables in comments only if the
variable is set (so no longer treat it as an error). Referencing unset
variables in non-comment code should be still treated as an error.
This would not break any existing queries, and the only problematic side
effect would be that "mistyped variables that should be hints" would be
harder to detect, but that should be a very rare issue.
I checked Hive shell and replaces it variables only if they are set, even
in non-comment code, but I think that it is better to treat this as en
error.
> fixed for all comments in variables
I would not go this way, as it could break some existing use cases. This
should be very rare, but as the original issue seems like a minor nuisance
to me, I think that it would be better to leave it as it is than to
introduce a breaking change.
On Mon, Jun 25, 2018 at 9:01 PM, Tim Armstrong <ta...@cloudera.com>
wrote:
> What is the original motivation for the JIRA? Is there a use case where
> the current behaviour is a problem?
>
> On Mon, Jun 25, 2018 at 11:38 AM, Adam Holley <ah...@cloudera.com>
> wrote:
>
>> Hello all,
>>
>> There is a Jira (https://issues.apache.org/jira/browse/IMPALA-3956) to
>> change Impala so that variables within comments are not replaced. However,
>> since Impala can use query hints in comments, fixing this as stated removes
>> the ability to use hints through variables. Does the community think this
>> should be fixed for all comments in variables, left alone, fixed in a
>> manner that would still allow query hints, or some other alternative?
>>
>> Thanks.
>>
>>
>> --
>> *Adam Holley* | Engineering
>> t. (512) 484-4060
>> cloudera.com <https://www.cloudera.com>
>>
>> [image: Cloudera] <https://www.cloudera.com/>
>>
>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>> ------------------------------
>>
>
>
Re: Request for feedback on variable substitution in comments
Posted by Tim Armstrong <ta...@cloudera.com>.
What is the original motivation for the JIRA? Is there a use case where the
current behaviour is a problem?
On Mon, Jun 25, 2018 at 11:38 AM, Adam Holley <ah...@cloudera.com> wrote:
> Hello all,
>
> There is a Jira (https://issues.apache.org/jira/browse/IMPALA-3956) to
> change Impala so that variables within comments are not replaced. However,
> since Impala can use query hints in comments, fixing this as stated removes
> the ability to use hints through variables. Does the community think this
> should be fixed for all comments in variables, left alone, fixed in a
> manner that would still allow query hints, or some other alternative?
>
> Thanks.
>
>
> --
> *Adam Holley* | Engineering
> t. (512) 484-4060
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------------------------------
>