You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/03/18 03:57:14 UTC

[GitHub] [iceberg] RussellSpitzer opened a new issue #2348: Iceberg Extensions Parser Should Allow Generic Expressions as Call Args

RussellSpitzer opened a new issue #2348:
URL: https://github.com/apache/iceberg/issues/2348


   As noted in #2341, we currently only support a very limited set of Spark expressions as call arguments. While internally we overcome this by manually modifying the spark parser code itself, I realize this isn't possible for most extension users.
   
   Ideally I think we should do something like importing a copy of Spark Grammar or some other scheme for delegation of call argument parsing. I'm not sure if this is possible but the procedures have a lot more utility if you can add function calls.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] aokolnychyi commented on issue #2348: Iceberg Extensions Parser Should Allow Generic Expressions as Call Args

Posted by GitBox <gi...@apache.org>.
aokolnychyi commented on issue #2348:
URL: https://github.com/apache/iceberg/issues/2348#issuecomment-803125217


   I am afraid this will be hard to achieve as we need to be able to parse all expressions in our extension parser.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] pan3793 commented on issue #2348: Iceberg Extensions Parser Should Allow Generic Expressions as Call Args

Posted by GitBox <gi...@apache.org>.
pan3793 commented on issue #2348:
URL: https://github.com/apache/iceberg/issues/2348#issuecomment-873822252


   > ... currently only support a very limited set of Spark expressions as call arguments.
   
   Hi, @RussellSpitzer, before finding the final solution to support generic expressions as call args, could you please list the supported expressions now? 
   
   The examples in documents still not work now (iceberg master branch with spark 3.1).
   
   ```
   E, statement: CALL iceberg.system.expire_snapshots(table => 'cdc_wenjuan.member', older_than => now(), retain_last => 1), time taken: 0.004 seconds
   Error: Error operating EXECUTE_STATEMENT: org.apache.spark.sql.catalyst.parser.extensions.IcebergParseException:
   mismatched input '(' expecting STRING(line 1, pos 85)
   
   == SQL ==
   CALL iceberg.system.expire_snapshots(table => 'cdc_wenjuan.member', older_than => now(), retain_last => 1)
   -------------------------------------------------------------------------------------^^^
   
   	at org.apache.spark.sql.catalyst.parser.extensions.IcebergParseException.withCommand(IcebergSparkSqlExtensionsParser.scala:294)
   	at org.apache.spark.sql.catalyst.parser.extensions.IcebergSparkSqlExtensionsParser.parse(IcebergSparkSqlExtensionsParser.scala:158)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] aokolnychyi commented on issue #2348: Iceberg Extensions Parser Should Allow Generic Expressions as Call Args

Posted by GitBox <gi...@apache.org>.
aokolnychyi commented on issue #2348:
URL: https://github.com/apache/iceberg/issues/2348#issuecomment-803133965


   How would we handle changes in the Spark grammar from version to version?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] RussellSpitzer commented on issue #2348: Iceberg Extensions Parser Should Allow Generic Expressions as Call Args

Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on issue #2348:
URL: https://github.com/apache/iceberg/issues/2348#issuecomment-803126081


   Something I considered would be if we just copied the Spark grammer, and then "import it", then our parser would get all the Spark things from that and we could still fail back to the delegate when we don't succeed. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] RussellSpitzer commented on issue #2348: Iceberg Extensions Parser Should Allow Generic Expressions as Call Args

Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on issue #2348:
URL: https://github.com/apache/iceberg/issues/2348#issuecomment-803154483


   I think the only conflicts are when the Spark Grammar invalidates something that was previously allowed, because otherwise we just accept .. maybe I'm missing something there ... extensions could be built to be spark version specific ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org