You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@vxquery.apache.org by Preston Carman <pr...@apache.org> on 2016/05/17 17:27:17 UTC
JSONiq constructors (next step)
As you (JSONiq GSOC students) look at extending the xquery-grammar.jj
[1] file for JSONiq, just remember to check the specification for
specifics [2]. The specification has a nice syntax block at the
beginning of the construction chapter. Also, check out the JavaCC
documentation [3] if you need to catch up on our parser.
Post if you have any questions. Also, post the final pull request for
the data model implementations.
[1] https://github.com/apache/vxquery/blob/master/vxquery-core/src/main/javacc/xquery-grammar.jj
[2] http://jsoniq.org/docs/JSONiqExtensionToXQuery/html-single/index.html#section-json-value-construction
[3] https://javacc.java.net/
Re: JSONiq constructors (next step)
Posted by christina pavlopoulou <cp...@ucr.edu>.
Hello Till,
I think I made the changes that you suggested in [1]. I removed the
tokens, and I put JsonConstructor inside the Constructor. I, also,
changed the previous implementation to accept the Expr() product as it
is in the specs. After the changes, I debug again the code and it seems
to parse it correctly.
[1] https://github.com/apache/vxquery/pull/49
Thank you,
Christina
On 5/29/2016 10:03 PM, Till Westmann wrote:
> Hi Riyafa, hi Christina,
>
> a few ideas/suggestions:
>
> (1) To be able to parse any of the constructors the JsonConstructors rule
> needs to be a production that can be reached from the Expr()
> production - as
> Christina has done in [1] by adding them to the PrimaryExpr() production.
> However, I think that we might want to add the JSONiq constructors as an
> alternative in the Constructor() production instead.
>
> (2) When parsing the ObjectConstructor I think that it might be
> simpler to
> create ASTNodes for the PairConstructors, to return those to from the
> rule
> to parse a single PairConstructor, and to then construct the list of
> pairs
> in the ObjectConstructor (instead of passing the LinkedHashMap down to
> the
> PairConstructors). This might cause a little more object creation/garbage
> collection for now, but I think that the resulting grammar changes
> will be
> simpler.
>
> (3) For the ArrayConstructor it might be simpler to create the rule
> the way
> it is written in the specification as
>
> "[" expr = Expr() "]"
>
> and then (a) build on the fact that an expression in XQuery returns a
> sequence and (b) use that sequence to construct the array. Restricting
> ourselves to Literals seems to cause more pain than benefit in this case.
>
> (4) The js:null literal should not be a special case for Object values.
> Instead, it should be a generally available literal in the Literal()
> production.
>
> (5) The XMLQuery Translator translates the ASTNodes into an Algebricks
> [2]
> plan. The plan consists of operators that (a) work on datasets and
> that (b)
> use functions to work on the individual items of these datasets.
> While constructors are special in the grammar (and have their own
> syntax),
> they are regular functions during query optimization and at runtime.
> So they
> will be translated to function calls in the translator
> (translateComputedElementConstructorNode is an example for this) that are
> usually invoked by an assign-operator.
>
> (6) Please try to keep the formatting in xquery-grammar.jj as close as
> possible to the current productions. We don't have an automated formatter
> for this one.
>
> Thoughts?
> Does this help or confuse?
>
> Cheers,
> Till
>
> [2] http://dl.acm.org/citation.cfm?id=2806941
>
> On 29 May 2016, at 20:17, Christina Pavlopoulou wrote:
>
>> Hi,
>>
>> I made some changes to the array constructor in grammar.jj in [1]. I
>> debug it and I get the array that I put as input, so I guess it
>> parses it correctly. I have the same question as Riyafa, regarding
>> the translator. It is not so clear to me how I can create a translate
>> method.
>>
>> [1] https://github.com/apache/vxquery/pull/48
>>
>> Thank you,
>> Christina
>>
>> On 05/29/2016 07:40 AM, Riyafa Abdul Hameed wrote:
>>> Hi,
>>>
>>> I have created a pull request with the Object constructor
>>> implementation[1]. Since I cannot create a duplicate token of kind
>>> "{", I
>>> have attempted to use <Lbrace>. When I build the code and try to
>>> debug to
>>> see if the parser recognizes an object, a parserexception is thrown.
>>> I have
>>> also tried using <LbraceExprEnclosure>. But as I understand both are
>>> not
>>> START_TAGs. How can I change the grammar so that token "{" would be
>>> recognized as a START_TAG to be processed by the MainModule() of the
>>> parser.
>>>
>>> Also a translate method for object constructor in the class
>>> XMLQueryTranslator needs to be created. I am not sure I understand
>>> how that
>>> should be done. Kindly provide any guidance.
>>>
>>> [1] https://github.com/apache/vxquery/pull/47
>>>
>>> Thank you.
>>>
>>> Yours sincerely,
>>> Riyafa
>>>
>>> On 24 May 2016 at 00:03, Riyafa Abdul Hameed <ri...@cse.mrt.ac.lk>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have been attempting to extend the xquery-grammar.jj file to support
>>>> JSONiq object constructor[1]. As I understand we need to create
>>>> classes
>>>> that extends ASTNode class in the package
>>>> org.apache.vxquery.xmlquery.ast
>>>> to be able to return ASTNode objects by the
>>>> ObjectConstructor--currently it
>>>> returns nothing. I hope it needs to be discussed what classes to be
>>>> created. Correct me if I am mistaken.
>>>>
>>>> [1] https://github.com/apache/vxquery/pull/44
>>>> <https://github.com/apache/vxquery/pull/40>
>>>>
>>>> Thank you.
>>>>
>>>> Yours sincerely,
>>>> Riyafa
>>>>
>>>> On 17 May 2016 at 22:57, Preston Carman <pr...@apache.org> wrote:
>>>>
>>>>> As you (JSONiq GSOC students) look at extending the xquery-grammar.jj
>>>>> [1] file for JSONiq, just remember to check the specification for
>>>>> specifics [2]. The specification has a nice syntax block at the
>>>>> beginning of the construction chapter. Also, check out the JavaCC
>>>>> documentation [3] if you need to catch up on our parser.
>>>>>
>>>>> Post if you have any questions. Also, post the final pull request for
>>>>> the data model implementations.
>>>>>
>>>>> [1]
>>>>> https://github.com/apache/vxquery/blob/master/vxquery-core/src/main/javacc/xquery-grammar.jj
>>>>>
>>>>> [2]
>>>>> http://jsoniq.org/docs/JSONiqExtensionToXQuery/html-single/index.html#section-json-value-construction
>>>>>
>>>>> [3] https://javacc.java.net/
>>>>>
>>>>
>>>>
>>>> --
>>>> Riyafa Abdul Hameed
>>>> Undergraduate, University of Moratuwa
>>>>
>>>> Email: riyafa.12@cse.mrt.ac.lk
>>>> Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/>
>>>> <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa>
>>>> <http://twitter.com/Riyafa1>
>>>>
>>>
>>>
Re: JSONiq constructors (next step)
Posted by Riyafa Abdul Hameed <ri...@cse.mrt.ac.lk>.
Hi Till,
Thank you. Yes it makes sense. I have committed the changes [1].
[1] https://github.com/apache/vxquery/pull/47
Thanks again.
Yours sincerely,
Riyafa
On 31 May 2016 at 01:36, Till Westmann <ti...@apache.org> wrote:
> Hi Riyafa,
>
> thinking a little more about it, it would actually be better not to do the
> duplicate checking for the keys in the parser (or the translator) at all
> and
> to instead defer it to runtime. I.e. we would just create an
> ObjectConstrutor ASTNode with a list of PairConstructor ASTNodes for now.
>
> The reason is, that at some point we will be able to compute the key values
> at runtime (as we’ll support an arbitrary expression for the key). If we
> compute them at runtime, there is not way to find out if a key is a
> duplicate until we’ve actually computed it.
>
> Does this make sense?
>
> Cheers,
> Till
>
>
> On 30 May 2016, at 8:17, Riyafa Abdul Hameed wrote:
>
> Hi Till,
>>
>> Thank you for the suggestions. I have committed the changes based on them
>> except for creating ASTNodes for the PairConstructors--in this case I have
>> been thinking that when creating a list of the PairConstructors in the
>> ObjectConstructor I will have to iterate through the PairConstructors to
>> check whether the keys (each one appearing as a field in the
>> PairConstructor) are equal before throwing an exception. But if a Map is
>> used I can maintain a unique set of keys and throw an exception as soon as
>> one is detected.
>>
>> Shall I proceed to create ASTNodes for the PairConstructors?
>>
>> Thanks again.
>>
>> Yours sincerely,
>> Riyafa
>>
>> On 30 May 2016 at 10:33, Till Westmann <ti...@apache.org> wrote:
>>
>> Hi Riyafa, hi Christina,
>>>
>>> a few ideas/suggestions:
>>>
>>> (1) To be able to parse any of the constructors the JsonConstructors rule
>>> needs to be a production that can be reached from the Expr() production -
>>> as
>>> Christina has done in [1] by adding them to the PrimaryExpr() production.
>>> However, I think that we might want to add the JSONiq constructors as an
>>> alternative in the Constructor() production instead.
>>>
>>> (2) When parsing the ObjectConstructor I think that it might be simpler
>>> to
>>> create ASTNodes for the PairConstructors, to return those to from the
>>> rule
>>> to parse a single PairConstructor, and to then construct the list of
>>> pairs
>>> in the ObjectConstructor (instead of passing the LinkedHashMap down to
>>> the
>>> PairConstructors). This might cause a little more object creation/garbage
>>> collection for now, but I think that the resulting grammar changes will
>>> be
>>> simpler.
>>>
>>> (3) For the ArrayConstructor it might be simpler to create the rule the
>>> way
>>> it is written in the specification as
>>>
>>> "[" expr = Expr() "]"
>>>
>>> and then (a) build on the fact that an expression in XQuery returns a
>>> sequence and (b) use that sequence to construct the array. Restricting
>>> ourselves to Literals seems to cause more pain than benefit in this case.
>>>
>>> (4) The js:null literal should not be a special case for Object values.
>>> Instead, it should be a generally available literal in the Literal()
>>> production.
>>>
>>> (5) The XMLQuery Translator translates the ASTNodes into an Algebricks
>>> [2]
>>> plan. The plan consists of operators that (a) work on datasets and that
>>> (b)
>>> use functions to work on the individual items of these datasets.
>>> While constructors are special in the grammar (and have their own
>>> syntax),
>>> they are regular functions during query optimization and at runtime. So
>>> they
>>> will be translated to function calls in the translator
>>> (translateComputedElementConstructorNode is an example for this) that are
>>> usually invoked by an assign-operator.
>>>
>>> (6) Please try to keep the formatting in xquery-grammar.jj as close as
>>> possible to the current productions. We don't have an automated formatter
>>> for this one.
>>>
>>> Thoughts?
>>> Does this help or confuse?
>>>
>>> Cheers,
>>> Till
>>>
>>> [2] http://dl.acm.org/citation.cfm?id=2806941
>>>
>>>
>>> On 29 May 2016, at 20:17, Christina Pavlopoulou wrote:
>>>
>>> Hi,
>>>
>>>>
>>>> I made some changes to the array constructor in grammar.jj in [1]. I
>>>> debug it and I get the array that I put as input, so I guess it parses
>>>> it
>>>> correctly. I have the same question as Riyafa, regarding the
>>>> translator. It
>>>> is not so clear to me how I can create a translate method.
>>>>
>>>> [1] https://github.com/apache/vxquery/pull/48
>>>>
>>>> Thank you,
>>>> Christina
>>>>
>>>> On 05/29/2016 07:40 AM, Riyafa Abdul Hameed wrote:
>>>>
>>>> Hi,
>>>>>
>>>>> I have created a pull request with the Object constructor
>>>>> implementation[1]. Since I cannot create a duplicate token of kind
>>>>> "{", I
>>>>> have attempted to use <Lbrace>. When I build the code and try to debug
>>>>> to
>>>>> see if the parser recognizes an object, a parserexception is thrown. I
>>>>> have
>>>>> also tried using <LbraceExprEnclosure>. But as I understand both are
>>>>> not
>>>>> START_TAGs. How can I change the grammar so that token "{" would be
>>>>> recognized as a START_TAG to be processed by the MainModule() of the
>>>>> parser.
>>>>>
>>>>> Also a translate method for object constructor in the class
>>>>> XMLQueryTranslator needs to be created. I am not sure I understand how
>>>>> that
>>>>> should be done. Kindly provide any guidance.
>>>>>
>>>>> [1] https://github.com/apache/vxquery/pull/47
>>>>>
>>>>> Thank you.
>>>>>
>>>>> Yours sincerely,
>>>>> Riyafa
>>>>>
>>>>> On 24 May 2016 at 00:03, Riyafa Abdul Hameed <ri...@cse.mrt.ac.lk>
>>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>>>
>>>>>> I have been attempting to extend the xquery-grammar.jj file to support
>>>>>> JSONiq object constructor[1]. As I understand we need to create
>>>>>> classes
>>>>>> that extends ASTNode class in the package
>>>>>> org.apache.vxquery.xmlquery.ast
>>>>>> to be able to return ASTNode objects by the
>>>>>> ObjectConstructor--currently it
>>>>>> returns nothing. I hope it needs to be discussed what classes to be
>>>>>> created. Correct me if I am mistaken.
>>>>>>
>>>>>> [1] https://github.com/apache/vxquery/pull/44
>>>>>> <https://github.com/apache/vxquery/pull/40>
>>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>> Yours sincerely,
>>>>>> Riyafa
>>>>>>
>>>>>> On 17 May 2016 at 22:57, Preston Carman <pr...@apache.org> wrote:
>>>>>>
>>>>>> As you (JSONiq GSOC students) look at extending the xquery-grammar.jj
>>>>>>
>>>>>>> [1] file for JSONiq, just remember to check the specification for
>>>>>>> specifics [2]. The specification has a nice syntax block at the
>>>>>>> beginning of the construction chapter. Also, check out the JavaCC
>>>>>>> documentation [3] if you need to catch up on our parser.
>>>>>>>
>>>>>>> Post if you have any questions. Also, post the final pull request for
>>>>>>> the data model implementations.
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>>
>>>>>>> https://github.com/apache/vxquery/blob/master/vxquery-core/src/main/javacc/xquery-grammar.jj
>>>>>>> [2]
>>>>>>>
>>>>>>>
>>>>>>> http://jsoniq.org/docs/JSONiqExtensionToXQuery/html-single/index.html#section-json-value-construction
>>>>>>> [3] https://javacc.java.net/
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Riyafa Abdul Hameed
>>>>>> Undergraduate, University of Moratuwa
>>>>>>
>>>>>> Email: riyafa.12@cse.mrt.ac.lk
>>>>>> Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/>
>>>>>> <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa>
>>>>>> <http://twitter.com/Riyafa1>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>
>> --
>> Riyafa Abdul Hameed
>> Undergraduate, University of Moratuwa
>>
>> Email: riyafa.12@cse.mrt.ac.lk
>> Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/>
>> <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa>
>> <http://twitter.com/Riyafa1>
>>
>
--
Riyafa Abdul Hameed
Undergraduate, University of Moratuwa
Email: riyafa.12@cse.mrt.ac.lk
Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/>
<http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa>
<http://twitter.com/Riyafa1>
Re: JSONiq constructors (next step)
Posted by Till Westmann <ti...@apache.org>.
Hi Riyafa,
thinking a little more about it, it would actually be better not to do
the
duplicate checking for the keys in the parser (or the translator) at all
and
to instead defer it to runtime. I.e. we would just create an
ObjectConstrutor ASTNode with a list of PairConstructor ASTNodes for
now.
The reason is, that at some point we will be able to compute the key
values
at runtime (as we’ll support an arbitrary expression for the key). If
we
compute them at runtime, there is not way to find out if a key is a
duplicate until we’ve actually computed it.
Does this make sense?
Cheers,
Till
On 30 May 2016, at 8:17, Riyafa Abdul Hameed wrote:
> Hi Till,
>
> Thank you for the suggestions. I have committed the changes based on
> them
> except for creating ASTNodes for the PairConstructors--in this case I
> have
> been thinking that when creating a list of the PairConstructors in the
> ObjectConstructor I will have to iterate through the PairConstructors
> to
> check whether the keys (each one appearing as a field in the
> PairConstructor) are equal before throwing an exception. But if a Map
> is
> used I can maintain a unique set of keys and throw an exception as
> soon as
> one is detected.
>
> Shall I proceed to create ASTNodes for the PairConstructors?
>
> Thanks again.
>
> Yours sincerely,
> Riyafa
>
> On 30 May 2016 at 10:33, Till Westmann <ti...@apache.org> wrote:
>
>> Hi Riyafa, hi Christina,
>>
>> a few ideas/suggestions:
>>
>> (1) To be able to parse any of the constructors the JsonConstructors
>> rule
>> needs to be a production that can be reached from the Expr()
>> production -
>> as
>> Christina has done in [1] by adding them to the PrimaryExpr()
>> production.
>> However, I think that we might want to add the JSONiq constructors as
>> an
>> alternative in the Constructor() production instead.
>>
>> (2) When parsing the ObjectConstructor I think that it might be
>> simpler to
>> create ASTNodes for the PairConstructors, to return those to from the
>> rule
>> to parse a single PairConstructor, and to then construct the list of
>> pairs
>> in the ObjectConstructor (instead of passing the LinkedHashMap down
>> to the
>> PairConstructors). This might cause a little more object
>> creation/garbage
>> collection for now, but I think that the resulting grammar changes
>> will be
>> simpler.
>>
>> (3) For the ArrayConstructor it might be simpler to create the rule
>> the way
>> it is written in the specification as
>>
>> "[" expr = Expr() "]"
>>
>> and then (a) build on the fact that an expression in XQuery returns a
>> sequence and (b) use that sequence to construct the array.
>> Restricting
>> ourselves to Literals seems to cause more pain than benefit in this
>> case.
>>
>> (4) The js:null literal should not be a special case for Object
>> values.
>> Instead, it should be a generally available literal in the Literal()
>> production.
>>
>> (5) The XMLQuery Translator translates the ASTNodes into an
>> Algebricks [2]
>> plan. The plan consists of operators that (a) work on datasets and
>> that (b)
>> use functions to work on the individual items of these datasets.
>> While constructors are special in the grammar (and have their own
>> syntax),
>> they are regular functions during query optimization and at runtime.
>> So
>> they
>> will be translated to function calls in the translator
>> (translateComputedElementConstructorNode is an example for this) that
>> are
>> usually invoked by an assign-operator.
>>
>> (6) Please try to keep the formatting in xquery-grammar.jj as close
>> as
>> possible to the current productions. We don't have an automated
>> formatter
>> for this one.
>>
>> Thoughts?
>> Does this help or confuse?
>>
>> Cheers,
>> Till
>>
>> [2] http://dl.acm.org/citation.cfm?id=2806941
>>
>>
>> On 29 May 2016, at 20:17, Christina Pavlopoulou wrote:
>>
>> Hi,
>>>
>>> I made some changes to the array constructor in grammar.jj in [1]. I
>>> debug it and I get the array that I put as input, so I guess it
>>> parses it
>>> correctly. I have the same question as Riyafa, regarding the
>>> translator. It
>>> is not so clear to me how I can create a translate method.
>>>
>>> [1] https://github.com/apache/vxquery/pull/48
>>>
>>> Thank you,
>>> Christina
>>>
>>> On 05/29/2016 07:40 AM, Riyafa Abdul Hameed wrote:
>>>
>>>> Hi,
>>>>
>>>> I have created a pull request with the Object constructor
>>>> implementation[1]. Since I cannot create a duplicate token of kind
>>>> "{", I
>>>> have attempted to use <Lbrace>. When I build the code and try to
>>>> debug to
>>>> see if the parser recognizes an object, a parserexception is
>>>> thrown. I
>>>> have
>>>> also tried using <LbraceExprEnclosure>. But as I understand both
>>>> are not
>>>> START_TAGs. How can I change the grammar so that token "{" would be
>>>> recognized as a START_TAG to be processed by the MainModule() of
>>>> the
>>>> parser.
>>>>
>>>> Also a translate method for object constructor in the class
>>>> XMLQueryTranslator needs to be created. I am not sure I understand
>>>> how
>>>> that
>>>> should be done. Kindly provide any guidance.
>>>>
>>>> [1] https://github.com/apache/vxquery/pull/47
>>>>
>>>> Thank you.
>>>>
>>>> Yours sincerely,
>>>> Riyafa
>>>>
>>>> On 24 May 2016 at 00:03, Riyafa Abdul Hameed
>>>> <ri...@cse.mrt.ac.lk>
>>>> wrote:
>>>>
>>>> Hi,
>>>>>
>>>>> I have been attempting to extend the xquery-grammar.jj file to
>>>>> support
>>>>> JSONiq object constructor[1]. As I understand we need to create
>>>>> classes
>>>>> that extends ASTNode class in the package
>>>>> org.apache.vxquery.xmlquery.ast
>>>>> to be able to return ASTNode objects by the
>>>>> ObjectConstructor--currently it
>>>>> returns nothing. I hope it needs to be discussed what classes to
>>>>> be
>>>>> created. Correct me if I am mistaken.
>>>>>
>>>>> [1] https://github.com/apache/vxquery/pull/44
>>>>> <https://github.com/apache/vxquery/pull/40>
>>>>>
>>>>> Thank you.
>>>>>
>>>>> Yours sincerely,
>>>>> Riyafa
>>>>>
>>>>> On 17 May 2016 at 22:57, Preston Carman <pr...@apache.org>
>>>>> wrote:
>>>>>
>>>>> As you (JSONiq GSOC students) look at extending the
>>>>> xquery-grammar.jj
>>>>>> [1] file for JSONiq, just remember to check the specification for
>>>>>> specifics [2]. The specification has a nice syntax block at the
>>>>>> beginning of the construction chapter. Also, check out the JavaCC
>>>>>> documentation [3] if you need to catch up on our parser.
>>>>>>
>>>>>> Post if you have any questions. Also, post the final pull request
>>>>>> for
>>>>>> the data model implementations.
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> https://github.com/apache/vxquery/blob/master/vxquery-core/src/main/javacc/xquery-grammar.jj
>>>>>> [2]
>>>>>>
>>>>>> http://jsoniq.org/docs/JSONiqExtensionToXQuery/html-single/index.html#section-json-value-construction
>>>>>> [3] https://javacc.java.net/
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Riyafa Abdul Hameed
>>>>> Undergraduate, University of Moratuwa
>>>>>
>>>>> Email: riyafa.12@cse.mrt.ac.lk
>>>>> Website: https://riyafa.wordpress.com/
>>>>> <http://riyafa.wordpress.com/>
>>>>> <http://facebook.com/riyafa.ahf>
>>>>> <http://lk.linkedin.com/in/riyafa>
>>>>> <http://twitter.com/Riyafa1>
>>>>>
>>>>>
>>>>
>>>>
>
>
> --
> Riyafa Abdul Hameed
> Undergraduate, University of Moratuwa
>
> Email: riyafa.12@cse.mrt.ac.lk
> Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/>
> <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa>
> <http://twitter.com/Riyafa1>
Re: JSONiq constructors (next step)
Posted by Riyafa Abdul Hameed <ri...@cse.mrt.ac.lk>.
Hi Till,
Thank you for the suggestions. I have committed the changes based on them
except for creating ASTNodes for the PairConstructors--in this case I have
been thinking that when creating a list of the PairConstructors in the
ObjectConstructor I will have to iterate through the PairConstructors to
check whether the keys (each one appearing as a field in the
PairConstructor) are equal before throwing an exception. But if a Map is
used I can maintain a unique set of keys and throw an exception as soon as
one is detected.
Shall I proceed to create ASTNodes for the PairConstructors?
Thanks again.
Yours sincerely,
Riyafa
On 30 May 2016 at 10:33, Till Westmann <ti...@apache.org> wrote:
> Hi Riyafa, hi Christina,
>
> a few ideas/suggestions:
>
> (1) To be able to parse any of the constructors the JsonConstructors rule
> needs to be a production that can be reached from the Expr() production -
> as
> Christina has done in [1] by adding them to the PrimaryExpr() production.
> However, I think that we might want to add the JSONiq constructors as an
> alternative in the Constructor() production instead.
>
> (2) When parsing the ObjectConstructor I think that it might be simpler to
> create ASTNodes for the PairConstructors, to return those to from the rule
> to parse a single PairConstructor, and to then construct the list of pairs
> in the ObjectConstructor (instead of passing the LinkedHashMap down to the
> PairConstructors). This might cause a little more object creation/garbage
> collection for now, but I think that the resulting grammar changes will be
> simpler.
>
> (3) For the ArrayConstructor it might be simpler to create the rule the way
> it is written in the specification as
>
> "[" expr = Expr() "]"
>
> and then (a) build on the fact that an expression in XQuery returns a
> sequence and (b) use that sequence to construct the array. Restricting
> ourselves to Literals seems to cause more pain than benefit in this case.
>
> (4) The js:null literal should not be a special case for Object values.
> Instead, it should be a generally available literal in the Literal()
> production.
>
> (5) The XMLQuery Translator translates the ASTNodes into an Algebricks [2]
> plan. The plan consists of operators that (a) work on datasets and that (b)
> use functions to work on the individual items of these datasets.
> While constructors are special in the grammar (and have their own syntax),
> they are regular functions during query optimization and at runtime. So
> they
> will be translated to function calls in the translator
> (translateComputedElementConstructorNode is an example for this) that are
> usually invoked by an assign-operator.
>
> (6) Please try to keep the formatting in xquery-grammar.jj as close as
> possible to the current productions. We don't have an automated formatter
> for this one.
>
> Thoughts?
> Does this help or confuse?
>
> Cheers,
> Till
>
> [2] http://dl.acm.org/citation.cfm?id=2806941
>
>
> On 29 May 2016, at 20:17, Christina Pavlopoulou wrote:
>
> Hi,
>>
>> I made some changes to the array constructor in grammar.jj in [1]. I
>> debug it and I get the array that I put as input, so I guess it parses it
>> correctly. I have the same question as Riyafa, regarding the translator. It
>> is not so clear to me how I can create a translate method.
>>
>> [1] https://github.com/apache/vxquery/pull/48
>>
>> Thank you,
>> Christina
>>
>> On 05/29/2016 07:40 AM, Riyafa Abdul Hameed wrote:
>>
>>> Hi,
>>>
>>> I have created a pull request with the Object constructor
>>> implementation[1]. Since I cannot create a duplicate token of kind "{", I
>>> have attempted to use <Lbrace>. When I build the code and try to debug to
>>> see if the parser recognizes an object, a parserexception is thrown. I
>>> have
>>> also tried using <LbraceExprEnclosure>. But as I understand both are not
>>> START_TAGs. How can I change the grammar so that token "{" would be
>>> recognized as a START_TAG to be processed by the MainModule() of the
>>> parser.
>>>
>>> Also a translate method for object constructor in the class
>>> XMLQueryTranslator needs to be created. I am not sure I understand how
>>> that
>>> should be done. Kindly provide any guidance.
>>>
>>> [1] https://github.com/apache/vxquery/pull/47
>>>
>>> Thank you.
>>>
>>> Yours sincerely,
>>> Riyafa
>>>
>>> On 24 May 2016 at 00:03, Riyafa Abdul Hameed <ri...@cse.mrt.ac.lk>
>>> wrote:
>>>
>>> Hi,
>>>>
>>>> I have been attempting to extend the xquery-grammar.jj file to support
>>>> JSONiq object constructor[1]. As I understand we need to create classes
>>>> that extends ASTNode class in the package
>>>> org.apache.vxquery.xmlquery.ast
>>>> to be able to return ASTNode objects by the
>>>> ObjectConstructor--currently it
>>>> returns nothing. I hope it needs to be discussed what classes to be
>>>> created. Correct me if I am mistaken.
>>>>
>>>> [1] https://github.com/apache/vxquery/pull/44
>>>> <https://github.com/apache/vxquery/pull/40>
>>>>
>>>> Thank you.
>>>>
>>>> Yours sincerely,
>>>> Riyafa
>>>>
>>>> On 17 May 2016 at 22:57, Preston Carman <pr...@apache.org> wrote:
>>>>
>>>> As you (JSONiq GSOC students) look at extending the xquery-grammar.jj
>>>>> [1] file for JSONiq, just remember to check the specification for
>>>>> specifics [2]. The specification has a nice syntax block at the
>>>>> beginning of the construction chapter. Also, check out the JavaCC
>>>>> documentation [3] if you need to catch up on our parser.
>>>>>
>>>>> Post if you have any questions. Also, post the final pull request for
>>>>> the data model implementations.
>>>>>
>>>>> [1]
>>>>>
>>>>> https://github.com/apache/vxquery/blob/master/vxquery-core/src/main/javacc/xquery-grammar.jj
>>>>> [2]
>>>>>
>>>>> http://jsoniq.org/docs/JSONiqExtensionToXQuery/html-single/index.html#section-json-value-construction
>>>>> [3] https://javacc.java.net/
>>>>>
>>>>>
>>>>
>>>> --
>>>> Riyafa Abdul Hameed
>>>> Undergraduate, University of Moratuwa
>>>>
>>>> Email: riyafa.12@cse.mrt.ac.lk
>>>> Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/>
>>>> <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa>
>>>> <http://twitter.com/Riyafa1>
>>>>
>>>>
>>>
>>>
--
Riyafa Abdul Hameed
Undergraduate, University of Moratuwa
Email: riyafa.12@cse.mrt.ac.lk
Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/>
<http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa>
<http://twitter.com/Riyafa1>
Re: JSONiq constructors (next step)
Posted by Till Westmann <ti...@apache.org>.
Hi Riyafa, hi Christina,
a few ideas/suggestions:
(1) To be able to parse any of the constructors the JsonConstructors
rule
needs to be a production that can be reached from the Expr() production
- as
Christina has done in [1] by adding them to the PrimaryExpr()
production.
However, I think that we might want to add the JSONiq constructors as an
alternative in the Constructor() production instead.
(2) When parsing the ObjectConstructor I think that it might be simpler
to
create ASTNodes for the PairConstructors, to return those to from the
rule
to parse a single PairConstructor, and to then construct the list of
pairs
in the ObjectConstructor (instead of passing the LinkedHashMap down to
the
PairConstructors). This might cause a little more object
creation/garbage
collection for now, but I think that the resulting grammar changes will
be
simpler.
(3) For the ArrayConstructor it might be simpler to create the rule the
way
it is written in the specification as
"[" expr = Expr() "]"
and then (a) build on the fact that an expression in XQuery returns a
sequence and (b) use that sequence to construct the array. Restricting
ourselves to Literals seems to cause more pain than benefit in this
case.
(4) The js:null literal should not be a special case for Object values.
Instead, it should be a generally available literal in the Literal()
production.
(5) The XMLQuery Translator translates the ASTNodes into an Algebricks
[2]
plan. The plan consists of operators that (a) work on datasets and that
(b)
use functions to work on the individual items of these datasets.
While constructors are special in the grammar (and have their own
syntax),
they are regular functions during query optimization and at runtime. So
they
will be translated to function calls in the translator
(translateComputedElementConstructorNode is an example for this) that
are
usually invoked by an assign-operator.
(6) Please try to keep the formatting in xquery-grammar.jj as close as
possible to the current productions. We don't have an automated
formatter
for this one.
Thoughts?
Does this help or confuse?
Cheers,
Till
[2] http://dl.acm.org/citation.cfm?id=2806941
On 29 May 2016, at 20:17, Christina Pavlopoulou wrote:
> Hi,
>
> I made some changes to the array constructor in grammar.jj in [1]. I
> debug it and I get the array that I put as input, so I guess it parses
> it correctly. I have the same question as Riyafa, regarding the
> translator. It is not so clear to me how I can create a translate
> method.
>
> [1] https://github.com/apache/vxquery/pull/48
>
> Thank you,
> Christina
>
> On 05/29/2016 07:40 AM, Riyafa Abdul Hameed wrote:
>> Hi,
>>
>> I have created a pull request with the Object constructor
>> implementation[1]. Since I cannot create a duplicate token of kind
>> "{", I
>> have attempted to use <Lbrace>. When I build the code and try to
>> debug to
>> see if the parser recognizes an object, a parserexception is thrown.
>> I have
>> also tried using <LbraceExprEnclosure>. But as I understand both are
>> not
>> START_TAGs. How can I change the grammar so that token "{" would be
>> recognized as a START_TAG to be processed by the MainModule() of the
>> parser.
>>
>> Also a translate method for object constructor in the class
>> XMLQueryTranslator needs to be created. I am not sure I understand
>> how that
>> should be done. Kindly provide any guidance.
>>
>> [1] https://github.com/apache/vxquery/pull/47
>>
>> Thank you.
>>
>> Yours sincerely,
>> Riyafa
>>
>> On 24 May 2016 at 00:03, Riyafa Abdul Hameed
>> <ri...@cse.mrt.ac.lk>
>> wrote:
>>
>>> Hi,
>>>
>>> I have been attempting to extend the xquery-grammar.jj file to
>>> support
>>> JSONiq object constructor[1]. As I understand we need to create
>>> classes
>>> that extends ASTNode class in the package
>>> org.apache.vxquery.xmlquery.ast
>>> to be able to return ASTNode objects by the
>>> ObjectConstructor--currently it
>>> returns nothing. I hope it needs to be discussed what classes to be
>>> created. Correct me if I am mistaken.
>>>
>>> [1] https://github.com/apache/vxquery/pull/44
>>> <https://github.com/apache/vxquery/pull/40>
>>>
>>> Thank you.
>>>
>>> Yours sincerely,
>>> Riyafa
>>>
>>> On 17 May 2016 at 22:57, Preston Carman <pr...@apache.org> wrote:
>>>
>>>> As you (JSONiq GSOC students) look at extending the
>>>> xquery-grammar.jj
>>>> [1] file for JSONiq, just remember to check the specification for
>>>> specifics [2]. The specification has a nice syntax block at the
>>>> beginning of the construction chapter. Also, check out the JavaCC
>>>> documentation [3] if you need to catch up on our parser.
>>>>
>>>> Post if you have any questions. Also, post the final pull request
>>>> for
>>>> the data model implementations.
>>>>
>>>> [1]
>>>> https://github.com/apache/vxquery/blob/master/vxquery-core/src/main/javacc/xquery-grammar.jj
>>>> [2]
>>>> http://jsoniq.org/docs/JSONiqExtensionToXQuery/html-single/index.html#section-json-value-construction
>>>> [3] https://javacc.java.net/
>>>>
>>>
>>>
>>> --
>>> Riyafa Abdul Hameed
>>> Undergraduate, University of Moratuwa
>>>
>>> Email: riyafa.12@cse.mrt.ac.lk
>>> Website: https://riyafa.wordpress.com/
>>> <http://riyafa.wordpress.com/>
>>> <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa>
>>> <http://twitter.com/Riyafa1>
>>>
>>
>>
Re: JSONiq constructors (next step)
Posted by Christina Pavlopoulou <cp...@ucr.edu>.
Hi,
I made some changes to the array constructor in grammar.jj in [1]. I
debug it and I get the array that I put as input, so I guess it parses
it correctly. I have the same question as Riyafa, regarding the
translator. It is not so clear to me how I can create a translate method.
[1] https://github.com/apache/vxquery/pull/48
Thank you,
Christina
On 05/29/2016 07:40 AM, Riyafa Abdul Hameed wrote:
> Hi,
>
> I have created a pull request with the Object constructor
> implementation[1]. Since I cannot create a duplicate token of kind "{", I
> have attempted to use <Lbrace>. When I build the code and try to debug to
> see if the parser recognizes an object, a parserexception is thrown. I have
> also tried using <LbraceExprEnclosure>. But as I understand both are not
> START_TAGs. How can I change the grammar so that token "{" would be
> recognized as a START_TAG to be processed by the MainModule() of the
> parser.
>
> Also a translate method for object constructor in the class
> XMLQueryTranslator needs to be created. I am not sure I understand how that
> should be done. Kindly provide any guidance.
>
> [1] https://github.com/apache/vxquery/pull/47
>
> Thank you.
>
> Yours sincerely,
> Riyafa
>
> On 24 May 2016 at 00:03, Riyafa Abdul Hameed <ri...@cse.mrt.ac.lk>
> wrote:
>
>> Hi,
>>
>> I have been attempting to extend the xquery-grammar.jj file to support
>> JSONiq object constructor[1]. As I understand we need to create classes
>> that extends ASTNode class in the package org.apache.vxquery.xmlquery.ast
>> to be able to return ASTNode objects by the ObjectConstructor--currently it
>> returns nothing. I hope it needs to be discussed what classes to be
>> created. Correct me if I am mistaken.
>>
>> [1] https://github.com/apache/vxquery/pull/44
>> <https://github.com/apache/vxquery/pull/40>
>>
>> Thank you.
>>
>> Yours sincerely,
>> Riyafa
>>
>> On 17 May 2016 at 22:57, Preston Carman <pr...@apache.org> wrote:
>>
>>> As you (JSONiq GSOC students) look at extending the xquery-grammar.jj
>>> [1] file for JSONiq, just remember to check the specification for
>>> specifics [2]. The specification has a nice syntax block at the
>>> beginning of the construction chapter. Also, check out the JavaCC
>>> documentation [3] if you need to catch up on our parser.
>>>
>>> Post if you have any questions. Also, post the final pull request for
>>> the data model implementations.
>>>
>>> [1]
>>> https://github.com/apache/vxquery/blob/master/vxquery-core/src/main/javacc/xquery-grammar.jj
>>> [2]
>>> http://jsoniq.org/docs/JSONiqExtensionToXQuery/html-single/index.html#section-json-value-construction
>>> [3] https://javacc.java.net/
>>>
>>
>>
>> --
>> Riyafa Abdul Hameed
>> Undergraduate, University of Moratuwa
>>
>> Email: riyafa.12@cse.mrt.ac.lk
>> Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/>
>> <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa>
>> <http://twitter.com/Riyafa1>
>>
>
>
Re: JSONiq constructors (next step)
Posted by Riyafa Abdul Hameed <ri...@cse.mrt.ac.lk>.
Hi,
I have created a pull request with the Object constructor
implementation[1]. Since I cannot create a duplicate token of kind "{", I
have attempted to use <Lbrace>. When I build the code and try to debug to
see if the parser recognizes an object, a parserexception is thrown. I have
also tried using <LbraceExprEnclosure>. But as I understand both are not
START_TAGs. How can I change the grammar so that token "{" would be
recognized as a START_TAG to be processed by the MainModule() of the
parser.
Also a translate method for object constructor in the class
XMLQueryTranslator needs to be created. I am not sure I understand how that
should be done. Kindly provide any guidance.
[1] https://github.com/apache/vxquery/pull/47
Thank you.
Yours sincerely,
Riyafa
On 24 May 2016 at 00:03, Riyafa Abdul Hameed <ri...@cse.mrt.ac.lk>
wrote:
> Hi,
>
> I have been attempting to extend the xquery-grammar.jj file to support
> JSONiq object constructor[1]. As I understand we need to create classes
> that extends ASTNode class in the package org.apache.vxquery.xmlquery.ast
> to be able to return ASTNode objects by the ObjectConstructor--currently it
> returns nothing. I hope it needs to be discussed what classes to be
> created. Correct me if I am mistaken.
>
> [1] https://github.com/apache/vxquery/pull/44
> <https://github.com/apache/vxquery/pull/40>
>
> Thank you.
>
> Yours sincerely,
> Riyafa
>
> On 17 May 2016 at 22:57, Preston Carman <pr...@apache.org> wrote:
>
>> As you (JSONiq GSOC students) look at extending the xquery-grammar.jj
>> [1] file for JSONiq, just remember to check the specification for
>> specifics [2]. The specification has a nice syntax block at the
>> beginning of the construction chapter. Also, check out the JavaCC
>> documentation [3] if you need to catch up on our parser.
>>
>> Post if you have any questions. Also, post the final pull request for
>> the data model implementations.
>>
>> [1]
>> https://github.com/apache/vxquery/blob/master/vxquery-core/src/main/javacc/xquery-grammar.jj
>> [2]
>> http://jsoniq.org/docs/JSONiqExtensionToXQuery/html-single/index.html#section-json-value-construction
>> [3] https://javacc.java.net/
>>
>
>
>
> --
> Riyafa Abdul Hameed
> Undergraduate, University of Moratuwa
>
> Email: riyafa.12@cse.mrt.ac.lk
> Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/>
> <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa>
> <http://twitter.com/Riyafa1>
>
--
Riyafa Abdul Hameed
Undergraduate, University of Moratuwa
Email: riyafa.12@cse.mrt.ac.lk
Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/>
<http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa>
<http://twitter.com/Riyafa1>
Re: JSONiq constructors (next step)
Posted by Riyafa Abdul Hameed <ri...@cse.mrt.ac.lk>.
Hi,
I have been attempting to extend the xquery-grammar.jj file to support
JSONiq object constructor[1]. As I understand we need to create classes
that extends ASTNode class in the package org.apache.vxquery.xmlquery.ast
to be able to return ASTNode objects by the ObjectConstructor--currently it
returns nothing. I hope it needs to be discussed what classes to be
created. Correct me if I am mistaken.
[1] https://github.com/apache/vxquery/pull/44
<https://github.com/apache/vxquery/pull/40>
Thank you.
Yours sincerely,
Riyafa
On 17 May 2016 at 22:57, Preston Carman <pr...@apache.org> wrote:
> As you (JSONiq GSOC students) look at extending the xquery-grammar.jj
> [1] file for JSONiq, just remember to check the specification for
> specifics [2]. The specification has a nice syntax block at the
> beginning of the construction chapter. Also, check out the JavaCC
> documentation [3] if you need to catch up on our parser.
>
> Post if you have any questions. Also, post the final pull request for
> the data model implementations.
>
> [1]
> https://github.com/apache/vxquery/blob/master/vxquery-core/src/main/javacc/xquery-grammar.jj
> [2]
> http://jsoniq.org/docs/JSONiqExtensionToXQuery/html-single/index.html#section-json-value-construction
> [3] https://javacc.java.net/
>
--
Riyafa Abdul Hameed
Undergraduate, University of Moratuwa
Email: riyafa.12@cse.mrt.ac.lk
Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/>
<http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa>
<http://twitter.com/Riyafa1>