You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@asterixdb.apache.org by Ian Maxon <im...@uci.edu> on 2017/04/12 20:41:09 UTC
When is it appropirate to add reserved words to the AQL/SQL++ Grammar?
Hey all,
I was working on a patch that would add a 'flush dataset' DDL command
(mainly for testing), and of course this would require adding 'flush'
as a new reserved word. What is the consensus on when this would be
permitted at this point? There are other ways to do this, of course, I
could expose this functionality through a separate API, and I am not
really partial to one solution or the other.
Thanks,
- Ian
Re: When is it appropirate to add reserved words to the AQL/SQL++ Grammar?
Posted by Yingyi Bu <bu...@gmail.com>.
+1 for HTTP API.
The end point can be easily removed from a real deployment by using a
config parameter.
Best,
Yingyi
On Wed, Apr 12, 2017 at 6:12 PM, Till Westmann <ti...@apache.org> wrote:
> Coming back to the question in the subject: If we need such functionality
> e.g. for testing of performance measurements, how should we expose it?
> Should we
> - extend the language with new keywords or
> - introduce on keyword for this that is followed by some parameters or
> - expose new HTTP APIs or
> - …
>
> Thoughts?
>
> Cheers,
> Till
>
>
> On 12 Apr 2017, at 15:11, Ian Maxon wrote:
>
> Yes, more or less. I'm not sure that I would force the buffer cache,
>> but at least I want all of the "object" representations outside of
>> that to have vanished such that their state must be regenerated from
>> the at-rest format.
>>
>> On Wed, Apr 12, 2017 at 2:41 PM, Mike Carey <dt...@gmail.com> wrote:
>>
>>> So you want to basically quiesce a dataset "all the way" - so that it's
>>> not
>>> open at all, and perhaps so that the buffer pool also has none of its
>>> pages
>>> resident?
>>>
>>>
>>>
>>> On 4/12/17 2:26 PM, Ian Maxon wrote:
>>>
>>>>
>>>> That's a little different than from what I'd like to do. Basically I
>>>> want to evict the dataset entirely. My alterior motive here is to test
>>>> the correctness of persisted storage artifacts as right now in many
>>>> cases this is not covered. It has been a recurring theme in many bugs
>>>> that there is some stored artifact on disk that is written incorrectly
>>>> but never read during tests, because often the tests simply never
>>>> encounter the path in which this would occur (restart,eviction, etc.)
>>>>
>>>> On Wed, Apr 12, 2017 at 1:46 PM, Ildar Absalyamov
>>>> <il...@gmail.com> wrote:
>>>>
>>>>>
>>>>> Ian,
>>>>>
>>>>> There is an existing API, which does exactly that.
>>>>> https://github.com/apache/asterixdb/blob/master/asterixdb/
>>>>> asterix-app/src/main/java/org/apache/asterix/api/http/
>>>>> server/ConnectorApiServlet.java#L123
>>>>> <https://github.com/apache/asterixdb/blob/master/asterixdb/
>>>>> asterix-app/src/main/java/org/apache/asterix/api/http/
>>>>> server/ConnectorApiServlet.java#L123>
>>>>> I have been successfully using it in my experiments.
>>>>>
>>>>> On Apr 12, 2017, at 13:41, Ian Maxon <im...@uci.edu> wrote:
>>>>>>
>>>>>> Hey all,
>>>>>> I was working on a patch that would add a 'flush dataset' DDL command
>>>>>> (mainly for testing), and of course this would require adding 'flush'
>>>>>> as a new reserved word. What is the consensus on when this would be
>>>>>> permitted at this point? There are other ways to do this, of course, I
>>>>>> could expose this functionality through a separate API, and I am not
>>>>>> really partial to one solution or the other.
>>>>>>
>>>>>> Thanks,
>>>>>> - Ian
>>>>>>
>>>>>
>>>>> Best regards,
>>>>> Ildar
>>>>>
>>>>>
>>>
Re: When is it appropirate to add reserved words to the AQL/SQL++
Grammar?
Posted by Till Westmann <ti...@apache.org>.
Coming back to the question in the subject: If we need such
functionality e.g. for testing of performance measurements, how should
we expose it? Should we
- extend the language with new keywords or
- introduce on keyword for this that is followed by some parameters or
- expose new HTTP APIs or
- …
Thoughts?
Cheers,
Till
On 12 Apr 2017, at 15:11, Ian Maxon wrote:
> Yes, more or less. I'm not sure that I would force the buffer cache,
> but at least I want all of the "object" representations outside of
> that to have vanished such that their state must be regenerated from
> the at-rest format.
>
> On Wed, Apr 12, 2017 at 2:41 PM, Mike Carey <dt...@gmail.com> wrote:
>> So you want to basically quiesce a dataset "all the way" - so that
>> it's not
>> open at all, and perhaps so that the buffer pool also has none of its
>> pages
>> resident?
>>
>>
>>
>> On 4/12/17 2:26 PM, Ian Maxon wrote:
>>>
>>> That's a little different than from what I'd like to do. Basically I
>>> want to evict the dataset entirely. My alterior motive here is to
>>> test
>>> the correctness of persisted storage artifacts as right now in many
>>> cases this is not covered. It has been a recurring theme in many
>>> bugs
>>> that there is some stored artifact on disk that is written
>>> incorrectly
>>> but never read during tests, because often the tests simply never
>>> encounter the path in which this would occur (restart,eviction,
>>> etc.)
>>>
>>> On Wed, Apr 12, 2017 at 1:46 PM, Ildar Absalyamov
>>> <il...@gmail.com> wrote:
>>>>
>>>> Ian,
>>>>
>>>> There is an existing API, which does exactly that.
>>>> https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123
>>>> <https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123>
>>>> I have been successfully using it in my experiments.
>>>>
>>>>> On Apr 12, 2017, at 13:41, Ian Maxon <im...@uci.edu> wrote:
>>>>>
>>>>> Hey all,
>>>>> I was working on a patch that would add a 'flush dataset' DDL
>>>>> command
>>>>> (mainly for testing), and of course this would require adding
>>>>> 'flush'
>>>>> as a new reserved word. What is the consensus on when this would
>>>>> be
>>>>> permitted at this point? There are other ways to do this, of
>>>>> course, I
>>>>> could expose this functionality through a separate API, and I am
>>>>> not
>>>>> really partial to one solution or the other.
>>>>>
>>>>> Thanks,
>>>>> - Ian
>>>>
>>>> Best regards,
>>>> Ildar
>>>>
>>
Re: When is it appropirate to add reserved words to the AQL/SQL++ Grammar?
Posted by Ian Maxon <im...@uci.edu>.
Yes, more or less. I'm not sure that I would force the buffer cache,
but at least I want all of the "object" representations outside of
that to have vanished such that their state must be regenerated from
the at-rest format.
On Wed, Apr 12, 2017 at 2:41 PM, Mike Carey <dt...@gmail.com> wrote:
> So you want to basically quiesce a dataset "all the way" - so that it's not
> open at all, and perhaps so that the buffer pool also has none of its pages
> resident?
>
>
>
> On 4/12/17 2:26 PM, Ian Maxon wrote:
>>
>> That's a little different than from what I'd like to do. Basically I
>> want to evict the dataset entirely. My alterior motive here is to test
>> the correctness of persisted storage artifacts as right now in many
>> cases this is not covered. It has been a recurring theme in many bugs
>> that there is some stored artifact on disk that is written incorrectly
>> but never read during tests, because often the tests simply never
>> encounter the path in which this would occur (restart,eviction, etc.)
>>
>> On Wed, Apr 12, 2017 at 1:46 PM, Ildar Absalyamov
>> <il...@gmail.com> wrote:
>>>
>>> Ian,
>>>
>>> There is an existing API, which does exactly that.
>>> https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123
>>> <https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123>
>>> I have been successfully using it in my experiments.
>>>
>>>> On Apr 12, 2017, at 13:41, Ian Maxon <im...@uci.edu> wrote:
>>>>
>>>> Hey all,
>>>> I was working on a patch that would add a 'flush dataset' DDL command
>>>> (mainly for testing), and of course this would require adding 'flush'
>>>> as a new reserved word. What is the consensus on when this would be
>>>> permitted at this point? There are other ways to do this, of course, I
>>>> could expose this functionality through a separate API, and I am not
>>>> really partial to one solution or the other.
>>>>
>>>> Thanks,
>>>> - Ian
>>>
>>> Best regards,
>>> Ildar
>>>
>
Re: When is it appropirate to add reserved words to the AQL/SQL++
Grammar?
Posted by Mike Carey <dt...@gmail.com>.
So you want to basically quiesce a dataset "all the way" - so that it's
not open at all, and perhaps so that the buffer pool also has none of
its pages resident?
On 4/12/17 2:26 PM, Ian Maxon wrote:
> That's a little different than from what I'd like to do. Basically I
> want to evict the dataset entirely. My alterior motive here is to test
> the correctness of persisted storage artifacts as right now in many
> cases this is not covered. It has been a recurring theme in many bugs
> that there is some stored artifact on disk that is written incorrectly
> but never read during tests, because often the tests simply never
> encounter the path in which this would occur (restart,eviction, etc.)
>
> On Wed, Apr 12, 2017 at 1:46 PM, Ildar Absalyamov
> <il...@gmail.com> wrote:
>> Ian,
>>
>> There is an existing API, which does exactly that. https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123 <https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123>
>> I have been successfully using it in my experiments.
>>
>>> On Apr 12, 2017, at 13:41, Ian Maxon <im...@uci.edu> wrote:
>>>
>>> Hey all,
>>> I was working on a patch that would add a 'flush dataset' DDL command
>>> (mainly for testing), and of course this would require adding 'flush'
>>> as a new reserved word. What is the consensus on when this would be
>>> permitted at this point? There are other ways to do this, of course, I
>>> could expose this functionality through a separate API, and I am not
>>> really partial to one solution or the other.
>>>
>>> Thanks,
>>> - Ian
>> Best regards,
>> Ildar
>>
Re: When is it appropirate to add reserved words to the AQL/SQL++ Grammar?
Posted by Ian Maxon <im...@uci.edu>.
That's a little different than from what I'd like to do. Basically I
want to evict the dataset entirely. My alterior motive here is to test
the correctness of persisted storage artifacts as right now in many
cases this is not covered. It has been a recurring theme in many bugs
that there is some stored artifact on disk that is written incorrectly
but never read during tests, because often the tests simply never
encounter the path in which this would occur (restart,eviction, etc.)
On Wed, Apr 12, 2017 at 1:46 PM, Ildar Absalyamov
<il...@gmail.com> wrote:
> Ian,
>
> There is an existing API, which does exactly that. https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123 <https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123>
> I have been successfully using it in my experiments.
>
>> On Apr 12, 2017, at 13:41, Ian Maxon <im...@uci.edu> wrote:
>>
>> Hey all,
>> I was working on a patch that would add a 'flush dataset' DDL command
>> (mainly for testing), and of course this would require adding 'flush'
>> as a new reserved word. What is the consensus on when this would be
>> permitted at this point? There are other ways to do this, of course, I
>> could expose this functionality through a separate API, and I am not
>> really partial to one solution or the other.
>>
>> Thanks,
>> - Ian
>
> Best regards,
> Ildar
>
Re: When is it appropirate to add reserved words to the AQL/SQL++
Grammar?
Posted by Ildar Absalyamov <il...@gmail.com>.
Ian,
There is an existing API, which does exactly that. https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123 <https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123>
I have been successfully using it in my experiments.
> On Apr 12, 2017, at 13:41, Ian Maxon <im...@uci.edu> wrote:
>
> Hey all,
> I was working on a patch that would add a 'flush dataset' DDL command
> (mainly for testing), and of course this would require adding 'flush'
> as a new reserved word. What is the consensus on when this would be
> permitted at this point? There are other ways to do this, of course, I
> could expose this functionality through a separate API, and I am not
> really partial to one solution or the other.
>
> Thanks,
> - Ian
Best regards,
Ildar