You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Kirk Lund <kl...@pivotal.io> on 2016/09/14 17:11:05 UTC
Need help on querying
I'm trying to remove TypedJson and replace it with Jackson. Unfortunately,
it appears that some of the behavior for limiting how much data a query
returns has leaked into the TypedJson class in the form of
"queryCollectionsDepth".
Both GFSH and Pulse use the following to limit how much a query returns...
/**
* Number of rows queryData operation will return. By default it will be
1000
*/
private int queryResultSetLimit = ManagementConstants.DEFAULT_QUERY_LIMIT;
/**
* NUmber of elements to be shown in queryData operation if query results
contain collections like Map, List etc.
*/
private int queryCollectionsDepth =
TypedJson.DEFAULT_COLLECTION_ELEMENT_LIMIT;
These are passed into:
QueryDataFunction.queryData(query, members, limit, false,
queryResultSetLimit, queryCollectionsDepth);
1) queryResultSetLimit is fed into the query itself as a LIMIT
2) queryCollectionsDepth is apparently fed into TypedJson as some sort of
limit involving depth of recursing a collection -- this should probably
move into the query in some way.
Does anyone understand querying enough to help me move this behavior to
some proper place such as a querying class? I'm completely unfamiliar with
the querying code so I'm not sure where to even begin on #2.
Thanks,
Kirk
Re: Need help on querying
Posted by Kirk Lund <kl...@pivotal.io>.
That's what I was afraid of. So by moving to Jackson we will lose whatever
feature that parameter provides.
-Kirk
On Wed, Sep 14, 2016 at 11:36 AM, Jason Huynh <jh...@pivotal.io> wrote:
> Hi Kirk,
>
> I am not too familiar with the TypeJson class or how it was being used...
> for number 2, I am assuming the queryCollectionsDepth is a specific
> gfsh/function specific to gfsh. I don't think there is an equivalent oql
> keyword or call for that.
>
> -Jason
>
> On Wed, Sep 14, 2016 at 10:11 AM Kirk Lund <kl...@pivotal.io> wrote:
>
> > I'm trying to remove TypedJson and replace it with Jackson.
> Unfortunately,
> > it appears that some of the behavior for limiting how much data a query
> > returns has leaked into the TypedJson class in the form of
> > "queryCollectionsDepth".
> >
> > Both GFSH and Pulse use the following to limit how much a query
> returns...
> >
> > /**
> > * Number of rows queryData operation will return. By default it will
> be
> > 1000
> > */
> > private int queryResultSetLimit =
> > ManagementConstants.DEFAULT_QUERY_LIMIT;
> >
> > /**
> > * NUmber of elements to be shown in queryData operation if query
> results
> > contain collections like Map, List etc.
> > */
> > private int queryCollectionsDepth =
> > TypedJson.DEFAULT_COLLECTION_ELEMENT_LIMIT;
> >
> > These are passed into:
> >
> > QueryDataFunction.queryData(query, members, limit, false,
> > queryResultSetLimit, queryCollectionsDepth);
> >
> > 1) queryResultSetLimit is fed into the query itself as a LIMIT
> >
> > 2) queryCollectionsDepth is apparently fed into TypedJson as some sort of
> > limit involving depth of recursing a collection -- this should probably
> > move into the query in some way.
> >
> > Does anyone understand querying enough to help me move this behavior to
> > some proper place such as a querying class? I'm completely unfamiliar
> with
> > the querying code so I'm not sure where to even begin on #2.
> >
> > Thanks,
> > Kirk
> >
>
Re: Need help on querying
Posted by Swapnil Bawaskar <sb...@pivotal.io>.
> I think queryResultSetLimit might be a way for TypedJson to cut a collection
short.
Kirk, did you mean queryCollectionsDepth here?
I would like to argue that we do not need queryCollectionsDepth, because
returning only a subset of data without even telling the users that its a
subset, or allowing the users to control the parameter is just wrong.
On Wed, Sep 14, 2016 at 3:04 PM, Anthony Baker <ab...@pivotal.io> wrote:
> Perhaps a custom jackson serializer for Collections that stops after
> queryCollectionsDepth would do the trick.
>
> Anthony
>
>
> > On Sep 14, 2016, at 2:24 PM, Kirk Lund <kl...@pivotal.io> wrote:
> >
> > Yep, that method does add " LIMIT X" to the query where X
> > equals queryResultSetLimit.
> >
> > I think queryResultSetLimit might be a way for TypedJson to cut a
> > collection short. It's always 100 and if one of the fields of your value
> > object is a collection then I believe the resulting json will always
> > include up to 100 maximum elements of that collection. This seems to be
> use
> > for it. But it's hardcoded to 100 and not exposed to the user in any way.
> >
> > Thanks,
> > Kirk
> >
> >
> > On Wed, Sep 14, 2016 at 2:15 PM, John Blum <jb...@pivotal.io> wrote:
> >
> >> Hi Kirk-
> >>
> >> Also, TMK there is no specific API call and/or parameter to limit the
> >> result set of a query, rather it is typically expressed in the OQL
> query as
> >> so...
> >>
> >> SELECT * FROM /Region WHERE <predicate> LIMIT X
> >>
> >> Where 'X' is the desired result quantity.
> >>
> >> So, perhaps the question is more of what does the following API call
> >> translate into OQL as...
> >>
> >> QueryDataFunction.queryData(query, members, limit, false,
> >> queryResultSetLimit, queryCollectionsDepth);
> >>
> >> Hope this helps,
> >>
> >> John
> >>
> >>
> >>
> >> On Wed, Sep 14, 2016 at 11:36 AM, Jason Huynh <jh...@pivotal.io>
> wrote:
> >>
> >>> Hi Kirk,
> >>>
> >>> I am not too familiar with the TypeJson class or how it was being
> used...
> >>> for number 2, I am assuming the queryCollectionsDepth is a specific
> >>> gfsh/function specific to gfsh. I don't think there is an equivalent
> oql
> >>> keyword or call for that.
> >>>
> >>> -Jason
> >>>
> >>> On Wed, Sep 14, 2016 at 10:11 AM Kirk Lund <kl...@pivotal.io> wrote:
> >>>
> >>>> I'm trying to remove TypedJson and replace it with Jackson.
> >>> Unfortunately,
> >>>> it appears that some of the behavior for limiting how much data a
> query
> >>>> returns has leaked into the TypedJson class in the form of
> >>>> "queryCollectionsDepth".
> >>>>
> >>>> Both GFSH and Pulse use the following to limit how much a query
> >>> returns...
> >>>>
> >>>> /**
> >>>> * Number of rows queryData operation will return. By default it will
> >>> be
> >>>> 1000
> >>>> */
> >>>> private int queryResultSetLimit =
> >>>> ManagementConstants.DEFAULT_QUERY_LIMIT;
> >>>>
> >>>> /**
> >>>> * NUmber of elements to be shown in queryData operation if query
> >>> results
> >>>> contain collections like Map, List etc.
> >>>> */
> >>>> private int queryCollectionsDepth =
> >>>> TypedJson.DEFAULT_COLLECTION_ELEMENT_LIMIT;
> >>>>
> >>>> These are passed into:
> >>>>
> >>>> QueryDataFunction.queryData(query, members, limit, false,
> >>>> queryResultSetLimit, queryCollectionsDepth);
> >>>>
> >>>> 1) queryResultSetLimit is fed into the query itself as a LIMIT
> >>>>
> >>>> 2) queryCollectionsDepth is apparently fed into TypedJson as some sort
> >> of
> >>>> limit involving depth of recursing a collection -- this should
> probably
> >>>> move into the query in some way.
> >>>>
> >>>> Does anyone understand querying enough to help me move this behavior
> to
> >>>> some proper place such as a querying class? I'm completely unfamiliar
> >>> with
> >>>> the querying code so I'm not sure where to even begin on #2.
> >>>>
> >>>> Thanks,
> >>>> Kirk
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> -John
> >> 503-504-8657
> >> john.blum10101 (skype)
> >>
>
>
Re: Need help on querying
Posted by Anthony Baker <ab...@pivotal.io>.
Perhaps a custom jackson serializer for Collections that stops after queryCollectionsDepth would do the trick.
Anthony
> On Sep 14, 2016, at 2:24 PM, Kirk Lund <kl...@pivotal.io> wrote:
>
> Yep, that method does add " LIMIT X" to the query where X
> equals queryResultSetLimit.
>
> I think queryResultSetLimit might be a way for TypedJson to cut a
> collection short. It's always 100 and if one of the fields of your value
> object is a collection then I believe the resulting json will always
> include up to 100 maximum elements of that collection. This seems to be use
> for it. But it's hardcoded to 100 and not exposed to the user in any way.
>
> Thanks,
> Kirk
>
>
> On Wed, Sep 14, 2016 at 2:15 PM, John Blum <jb...@pivotal.io> wrote:
>
>> Hi Kirk-
>>
>> Also, TMK there is no specific API call and/or parameter to limit the
>> result set of a query, rather it is typically expressed in the OQL query as
>> so...
>>
>> SELECT * FROM /Region WHERE <predicate> LIMIT X
>>
>> Where 'X' is the desired result quantity.
>>
>> So, perhaps the question is more of what does the following API call
>> translate into OQL as...
>>
>> QueryDataFunction.queryData(query, members, limit, false,
>> queryResultSetLimit, queryCollectionsDepth);
>>
>> Hope this helps,
>>
>> John
>>
>>
>>
>> On Wed, Sep 14, 2016 at 11:36 AM, Jason Huynh <jh...@pivotal.io> wrote:
>>
>>> Hi Kirk,
>>>
>>> I am not too familiar with the TypeJson class or how it was being used...
>>> for number 2, I am assuming the queryCollectionsDepth is a specific
>>> gfsh/function specific to gfsh. I don't think there is an equivalent oql
>>> keyword or call for that.
>>>
>>> -Jason
>>>
>>> On Wed, Sep 14, 2016 at 10:11 AM Kirk Lund <kl...@pivotal.io> wrote:
>>>
>>>> I'm trying to remove TypedJson and replace it with Jackson.
>>> Unfortunately,
>>>> it appears that some of the behavior for limiting how much data a query
>>>> returns has leaked into the TypedJson class in the form of
>>>> "queryCollectionsDepth".
>>>>
>>>> Both GFSH and Pulse use the following to limit how much a query
>>> returns...
>>>>
>>>> /**
>>>> * Number of rows queryData operation will return. By default it will
>>> be
>>>> 1000
>>>> */
>>>> private int queryResultSetLimit =
>>>> ManagementConstants.DEFAULT_QUERY_LIMIT;
>>>>
>>>> /**
>>>> * NUmber of elements to be shown in queryData operation if query
>>> results
>>>> contain collections like Map, List etc.
>>>> */
>>>> private int queryCollectionsDepth =
>>>> TypedJson.DEFAULT_COLLECTION_ELEMENT_LIMIT;
>>>>
>>>> These are passed into:
>>>>
>>>> QueryDataFunction.queryData(query, members, limit, false,
>>>> queryResultSetLimit, queryCollectionsDepth);
>>>>
>>>> 1) queryResultSetLimit is fed into the query itself as a LIMIT
>>>>
>>>> 2) queryCollectionsDepth is apparently fed into TypedJson as some sort
>> of
>>>> limit involving depth of recursing a collection -- this should probably
>>>> move into the query in some way.
>>>>
>>>> Does anyone understand querying enough to help me move this behavior to
>>>> some proper place such as a querying class? I'm completely unfamiliar
>>> with
>>>> the querying code so I'm not sure where to even begin on #2.
>>>>
>>>> Thanks,
>>>> Kirk
>>>>
>>>
>>
>>
>>
>> --
>> -John
>> 503-504-8657
>> john.blum10101 (skype)
>>
Re: Need help on querying
Posted by Kirk Lund <kl...@pivotal.io>.
Yep, that method does add " LIMIT X" to the query where X
equals queryResultSetLimit.
I think queryResultSetLimit might be a way for TypedJson to cut a
collection short. It's always 100 and if one of the fields of your value
object is a collection then I believe the resulting json will always
include up to 100 maximum elements of that collection. This seems to be use
for it. But it's hardcoded to 100 and not exposed to the user in any way.
Thanks,
Kirk
On Wed, Sep 14, 2016 at 2:15 PM, John Blum <jb...@pivotal.io> wrote:
> Hi Kirk-
>
> Also, TMK there is no specific API call and/or parameter to limit the
> result set of a query, rather it is typically expressed in the OQL query as
> so...
>
> SELECT * FROM /Region WHERE <predicate> LIMIT X
>
> Where 'X' is the desired result quantity.
>
> So, perhaps the question is more of what does the following API call
> translate into OQL as...
>
> QueryDataFunction.queryData(query, members, limit, false,
> queryResultSetLimit, queryCollectionsDepth);
>
> Hope this helps,
>
> John
>
>
>
> On Wed, Sep 14, 2016 at 11:36 AM, Jason Huynh <jh...@pivotal.io> wrote:
>
> > Hi Kirk,
> >
> > I am not too familiar with the TypeJson class or how it was being used...
> > for number 2, I am assuming the queryCollectionsDepth is a specific
> > gfsh/function specific to gfsh. I don't think there is an equivalent oql
> > keyword or call for that.
> >
> > -Jason
> >
> > On Wed, Sep 14, 2016 at 10:11 AM Kirk Lund <kl...@pivotal.io> wrote:
> >
> > > I'm trying to remove TypedJson and replace it with Jackson.
> > Unfortunately,
> > > it appears that some of the behavior for limiting how much data a query
> > > returns has leaked into the TypedJson class in the form of
> > > "queryCollectionsDepth".
> > >
> > > Both GFSH and Pulse use the following to limit how much a query
> > returns...
> > >
> > > /**
> > > * Number of rows queryData operation will return. By default it will
> > be
> > > 1000
> > > */
> > > private int queryResultSetLimit =
> > > ManagementConstants.DEFAULT_QUERY_LIMIT;
> > >
> > > /**
> > > * NUmber of elements to be shown in queryData operation if query
> > results
> > > contain collections like Map, List etc.
> > > */
> > > private int queryCollectionsDepth =
> > > TypedJson.DEFAULT_COLLECTION_ELEMENT_LIMIT;
> > >
> > > These are passed into:
> > >
> > > QueryDataFunction.queryData(query, members, limit, false,
> > > queryResultSetLimit, queryCollectionsDepth);
> > >
> > > 1) queryResultSetLimit is fed into the query itself as a LIMIT
> > >
> > > 2) queryCollectionsDepth is apparently fed into TypedJson as some sort
> of
> > > limit involving depth of recursing a collection -- this should probably
> > > move into the query in some way.
> > >
> > > Does anyone understand querying enough to help me move this behavior to
> > > some proper place such as a querying class? I'm completely unfamiliar
> > with
> > > the querying code so I'm not sure where to even begin on #2.
> > >
> > > Thanks,
> > > Kirk
> > >
> >
>
>
>
> --
> -John
> 503-504-8657
> john.blum10101 (skype)
>
Re: Need help on querying
Posted by John Blum <jb...@pivotal.io>.
Hi Kirk-
Also, TMK there is no specific API call and/or parameter to limit the
result set of a query, rather it is typically expressed in the OQL query as
so...
SELECT * FROM /Region WHERE <predicate> LIMIT X
Where 'X' is the desired result quantity.
So, perhaps the question is more of what does the following API call
translate into OQL as...
QueryDataFunction.queryData(query, members, limit, false,
queryResultSetLimit, queryCollectionsDepth);
Hope this helps,
John
On Wed, Sep 14, 2016 at 11:36 AM, Jason Huynh <jh...@pivotal.io> wrote:
> Hi Kirk,
>
> I am not too familiar with the TypeJson class or how it was being used...
> for number 2, I am assuming the queryCollectionsDepth is a specific
> gfsh/function specific to gfsh. I don't think there is an equivalent oql
> keyword or call for that.
>
> -Jason
>
> On Wed, Sep 14, 2016 at 10:11 AM Kirk Lund <kl...@pivotal.io> wrote:
>
> > I'm trying to remove TypedJson and replace it with Jackson.
> Unfortunately,
> > it appears that some of the behavior for limiting how much data a query
> > returns has leaked into the TypedJson class in the form of
> > "queryCollectionsDepth".
> >
> > Both GFSH and Pulse use the following to limit how much a query
> returns...
> >
> > /**
> > * Number of rows queryData operation will return. By default it will
> be
> > 1000
> > */
> > private int queryResultSetLimit =
> > ManagementConstants.DEFAULT_QUERY_LIMIT;
> >
> > /**
> > * NUmber of elements to be shown in queryData operation if query
> results
> > contain collections like Map, List etc.
> > */
> > private int queryCollectionsDepth =
> > TypedJson.DEFAULT_COLLECTION_ELEMENT_LIMIT;
> >
> > These are passed into:
> >
> > QueryDataFunction.queryData(query, members, limit, false,
> > queryResultSetLimit, queryCollectionsDepth);
> >
> > 1) queryResultSetLimit is fed into the query itself as a LIMIT
> >
> > 2) queryCollectionsDepth is apparently fed into TypedJson as some sort of
> > limit involving depth of recursing a collection -- this should probably
> > move into the query in some way.
> >
> > Does anyone understand querying enough to help me move this behavior to
> > some proper place such as a querying class? I'm completely unfamiliar
> with
> > the querying code so I'm not sure where to even begin on #2.
> >
> > Thanks,
> > Kirk
> >
>
--
-John
503-504-8657
john.blum10101 (skype)
Re: Need help on querying
Posted by Jason Huynh <jh...@pivotal.io>.
Hi Kirk,
I am not too familiar with the TypeJson class or how it was being used...
for number 2, I am assuming the queryCollectionsDepth is a specific
gfsh/function specific to gfsh. I don't think there is an equivalent oql
keyword or call for that.
-Jason
On Wed, Sep 14, 2016 at 10:11 AM Kirk Lund <kl...@pivotal.io> wrote:
> I'm trying to remove TypedJson and replace it with Jackson. Unfortunately,
> it appears that some of the behavior for limiting how much data a query
> returns has leaked into the TypedJson class in the form of
> "queryCollectionsDepth".
>
> Both GFSH and Pulse use the following to limit how much a query returns...
>
> /**
> * Number of rows queryData operation will return. By default it will be
> 1000
> */
> private int queryResultSetLimit =
> ManagementConstants.DEFAULT_QUERY_LIMIT;
>
> /**
> * NUmber of elements to be shown in queryData operation if query results
> contain collections like Map, List etc.
> */
> private int queryCollectionsDepth =
> TypedJson.DEFAULT_COLLECTION_ELEMENT_LIMIT;
>
> These are passed into:
>
> QueryDataFunction.queryData(query, members, limit, false,
> queryResultSetLimit, queryCollectionsDepth);
>
> 1) queryResultSetLimit is fed into the query itself as a LIMIT
>
> 2) queryCollectionsDepth is apparently fed into TypedJson as some sort of
> limit involving depth of recursing a collection -- this should probably
> move into the query in some way.
>
> Does anyone understand querying enough to help me move this behavior to
> some proper place such as a querying class? I'm completely unfamiliar with
> the querying code so I'm not sure where to even begin on #2.
>
> Thanks,
> Kirk
>