You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by Kathey Marsden <km...@sbcglobal.net> on 2011/10/13 00:48:53 UTC
Fwd: [jira] [Created] (INFRA-4026) Jira no longer allows export of
more than 1000 issues. Gives HTTP Status 403 - You are not allowed to get
a result set of more than 1000 results.
FYI, I filed this IFRA issue which has some impact on the Derby Jira VTI
in our demos.
I suppose the demo will still work but now Jira export is limited to
1000 issues. Is a DERBY issue in order as well?
Thanks
Kathey
-------- Original Message --------
Subject: [jira] [Created] (INFRA-4026) Jira no longer allows export of
more than 1000 issues. Gives HTTP Status 403 - You are not allowed to
get a result set of more than 1000 results.
Date: Wed, 12 Oct 2011 22:45:12 +0000 (UTC)
From: Kathey Marsden (Created) (JIRA) <ji...@apache.org>
To: kmarsdenderby@sbcglobal.net
Jira no longer allows export of more than 1000 issues. Gives HTTP Status 403 - You are not allowed to get a result set of more than 1000 results.
--------------------------------------------------------------------------------------------------------------------------------------------------
Key: INFRA-4026
URL: https://issues.apache.org/jira/browse/INFRA-4026
Project: Infrastructure
Issue Type: Bug
Security Level: public (Regular issues)
Components: JIRA
Reporter: Kathey Marsden
Occasionally I will export the DERBY Jira issues so I can perform queries on them with Derby Jira VTI provided in the Derby Demos. See http://wiki.apache.org/db-derby/HowToRunJiraVtiReports. Once the issues are dumped I can run lots of analytic queries with the VTI without taxing Jira and without manual intervention.
I have noticed that recently though, that the maximum rows returned from Jira has been reduced to 1000.
e.g.
https://issues.apache.org/jira/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?pid=10594&sorter/field=issuekey&sorter/order=DESC&tempMax=6000&reset=true&decorator=none
now yields an error:
HTTP Status 403 - You are not allowed to get a result set of more than 1000 results. Current search returns 5450 results
type Status report
message You are not allowed to get a result set of more than 1000 results. Current search returns 5450 results
description Access to the specified resource (You are not allowed to get a result set of more than 1000 results. Current search returns 5450 results) has been forbidden.
Can you please restore the previous defaults so the data dump can occur ?
.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Fwd: [jira] [Created] (INFRA-4026) Jira no longer allows export
of more than 1000 issues. Gives HTTP Status 403 - You are not allowed to
get a result set of more than 1000 results.
Posted by Kathey Marsden <km...@sbcglobal.net>.
>>
>> But if you can express your query using JQL, why not go directly to
>> JIRA?
> I think that the problem is that there are some queries (particularly
> those involving NOT) which can't be constructed using the existing
> JIRA gui. You can say "I want all issues with the following fields
> turned on" but you can't say "I want all issues with the following
> field NOT turned on".
>
> Thanks,
> -Rick
>> I see the point of caching and dataset rip-offs (i.e. you populate a
>> table and continue to refine the results/searches locally), are there
>> other use-cases where people think they would prefer Derby+VTI over
>> JIRA+JQL?
>>
>>
I know of a couple of tools where it is important to be able to do
queries joining JIRA data with local private DERBY tables. A VTI with
JQL would be nice for that sort of thing, but NOT is a requirement and
I actually still think might actually pull more data over time with
the number of queries than just creating the cache table periodically.
One of these very old tools just dumps the data from Jira into a
JIRA_ISSUES table and does queries. I think that ancient app will just
have to become smarter about updating the data in the table
incrementally to maintain the same functionality.
Personal problems aside, I don't know how much the JIRA VTI demo or any
kind of JIRA data download and interface to Derby is being used by the
user community. I have used this sort of thing for analyzing issues
for backport and generating metrics. I know they can be useful to
manage support and development work, joining private customer data with
the public data in Jira. I could imagine it might be useful to other
development groups in the same capacity, so the Jira VTI might be a good
demo for that reason, but my gut feeling is that demo is not used that
often (of course could totally be wrong).
Kathey
Re: Fwd: [jira] [Created] (INFRA-4026) Jira no longer allows export
of more than 1000 issues. Gives HTTP Status 403 - You are not allowed to
get a result set of more than 1000 results.
Posted by Rick Hillegas <ri...@oracle.com>.
On 10/14/11 6:13 AM, Kristian Waagan wrote:
> On 14.10.11 14:28, Rick Hillegas wrote:
>> On 10/14/11 1:38 AM, Kristian Waagan wrote:
>>> On 10/13/11 02:47 PM, Rick Hillegas wrote:
>>>> On 10/13/11 3:12 AM, Kristian Waagan wrote:
>>>>> On 10/13/11 12:48 AM, Kathey Marsden wrote:
>>>>>> FYI, I filed this IFRA issue which has some impact on the Derby
>>>>>> Jira VTI in our demos.
>>>>>> I suppose the demo will still work but now Jira export is limited
>>>>>> to 1000 issues. Is a DERBY issue in order as well?
>>>>>
>>>>> Working around that limit is possible in this case [1] - the
>>>>> question is if we want to do that?
>>>>>
>>>>> If people are using this demo extensively as their own private
>>>>> "Derby JIRA cache", I think some effort should be invested into
>>>>> writing something that imposes less load on the ASF JIRA instance.
>>>>>
>>>>> Writing something solid here may require quite a bit of effort.
>>>>> Without having given this extensive thought, I think parsing emails
>>>>> as suggested on infra is the kindest option. This would probably
>>>>> suit committers well.
>>>>> Maybe the activity stream (as an RSS feed) could be used too, but in
>>>>> that case it may be harder to make sure no updates are missed.
>>>>> A third option is to use the SOAP API. One has to be able to define
>>>>> a suitable search (for instance 'give me all updates on Derby from
>>>>> date X to date Y'), but I suspect you'd have to download all the
>>>>> issue information instead of just what changed in that period. The
>>>>> upside with this approach compared to the RSS one, is that you
>>>>> decide when you want to update your copy.
>>>> Now that we (by which I mean Kristian) know how to use the SOAP API,
>>>> it may be possible to write a restricted vti that wraps the SOAP API.
>>>> I believe that the existing vti is used to dramatically reduce the
>>>> set of issues you have to look at by applying restrictions which
>>>> cannot be expressed in the JIRA gui.
>>>
>>> What are examples of such restrictions?
>> Hi Kristian,
>>
>> Here is a sample query which uses the existing jira xml vti (from
>> http://wiki.apache.org/db-derby/HowToRunJiraVtiReports):
>>
>> select s."key", s."assignee", s."updated" , s."title"
>> from table( "apacheVanillaJiraReport"(
>> 'file:///kmarsden/projects/jirareports/all_JIRA_ISSUES.xml' ) ) s where
>> "resolution" = 'Unresolved' and "assignee" != 'Unassigned' and NOT
>> "updated" like '%2008%' order by "assignee";
>
> Thanks, Rick.
>
> This translates roughly to the following JQL (I changed the date):
> project = 'DERBY' AND resolution = 'Unresolved' AND assignee is not
> NULL AND updated < '2011-01-01'
>
> (returns 25 issues by the way)
>
>>
>> There are two interesting things about this query:
>>
>> 1) Projection (fancy term for "SELECT list")- The SELECT list is just a
>> few of the columns in the vti. That is, the width of the query result is
>> considerably narrower than the full width of the vti.
>>
>> 2) Restriction (fancy term for pieces of the WHERE clause) - A couple
>> terms in the WHERE clause could be pushed into the Store if the query
>> were directed against a Derby table rather than a vti. These terms
>> reduce the length of the result set coming back to the SQL interpreter.
>> These terms are:
>>
>> "resolution" = 'Unresolved'
>> "assignee" != 'Unassigned'
>>
>> If the query were issued against a restricted vti which wrapped the SOAP
>> api, then the projection and the restriction could be pushed into the
>> vti. That is, Derby would tell the vti to fetch only 4 columns and to
>> throw away any rows which didn't match the two WHERE clause terms above.
>> Presumably, this information could be passed through the SOAP api to
>> limit the width and length of the result set being requested from JIRA.
>
> The projection cannot be used to download less data with the SOAP API.
> The restrictions would be helpful though, but then we need a SQL ->
> JQL translator, or we can cheat and accept the JQL as an argument to
> the VTI.
>
> I haven't looked at how the VTI restrictions are represented, but for
> the sake of a demo it should be feasible to write some code that
> converts the VTI restrictions into JQL.
Hi Kristian,
The restriction is represented as a org.apache.derby.vti.Restriction.
That class has a toSQL() method which turns the restriction into SQL
suitable for sending to a foreign rdbms. JQL may be enough like SQL that
the existing toSQL() output will be good enough. If not, turning a
Restriction into a JQL WHERE clause should be straightforward (tm).
>
> But if you can express your query using JQL, why not go directly to JIRA?
I think that the problem is that there are some queries (particularly
those involving NOT) which can't be constructed using the existing JIRA
gui. You can say "I want all issues with the following fields turned on"
but you can't say "I want all issues with the following field NOT turned
on".
Thanks,
-Rick
> I see the point of caching and dataset rip-offs (i.e. you populate a
> table and continue to refine the results/searches locally), are there
> other use-cases where people think they would prefer Derby+VTI over
> JIRA+JQL?
>
>
Re: Fwd: [jira] [Created] (INFRA-4026) Jira no longer allows export
of more than 1000 issues. Gives HTTP Status 403 - You are not allowed to
get a result set of more than 1000 results.
Posted by Kristian Waagan <kr...@oracle.com>.
On 14.10.11 14:28, Rick Hillegas wrote:
> On 10/14/11 1:38 AM, Kristian Waagan wrote:
>> On 10/13/11 02:47 PM, Rick Hillegas wrote:
>>> On 10/13/11 3:12 AM, Kristian Waagan wrote:
>>>> On 10/13/11 12:48 AM, Kathey Marsden wrote:
>>>>> FYI, I filed this IFRA issue which has some impact on the Derby
>>>>> Jira VTI in our demos.
>>>>> I suppose the demo will still work but now Jira export is limited
>>>>> to 1000 issues. Is a DERBY issue in order as well?
>>>>
>>>> Working around that limit is possible in this case [1] - the
>>>> question is if we want to do that?
>>>>
>>>> If people are using this demo extensively as their own private
>>>> "Derby JIRA cache", I think some effort should be invested into
>>>> writing something that imposes less load on the ASF JIRA instance.
>>>>
>>>> Writing something solid here may require quite a bit of effort.
>>>> Without having given this extensive thought, I think parsing emails
>>>> as suggested on infra is the kindest option. This would probably
>>>> suit committers well.
>>>> Maybe the activity stream (as an RSS feed) could be used too, but in
>>>> that case it may be harder to make sure no updates are missed.
>>>> A third option is to use the SOAP API. One has to be able to define
>>>> a suitable search (for instance 'give me all updates on Derby from
>>>> date X to date Y'), but I suspect you'd have to download all the
>>>> issue information instead of just what changed in that period. The
>>>> upside with this approach compared to the RSS one, is that you
>>>> decide when you want to update your copy.
>>> Now that we (by which I mean Kristian) know how to use the SOAP API,
>>> it may be possible to write a restricted vti that wraps the SOAP API.
>>> I believe that the existing vti is used to dramatically reduce the
>>> set of issues you have to look at by applying restrictions which
>>> cannot be expressed in the JIRA gui.
>>
>> What are examples of such restrictions?
> Hi Kristian,
>
> Here is a sample query which uses the existing jira xml vti (from
> http://wiki.apache.org/db-derby/HowToRunJiraVtiReports):
>
> select s."key", s."assignee", s."updated" , s."title"
> from table( "apacheVanillaJiraReport"(
> 'file:///kmarsden/projects/jirareports/all_JIRA_ISSUES.xml' ) ) s where
> "resolution" = 'Unresolved' and "assignee" != 'Unassigned' and NOT
> "updated" like '%2008%' order by "assignee";
Thanks, Rick.
This translates roughly to the following JQL (I changed the date):
project = 'DERBY' AND resolution = 'Unresolved' AND assignee is not NULL
AND updated < '2011-01-01'
(returns 25 issues by the way)
>
> There are two interesting things about this query:
>
> 1) Projection (fancy term for "SELECT list")- The SELECT list is just a
> few of the columns in the vti. That is, the width of the query result is
> considerably narrower than the full width of the vti.
>
> 2) Restriction (fancy term for pieces of the WHERE clause) - A couple
> terms in the WHERE clause could be pushed into the Store if the query
> were directed against a Derby table rather than a vti. These terms
> reduce the length of the result set coming back to the SQL interpreter.
> These terms are:
>
> "resolution" = 'Unresolved'
> "assignee" != 'Unassigned'
>
> If the query were issued against a restricted vti which wrapped the SOAP
> api, then the projection and the restriction could be pushed into the
> vti. That is, Derby would tell the vti to fetch only 4 columns and to
> throw away any rows which didn't match the two WHERE clause terms above.
> Presumably, this information could be passed through the SOAP api to
> limit the width and length of the result set being requested from JIRA.
The projection cannot be used to download less data with the SOAP API.
The restrictions would be helpful though, but then we need a SQL -> JQL
translator, or we can cheat and accept the JQL as an argument to the VTI.
I haven't looked at how the VTI restrictions are represented, but for
the sake of a demo it should be feasible to write some code that
converts the VTI restrictions into JQL.
But if you can express your query using JQL, why not go directly to JIRA?
I see the point of caching and dataset rip-offs (i.e. you populate a
table and continue to refine the results/searches locally), are there
other use-cases where people think they would prefer Derby+VTI over
JIRA+JQL?
--
Kristian
>
> Thanks,
> -Rick
>
>
>>
>>> If I understand correctly, those restrictions can be expressed in the
>>> SOAP API. By pushing those restrictions into the SOAP query, we would
>>> dramatically reduce the data set being requested from JIRA in the
>>> first place. That should satisfy the objections of the INFRA group.
>>
>> I'm still a little puzzled by how this demo/tool is being used.
>> There's a big difference if we go from parsing a local XML file
>> multiple times, to accessing the JIRA instance through the SOAP API on
>> every invocation (for instance as you are refining your SQL query).
>>
>> FYI, pulling in SOAP requires some extra libraries. A tool as
>> described above would probably be a Maven project like the SOAP client
>> used to generate parts of the release notes.
>>
>>
>> Regards,
>
Re: Fwd: [jira] [Created] (INFRA-4026) Jira no longer allows export
of more than 1000 issues. Gives HTTP Status 403 - You are not allowed to
get a result set of more than 1000 results.
Posted by Rick Hillegas <ri...@oracle.com>.
On 10/14/11 1:38 AM, Kristian Waagan wrote:
> On 10/13/11 02:47 PM, Rick Hillegas wrote:
>> On 10/13/11 3:12 AM, Kristian Waagan wrote:
>>> On 10/13/11 12:48 AM, Kathey Marsden wrote:
>>>> FYI, I filed this IFRA issue which has some impact on the Derby
>>>> Jira VTI in our demos.
>>>> I suppose the demo will still work but now Jira export is limited
>>>> to 1000 issues. Is a DERBY issue in order as well?
>>>
>>> Working around that limit is possible in this case [1] - the
>>> question is if we want to do that?
>>>
>>> If people are using this demo extensively as their own private
>>> "Derby JIRA cache", I think some effort should be invested into
>>> writing something that imposes less load on the ASF JIRA instance.
>>>
>>> Writing something solid here may require quite a bit of effort.
>>> Without having given this extensive thought, I think parsing emails
>>> as suggested on infra is the kindest option. This would probably
>>> suit committers well.
>>> Maybe the activity stream (as an RSS feed) could be used too, but in
>>> that case it may be harder to make sure no updates are missed.
>>> A third option is to use the SOAP API. One has to be able to define
>>> a suitable search (for instance 'give me all updates on Derby from
>>> date X to date Y'), but I suspect you'd have to download all the
>>> issue information instead of just what changed in that period. The
>>> upside with this approach compared to the RSS one, is that you
>>> decide when you want to update your copy.
>> Now that we (by which I mean Kristian) know how to use the SOAP API,
>> it may be possible to write a restricted vti that wraps the SOAP API.
>> I believe that the existing vti is used to dramatically reduce the
>> set of issues you have to look at by applying restrictions which
>> cannot be expressed in the JIRA gui.
>
> What are examples of such restrictions?
Hi Kristian,
Here is a sample query which uses the existing jira xml vti (from
http://wiki.apache.org/db-derby/HowToRunJiraVtiReports):
select s."key", s."assignee", s."updated" , s."title"
from table( "apacheVanillaJiraReport"(
'file:///kmarsden/projects/jirareports/all_JIRA_ISSUES.xml' ) ) s where
"resolution" = 'Unresolved' and "assignee" != 'Unassigned' and NOT
"updated" like '%2008%' order by "assignee";
There are two interesting things about this query:
1) Projection (fancy term for "SELECT list")- The SELECT list is just a
few of the columns in the vti. That is, the width of the query result is
considerably narrower than the full width of the vti.
2) Restriction (fancy term for pieces of the WHERE clause) - A couple
terms in the WHERE clause could be pushed into the Store if the query
were directed against a Derby table rather than a vti. These terms
reduce the length of the result set coming back to the SQL interpreter.
These terms are:
"resolution" = 'Unresolved'
"assignee" != 'Unassigned'
If the query were issued against a restricted vti which wrapped the SOAP
api, then the projection and the restriction could be pushed into the
vti. That is, Derby would tell the vti to fetch only 4 columns and to
throw away any rows which didn't match the two WHERE clause terms above.
Presumably, this information could be passed through the SOAP api to
limit the width and length of the result set being requested from JIRA.
Thanks,
-Rick
>
>> If I understand correctly, those restrictions can be expressed in the
>> SOAP API. By pushing those restrictions into the SOAP query, we would
>> dramatically reduce the data set being requested from JIRA in the
>> first place. That should satisfy the objections of the INFRA group.
>
> I'm still a little puzzled by how this demo/tool is being used.
> There's a big difference if we go from parsing a local XML file
> multiple times, to accessing the JIRA instance through the SOAP API on
> every invocation (for instance as you are refining your SQL query).
>
> FYI, pulling in SOAP requires some extra libraries. A tool as
> described above would probably be a Maven project like the SOAP client
> used to generate parts of the release notes.
>
>
> Regards,
Re: Fwd: [jira] [Created] (INFRA-4026) Jira no longer allows export
of more than 1000 issues. Gives HTTP Status 403 - You are not allowed to
get a result set of more than 1000 results.
Posted by Kristian Waagan <kr...@oracle.com>.
On 10/13/11 02:47 PM, Rick Hillegas wrote:
> On 10/13/11 3:12 AM, Kristian Waagan wrote:
>> On 10/13/11 12:48 AM, Kathey Marsden wrote:
>>> FYI, I filed this IFRA issue which has some impact on the Derby Jira
>>> VTI in our demos.
>>> I suppose the demo will still work but now Jira export is limited to
>>> 1000 issues. Is a DERBY issue in order as well?
>>
>> Working around that limit is possible in this case [1] - the question
>> is if we want to do that?
>>
>> If people are using this demo extensively as their own private "Derby
>> JIRA cache", I think some effort should be invested into writing
>> something that imposes less load on the ASF JIRA instance.
>>
>> Writing something solid here may require quite a bit of effort.
>> Without having given this extensive thought, I think parsing emails
>> as suggested on infra is the kindest option. This would probably suit
>> committers well.
>> Maybe the activity stream (as an RSS feed) could be used too, but in
>> that case it may be harder to make sure no updates are missed.
>> A third option is to use the SOAP API. One has to be able to define a
>> suitable search (for instance 'give me all updates on Derby from date
>> X to date Y'), but I suspect you'd have to download all the issue
>> information instead of just what changed in that period. The upside
>> with this approach compared to the RSS one, is that you decide when
>> you want to update your copy.
> Now that we (by which I mean Kristian) know how to use the SOAP API,
> it may be possible to write a restricted vti that wraps the SOAP API.
> I believe that the existing vti is used to dramatically reduce the set
> of issues you have to look at by applying restrictions which cannot be
> expressed in the JIRA gui.
What are examples of such restrictions?
> If I understand correctly, those restrictions can be expressed in the
> SOAP API. By pushing those restrictions into the SOAP query, we would
> dramatically reduce the data set being requested from JIRA in the
> first place. That should satisfy the objections of the INFRA group.
I'm still a little puzzled by how this demo/tool is being used. There's
a big difference if we go from parsing a local XML file multiple times,
to accessing the JIRA instance through the SOAP API on every invocation
(for instance as you are refining your SQL query).
FYI, pulling in SOAP requires some extra libraries. A tool as described
above would probably be a Maven project like the SOAP client used to
generate parts of the release notes.
Regards,
--
Kristian
>
> Thanks,
> -Rick
>>
>> All solutions require some kind of initial database population
>> functionality, which can be "abused" in the same way as the current
>> approach (before the JIRA limit was in place).
>>
>> The requirements for the "permanent Derby JIRA cache" and the demo
>> differ, and it may be unwise to include something that will reach out
>> to the ASF JIRA instance out of the box. If I understand this
>> correctly, you currently have to manually obtain the XML file to be
>> parsed by the VTI.
>>
>>
>> --
>> Kristian
>>
>> [1] This would be a two-step task, it requires changes both on how we
>> interact with JIRA and how the VTI works.
>>
>>>
>>> Thanks
>>>
>>>
>>> Kathey
>>>
>>> -------- Original Message --------
>>> Subject: [jira] [Created] (INFRA-4026) Jira no longer allows
>>> export of more than 1000 issues. Gives HTTP Status 403 - You are not
>>> allowed to get a result set of more than 1000 results.
>>> Date: Wed, 12 Oct 2011 22:45:12 +0000 (UTC)
>>> From: Kathey Marsden (Created) (JIRA) <ji...@apache.org>
>>> <ma...@apache.org>
>>> To: kmarsdenderby@sbcglobal.net
>>> <ma...@sbcglobal.net>
>>>
>>>
>>>
>>> Jira no longer allows export of more than 1000 issues. Gives HTTP
>>> Status 403 - You are not allowed to get a result set of more than
>>> 1000 results.
>>> --------------------------------------------------------------------------------------------------------------------------------------------------
>>>
>>>
>>> Key: INFRA-4026
>>> URL:https://issues.apache.org/jira/browse/INFRA-4026
>>> Project: Infrastructure
>>> Issue Type: Bug
>>> Security Level: public (Regular issues)
>>> Components: JIRA
>>> Reporter: Kathey Marsden
>>>
>>>
>>> Occasionally I will export the DERBY Jira issues so I can perform
>>> queries on them with Derby Jira VTI provided in the Derby Demos.
>>> Seehttp://wiki.apache.org/db-derby/HowToRunJiraVtiReports. Once the
>>> issues are dumped I can run lots of analytic queries with the VTI
>>> without taxing Jira and without manual intervention.
>>>
>>> I have noticed that recently though, that the maximum rows returned
>>> from Jira has been reduced to 1000.
>>>
>>> e.g.
>>> https://issues.apache.org/jira/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?pid=10594&sorter/field=issuekey&sorter/order=DESC&tempMax=6000&reset=true&decorator=none
>>> <https://issues.apache.org/jira/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?pid=10594&sorter/field=issuekey&sorter/order=DESC&tempMax=6000&reset=true&decorator=none>
>>>
>>>
>>> now yields an error:
>>> HTTP Status 403 - You are not allowed to get a result set of more
>>> than 1000 results. Current search returns 5450 results
>>>
>>> type Status report
>>>
>>> message You are not allowed to get a result set of more than 1000
>>> results. Current search returns 5450 results
>>>
>>> description Access to the specified resource (You are not allowed to
>>> get a result set of more than 1000 results. Current search returns
>>> 5450 results) has been forbidden.
>>>
>>> Can you please restore the previous defaults so the data dump can
>>> occur ?
>>>
>>>
>>>
>>>
>>> .
>>>
>>>
>>> --
>>> This message is automatically generated by JIRA.
>>> If you think it was sent incorrectly, please contact your JIRA
>>> administrators:https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
>>> <https://issues.apache.org/jira/secure/ContactAdministrators%21default.jspa>
>>>
>>> For more information on JIRA,
>>> see:http://www.atlassian.com/software/jira
>>>
>>>
>>>
>>
>
Re: Fwd: [jira] [Created] (INFRA-4026) Jira no longer allows export
of more than 1000 issues. Gives HTTP Status 403 - You are not allowed to
get a result set of more than 1000 results.
Posted by Rick Hillegas <ri...@oracle.com>.
On 10/13/11 3:12 AM, Kristian Waagan wrote:
> On 10/13/11 12:48 AM, Kathey Marsden wrote:
>> FYI, I filed this IFRA issue which has some impact on the Derby Jira
>> VTI in our demos.
>> I suppose the demo will still work but now Jira export is limited to
>> 1000 issues. Is a DERBY issue in order as well?
>
> Working around that limit is possible in this case [1] - the question
> is if we want to do that?
>
> If people are using this demo extensively as their own private "Derby
> JIRA cache", I think some effort should be invested into writing
> something that imposes less load on the ASF JIRA instance.
>
> Writing something solid here may require quite a bit of effort.
> Without having given this extensive thought, I think parsing emails as
> suggested on infra is the kindest option. This would probably suit
> committers well.
> Maybe the activity stream (as an RSS feed) could be used too, but in
> that case it may be harder to make sure no updates are missed.
> A third option is to use the SOAP API. One has to be able to define a
> suitable search (for instance 'give me all updates on Derby from date
> X to date Y'), but I suspect you'd have to download all the issue
> information instead of just what changed in that period. The upside
> with this approach compared to the RSS one, is that you decide when
> you want to update your copy.
Now that we (by which I mean Kristian) know how to use the SOAP API, it
may be possible to write a restricted vti that wraps the SOAP API. I
believe that the existing vti is used to dramatically reduce the set of
issues you have to look at by applying restrictions which cannot be
expressed in the JIRA gui. If I understand correctly, those restrictions
can be expressed in the SOAP API. By pushing those restrictions into the
SOAP query, we would dramatically reduce the data set being requested
from JIRA in the first place. That should satisfy the objections of the
INFRA group.
Thanks,
-Rick
>
> All solutions require some kind of initial database population
> functionality, which can be "abused" in the same way as the current
> approach (before the JIRA limit was in place).
>
> The requirements for the "permanent Derby JIRA cache" and the demo
> differ, and it may be unwise to include something that will reach out
> to the ASF JIRA instance out of the box. If I understand this
> correctly, you currently have to manually obtain the XML file to be
> parsed by the VTI.
>
>
> --
> Kristian
>
> [1] This would be a two-step task, it requires changes both on how we
> interact with JIRA and how the VTI works.
>
>>
>> Thanks
>>
>>
>> Kathey
>>
>> -------- Original Message --------
>> Subject: [jira] [Created] (INFRA-4026) Jira no longer allows export
>> of more than 1000 issues. Gives HTTP Status 403 - You are not allowed
>> to get a result set of more than 1000 results.
>> Date: Wed, 12 Oct 2011 22:45:12 +0000 (UTC)
>> From: Kathey Marsden (Created) (JIRA) <ji...@apache.org>
>> <ma...@apache.org>
>> To: kmarsdenderby@sbcglobal.net <ma...@sbcglobal.net>
>>
>>
>>
>> Jira no longer allows export of more than 1000 issues. Gives HTTP Status 403 - You are not allowed to get a result set of more than 1000 results.
>> --------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> Key: INFRA-4026
>> URL:https://issues.apache.org/jira/browse/INFRA-4026
>> Project: Infrastructure
>> Issue Type: Bug
>> Security Level: public (Regular issues)
>> Components: JIRA
>> Reporter: Kathey Marsden
>>
>>
>> Occasionally I will export the DERBY Jira issues so I can perform queries on them with Derby Jira VTI provided in the Derby Demos. Seehttp://wiki.apache.org/db-derby/HowToRunJiraVtiReports. Once the issues are dumped I can run lots of analytic queries with the VTI without taxing Jira and without manual intervention.
>>
>> I have noticed that recently though, that the maximum rows returned from Jira has been reduced to 1000.
>>
>> e.g.
>> https://issues.apache.org/jira/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?pid=10594&sorter/field=issuekey&sorter/order=DESC&tempMax=6000&reset=true&decorator=none <https://issues.apache.org/jira/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?pid=10594&sorter/field=issuekey&sorter/order=DESC&tempMax=6000&reset=true&decorator=none>
>>
>> now yields an error:
>> HTTP Status 403 - You are not allowed to get a result set of more than 1000 results. Current search returns 5450 results
>>
>> type Status report
>>
>> message You are not allowed to get a result set of more than 1000 results. Current search returns 5450 results
>>
>> description Access to the specified resource (You are not allowed to get a result set of more than 1000 results. Current search returns 5450 results) has been forbidden.
>>
>> Can you please restore the previous defaults so the data dump can occur ?
>>
>>
>>
>>
>> .
>>
>>
>> --
>> This message is automatically generated by JIRA.
>> If you think it was sent incorrectly, please contact your JIRA administrators:https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa <https://issues.apache.org/jira/secure/ContactAdministrators%21default.jspa>
>> For more information on JIRA, see:http://www.atlassian.com/software/jira
>>
>>
>>
>
Re: Fwd: [jira] [Created] (INFRA-4026) Jira no longer allows export
of more than 1000 issues. Gives HTTP Status 403 - You are not allowed to
get a result set of more than 1000 results.
Posted by Kristian Waagan <kr...@oracle.com>.
On 10/13/11 12:48 AM, Kathey Marsden wrote:
> FYI, I filed this IFRA issue which has some impact on the Derby Jira
> VTI in our demos.
> I suppose the demo will still work but now Jira export is limited to
> 1000 issues. Is a DERBY issue in order as well?
Working around that limit is possible in this case [1] - the question is
if we want to do that?
If people are using this demo extensively as their own private "Derby
JIRA cache", I think some effort should be invested into writing
something that imposes less load on the ASF JIRA instance.
Writing something solid here may require quite a bit of effort.
Without having given this extensive thought, I think parsing emails as
suggested on infra is the kindest option. This would probably suit
committers well.
Maybe the activity stream (as an RSS feed) could be used too, but in
that case it may be harder to make sure no updates are missed.
A third option is to use the SOAP API. One has to be able to define a
suitable search (for instance 'give me all updates on Derby from date X
to date Y'), but I suspect you'd have to download all the issue
information instead of just what changed in that period. The upside with
this approach compared to the RSS one, is that you decide when you want
to update your copy.
All solutions require some kind of initial database population
functionality, which can be "abused" in the same way as the current
approach (before the JIRA limit was in place).
The requirements for the "permanent Derby JIRA cache" and the demo
differ, and it may be unwise to include something that will reach out to
the ASF JIRA instance out of the box. If I understand this correctly,
you currently have to manually obtain the XML file to be parsed by the VTI.
--
Kristian
[1] This would be a two-step task, it requires changes both on how we
interact with JIRA and how the VTI works.
>
> Thanks
>
>
> Kathey
>
> -------- Original Message --------
> Subject: [jira] [Created] (INFRA-4026) Jira no longer allows export
> of more than 1000 issues. Gives HTTP Status 403 - You are not allowed
> to get a result set of more than 1000 results.
> Date: Wed, 12 Oct 2011 22:45:12 +0000 (UTC)
> From: Kathey Marsden (Created) (JIRA) <ji...@apache.org>
> <ma...@apache.org>
> To: kmarsdenderby@sbcglobal.net <ma...@sbcglobal.net>
>
>
>
> Jira no longer allows export of more than 1000 issues. Gives HTTP Status 403 - You are not allowed to get a result set of more than 1000 results.
> --------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: INFRA-4026
> URL:https://issues.apache.org/jira/browse/INFRA-4026
> Project: Infrastructure
> Issue Type: Bug
> Security Level: public (Regular issues)
> Components: JIRA
> Reporter: Kathey Marsden
>
>
> Occasionally I will export the DERBY Jira issues so I can perform queries on them with Derby Jira VTI provided in the Derby Demos. Seehttp://wiki.apache.org/db-derby/HowToRunJiraVtiReports. Once the issues are dumped I can run lots of analytic queries with the VTI without taxing Jira and without manual intervention.
>
> I have noticed that recently though, that the maximum rows returned from Jira has been reduced to 1000.
>
> e.g.
> https://issues.apache.org/jira/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?pid=10594&sorter/field=issuekey&sorter/order=DESC&tempMax=6000&reset=true&decorator=none <https://issues.apache.org/jira/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?pid=10594&sorter/field=issuekey&sorter/order=DESC&tempMax=6000&reset=true&decorator=none>
>
> now yields an error:
> HTTP Status 403 - You are not allowed to get a result set of more than 1000 results. Current search returns 5450 results
>
> type Status report
>
> message You are not allowed to get a result set of more than 1000 results. Current search returns 5450 results
>
> description Access to the specified resource (You are not allowed to get a result set of more than 1000 results. Current search returns 5450 results) has been forbidden.
>
> Can you please restore the previous defaults so the data dump can occur ?
>
>
>
>
> .
>
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators:https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa <https://issues.apache.org/jira/secure/ContactAdministrators%21default.jspa>
> For more information on JIRA, see:http://www.atlassian.com/software/jira
>
>
>