You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Pierre Grenon <pg...@horizon-asset.co.uk> on 2019/08/06 06:15:18 UTC
Sensible size limit for SPARQL update payload to Fuseki2?
Hi,
Maybe a long shot but thought I'd ask.
I'm sending updates to a Fuseki2 from an RDFLib/SPARQLWrapper based client. This POSTs an INSERT string to an update endpoint. I get back an error 500 for strings over a certain large size which (the limit) I haven't tried to figure out.
Is there a theoretical, or other, reason why the limit exists and a strategy to adopt besides fine tuning the string size?
Many thanks and best regards,
Pierre
THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
"THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
(AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
(https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
HORIZON ASSET LLP IS AUTHORISED AND REGULATED
BY THE FINANCIAL CONDUCT AUTHORITY.
RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Pierre Grenon <pg...@horizon-asset.co.uk>.
Thanks, Andy.
I'll drop the jetty config attempt. (I surmise that the concern with bumping very high the form size would cause 'performance' issues.) I can somewhat toy with the jetty locally for testing but in any event it becomes tedious when working with remote servers deployed in varied ways. So, I'll keep the strategy client side.
With many thanks,
Pierre
THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
"THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
(AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
(https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
HORIZON ASSET LLP IS AUTHORISED AND REGULATED
BY THE FINANCIAL CONDUCT AUTHORITY.
From: Andy Seaborne [mailto:andy@apache.org]
Sent: 07 August 2019 09:45
To: users@jena.apache.org
Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
Pierre,
RDFLib/SPARQLWrapper is using an HTML form upload.
The scalable way is to POST with "Content-type:
application/sparql-update" and the INSERT in the body, then it will
stream - directly reading the update from the HTTP input stream with no
HTML Form (Request.extractFormParameters) on the execution path.
Fro an HTML form, the entire requests ends up in memory - its the way
that HTML forms have to handled to see all the name=value pairs in the
form. Incidentally, the same is true in the client.
The default form size is already bumped up to 10M from the Jetty default
of 200K.
If the server is running in verbose mode, the entire SPARQL update is
read in for logging/debugging purposes.
The default jetty configuration is in code. For the form size, that is
JettyFusekiWebapp.createWebApp which is 10M - we can make that default
bigger but not 101M which is the request.
Otherwise, break the request into parts and send multiple requests.
Andy
On 07/08/2019 08:49, Pierre Grenon wrote:
> Thank you, Lorenz.
>
> I did as you suggest and made the changes indicated.
>
> Fuseki started and seems to have accepted the jetty config. But then when trying to send the update the same error occurs and the limit seems unmodified (I used <Arg>200000000</Arg>).
>
> Caused by: java.lang.IllegalStateException: Form too large: 100948991 > 10000000
> at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:545)
> at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:475)
> at org.eclipse.jetty.server.Request.getParameters(Request.java:386)
> ... 50 more
>
> Can it be that the config does not override some default set elsewhere in Fuseki?
>
> I’ll try to figure if I’m not doing something else wrong…
>
> Many thanks,
> Pierre
>
> For reference:
> https://www.eclipse.org/jetty/documentation/current/configuring-form-size.html
RE: RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Pierre Grenon <pg...@horizon-asset.co.uk>.
As indicated by Andy, not with that library but just using requests work -- see follow up email
Best,
P
THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
"THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
(AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
(https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
HORIZON ASSET LLP IS AUTHORISED AND REGULATED
BY THE FINANCIAL CONDUCT AUTHORITY.
> -----Original Message-----
> From: Laura Morales [mailto:lauretas@mail.com]
> Sent: 07 August 2019 12:19
> To: users@jena.apache.org
> Cc: users@jena.apache.org
> Subject: Re: RE: Sensible size limit for SPARQL update payload to Fuseki2?
>
> > from SPARQLWrapper import SPARQLWrapper
> >
> > # a lot longer
> > myString = "INSERT DATA { <not> <noot> <noot> }"
> >
> > def insertFromString(url, sparql):
> > endpoint = SPARQLWrapper(url)
> > endpoint.setQuery(sparql)
> > endpoint.method = 'POST'
> > endpoint.query()
> >
> > insertFromString('http://localhost:3030/myDS/update', myString)
>
>
> can you change the html headers and see if it works? Something like this:
>
> endpoint.addCustomHttpHeader("Content-Type", "application/sparql-
> update")
Re: RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Laura Morales <la...@mail.com>.
> from SPARQLWrapper import SPARQLWrapper
>
> # a lot longer
> myString = "INSERT DATA { <not> <noot> <noot> }"
>
> def insertFromString(url, sparql):
> endpoint = SPARQLWrapper(url)
> endpoint.setQuery(sparql)
> endpoint.method = 'POST'
> endpoint.query()
>
> insertFromString('http://localhost:3030/myDS/update', myString)
can you change the html headers and see if it works? Something like this:
endpoint.addCustomHttpHeader("Content-Type", "application/sparql-update")
RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Pierre Grenon <pg...@horizon-asset.co.uk>.
Following Andy's suggestion works perfectly indeed --- many thanks again!
Best,
Pierre
import requests
THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
"THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
(AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
(https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
HORIZON ASSET LLP IS AUTHORISED AND REGULATED
BY THE FINANCIAL CONDUCT AUTHORITY.
def insertFromString(url, sparql):
req_ = requests.post(url, headers = {"Content-Type":"application/sparql-update"}, data = sparql)
resp_ = req_.content
return(resp_)
insertFromString('http://localhost:3030/myDS/update', myString)
Then
queryFn(qls2, "select (Count(?x) as ?noot) where {?x ?y ?z}")
noot.value
0 899401
> -----Original Message-----
> From: Pierre Grenon
> Sent: 07 August 2019 12:10
> To: 'users@jena.apache.org'
> Subject: RE: Sensible size limit for SPARQL update payload to Fuseki2?
>
>
> FYC I basically do this
>
> from SPARQLWrapper import SPARQLWrapper
>
> # a lot longer
> myString = "INSERT DATA { <not> <noot> <noot> }"
>
> def insertFromString(url, sparql):
> endpoint = SPARQLWrapper(url)
> endpoint.setQuery(sparql)
> endpoint.method = 'POST'
> endpoint.query()
>
> insertFromString('http://localhost:3030/myDS/update', myString)
>
>
> From: Laura Morales [mailto:lauretas@mail.com]
> Sent: 07 August 2019 10:29
> To: users@jena.apache.org
> Cc: users@jena.apache.org
> Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
>
> Basically, this request should work?
>
> POST /database HTTP/1.1
> Host: example.com
> Content-Type: application/sparql-update
>
> INSERT DATA { < 100 MB of triples > }
>
>
>
>
> > Sent: Wednesday, August 07, 2019 at 10:44 AM
> > From: "Andy Seaborne" <an...@apache.org>
> > To: users@jena.apache.org
> > Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
> >
> > Pierre,
> >
> > RDFLib/SPARQLWrapper is using an HTML form upload.
> >
> > The scalable way is to POST with "Content-type:
> > application/sparql-update" and the INSERT in the body, then it will
> > stream - directly reading the update from the HTTP input stream with no
> > HTML Form (Request.extractFormParameters) on the execution path.
> >
> > Fro an HTML form, the entire requests ends up in memory - its the way
> > that HTML forms have to handled to see all the name=value pairs in the
> > form. Incidentally, the same is true in the client.
> >
> > The default form size is already bumped up to 10M from the Jetty default
> > of 200K.
> >
> > If the server is running in verbose mode, the entire SPARQL update is
> > read in for logging/debugging purposes.
> >
> > The default jetty configuration is in code. For the form size, that is
> > JettyFusekiWebapp.createWebApp which is 10M - we can make that
> default
> > bigger but not 101M which is the request.
> >
> > Otherwise, break the request into parts and send multiple requests.
> >
> > Andy
> >
> > On 07/08/2019 08:49, Pierre Grenon wrote:
> > > Thank you, Lorenz.
> > >
> > > I did as you suggest and made the changes indicated.
> > >
> > > Fuseki started and seems to have accepted the jetty config. But then
> when trying to send the update the same error occurs and the limit seems
> unmodified (I used <Arg>200000000</Arg>).
> > >
> > > Caused by: java.lang.IllegalStateException: Form too large: 100948991 >
> 10000000
> > > at
> org.eclipse.jetty.server.Request.extractFormParameters(Request.java:545)
> > > at
> org.eclipse.jetty.server.Request.extractContentParameters(Request.java:475)
> > > at org.eclipse.jetty.server.Request.getParameters(Request.java:386)
> > > ... 50 more
> > >
> > > Can it be that the config does not override some default set elsewhere in
> Fuseki?
> > >
> > > I’ll try to figure if I’m not doing something else wrong…
> > >
> > > Many thanks,
> > > Pierre
> > >
> > > For reference:
> > > https://www.eclipse.org/jetty/documentation/current/configuring-form-
> size.html
> >
RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Pierre Grenon <pg...@horizon-asset.co.uk>.
FYC I basically do this
from SPARQLWrapper import SPARQLWrapper
# a lot longer
myString = "INSERT DATA { <not> <noot> <noot> }"
def insertFromString(url, sparql):
endpoint = SPARQLWrapper(url)
endpoint.setQuery(sparql)
endpoint.method = 'POST'
endpoint.query()
insertFromString('http://localhost:3030/myDS/update', myString)
THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
"THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
(AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
(https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
HORIZON ASSET LLP IS AUTHORISED AND REGULATED
BY THE FINANCIAL CONDUCT AUTHORITY.
From: Laura Morales [mailto:lauretas@mail.com]
Sent: 07 August 2019 10:29
To: users@jena.apache.org
Cc: users@jena.apache.org
Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
Basically, this request should work?
POST /database HTTP/1.1
Host: example.com
Content-Type: application/sparql-update
INSERT DATA { < 100 MB of triples > }
> Sent: Wednesday, August 07, 2019 at 10:44 AM
> From: "Andy Seaborne" <an...@apache.org>
> To: users@jena.apache.org
> Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
>
> Pierre,
>
> RDFLib/SPARQLWrapper is using an HTML form upload.
>
> The scalable way is to POST with "Content-type:
> application/sparql-update" and the INSERT in the body, then it will
> stream - directly reading the update from the HTTP input stream with no
> HTML Form (Request.extractFormParameters) on the execution path.
>
> Fro an HTML form, the entire requests ends up in memory - its the way
> that HTML forms have to handled to see all the name=value pairs in the
> form. Incidentally, the same is true in the client.
>
> The default form size is already bumped up to 10M from the Jetty default
> of 200K.
>
> If the server is running in verbose mode, the entire SPARQL update is
> read in for logging/debugging purposes.
>
> The default jetty configuration is in code. For the form size, that is
> JettyFusekiWebapp.createWebApp which is 10M - we can make that default
> bigger but not 101M which is the request.
>
> Otherwise, break the request into parts and send multiple requests.
>
> Andy
>
> On 07/08/2019 08:49, Pierre Grenon wrote:
> > Thank you, Lorenz.
> >
> > I did as you suggest and made the changes indicated.
> >
> > Fuseki started and seems to have accepted the jetty config. But then when trying to send the update the same error occurs and the limit seems unmodified (I used <Arg>200000000</Arg>).
> >
> > Caused by: java.lang.IllegalStateException: Form too large: 100948991 > 10000000
> > at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:545)
> > at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:475)
> > at org.eclipse.jetty.server.Request.getParameters(Request.java:386)
> > ... 50 more
> >
> > Can it be that the config does not override some default set elsewhere in Fuseki?
> >
> > I’ll try to figure if I’m not doing something else wrong…
> >
> > Many thanks,
> > Pierre
> >
> > For reference:
> > https://www.eclipse.org/jetty/documentation/current/configuring-form-size.html
>
Re: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Laura Morales <la...@mail.com>.
Basically, this request should work?
POST /database HTTP/1.1
Host: example.com
Content-Type: application/sparql-update
INSERT DATA { < 100 MB of triples > }
> Sent: Wednesday, August 07, 2019 at 10:44 AM
> From: "Andy Seaborne" <an...@apache.org>
> To: users@jena.apache.org
> Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
>
> Pierre,
>
> RDFLib/SPARQLWrapper is using an HTML form upload.
>
> The scalable way is to POST with "Content-type:
> application/sparql-update" and the INSERT in the body, then it will
> stream - directly reading the update from the HTTP input stream with no
> HTML Form (Request.extractFormParameters) on the execution path.
>
> Fro an HTML form, the entire requests ends up in memory - its the way
> that HTML forms have to handled to see all the name=value pairs in the
> form. Incidentally, the same is true in the client.
>
> The default form size is already bumped up to 10M from the Jetty default
> of 200K.
>
> If the server is running in verbose mode, the entire SPARQL update is
> read in for logging/debugging purposes.
>
> The default jetty configuration is in code. For the form size, that is
> JettyFusekiWebapp.createWebApp which is 10M - we can make that default
> bigger but not 101M which is the request.
>
> Otherwise, break the request into parts and send multiple requests.
>
> Andy
>
> On 07/08/2019 08:49, Pierre Grenon wrote:
> > Thank you, Lorenz.
> >
> > I did as you suggest and made the changes indicated.
> >
> > Fuseki started and seems to have accepted the jetty config. But then when trying to send the update the same error occurs and the limit seems unmodified (I used <Arg>200000000</Arg>).
> >
> > Caused by: java.lang.IllegalStateException: Form too large: 100948991 > 10000000
> > at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:545)
> > at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:475)
> > at org.eclipse.jetty.server.Request.getParameters(Request.java:386)
> > ... 50 more
> >
> > Can it be that the config does not override some default set elsewhere in Fuseki?
> >
> > I’ll try to figure if I’m not doing something else wrong…
> >
> > Many thanks,
> > Pierre
> >
> > For reference:
> > https://www.eclipse.org/jetty/documentation/current/configuring-form-size.html
>
Re: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Andy Seaborne <an...@apache.org>.
Pierre,
RDFLib/SPARQLWrapper is using an HTML form upload.
The scalable way is to POST with "Content-type:
application/sparql-update" and the INSERT in the body, then it will
stream - directly reading the update from the HTTP input stream with no
HTML Form (Request.extractFormParameters) on the execution path.
Fro an HTML form, the entire requests ends up in memory - its the way
that HTML forms have to handled to see all the name=value pairs in the
form. Incidentally, the same is true in the client.
The default form size is already bumped up to 10M from the Jetty default
of 200K.
If the server is running in verbose mode, the entire SPARQL update is
read in for logging/debugging purposes.
The default jetty configuration is in code. For the form size, that is
JettyFusekiWebapp.createWebApp which is 10M - we can make that default
bigger but not 101M which is the request.
Otherwise, break the request into parts and send multiple requests.
Andy
On 07/08/2019 08:49, Pierre Grenon wrote:
> Thank you, Lorenz.
>
> I did as you suggest and made the changes indicated.
>
> Fuseki started and seems to have accepted the jetty config. But then when trying to send the update the same error occurs and the limit seems unmodified (I used <Arg>200000000</Arg>).
>
> Caused by: java.lang.IllegalStateException: Form too large: 100948991 > 10000000
> at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:545)
> at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:475)
> at org.eclipse.jetty.server.Request.getParameters(Request.java:386)
> ... 50 more
>
> Can it be that the config does not override some default set elsewhere in Fuseki?
>
> I’ll try to figure if I’m not doing something else wrong…
>
> Many thanks,
> Pierre
>
> For reference:
> https://www.eclipse.org/jetty/documentation/current/configuring-form-size.html
Re: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Martynas Jusevičius <ma...@atomgraph.com>.
Pierre,
what are you trying to do? Does the INSERT contain some variables/do
some pattern matching?
If not (e.g. it's INSERT DATA), then you might be better off using the
Graph Store Protocol:
https://jena.apache.org/documentation/fuseki2/soh.html#soh-sparql-http
On Wed, Aug 7, 2019 at 9:49 AM Pierre Grenon
<pg...@horizon-asset.co.uk> wrote:
>
> Thank you, Lorenz.
>
> I did as you suggest and made the changes indicated.
>
> Fuseki started and seems to have accepted the jetty config. But then when trying to send the update the same error occurs and the limit seems unmodified (I used <Arg>200000000</Arg>).
>
> Caused by: java.lang.IllegalStateException: Form too large: 100948991 > 10000000
> at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:545)
> at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:475)
> at org.eclipse.jetty.server.Request.getParameters(Request.java:386)
> ... 50 more
>
> Can it be that the config does not override some default set elsewhere in Fuseki?
>
> I’ll try to figure if I’m not doing something else wrong…
>
> Many thanks,
> Pierre
>
> For reference:
> https://www.eclipse.org/jetty/documentation/current/configuring-form-size.html
>
>
>
> From: Lorenz Buehmann [mailto:buehmann@informatik.uni-leipzig.de]
> Sent: 07 August 2019 08:08
> To: users@jena.apache.org
> Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
>
>
>
> > <?xml version="1.0"?>
> > <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd<http://www.eclipse.org/jetty/configure_9_3.dtd>">
> >
> > <Configure id="Server" class="org.eclipse.jetty.server.Server">
> > <Call name="setAttribute">
> > <Arg>org.eclipse.jetty.server.Request.maxFormContentSize</Arg>
> > <Arg><Property name="jetty.maxFormContentSize" default="1000000"/></Arg>
> > </Call></Configure>
> >
> > [2019-08-06 17:07:48] Server ERROR SPARQLServer: Failed to configure server: 0
> > java.lang.ArrayIndexOutOfBoundsException: 0
> > at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.configServer(JettyFusekiWebapp.java:297)
> > at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.buildServerWebapp(JettyFusekiWebapp.java:243)
> > at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.<init>(http://JettyFusekiWebapp.java:99<http://JettyFusekiWebapp.java:99>)
> > at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.initializeServer(JettyFusekiWebapp.java:94)
> > at org.apache.jena.fuseki.cmd.FusekiCmd.runFuseki(FusekiCmd.java:371)
> > at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:356)
> > at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
> > at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
> > at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
> > at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:104)
> > at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:67)
> >
> > So I suppose I need a complete jetty config file rather than a snippet (unless the above is erroneous anyway). I wasn't able to find the default jetty configuration file in the jars.
> >
> > I found this https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml<https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml>
> > But it mentions needing configuring further things and I have no clue how to adapt it.
>
> if you don't need HTTPS resp. SSL, you can just remove everything that
> configures the SSL stuff:
> https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml#L179-L285<https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml#L179-L285>
>
>
> And then add
>
> <Call name="setAttribute">
> <Arg>org.eclipse.jetty.server.Request.maxFormContentSize</Arg>
> <Arg>2000000</Arg>
> </Call>
>
> with whatever size you need.
>
> And don't forget to change the port, in the file it's configured with
> 8082 compared to default 3030.
>
> THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
> IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
> IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
> E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
> MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
>
> IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
> MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
> "THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
> MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
> FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
> (AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
> (https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
> COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
> AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
> ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
> PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
>
> HORIZON ASSET LLP IS AUTHORISED AND REGULATED
> BY THE FINANCIAL CONDUCT AUTHORITY.
>
>
RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Pierre Grenon <pg...@horizon-asset.co.uk>.
Thank you, Lorenz.
I did as you suggest and made the changes indicated.
Fuseki started and seems to have accepted the jetty config. But then when trying to send the update the same error occurs and the limit seems unmodified (I used <Arg>200000000</Arg>).
Caused by: java.lang.IllegalStateException: Form too large: 100948991 > 10000000
at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:545)
at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:475)
at org.eclipse.jetty.server.Request.getParameters(Request.java:386)
... 50 more
Can it be that the config does not override some default set elsewhere in Fuseki?
I’ll try to figure if I’m not doing something else wrong…
Many thanks,
Pierre
For reference:
https://www.eclipse.org/jetty/documentation/current/configuring-form-size.html
From: Lorenz Buehmann [mailto:buehmann@informatik.uni-leipzig.de]
Sent: 07 August 2019 08:08
To: users@jena.apache.org
Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
> <?xml version="1.0"?>
> <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd<http://www.eclipse.org/jetty/configure_9_3.dtd>">
>
> <Configure id="Server" class="org.eclipse.jetty.server.Server">
> <Call name="setAttribute">
> <Arg>org.eclipse.jetty.server.Request.maxFormContentSize</Arg>
> <Arg><Property name="jetty.maxFormContentSize" default="1000000"/></Arg>
> </Call></Configure>
>
> [2019-08-06 17:07:48] Server ERROR SPARQLServer: Failed to configure server: 0
> java.lang.ArrayIndexOutOfBoundsException: 0
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.configServer(JettyFusekiWebapp.java:297)
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.buildServerWebapp(JettyFusekiWebapp.java:243)
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.<init>(http://JettyFusekiWebapp.java:99<http://JettyFusekiWebapp.java:99>)
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.initializeServer(JettyFusekiWebapp.java:94)
> at org.apache.jena.fuseki.cmd.FusekiCmd.runFuseki(FusekiCmd.java:371)
> at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:356)
> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
> at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:104)
> at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:67)
>
> So I suppose I need a complete jetty config file rather than a snippet (unless the above is erroneous anyway). I wasn't able to find the default jetty configuration file in the jars.
>
> I found this https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml<https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml>
> But it mentions needing configuring further things and I have no clue how to adapt it.
if you don't need HTTPS resp. SSL, you can just remove everything that
configures the SSL stuff:
https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml#L179-L285<https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml#L179-L285>
And then add
<Call name="setAttribute">
<Arg>org.eclipse.jetty.server.Request.maxFormContentSize</Arg>
<Arg>2000000</Arg>
</Call>
with whatever size you need.
And don't forget to change the port, in the file it's configured with
8082 compared to default 3030.
THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
"THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
(AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
(https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
HORIZON ASSET LLP IS AUTHORISED AND REGULATED
BY THE FINANCIAL CONDUCT AUTHORITY.
Re: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Lorenz Buehmann <bu...@informatik.uni-leipzig.de>.
> <?xml version="1.0"?>
> <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">
>
> <Configure id="Server" class="org.eclipse.jetty.server.Server">
> <Call name="setAttribute">
> <Arg>org.eclipse.jetty.server.Request.maxFormContentSize</Arg>
> <Arg><Property name="jetty.maxFormContentSize" default="1000000"/></Arg>
> </Call></Configure>
>
> [2019-08-06 17:07:48] Server ERROR SPARQLServer: Failed to configure server: 0
> java.lang.ArrayIndexOutOfBoundsException: 0
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.configServer(JettyFusekiWebapp.java:297)
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.buildServerWebapp(JettyFusekiWebapp.java:243)
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.<init>(JettyFusekiWebapp.java:99)
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.initializeServer(JettyFusekiWebapp.java:94)
> at org.apache.jena.fuseki.cmd.FusekiCmd.runFuseki(FusekiCmd.java:371)
> at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:356)
> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
> at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:104)
> at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:67)
>
> So I suppose I need a complete jetty config file rather than a snippet (unless the above is erroneous anyway). I wasn't able to find the default jetty configuration file in the jars.
>
> I found this https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml
> But it mentions needing configuring further things and I have no clue how to adapt it.
if you don't need HTTPS resp. SSL, you can just remove everything that
configures the SSL stuff:
https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml#L179-L285
And then add
<Call name="setAttribute">
<Arg>org.eclipse.jetty.server.Request.maxFormContentSize</Arg>
<Arg>2000000</Arg>
</Call>
with whatever size you need.
And don't forget to change the port, in the file it's configured with
8082 compared to default 3030.
RE: RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Pierre Grenon <pg...@horizon-asset.co.uk>.
Thank you, Laura, that’s a good point too.
Meanwhile, I guess the answer to the original question, judging from the error indicating the default jetty form size in fuseki is that the limit is 10Mb.
I’ll chunk away or maybe work around with the LOAD requirements, although I might open a new thread specifically on jetty if I get very frustrated.
With many thanks and best regards,
Pierre
THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
"THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
(AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
(https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
HORIZON ASSET LLP IS AUTHORISED AND REGULATED
BY THE FINANCIAL CONDUCT AUTHORITY.
From: Laura Morales [mailto:lauretas@mail.com]
Sent: 06 August 2019 18:29
To: users@jena.apache.org
Cc: 'users@jena.apache.org'
Subject: Re: RE: Sensible size limit for SPARQL update payload to Fuseki2?
I don't know myself how to configure jetty, even less with Fuseki. I hope for you somebody else on the list does.
Regarding LOAD however, you should be able to use HTTP URIs.
> Sent: Tuesday, August 06, 2019 at 7:01 PM
> From: "Pierre Grenon" <pg...@horizon-asset.co.uk>
> To: "'users@jena.apache.org'" <us...@jena.apache.org>
> Subject: RE: Sensible size limit for SPARQL update payload to Fuseki2?
>
> Ok, so apologies for kinda spamming the list with this.
>
> 1. Laura, I agree with the options you listed below. Although:
> - LOAD, in my experience, requires access to the file system where fuseki is running and I do not have that
> - SOH has that same requirement and also requires me to ssh into the machine, which I don't want to have to do programmatically
> - chunking is my likely work around although it is suboptimal (I serialize an RDFLib in memory graph)
>
> 2. After looking around and doing a bit of archaeology,
>
> https://jena.markmail.org/message/nmtny6wlnvzltws7?q=maxFormContentSize<https://jena.markmail.org/message/nmtny6wlnvzltws7?q=maxFormContentSize>
> (At first seeing this I thought it used to be called 'Fuseky'! I can only recall Joseki)
>
> it seems that the principled approach is to run fuseki with a customised jetty configuration.
>
> http://jena.apache.org/documentation/fuseki2/data-access-control#jetty-configuration<http://jena.apache.org/documentation/fuseki2/data-access-control#jetty-configuration>
> " Server command line: --jetty=jetty.xml."
> -> is wrong
>
> This:
> > fuseki-server --jetty-config=jetty.xml
> Worked for me.
>
> However, I do not know what to put in jetty.xml
>
> https://www.eclipse.org/jetty/documentation/current/setting-form-size.html<https://www.eclipse.org/jetty/documentation/current/setting-form-size.html>
>
> I tried the following snippet but it broke
>
> <?xml version="1.0"?>
> <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd<http://www.eclipse.org/jetty/configure_9_3.dtd>">
>
> <Configure id="Server" class="org.eclipse.jetty.server.Server">
> <Call name="setAttribute">
> <Arg>org.eclipse.jetty.server.Request.maxFormContentSize</Arg>
> <Arg><Property name="jetty.maxFormContentSize" default="1000000"/></Arg>
> </Call></Configure>
>
> [2019-08-06 17:07:48] Server ERROR SPARQLServer: Failed to configure server: 0
> java.lang.ArrayIndexOutOfBoundsException: 0
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.configServer(JettyFusekiWebapp.java:297)
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.buildServerWebapp(JettyFusekiWebapp.java:243)
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.<init>(http://JettyFusekiWebapp.java:99<http://JettyFusekiWebapp.java:99>)
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.initializeServer(JettyFusekiWebapp.java:94)
> at org.apache.jena.fuseki.cmd.FusekiCmd.runFuseki(FusekiCmd.java:371)
> at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:356)
> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
> at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:104)
> at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:67)
>
> So I suppose I need a complete jetty config file rather than a snippet (unless the above is erroneous anyway). I wasn't able to find the default jetty configuration file in the jars.
>
> I found this https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml<https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml>
> But it mentions needing configuring further things and I have no clue how to adapt it.
>
> Any pointer, walkthrough or further help most appreciated.
>
> With many thanks and kind regards,
> Pierre
Re: RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Laura Morales <la...@mail.com>.
I don't know myself how to configure jetty, even less with Fuseki. I hope for you somebody else on the list does.
Regarding LOAD however, you should be able to use HTTP URIs.
> Sent: Tuesday, August 06, 2019 at 7:01 PM
> From: "Pierre Grenon" <pg...@horizon-asset.co.uk>
> To: "'users@jena.apache.org'" <us...@jena.apache.org>
> Subject: RE: Sensible size limit for SPARQL update payload to Fuseki2?
>
> Ok, so apologies for kinda spamming the list with this.
>
> 1. Laura, I agree with the options you listed below. Although:
> - LOAD, in my experience, requires access to the file system where fuseki is running and I do not have that
> - SOH has that same requirement and also requires me to ssh into the machine, which I don't want to have to do programmatically
> - chunking is my likely work around although it is suboptimal (I serialize an RDFLib in memory graph)
>
> 2. After looking around and doing a bit of archaeology,
>
> https://jena.markmail.org/message/nmtny6wlnvzltws7?q=maxFormContentSize
> (At first seeing this I thought it used to be called 'Fuseky'! I can only recall Joseki)
>
> it seems that the principled approach is to run fuseki with a customised jetty configuration.
>
> http://jena.apache.org/documentation/fuseki2/data-access-control#jetty-configuration
> " Server command line: --jetty=jetty.xml."
> -> is wrong
>
> This:
> > fuseki-server --jetty-config=jetty.xml
> Worked for me.
>
> However, I do not know what to put in jetty.xml
>
> https://www.eclipse.org/jetty/documentation/current/setting-form-size.html
>
> I tried the following snippet but it broke
>
> <?xml version="1.0"?>
> <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">
>
> <Configure id="Server" class="org.eclipse.jetty.server.Server">
> <Call name="setAttribute">
> <Arg>org.eclipse.jetty.server.Request.maxFormContentSize</Arg>
> <Arg><Property name="jetty.maxFormContentSize" default="1000000"/></Arg>
> </Call></Configure>
>
> [2019-08-06 17:07:48] Server ERROR SPARQLServer: Failed to configure server: 0
> java.lang.ArrayIndexOutOfBoundsException: 0
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.configServer(JettyFusekiWebapp.java:297)
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.buildServerWebapp(JettyFusekiWebapp.java:243)
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.<init>(JettyFusekiWebapp.java:99)
> at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.initializeServer(JettyFusekiWebapp.java:94)
> at org.apache.jena.fuseki.cmd.FusekiCmd.runFuseki(FusekiCmd.java:371)
> at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:356)
> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
> at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:104)
> at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:67)
>
> So I suppose I need a complete jetty config file rather than a snippet (unless the above is erroneous anyway). I wasn't able to find the default jetty configuration file in the jars.
>
> I found this https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml
> But it mentions needing configuring further things and I have no clue how to adapt it.
>
> Any pointer, walkthrough or further help most appreciated.
>
> With many thanks and kind regards,
> Pierre
RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Pierre Grenon <pg...@horizon-asset.co.uk>.
Ok, so apologies for kinda spamming the list with this.
1. Laura, I agree with the options you listed below. Although:
- LOAD, in my experience, requires access to the file system where fuseki is running and I do not have that
- SOH has that same requirement and also requires me to ssh into the machine, which I don't want to have to do programmatically
- chunking is my likely work around although it is suboptimal (I serialize an RDFLib in memory graph)
2. After looking around and doing a bit of archaeology,
https://jena.markmail.org/message/nmtny6wlnvzltws7?q=maxFormContentSize
(At first seeing this I thought it used to be called 'Fuseky'! I can only recall Joseki)
it seems that the principled approach is to run fuseki with a customised jetty configuration.
http://jena.apache.org/documentation/fuseki2/data-access-control#jetty-configuration
" Server command line: --jetty=jetty.xml."
-> is wrong
This:
> fuseki-server --jetty-config=jetty.xml
Worked for me.
However, I do not know what to put in jetty.xml
https://www.eclipse.org/jetty/documentation/current/setting-form-size.html
I tried the following snippet but it broke
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<Call name="setAttribute">
<Arg>org.eclipse.jetty.server.Request.maxFormContentSize</Arg>
<Arg><Property name="jetty.maxFormContentSize" default="1000000"/></Arg>
</Call></Configure>
[2019-08-06 17:07:48] Server ERROR SPARQLServer: Failed to configure server: 0
java.lang.ArrayIndexOutOfBoundsException: 0
at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.configServer(JettyFusekiWebapp.java:297)
at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.buildServerWebapp(JettyFusekiWebapp.java:243)
at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.<init>(JettyFusekiWebapp.java:99)
at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.initializeServer(JettyFusekiWebapp.java:94)
at org.apache.jena.fuseki.cmd.FusekiCmd.runFuseki(FusekiCmd.java:371)
at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:356)
at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:104)
at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:67)
So I suppose I need a complete jetty config file rather than a snippet (unless the above is erroneous anyway). I wasn't able to find the default jetty configuration file in the jars.
I found this https://github.com/apache/jena/blob/master/jena-fuseki2/examples/fuseki-jetty-https.xml
But it mentions needing configuring further things and I have no clue how to adapt it.
Any pointer, walkthrough or further help most appreciated.
With many thanks and kind regards,
Pierre
THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
"THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
(AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
(https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
HORIZON ASSET LLP IS AUTHORISED AND REGULATED
BY THE FINANCIAL CONDUCT AUTHORITY.
From: Laura Morales [mailto:lauretas@mail.com]
Sent: 06 August 2019 12:25
To: users@jena.apache.org
Cc: 'users@jena.apache.org'
Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
Your best option is to look at the Fuseki logs for the exact error. I've personally never POSTed so much data to Fuseki, but I feel like it should not be be a problem unless something is timing out the connection, or truncating the POST data, or your triples contain syntax errors. Another option is to try the LOAD operation (https://www.w3.org/TR/sparql11-update/#load), or the SOH command line tools (https://jena.apache.org/documentation/fuseki2/soh.html). What I would do, personally speaking, is find a way to chunk your data and send multiple requests (even if, as I said, 85MB should work. It's not a huge file after all).
> Sent: Tuesday, August 06, 2019 at 12:39 PM
> From: "Pierre Grenon" <pg...@horizon-asset.co.uk>
> To: "'users@jena.apache.org'" <us...@jena.apache.org>
> Subject: RE: RE: Sensible size limit for SPARQL update payload to Fuseki2?
>
> > Can you share your query?
>
> Afraid I can't
>
> It looks like :
>
> <PREFIX>
>
> INSERT DATA {
>
> <9000K triples>
>
> }
>
> > I don't understand if you're trying to insert a single literal string that is 85MB in size, or if you're trying to load 900K triples that are 85MB in total.
>
> Second one. I'm not trying to load a single triple with a 85Mb object literal but I am trying to perform a single INSERT operation of 900k triples.
>
> Many thanks,
> Pierre
Re: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Laura Morales <la...@mail.com>.
Your best option is to look at the Fuseki logs for the exact error. I've personally never POSTed so much data to Fuseki, but I feel like it should not be be a problem unless something is timing out the connection, or truncating the POST data, or your triples contain syntax errors. Another option is to try the LOAD operation (https://www.w3.org/TR/sparql11-update/#load), or the SOH command line tools (https://jena.apache.org/documentation/fuseki2/soh.html). What I would do, personally speaking, is find a way to chunk your data and send multiple requests (even if, as I said, 85MB should work. It's not a huge file after all).
> Sent: Tuesday, August 06, 2019 at 12:39 PM
> From: "Pierre Grenon" <pg...@horizon-asset.co.uk>
> To: "'users@jena.apache.org'" <us...@jena.apache.org>
> Subject: RE: RE: Sensible size limit for SPARQL update payload to Fuseki2?
>
> > Can you share your query?
>
> Afraid I can't
>
> It looks like :
>
> <PREFIX>
>
> INSERT DATA {
>
> <9000K triples>
>
> }
>
> > I don't understand if you're trying to insert a single literal string that is 85MB in size, or if you're trying to load 900K triples that are 85MB in total.
>
> Second one. I'm not trying to load a single triple with a 85Mb object literal but I am trying to perform a single INSERT operation of 900k triples.
>
> Many thanks,
> Pierre
RE: RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Pierre Grenon <pg...@horizon-asset.co.uk>.
> Can you share your query?
Afraid I can't
It looks like :
<PREFIX>
INSERT DATA {
<9000K triples>
}
> I don't understand if you're trying to insert a single literal string that is 85MB in size, or if you're trying to load 900K triples that are 85MB in total.
Second one. I'm not trying to load a single triple with a 85Mb object literal but I am trying to perform a single INSERT operation of 900k triples.
Many thanks,
Pierre
THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
"THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
(AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
(https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
HORIZON ASSET LLP IS AUTHORISED AND REGULATED
BY THE FINANCIAL CONDUCT AUTHORITY.
From: Laura Morales [mailto:lauretas@mail.com]
Sent: 06 August 2019 09:12
To: users@jena.apache.org
Cc: users@jena.apache.org
Subject: Re: RE: Sensible size limit for SPARQL update payload to Fuseki2?
Can you share your query? I don't understand if you're trying to insert a single literal string that is 85MB in size, or if you're trying to load 900K triples that are 85MB in total.
> Sent: Tuesday, August 06, 2019 at 9:52 AM
> From: "Pierre Grenon" <pg...@horizon-asset.co.uk>
> To: "users@jena.apache.org" <us...@jena.apache.org>
> Subject: RE: Sensible size limit for SPARQL update payload to Fuseki2?
>
> Quick follow up --
>
> The web fuseki interface is happy loading the turtle equivalent. Takes more time (~10-15 sec) than it takes for the programmatic way to return an error (<2 sec), it's about 900k triples.
>
> So I was thinking of splitting my INSERT and was wondering if there is a reasonable chunk size.
>
> As said, will try to look at the logs anyway.
>
> Best,
> Pierre
>
> THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
> IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
> IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
> E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
> MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
>
> IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
> MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
> "THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
> MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
> FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
> (AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
> (https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
> COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
> AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
> ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
> PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
>
> HORIZON ASSET LLP IS AUTHORISED AND REGULATED
> BY THE FINANCIAL CONDUCT AUTHORITY.
>
>
> > -----Original Message-----
> > From: Pierre Grenon
> > Sent: 06 August 2019 08:47
> > To: 'users@jena.apache.org'
> > Subject: RE: Sensible size limit for SPARQL update payload to Fuseki2?
> >
> > Hi,
> >
> > Thanks for your answer.
> >
> > I'll look into the server's logs if I can. I am using this approach because the
> > server is in fact remote. So I read into a string that I then pass to POST. The
> > query when saved to file is ~ 85Mb.
> >
> > I guess I was lazy and hoping for an easy answer if this clogged up Fuseki in
> > known ways and whether that might be addressed through config (memory,
> > thread, whatnots).
> >
> > Will double check a few things and report then.
> >
> > With many thanks and best regards,
> > Pierre
> >
> >
> > From: Laura Morales [mailto:lauretas@mail.com]
> > Sent: 06 August 2019 08:19
> > To: users@jena.apache.org
> > Cc: users@jena.apache.org
> > Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
> >
> > How long is your query?? Personally I'm not aware of any such limitations,
> > especially when POSTing, but other people here definitely know better than
> > me if there is one or not. If there is a limit, let's say even just 1MB, you need
> > a *very* long query to break it. 500 is a general "internal error" code, it's not
> > specific to query length. It could be something else (look at the logs). Did you
> > try to run the query from the Fuseki web interface, or even with another
> > database entirely? It could help you debug it.
> >
> >
> >
> > > Sent: Tuesday, August 06, 2019 at 8:15 AM
> > > From: "Pierre Grenon" <pg...@horizon-asset.co.uk>
> > > To: "users@jena.apache.org" <us...@jena.apache.org>
> > > Subject: Sensible size limit for SPARQL update payload to Fuseki2?
> > >
> > > Hi,
> > >
> > > Maybe a long shot but thought I'd ask.
> > >
> > > I'm sending updates to a Fuseki2 from an RDFLib/SPARQLWrapper based
> > client. This POSTs an INSERT string to an update endpoint. I get back an error
> > 500 for strings over a certain large size which (the limit) I haven't tried to
> > figure out.
> > >
> > > Is there a theoretical, or other, reason why the limit exists and a strategy
> > to adopt besides fine tuning the string size?
> > >
> > > Many thanks and best regards,
> > > Pierre
> > >
> > > THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED
> > INFORMATION.
> > > IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
> > > IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
> > > E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF
> > THE
> > > MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
> > >
> > > IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S
> > EMPLOYEES
> > > MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA
> > HANDBOOK AS
> > > "THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN
> > AN INVESTMENT
> > > MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH
> > MEETINGS, THE
> > > FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE
> > INFORMATION
> > > (AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO
> > 596/2014).
> > > (https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
> > > COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF
> > REGULATION
> > > AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR
> > INFORMATION
> > > ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET
> > SOUNDINGS,
> > > PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
> > >
> > > HORIZON ASSET LLP IS AUTHORISED AND REGULATED
> > > BY THE FINANCIAL CONDUCT AUTHORITY.
> > >
> > >
> > >
>
Re: RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Laura Morales <la...@mail.com>.
Can you share your query? I don't understand if you're trying to insert a single literal string that is 85MB in size, or if you're trying to load 900K triples that are 85MB in total.
> Sent: Tuesday, August 06, 2019 at 9:52 AM
> From: "Pierre Grenon" <pg...@horizon-asset.co.uk>
> To: "users@jena.apache.org" <us...@jena.apache.org>
> Subject: RE: Sensible size limit for SPARQL update payload to Fuseki2?
>
> Quick follow up --
>
> The web fuseki interface is happy loading the turtle equivalent. Takes more time (~10-15 sec) than it takes for the programmatic way to return an error (<2 sec), it's about 900k triples.
>
> So I was thinking of splitting my INSERT and was wondering if there is a reasonable chunk size.
>
> As said, will try to look at the logs anyway.
>
> Best,
> Pierre
>
> THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
> IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
> IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
> E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
> MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
>
> IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
> MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
> "THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
> MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
> FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
> (AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
> (https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
> COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
> AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
> ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
> PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
>
> HORIZON ASSET LLP IS AUTHORISED AND REGULATED
> BY THE FINANCIAL CONDUCT AUTHORITY.
>
>
> > -----Original Message-----
> > From: Pierre Grenon
> > Sent: 06 August 2019 08:47
> > To: 'users@jena.apache.org'
> > Subject: RE: Sensible size limit for SPARQL update payload to Fuseki2?
> >
> > Hi,
> >
> > Thanks for your answer.
> >
> > I'll look into the server's logs if I can. I am using this approach because the
> > server is in fact remote. So I read into a string that I then pass to POST. The
> > query when saved to file is ~ 85Mb.
> >
> > I guess I was lazy and hoping for an easy answer if this clogged up Fuseki in
> > known ways and whether that might be addressed through config (memory,
> > thread, whatnots).
> >
> > Will double check a few things and report then.
> >
> > With many thanks and best regards,
> > Pierre
> >
> >
> > From: Laura Morales [mailto:lauretas@mail.com]
> > Sent: 06 August 2019 08:19
> > To: users@jena.apache.org
> > Cc: users@jena.apache.org
> > Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
> >
> > How long is your query?? Personally I'm not aware of any such limitations,
> > especially when POSTing, but other people here definitely know better than
> > me if there is one or not. If there is a limit, let's say even just 1MB, you need
> > a *very* long query to break it. 500 is a general "internal error" code, it's not
> > specific to query length. It could be something else (look at the logs). Did you
> > try to run the query from the Fuseki web interface, or even with another
> > database entirely? It could help you debug it.
> >
> >
> >
> > > Sent: Tuesday, August 06, 2019 at 8:15 AM
> > > From: "Pierre Grenon" <pg...@horizon-asset.co.uk>
> > > To: "users@jena.apache.org" <us...@jena.apache.org>
> > > Subject: Sensible size limit for SPARQL update payload to Fuseki2?
> > >
> > > Hi,
> > >
> > > Maybe a long shot but thought I'd ask.
> > >
> > > I'm sending updates to a Fuseki2 from an RDFLib/SPARQLWrapper based
> > client. This POSTs an INSERT string to an update endpoint. I get back an error
> > 500 for strings over a certain large size which (the limit) I haven't tried to
> > figure out.
> > >
> > > Is there a theoretical, or other, reason why the limit exists and a strategy
> > to adopt besides fine tuning the string size?
> > >
> > > Many thanks and best regards,
> > > Pierre
> > >
> > > THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED
> > INFORMATION.
> > > IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
> > > IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
> > > E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF
> > THE
> > > MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
> > >
> > > IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S
> > EMPLOYEES
> > > MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA
> > HANDBOOK AS
> > > "THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN
> > AN INVESTMENT
> > > MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH
> > MEETINGS, THE
> > > FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE
> > INFORMATION
> > > (AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO
> > 596/2014).
> > > (https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
> > > COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF
> > REGULATION
> > > AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR
> > INFORMATION
> > > ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET
> > SOUNDINGS,
> > > PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
> > >
> > > HORIZON ASSET LLP IS AUTHORISED AND REGULATED
> > > BY THE FINANCIAL CONDUCT AUTHORITY.
> > >
> > >
> > >
>
RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Pierre Grenon <pg...@horizon-asset.co.uk>.
Quick follow up --
The web fuseki interface is happy loading the turtle equivalent. Takes more time (~10-15 sec) than it takes for the programmatic way to return an error (<2 sec), it's about 900k triples.
So I was thinking of splitting my INSERT and was wondering if there is a reasonable chunk size.
As said, will try to look at the logs anyway.
Best,
Pierre
THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
"THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
(AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
(https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
HORIZON ASSET LLP IS AUTHORISED AND REGULATED
BY THE FINANCIAL CONDUCT AUTHORITY.
> -----Original Message-----
> From: Pierre Grenon
> Sent: 06 August 2019 08:47
> To: 'users@jena.apache.org'
> Subject: RE: Sensible size limit for SPARQL update payload to Fuseki2?
>
> Hi,
>
> Thanks for your answer.
>
> I'll look into the server's logs if I can. I am using this approach because the
> server is in fact remote. So I read into a string that I then pass to POST. The
> query when saved to file is ~ 85Mb.
>
> I guess I was lazy and hoping for an easy answer if this clogged up Fuseki in
> known ways and whether that might be addressed through config (memory,
> thread, whatnots).
>
> Will double check a few things and report then.
>
> With many thanks and best regards,
> Pierre
>
>
> From: Laura Morales [mailto:lauretas@mail.com]
> Sent: 06 August 2019 08:19
> To: users@jena.apache.org
> Cc: users@jena.apache.org
> Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
>
> How long is your query?? Personally I'm not aware of any such limitations,
> especially when POSTing, but other people here definitely know better than
> me if there is one or not. If there is a limit, let's say even just 1MB, you need
> a *very* long query to break it. 500 is a general "internal error" code, it's not
> specific to query length. It could be something else (look at the logs). Did you
> try to run the query from the Fuseki web interface, or even with another
> database entirely? It could help you debug it.
>
>
>
> > Sent: Tuesday, August 06, 2019 at 8:15 AM
> > From: "Pierre Grenon" <pg...@horizon-asset.co.uk>
> > To: "users@jena.apache.org" <us...@jena.apache.org>
> > Subject: Sensible size limit for SPARQL update payload to Fuseki2?
> >
> > Hi,
> >
> > Maybe a long shot but thought I'd ask.
> >
> > I'm sending updates to a Fuseki2 from an RDFLib/SPARQLWrapper based
> client. This POSTs an INSERT string to an update endpoint. I get back an error
> 500 for strings over a certain large size which (the limit) I haven't tried to
> figure out.
> >
> > Is there a theoretical, or other, reason why the limit exists and a strategy
> to adopt besides fine tuning the string size?
> >
> > Many thanks and best regards,
> > Pierre
> >
> > THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED
> INFORMATION.
> > IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
> > IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
> > E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF
> THE
> > MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
> >
> > IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S
> EMPLOYEES
> > MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA
> HANDBOOK AS
> > "THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN
> AN INVESTMENT
> > MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH
> MEETINGS, THE
> > FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE
> INFORMATION
> > (AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO
> 596/2014).
> > (https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
> > COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF
> REGULATION
> > AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR
> INFORMATION
> > ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET
> SOUNDINGS,
> > PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
> >
> > HORIZON ASSET LLP IS AUTHORISED AND REGULATED
> > BY THE FINANCIAL CONDUCT AUTHORITY.
> >
> >
> >
RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Pierre Grenon <pg...@horizon-asset.co.uk>.
Oioioi --- tried on localhost
Caused by: java.lang.IllegalStateException: Form too large: 100948991 > 10000000
Which seems to be Jetty related https://www.xwiki.org/xwiki/bin/view/FAQ/FormTooLargeError
Haven’t figured out yet how to pass that to fuseki-server...
Thanks,
Pierre
[2019-08-06 13:33:15] Fuseki WARN [40796] RC = 500 : 400: Unable to parse form content
org.eclipse.jetty.http.BadMessageException: 400: Unable to parse form content
at org.eclipse.jetty.server.Request.getParameters(Request.java:390)
at org.eclipse.jetty.server.Request.getParameterValues(Request.java:1059)
at javax.servlet.ServletRequestWrapper.getParameterValues(ServletRequestWrapper.java:221)
at org.apache.jena.fuseki.servlets.SPARQL_Protocol.countParamOccurences(SPARQL_Protocol.java:105)
at org.apache.jena.fuseki.servlets.SPARQL_Update.validate(SPARQL_Update.java:142)
at org.apache.jena.fuseki.servlets.ActionService.executeLifecycle(ActionService.java:234)
at org.apache.jena.fuseki.servlets.ActionService.execCommonWorker(ActionService.java:134)
at org.apache.jena.fuseki.servlets.ActionBase.doCommon(ActionBase.java:74)
at org.apache.jena.fuseki.servlets.FusekiFilter.doFilter(FusekiFilter.java:73)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.jena.fuseki.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:285)
at org.apache.jena.fuseki.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:248)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:690)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:503)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.IllegalStateException: Form too large: 100948991 > 10000000
at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:545)
at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:475)
at org.eclipse.jetty.server.Request.getParameters(Request.java:386)
... 55 more
[2019-08-06 13:33:15] Fuseki INFO [40796] 500 400: Unable to parse form content (1 ms)
THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
"THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
(AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
(https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
HORIZON ASSET LLP IS AUTHORISED AND REGULATED
BY THE FINANCIAL CONDUCT AUTHORITY.
From: Laura Morales [mailto:lauretas@mail.com]
Sent: 06 August 2019 08:19
To: users@jena.apache.org
Cc: users@jena.apache.org
Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
How long is your query?? Personally I'm not aware of any such limitations, especially when POSTing, but other people here definitely know better than me if there is one or not. If there is a limit, let's say even just 1MB, you need a *very* long query to break it. 500 is a general "internal error" code, it's not specific to query length. It could be something else (look at the logs). Did you try to run the query from the Fuseki web interface, or even with another database entirely? It could help you debug it.
> Sent: Tuesday, August 06, 2019 at 8:15 AM
> From: "Pierre Grenon" <pg...@horizon-asset.co.uk>
> To: "users@jena.apache.org" <us...@jena.apache.org>
> Subject: Sensible size limit for SPARQL update payload to Fuseki2?
>
> Hi,
>
> Maybe a long shot but thought I'd ask.
>
> I'm sending updates to a Fuseki2 from an RDFLib/SPARQLWrapper based client. This POSTs an INSERT string to an update endpoint. I get back an error 500 for strings over a certain large size which (the limit) I haven't tried to figure out.
>
> Is there a theoretical, or other, reason why the limit exists and a strategy to adopt besides fine tuning the string size?
>
> Many thanks and best regards,
> Pierre
>
> THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
> IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
> IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
> E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
> MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
>
> IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
> MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
> "THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
> MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
> FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
> (AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
> (https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
> COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
> AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
> ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
> PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
>
> HORIZON ASSET LLP IS AUTHORISED AND REGULATED
> BY THE FINANCIAL CONDUCT AUTHORITY.
>
>
>
RE: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Pierre Grenon <pg...@horizon-asset.co.uk>.
Hi,
Thanks for your answer.
I'll look into the server's logs if I can. I am using this approach because the server is in fact remote. So I read into a string that I then pass to POST. The query when saved to file is ~ 85Mb.
I guess I was lazy and hoping for an easy answer if this clogged up Fuseki in known ways and whether that might be addressed through config (memory, thread, whatnots).
Will double check a few things and report then.
With many thanks and best regards,
Pierre
THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
"THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
(AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
(https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
HORIZON ASSET LLP IS AUTHORISED AND REGULATED
BY THE FINANCIAL CONDUCT AUTHORITY.
From: Laura Morales [mailto:lauretas@mail.com]
Sent: 06 August 2019 08:19
To: users@jena.apache.org
Cc: users@jena.apache.org
Subject: Re: Sensible size limit for SPARQL update payload to Fuseki2?
How long is your query?? Personally I'm not aware of any such limitations, especially when POSTing, but other people here definitely know better than me if there is one or not. If there is a limit, let's say even just 1MB, you need a *very* long query to break it. 500 is a general "internal error" code, it's not specific to query length. It could be something else (look at the logs). Did you try to run the query from the Fuseki web interface, or even with another database entirely? It could help you debug it.
> Sent: Tuesday, August 06, 2019 at 8:15 AM
> From: "Pierre Grenon" <pg...@horizon-asset.co.uk>
> To: "users@jena.apache.org" <us...@jena.apache.org>
> Subject: Sensible size limit for SPARQL update payload to Fuseki2?
>
> Hi,
>
> Maybe a long shot but thought I'd ask.
>
> I'm sending updates to a Fuseki2 from an RDFLib/SPARQLWrapper based client. This POSTs an INSERT string to an update endpoint. I get back an error 500 for strings over a certain large size which (the limit) I haven't tried to figure out.
>
> Is there a theoretical, or other, reason why the limit exists and a strategy to adopt besides fine tuning the string size?
>
> Many thanks and best regards,
> Pierre
>
> THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
> IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
> IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
> E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
> MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
>
> IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
> MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
> "THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
> MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
> FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
> (AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
> (https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
> COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
> AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
> ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
> PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
>
> HORIZON ASSET LLP IS AUTHORISED AND REGULATED
> BY THE FINANCIAL CONDUCT AUTHORITY.
>
>
>
Re: Sensible size limit for SPARQL update payload to Fuseki2?
Posted by Laura Morales <la...@mail.com>.
How long is your query?? Personally I'm not aware of any such limitations, especially when POSTing, but other people here definitely know better than me if there is one or not. If there is a limit, let's say even just 1MB, you need a *very* long query to break it. 500 is a general "internal error" code, it's not specific to query length. It could be something else (look at the logs). Did you try to run the query from the Fuseki web interface, or even with another database entirely? It could help you debug it.
> Sent: Tuesday, August 06, 2019 at 8:15 AM
> From: "Pierre Grenon" <pg...@horizon-asset.co.uk>
> To: "users@jena.apache.org" <us...@jena.apache.org>
> Subject: Sensible size limit for SPARQL update payload to Fuseki2?
>
> Hi,
>
> Maybe a long shot but thought I'd ask.
>
> I'm sending updates to a Fuseki2 from an RDFLib/SPARQLWrapper based client. This POSTs an INSERT string to an update endpoint. I get back an error 500 for strings over a certain large size which (the limit) I haven't tried to figure out.
>
> Is there a theoretical, or other, reason why the limit exists and a strategy to adopt besides fine tuning the string size?
>
> Many thanks and best regards,
> Pierre
>
> THIS E-MAIL MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED INFORMATION.
> IF YOU ARE NOT THE INTENDED RECIPIENT (OR HAVE RECEIVED THIS E-MAIL
> IN ERROR) PLEASE NOTIFY THE SENDER IMMEDIATELY AND DESTROY THIS
> E-MAIL. ANY UNAUTHORISED COPYING, DISCLOSURE OR DISTRIBUTION OF THE
> MATERIAL IN THIS E-MAIL IS STRICTLY FORBIDDEN.
>
> IN ACCORDANCE WITH MIFID II RULES ON INDUCEMENTS, THE FIRM'S EMPLOYEES
> MAY ATTEND CORPORATE ACCESS EVENTS (DEFINED IN THE FCA HANDBOOK AS
> "THE SERVICE OF ARRANGING OR BRINGING ABOUT CONTACT BETWEEN AN INVESTMENT
> MANAGER AND AN ISSUER OR POTENTIAL ISSUER"). DURING SUCH MEETINGS, THE
> FIRM'S EMPLOYEES MAY ON NO ACCOUNT BE IN RECEIPT OF INSIDE INFORMATION
> (AS DESCRIBED IN ARTICLE 7 OF THE MARKET ABUSE REGULATION (EU) NO 596/2014).
> (https://www.handbook.fca.org.uk/handbook/glossary/G3532m.html)
> COMPANIES WHO DISCLOSE INSIDE INFORMATION ARE IN BREACH OF REGULATION
> AND MUST IMMEDIATELY AND CLEARLY NOTIFY ALL ATTENDEES. FOR INFORMATION
> ON THE FIRM'S POLICY IN RELATION TO ITS PARTICIPATION IN MARKET SOUNDINGS,
> PLEASE SEE https://www.horizon-asset.co.uk/market-soundings/.
>
> HORIZON ASSET LLP IS AUTHORISED AND REGULATED
> BY THE FINANCIAL CONDUCT AUTHORITY.
>
>
>