You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Bart van Leeuwen <ba...@netage.nl> on 2015/11/28 23:10:54 UTC
DatasetAccessor content-length issue
Hi,
I've been working with Dydra to the Graph Store protocol to upload data.
I ran into a issue with the 3.0.0 version of Jena.
dydraGS = DatasetAccessorFactory.createHTTP("
http://dydra.com/user/repository/service", new
SimpleAuthenticator("<dydrakey>", null));
Model mod = dydraGS.getModel();
mod.removeAll();
dydraGS.add("http://example.com",mod);
This results in:
Exception in thread "main" org.apache.jena.atlas.web.HttpException: 411 -
Length Required
at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1107)
at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:718)
at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:686)
at
org.apache.jena.web.DatasetGraphAccessorHTTP.doPost(DatasetGraphAccessorHTTP.java:209)
at
org.apache.jena.web.DatasetGraphAccessorHTTP.httpPost(DatasetGraphAccessorHTTP.java:204)
at org.apache.jena.web.DatasetAdapter.add(DatasetAdapter.java:98)
at com.example.test.App.main(App.java:29)
James Anderson debugged the problem to a missing 'Content-Length' header.
Where should I report this ?
Another 'feature' which is missing is the ability to set the content-type
from datasetaccessor, this is available in the underlying classes.
Sending N3 will greatly improve performance
Met Vriendelijke Groet / With Kind Regards
Bart van Leeuwen
##############################################################
# twitter: @semanticfire
# netage.nl
# http://netage.nl
# M.A. Reinaldaweg 79
# 3461AJ Linschoten
# tel. +31(0)6-53182997
##############################################################
Met Vriendelijke Groet / With Kind Regards
Bart van Leeuwen
##############################################################
# twitter: @semanticfire
# netage.nl
# http://netage.nl
# M.A. Reinaldaweg 79
# 3461AJ Linschoten
# tel. +31(0)6-53182997
##############################################################
Re: DatasetAccessor content-length issue
Posted by Andy Seaborne <an...@apache.org>.
On 29/11/15 04:04, james anderson wrote:
> good morning;
>
>> On 2015-11-29, at 00:20, Andy Seaborne <an...@apache.org> wrote:
>>
>> Hi Bart,
>>
>> It's come up before. It's a feature of Dydra; there is nothing wrong with an HTTP request without a Content-Length. A server can reject any request it does not like in HTTP.
>>
>> [*] Their reason -
>> http://mail-archives.apache.org/mod_mbox/jena-users/201312.mbox/%3C52E748F6-304D-40FE-98FB-4939E62D46AE@dydra.com%3E
>>
>> though there must be more to the story because for the given reason the Content-Length is contractual so it needs to be verified. Hence if it is buffered server-side to the the agree limit for verification, the approach 4xx can be sent then. Content-Length is a hint, not a guarantee. Spot the DOS attack!
>
> in the sense here, it is more significant that the constraint be enforced than that it be verified, but that note was a thorough disclosure of the observed usage and the consequence.
> at some point, it started making less sense to repeat 512MB+1 uploads to just fail the request in the end.
> it is true, we could be more permissive, to perform speculative caching in cases where, as noted below, the client is not able to compute a length.
>
Well, to try to solve Bart's situation ...
Does that mean you don't accept chunked encoding at all?
Requiring Content-Length for non-chunked it quite reasonable and normal
(you can't cache connections otherwise). curl by default does not use
chunked encoding (you have to ask for it).
Andy
>>
>>> Another 'feature' which is missing is the ability to set the content-type
>>> from datasetaccessor, this is available in the underlying classes.
>>> Sending N3 will greatly improve performance
>>
>> You can on the underlying DatasetGraphAccessorHTTP but it is not properly accessible.
>>
>> Noted - and related. Setting to a non-pretty version of Turtle, or using N-Triples will be better yet if streaming support were available.
>>
>> Streaming data means Content-type isn't always available. e.g Uploading a 100 millions triples (yes, with TDB2, you can upload 100 million triples into a live server in a single transaction).
>>
>> Adding a Content-Length would require knowing the Content-Length but that is not available when writing RDF from a model.
>>
>> Andy
>>
>> On 28/11/15 22:10, Bart van Leeuwen wrote:
>>> Hi,
>>>
>>> I've been working with Dydra to the Graph Store protocol to upload data.
>>> I ran into a issue with the 3.0.0 version of Jena.
>>>
>>> dydraGS = DatasetAccessorFactory.createHTTP("
>>> http://dydra.com/user/repository/service", new
>>> SimpleAuthenticator("<dydrakey>", null));
>>>
>>> Model mod = dydraGS.getModel();
>>>
>>> mod.removeAll();
>>>
>>> dydraGS.add("http://example.com",mod);
>>>
>>> This results in:
>>>
>>> Exception in thread "main" org.apache.jena.atlas.web.HttpException: 411 -
>>> Length Required
>>> at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1107)
>>> at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:718)
>>> at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:686)
>>> at
>>> org.apache.jena.web.DatasetGraphAccessorHTTP.doPost(DatasetGraphAccessorHTTP.java:209)
>>> at
>>> org.apache.jena.web.DatasetGraphAccessorHTTP.httpPost(DatasetGraphAccessorHTTP.java:204)
>>> at org.apache.jena.web.DatasetAdapter.add(DatasetAdapter.java:98)
>>> at com.example.test.App.main(App.java:29)
>>>
>>>
>>> James Anderson debugged the problem to a missing 'Content-Length' header.
>>> Where should I report this ?
>>>
>>> Another 'feature' which is missing is the ability to set the content-type
>>> from datasetaccessor, this is available in the underlying classes.
>>> Sending N3 will greatly improve performance
>>>
>>> Met Vriendelijke Groet / With Kind Regards
>>> Bart van Leeuwen
>>>
>>> ##############################################################
>>> # twitter: @semanticfire
>>> # netage.nl
>>> # http://netage.nl
>>> # M.A. Reinaldaweg 79
>>> # 3461AJ Linschoten
>>> # tel. +31(0)6-53182997
>>> ##############################################################
>>> Met Vriendelijke Groet / With Kind Regards
>>> Bart van Leeuwen
>>>
>>> ##############################################################
>>> # twitter: @semanticfire
>>> # netage.nl
>>> # http://netage.nl
>>> # M.A. Reinaldaweg 79
>>> # 3461AJ Linschoten
>>> # tel. +31(0)6-53182997
>>> ##############################################################
>>>
>>
>
> ---
> james anderson | james@dydra.com | http://dydra.com
>
>
>
>
>
>
Re: DatasetAccessor content-length issue
Posted by james anderson <ja...@dydra.com>.
good morning;
> On 2015-11-29, at 00:20, Andy Seaborne <an...@apache.org> wrote:
>
> Hi Bart,
>
> It's come up before. It's a feature of Dydra; there is nothing wrong with an HTTP request without a Content-Length. A server can reject any request it does not like in HTTP.
>
> [*] Their reason -
> http://mail-archives.apache.org/mod_mbox/jena-users/201312.mbox/%3C52E748F6-304D-40FE-98FB-4939E62D46AE@dydra.com%3E
>
> though there must be more to the story because for the given reason the Content-Length is contractual so it needs to be verified. Hence if it is buffered server-side to the the agree limit for verification, the approach 4xx can be sent then. Content-Length is a hint, not a guarantee. Spot the DOS attack!
in the sense here, it is more significant that the constraint be enforced than that it be verified, but that note was a thorough disclosure of the observed usage and the consequence.
at some point, it started making less sense to repeat 512MB+1 uploads to just fail the request in the end.
it is true, we could be more permissive, to perform speculative caching in cases where, as noted below, the client is not able to compute a length.
>
> > Another 'feature' which is missing is the ability to set the content-type
> > from datasetaccessor, this is available in the underlying classes.
> > Sending N3 will greatly improve performance
>
> You can on the underlying DatasetGraphAccessorHTTP but it is not properly accessible.
>
> Noted - and related. Setting to a non-pretty version of Turtle, or using N-Triples will be better yet if streaming support were available.
>
> Streaming data means Content-type isn't always available. e.g Uploading a 100 millions triples (yes, with TDB2, you can upload 100 million triples into a live server in a single transaction).
>
> Adding a Content-Length would require knowing the Content-Length but that is not available when writing RDF from a model.
>
> Andy
>
> On 28/11/15 22:10, Bart van Leeuwen wrote:
>> Hi,
>>
>> I've been working with Dydra to the Graph Store protocol to upload data.
>> I ran into a issue with the 3.0.0 version of Jena.
>>
>> dydraGS = DatasetAccessorFactory.createHTTP("
>> http://dydra.com/user/repository/service", new
>> SimpleAuthenticator("<dydrakey>", null));
>>
>> Model mod = dydraGS.getModel();
>>
>> mod.removeAll();
>>
>> dydraGS.add("http://example.com",mod);
>>
>> This results in:
>>
>> Exception in thread "main" org.apache.jena.atlas.web.HttpException: 411 -
>> Length Required
>> at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1107)
>> at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:718)
>> at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:686)
>> at
>> org.apache.jena.web.DatasetGraphAccessorHTTP.doPost(DatasetGraphAccessorHTTP.java:209)
>> at
>> org.apache.jena.web.DatasetGraphAccessorHTTP.httpPost(DatasetGraphAccessorHTTP.java:204)
>> at org.apache.jena.web.DatasetAdapter.add(DatasetAdapter.java:98)
>> at com.example.test.App.main(App.java:29)
>>
>>
>> James Anderson debugged the problem to a missing 'Content-Length' header.
>> Where should I report this ?
>>
>> Another 'feature' which is missing is the ability to set the content-type
>> from datasetaccessor, this is available in the underlying classes.
>> Sending N3 will greatly improve performance
>>
>> Met Vriendelijke Groet / With Kind Regards
>> Bart van Leeuwen
>>
>> ##############################################################
>> # twitter: @semanticfire
>> # netage.nl
>> # http://netage.nl
>> # M.A. Reinaldaweg 79
>> # 3461AJ Linschoten
>> # tel. +31(0)6-53182997
>> ##############################################################
>> Met Vriendelijke Groet / With Kind Regards
>> Bart van Leeuwen
>>
>> ##############################################################
>> # twitter: @semanticfire
>> # netage.nl
>> # http://netage.nl
>> # M.A. Reinaldaweg 79
>> # 3461AJ Linschoten
>> # tel. +31(0)6-53182997
>> ##############################################################
>>
>
---
james anderson | james@dydra.com | http://dydra.com
Re: DatasetAccessor content-length issue
Posted by Andy Seaborne <an...@apache.org>.
Hi Bart,
It's come up before. It's a feature of Dydra; there is nothing wrong
with an HTTP request without a Content-Length. A server can reject any
request it does not like in HTTP.
[*] Their reason -
http://mail-archives.apache.org/mod_mbox/jena-users/201312.mbox/%3C52E748F6-304D-40FE-98FB-4939E62D46AE@dydra.com%3E
though there must be more to the story because for the given reason the
Content-Length is contractual so it needs to be verified. Hence if it
is buffered server-side to the the agree limit for verification, the
approach 4xx can be sent then. Content-Length is a hint, not a
guarantee. Spot the DOS attack!
> Another 'feature' which is missing is the ability to set the content-type
> from datasetaccessor, this is available in the underlying classes.
> Sending N3 will greatly improve performance
You can on the underlying DatasetGraphAccessorHTTP but it is not
properly accessible.
Noted - and related. Setting to a non-pretty version of Turtle, or
using N-Triples will be better yet if streaming support were available.
Streaming data means Content-type isn't always available. e.g Uploading
a 100 millions triples (yes, with TDB2, you can upload 100 million
triples into a live server in a single transaction).
Adding a Content-Length would require knowing the Content-Length but
that is not available when writing RDF from a model.
Andy
On 28/11/15 22:10, Bart van Leeuwen wrote:
> Hi,
>
> I've been working with Dydra to the Graph Store protocol to upload data.
> I ran into a issue with the 3.0.0 version of Jena.
>
> dydraGS = DatasetAccessorFactory.createHTTP("
> http://dydra.com/user/repository/service", new
> SimpleAuthenticator("<dydrakey>", null));
>
> Model mod = dydraGS.getModel();
>
> mod.removeAll();
>
> dydraGS.add("http://example.com",mod);
>
> This results in:
>
> Exception in thread "main" org.apache.jena.atlas.web.HttpException: 411 -
> Length Required
> at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1107)
> at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:718)
> at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:686)
> at
> org.apache.jena.web.DatasetGraphAccessorHTTP.doPost(DatasetGraphAccessorHTTP.java:209)
> at
> org.apache.jena.web.DatasetGraphAccessorHTTP.httpPost(DatasetGraphAccessorHTTP.java:204)
> at org.apache.jena.web.DatasetAdapter.add(DatasetAdapter.java:98)
> at com.example.test.App.main(App.java:29)
>
>
> James Anderson debugged the problem to a missing 'Content-Length' header.
> Where should I report this ?
>
> Another 'feature' which is missing is the ability to set the content-type
> from datasetaccessor, this is available in the underlying classes.
> Sending N3 will greatly improve performance
>
> Met Vriendelijke Groet / With Kind Regards
> Bart van Leeuwen
>
> ##############################################################
> # twitter: @semanticfire
> # netage.nl
> # http://netage.nl
> # M.A. Reinaldaweg 79
> # 3461AJ Linschoten
> # tel. +31(0)6-53182997
> ##############################################################
> Met Vriendelijke Groet / With Kind Regards
> Bart van Leeuwen
>
> ##############################################################
> # twitter: @semanticfire
> # netage.nl
> # http://netage.nl
> # M.A. Reinaldaweg 79
> # 3461AJ Linschoten
> # tel. +31(0)6-53182997
> ##############################################################
>