You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@sling.apache.org by John Logan <Jo...@texture.com> on 2016/10/06 18:15:25 UTC
Retrieving content length for file node without retrieving object
from S3 backend?
Hi! I have a Sling instance set up using the S3 backend and am making a request like this to perform a HEAD operation on a large file node:
curl -I 'https://sling.example.com/some_big_file'
I see a fair bit of delay to retrieve the headers. I did a little poking in the code and it looks like the StreamRendererServlet and SlingHttpServletRequestImpl might try to get the whole object from the S3 backend instead of performing only an amazonS3Client.getObjectMetadata() call to populate the headers.
Can someone confirm whether this is the case?
If so, is there a workaround that would allow me to retrieve the content length for the node without pulling the node content into the repository read cache?
If not, does it seem worthwhile for me to file an issue for this?
Thanks! John
Re: Retrieving content length for file node without retrieving object
from S3 backend?
Posted by Chetan Mehrotra <ch...@gmail.com>.
Hi John,
This issue is being addressed as part of OAK-4837. With that binary
should not get downloaded for just getting the metadata
Chetan Mehrotra
On Fri, Oct 7, 2016 at 3:00 AM, John Logan <Jo...@texture.com> wrote:
> I forgot, the other Oak modules are at 1.5.7 also. Here is the oak.txt file for the launchpad.
>
>
> [feature name=oak]
> oak.version=1.5.7
> org.apache.jackrabbit/oak-core/${oak.version}
> org.apache.jackrabbit/oak-commons/${oak.version}
> org.apache.jackrabbit/oak-lucene/${oak.version}
> org.apache.jackrabbit/oak-blob/${oak.version}
> org.apache.jackrabbit/oak-jcr/${oak.version}
> [artifacts startLevel=15 runModes=oak_tar]
> org.apache.jackrabbit/oak-segment/${oak.version}
> [artifacts startLevel=15 runModes=oak_mongo]
> org.apache.sling/org.apache.sling.jcr.oak.server/1.1.0
> jaas.classname="org.apache.jackrabbit.oak.spi.security.authentication.GuestLoginModule"
> jaas.classname="org.apache.jackrabbit.oak.security.authentication.user.LoginModuleImpl"
> jaas.classname="org.apache.jackrabbit.oak.security.authentication.token.TokenLoginModule"
> jaas.defaultRealmName="jackrabbit.oak"
> org.apache.jackrabbit.oak.security.authentication.AuthenticationConfigurationImpl
> org.apache.jackrabbit.oak.authentication.configSpiName="FelixJaasProvider"
> org.apache.jackrabbit.oak.security.user.UserConfigurationImpl
> org.apache.jackrabbit.oak.security.user.RandomAuthorizableNodeName
> org.apache.jackrabbit.oak.spi.security.user.action.DefaultAuthorizableActionProvider
> enabledActions=["org.apache.jackrabbit.oak.spi.security.user.action.AccessControlAction"]
> [configurations runModes=oak_tar]
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService
> [configurations runModes=oak_mongo]
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService
>
>
> ________________________________
> From: John Logan
> Sent: Thursday, October 6, 2016 2:07:17 PM
> To: users@sling.apache.org
> Subject: Re: Retrieving content length for file node without retrieving object from S3 backend?
>
>
> Hi Robert,
>
>
> By S3 backend, yes, I mean Oak using the S3SharedDataStore, which I believe in turn uses the class:
>
>
> org.apache.jackrabbit.oak.blob.cloud.aws.s3.S3Backend.java
>
>
> which appears to encapsulate all of the AWS SDK calls.
>
>
> I am using a copy of the Sling 9 launchpad to create my instance, and have modified the oak_s3.txt provisioning file as follows. Does this provide you with the information that you need?
>
>
> [feature name=oak_s3]
>
> [variables]
> oak.version=1.5.7
>
> [artifacts startLevel=5]
> joda-time/joda-time/2.9.4
>
> [artifacts startLevel=15]
> com.fasterxml.jackson.core/jackson-core/2.8.2
> com.fasterxml.jackson.core/jackson-databind/2.8.2
> com.fasterxml.jackson.core/jackson-annotations/2.8.2
> org.apache.servicemix.bundles/org.apache.servicemix.bundles.jaxp-ri/1.4.5_1
> com.amazonaws/aws-java-sdk-osgi/1.10.27
> org.apache.jackrabbit/oak-blob-cloud/${oak.version}
>
>
> ________________________________
> From: Robert Munteanu <ro...@apache.org>
> Sent: Thursday, October 6, 2016 2:01:21 PM
> To: users@sling.apache.org
> Subject: Re: Retrieving content length for file node without retrieving object from S3 backend?
>
> Hi Johnm
>
> On Thu, 2016-10-06 at 18:15 +0000, John Logan wrote:
>> Hi! I have a Sling instance set up using the S3 backend and am
>> making a request like this to perform a HEAD operation on a large
>> file node:
>
> What does the S3 backend mean? Oak using the S3DataStore ? It would
> also help if we knew the version of Oak you're suing.
>
> Thanks,
>
> Robert
Re: Retrieving content length for file node without retrieving object
from S3 backend?
Posted by John Logan <Jo...@texture.com>.
I forgot, the other Oak modules are at 1.5.7 also. Here is the oak.txt file for the launchpad.
[feature name=oak]
oak.version=1.5.7
org.apache.jackrabbit/oak-core/${oak.version}
org.apache.jackrabbit/oak-commons/${oak.version}
org.apache.jackrabbit/oak-lucene/${oak.version}
org.apache.jackrabbit/oak-blob/${oak.version}
org.apache.jackrabbit/oak-jcr/${oak.version}
[artifacts startLevel=15 runModes=oak_tar]
org.apache.jackrabbit/oak-segment/${oak.version}
[artifacts startLevel=15 runModes=oak_mongo]
org.apache.sling/org.apache.sling.jcr.oak.server/1.1.0
jaas.classname="org.apache.jackrabbit.oak.spi.security.authentication.GuestLoginModule"
jaas.classname="org.apache.jackrabbit.oak.security.authentication.user.LoginModuleImpl"
jaas.classname="org.apache.jackrabbit.oak.security.authentication.token.TokenLoginModule"
jaas.defaultRealmName="jackrabbit.oak"
org.apache.jackrabbit.oak.security.authentication.AuthenticationConfigurationImpl
org.apache.jackrabbit.oak.authentication.configSpiName="FelixJaasProvider"
org.apache.jackrabbit.oak.security.user.UserConfigurationImpl
org.apache.jackrabbit.oak.security.user.RandomAuthorizableNodeName
org.apache.jackrabbit.oak.spi.security.user.action.DefaultAuthorizableActionProvider
enabledActions=["org.apache.jackrabbit.oak.spi.security.user.action.AccessControlAction"]
[configurations runModes=oak_tar]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService
[configurations runModes=oak_mongo]
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService
________________________________
From: John Logan
Sent: Thursday, October 6, 2016 2:07:17 PM
To: users@sling.apache.org
Subject: Re: Retrieving content length for file node without retrieving object from S3 backend?
Hi Robert,
By S3 backend, yes, I mean Oak using the S3SharedDataStore, which I believe in turn uses the class:
org.apache.jackrabbit.oak.blob.cloud.aws.s3.S3Backend.java
which appears to encapsulate all of the AWS SDK calls.
I am using a copy of the Sling 9 launchpad to create my instance, and have modified the oak_s3.txt provisioning file as follows. Does this provide you with the information that you need?
[feature name=oak_s3]
[variables]
oak.version=1.5.7
[artifacts startLevel=5]
joda-time/joda-time/2.9.4
[artifacts startLevel=15]
com.fasterxml.jackson.core/jackson-core/2.8.2
com.fasterxml.jackson.core/jackson-databind/2.8.2
com.fasterxml.jackson.core/jackson-annotations/2.8.2
org.apache.servicemix.bundles/org.apache.servicemix.bundles.jaxp-ri/1.4.5_1
com.amazonaws/aws-java-sdk-osgi/1.10.27
org.apache.jackrabbit/oak-blob-cloud/${oak.version}
________________________________
From: Robert Munteanu <ro...@apache.org>
Sent: Thursday, October 6, 2016 2:01:21 PM
To: users@sling.apache.org
Subject: Re: Retrieving content length for file node without retrieving object from S3 backend?
Hi Johnm
On Thu, 2016-10-06 at 18:15 +0000, John Logan wrote:
> Hi! I have a Sling instance set up using the S3 backend and am
> making a request like this to perform a HEAD operation on a large
> file node:
What does the S3 backend mean? Oak using the S3DataStore ? It would
also help if we knew the version of Oak you're suing.
Thanks,
Robert
Re: Retrieving content length for file node without retrieving object
from S3 backend?
Posted by John Logan <Jo...@texture.com>.
Hi Robert,
By S3 backend, yes, I mean Oak using the S3SharedDataStore, which I believe in turn uses the class:
org.apache.jackrabbit.oak.blob.cloud.aws.s3.S3Backend.java
which appears to encapsulate all of the AWS SDK calls.
I am using a copy of the Sling 9 launchpad to create my instance, and have modified the oak_s3.txt provisioning file as follows. Does this provide you with the information that you need?
[feature name=oak_s3]
[variables]
oak.version=1.5.7
[artifacts startLevel=5]
joda-time/joda-time/2.9.4
[artifacts startLevel=15]
com.fasterxml.jackson.core/jackson-core/2.8.2
com.fasterxml.jackson.core/jackson-databind/2.8.2
com.fasterxml.jackson.core/jackson-annotations/2.8.2
org.apache.servicemix.bundles/org.apache.servicemix.bundles.jaxp-ri/1.4.5_1
com.amazonaws/aws-java-sdk-osgi/1.10.27
org.apache.jackrabbit/oak-blob-cloud/${oak.version}
________________________________
From: Robert Munteanu <ro...@apache.org>
Sent: Thursday, October 6, 2016 2:01:21 PM
To: users@sling.apache.org
Subject: Re: Retrieving content length for file node without retrieving object from S3 backend?
Hi Johnm
On Thu, 2016-10-06 at 18:15 +0000, John Logan wrote:
> Hi! I have a Sling instance set up using the S3 backend and am
> making a request like this to perform a HEAD operation on a large
> file node:
What does the S3 backend mean? Oak using the S3DataStore ? It would
also help if we knew the version of Oak you're suing.
Thanks,
Robert
Re: Retrieving content length for file node without retrieving
object from S3 backend?
Posted by Robert Munteanu <ro...@apache.org>.
Hi Johnm
On Thu, 2016-10-06 at 18:15 +0000, John Logan wrote:
> Hi!��I have a Sling instance set up using the S3 backend and am
> making a request like this to perform a HEAD operation on a large
> file node:
What does the S3 backend mean? Oak using the�S3DataStore ? It would
also help if we knew the version of Oak you're suing.
Thanks,
Robert