You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by Timothee Maret <tm...@apache.org> on 2016/12/20 09:40:31 UTC
[JCRVLT] extract package directly from an inputstream
Hi,
For Sling Content Distribution we are using VLT (by default) as a
serialisation. Currently, on the SCD receiving side, the VLT packages must
be written in a file to disk before being passed to the PackageManager API
and ultimately exported to the repository.
I am thinking that if the PackageManager was able to export directly from
an InputStream, we may speed up the SCD import by skipping the
write-to-temp-file operation.
Maybe the PackageManger could be extended with the following signature in
order to allow extracting from InputStream:
void PackageManager.extract(session, importOptions, InputStream);
wdyt ?
Regards,
Timothee
[0]
https://github.com/apache/sling/blob/trunk/contrib/extensions/distribution/core/src/main/java/org/apache/sling/distribution/serialization/impl/vlt/FileVaultContentSerializer.java#L108-L147
Re: [JCRVLT] extract package directly from an inputstream
Posted by Tobias Bocanegra <tr...@apache.org>.
Hi Timothee,
> I'll try to submit a patch as it seems this issue would be useful and has
been opened for a while.
thanks patches always welcome.
just keep in mind to honour the checks mentioned in the javadoc
(dependencies, subpackages, pacakgeid). also, I like to get rid of the old
jdk1.5 compatible zip support, so we can optimize for jdk7 64bit zips.
we can assume that the properties.xml come early in the zip, which is
usually the case if its built with package manager or the maven plugin. if
the zip is built "manually" there is a chance that the properties.xml comes
late in the stream. in this case we need to have a tmp-file backed input
stream, that we can reset if the memory storage gets too large.
so I think that we might need to solve JCRVLT-35 anyways, in order to
handle the streams.
regards, toby
https://issues.apache.org/jira/browse/JCRVLT-35
Re: [JCRVLT] extract package directly from an inputstream
Posted by Timothee Maret <tm...@apache.org>.
Hi Toby,
Thanks for the pointers. I think that JCRVLT-50 covers what I want. If some
more file copies can be avoided as part of JCRVLT-50, it'd be even better.
I'll try to submit a patch as it seems this issue would be useful and has
been opened for a while.
Regards,
Timothee
2016-12-21 6:26 GMT+01:00 Tobias Bocanegra <tr...@apache.org>:
> I was wrong. JCRVLT-35 is about something else.....
>
> regards, toby
>
> On Wed, Dec 21, 2016 at 12:12 PM, Tobias Bocanegra <tr...@apache.org>
> wrote:
>
>> Hi Timothee,
>>
>> there exist 2 issues related to this:
>>
>> JCRVLT-35 is about 'uploading' them directly from a stream (probably what
>> you want)
>> JCRVLT-50 is about avoiding to store the packages in the content
>> repository
>>
>> regards, toby
>>
>> - https://issues.apache.org/jira/browse/JCRVLT-35
>> - https://issues.apache.org/jira/browse/JCRVLT-50
>>
>> On Tue, Dec 20, 2016 at 6:40 PM, Timothee Maret <tm...@apache.org>
>> wrote:
>>
>>> Hi,
>>>
>>> For Sling Content Distribution we are using VLT (by default) as a
>>> serialisation. Currently, on the SCD receiving side, the VLT packages must
>>> be written in a file to disk before being passed to the PackageManager API
>>> and ultimately exported to the repository.
>>>
>>> I am thinking that if the PackageManager was able to export directly
>>> from an InputStream, we may speed up the SCD import by skipping the
>>> write-to-temp-file operation.
>>>
>>> Maybe the PackageManger could be extended with the following signature
>>> in order to allow extracting from InputStream:
>>>
>>> void PackageManager.extract(session, importOptions,
>>> InputStream);
>>>
>>> wdyt ?
>>>
>>> Regards,
>>>
>>> Timothee
>>>
>>> [0] https://github.com/apache/sling/blob/trunk/contrib/exten
>>> sions/distribution/core/src/main/java/org/apache/sling/distr
>>> ibution/serialization/impl/vlt/FileVaultContentSerializer.java#L108-L147
>>>
>>
>>
>
Re: [JCRVLT] extract package directly from an inputstream
Posted by Tobias Bocanegra <tr...@apache.org>.
I was wrong. JCRVLT-35 is about something else.....
regards, toby
On Wed, Dec 21, 2016 at 12:12 PM, Tobias Bocanegra <tr...@apache.org>
wrote:
> Hi Timothee,
>
> there exist 2 issues related to this:
>
> JCRVLT-35 is about 'uploading' them directly from a stream (probably what
> you want)
> JCRVLT-50 is about avoiding to store the packages in the content repository
>
> regards, toby
>
> - https://issues.apache.org/jira/browse/JCRVLT-35
> - https://issues.apache.org/jira/browse/JCRVLT-50
>
> On Tue, Dec 20, 2016 at 6:40 PM, Timothee Maret <tm...@apache.org> wrote:
>
>> Hi,
>>
>> For Sling Content Distribution we are using VLT (by default) as a
>> serialisation. Currently, on the SCD receiving side, the VLT packages must
>> be written in a file to disk before being passed to the PackageManager API
>> and ultimately exported to the repository.
>>
>> I am thinking that if the PackageManager was able to export directly from
>> an InputStream, we may speed up the SCD import by skipping the
>> write-to-temp-file operation.
>>
>> Maybe the PackageManger could be extended with the following signature in
>> order to allow extracting from InputStream:
>>
>> void PackageManager.extract(session, importOptions,
>> InputStream);
>>
>> wdyt ?
>>
>> Regards,
>>
>> Timothee
>>
>> [0] https://github.com/apache/sling/blob/trunk/contrib/exten
>> sions/distribution/core/src/main/java/org/apache/sling/
>> distribution/serialization/impl/vlt/FileVaultContentSeria
>> lizer.java#L108-L147
>>
>
>
Re: [JCRVLT] extract package directly from an inputstream
Posted by Tobias Bocanegra <tr...@apache.org>.
Hi Timothee,
there exist 2 issues related to this:
JCRVLT-35 is about 'uploading' them directly from a stream (probably what
you want)
JCRVLT-50 is about avoiding to store the packages in the content repository
regards, toby
- https://issues.apache.org/jira/browse/JCRVLT-35
- https://issues.apache.org/jira/browse/JCRVLT-50
On Tue, Dec 20, 2016 at 6:40 PM, Timothee Maret <tm...@apache.org> wrote:
> Hi,
>
> For Sling Content Distribution we are using VLT (by default) as a
> serialisation. Currently, on the SCD receiving side, the VLT packages must
> be written in a file to disk before being passed to the PackageManager API
> and ultimately exported to the repository.
>
> I am thinking that if the PackageManager was able to export directly from
> an InputStream, we may speed up the SCD import by skipping the
> write-to-temp-file operation.
>
> Maybe the PackageManger could be extended with the following signature in
> order to allow extracting from InputStream:
>
> void PackageManager.extract(session, importOptions, InputStream);
>
> wdyt ?
>
> Regards,
>
> Timothee
>
> [0] https://github.com/apache/sling/blob/trunk/contrib/
> extensions/distribution/core/src/main/java/org/apache/sling/distribution/
> serialization/impl/vlt/FileVaultContentSerializer.java#L108-L147
>