You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by Thomas Mueller <mu...@adobe.com> on 2014/06/26 15:05:24 UTC

Oak-run "backup" trouble (heap memory, external blob)

Hi,

I tried to backup a segmentstore using

java -mx2g -jar oak-run-1.0.2-SNAPSHOT.jar backup segmentstore s2
java.lang.OutOfMemoryError: Java heap space

I think 2 GB heap are quite a lot, is this expected? The repository is
relatively large (2260 tar files; 566 GB). So anyway, then I tried with 8
GB heap:

java -mx8g -jar oak-run-1.0.2-SNAPSHOT.jar backup segmentstore s2
Attempt to read external blob with blobId
[c184d2a3f1dbc709004a45ae6c5df7624c2ae653#32768] without specifying
BlobStore
  at 
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.getReference(SegmentB
lob.java:118)
  at 
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeBlob(SegmentWr
iter.java:706)
  at 
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeProperty(Segme
ntWriter.java:808)
  at 
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeProperty(Segme
ntWriter.java:796)

I think I can't specify a BlobStore on the command line... What should I
do?

Regards,
Thomas

Re: Oak-run "backup" trouble (heap memory, external blob)

Posted by Thomas Mueller <mu...@adobe.com>.
Created OAK-1921

On 26/06/14 15:58, "Thomas Mueller" <mu...@adobe.com> wrote:

>Hi,
>
>OK, makes sense. I'll create a Jira issue for the "external blob" case,
>and will find out more about the memory issue.
>
>Regards,
>Thomas
>
>
>
>
>On 26/06/14 15:55, "Jukka Zitting" <ju...@gmail.com> wrote:
>
>>Hi,
>>
>>On Thu, Jun 26, 2014 at 9:05 AM, Thomas Mueller <mu...@adobe.com>
>>wrote:
>>> I tried to backup a segmentstore using
>>>
>>> java -mx2g -jar oak-run-1.0.2-SNAPSHOT.jar backup segmentstore s2
>>> java.lang.OutOfMemoryError: Java heap space
>>>
>>> I think 2 GB heap are quite a lot, is this expected? The repository is
>>> relatively large (2260 tar files; 566 GB).
>>
>>Hmm, that's not expected. Can you try with
>>-XX:+HeapDumpOnOutOfMemoryError and take a look at what's using all
>>the extra memory?
>>
>>> java -mx8g -jar oak-run-1.0.2-SNAPSHOT.jar backup segmentstore s2
>>> Attempt to read external blob with blobId
>>> [c184d2a3f1dbc709004a45ae6c5df7624c2ae653#32768] without specifying
>>> BlobStore
>>> [...]
>>>
>>> I think I can't specify a BlobStore on the command line... What should
>>>I
>>> do?
>>
>>Right, it's not possible to configure a BlobStore on the command line.
>>I see two alternatives here:
>>
>>1) Adjust the backup code to work like compaction does, i.e. leave
>>external blobs as-is and perhaps output a message that informs the
>>user about the need to use a different mechanism to back up the
>>BlobStore contents
>>
>>2) Add command line options for configuring the BlobStore to be used
>>for accessing external blobs.
>>
>>BR,
>>
>>Jukka Zitting
>

Re: Oak-run "backup" trouble (heap memory, external blob)

Posted by Thomas Mueller <mu...@adobe.com>.
Hi,

OK, makes sense. I'll create a Jira issue for the "external blob" case,
and will find out more about the memory issue.

Regards,
Thomas




On 26/06/14 15:55, "Jukka Zitting" <ju...@gmail.com> wrote:

>Hi,
>
>On Thu, Jun 26, 2014 at 9:05 AM, Thomas Mueller <mu...@adobe.com> wrote:
>> I tried to backup a segmentstore using
>>
>> java -mx2g -jar oak-run-1.0.2-SNAPSHOT.jar backup segmentstore s2
>> java.lang.OutOfMemoryError: Java heap space
>>
>> I think 2 GB heap are quite a lot, is this expected? The repository is
>> relatively large (2260 tar files; 566 GB).
>
>Hmm, that's not expected. Can you try with
>-XX:+HeapDumpOnOutOfMemoryError and take a look at what's using all
>the extra memory?
>
>> java -mx8g -jar oak-run-1.0.2-SNAPSHOT.jar backup segmentstore s2
>> Attempt to read external blob with blobId
>> [c184d2a3f1dbc709004a45ae6c5df7624c2ae653#32768] without specifying
>> BlobStore
>> [...]
>>
>> I think I can't specify a BlobStore on the command line... What should I
>> do?
>
>Right, it's not possible to configure a BlobStore on the command line.
>I see two alternatives here:
>
>1) Adjust the backup code to work like compaction does, i.e. leave
>external blobs as-is and perhaps output a message that informs the
>user about the need to use a different mechanism to back up the
>BlobStore contents
>
>2) Add command line options for configuring the BlobStore to be used
>for accessing external blobs.
>
>BR,
>
>Jukka Zitting

Re: Oak-run "backup" trouble (heap memory, external blob)

Posted by Jukka Zitting <ju...@gmail.com>.
Hi,

On Thu, Jun 26, 2014 at 9:05 AM, Thomas Mueller <mu...@adobe.com> wrote:
> I tried to backup a segmentstore using
>
> java -mx2g -jar oak-run-1.0.2-SNAPSHOT.jar backup segmentstore s2
> java.lang.OutOfMemoryError: Java heap space
>
> I think 2 GB heap are quite a lot, is this expected? The repository is
> relatively large (2260 tar files; 566 GB).

Hmm, that's not expected. Can you try with
-XX:+HeapDumpOnOutOfMemoryError and take a look at what's using all
the extra memory?

> java -mx8g -jar oak-run-1.0.2-SNAPSHOT.jar backup segmentstore s2
> Attempt to read external blob with blobId
> [c184d2a3f1dbc709004a45ae6c5df7624c2ae653#32768] without specifying
> BlobStore
> [...]
>
> I think I can't specify a BlobStore on the command line... What should I
> do?

Right, it's not possible to configure a BlobStore on the command line.
I see two alternatives here:

1) Adjust the backup code to work like compaction does, i.e. leave
external blobs as-is and perhaps output a message that informs the
user about the need to use a different mechanism to back up the
BlobStore contents

2) Add command line options for configuring the BlobStore to be used
for accessing external blobs.

BR,

Jukka Zitting