You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@geode.apache.org by Sneha George <ge...@gmail.com> on 2017/10/27 04:29:47 UTC
Regarding storing documents in Apache Geode
Hello ,
Wanted to know if documents like ppts, pdfs and images can be stored in
cache using Apache Geode. If , so could you provide me some direction on
the implementation details.
Thanks,
Sneha
Re: Regarding storing documents in Apache Geode
Posted by Darrel Schneider <ds...@pivotal.io>.
If you do have large objects to store and want to avoid a large heap and
the increased GC pressure consider using an offheap geode region.
On Fri, Oct 27, 2017 at 9:29 AM, Anthony Baker <ab...@pivotal.io> wrote:
> Another factor to consider is the size of the files. Storing very large
> files on heap can create lots of GC pressure (and performance impact).
>
> Anthony
>
> On Oct 27, 2017, at 9:15 AM, Dan Smith <ds...@pivotal.io> wrote:
>
> Do you want to extract and query metadata in your documents, or are you
> just trying to store them some where? As John pointed out, if you are just
> trying to store them you can just to a region.put(document_name,
> document_bytes) to store your document in geode. The geode-examples
> <https://github.com/apache/geode-examples/> are a good place to start if
> you want to see some working code.
>
> If you want to extract metadata from the document, geode doesn't provide
> anything to do that itself. But you could probably use a combination of
> something like apache tika to extract text from your document and geode to
> store and index the text.
>
> -Dan
>
> On Thu, Oct 26, 2017 at 11:05 PM, John Blum <jb...@pivotal.io> wrote:
>
>> Alternatively, you can, and probably should, store the documents as a
>> byte array (in value) instead.
>>
>> On Fri, Oct 27, 2017 at 1:16 AM, Akihiro Kitada <ak...@pivotal.io>
>> wrote:
>>
>>> Hello,
>>>
>>> I believe you can store any types of document files if you can convert
>>> them into some types of java objects.
>>>
>>>
>>>
>>>
>>> --
>>> Akihiro Kitada | Staff Customer Engineer | +81 80 3716 3736
>>> <+81%2080-3716-3736>
>>> Support.Pivotal.io <http://support.pivotal.io/> | Mon-Fri 9:00am to
>>> 5:30pm JST | 1-877-477-2269 <(877)%20477-2269>
>>> [image: support] <https://support.pivotal.io/> [image: twitter]
>>> <https://twitter.com/pivotal> [image: linkedin]
>>> <https://www.linkedin.com/company/3048967> [image: facebook]
>>> <https://www.facebook.com/pivotalsoftware> [image: google plus]
>>> <https://plus.google.com/+Pivotal> [image: youtube]
>>> <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
>>>
>>>
>>> 2017-10-27 13:29 GMT+09:00 Sneha George <ge...@gmail.com>:
>>>
>>>> Hello ,
>>>>
>>>> Wanted to know if documents like ppts, pdfs and images can be stored in
>>>> cache using Apache Geode. If , so could you provide me some direction on
>>>> the implementation details.
>>>>
>>>> Thanks,
>>>> Sneha
>>>>
>>>
>>>
>>
>>
>> --
>> -John
>> john.blum10101 (skype)
>>
>
>
>
Re: Regarding storing documents in Apache Geode
Posted by Anthony Baker <ab...@pivotal.io>.
Another factor to consider is the size of the files. Storing very large files on heap can create lots of GC pressure (and performance impact).
Anthony
> On Oct 27, 2017, at 9:15 AM, Dan Smith <ds...@pivotal.io> wrote:
>
> Do you want to extract and query metadata in your documents, or are you just trying to store them some where? As John pointed out, if you are just trying to store them you can just to a region.put(document_name, document_bytes) to store your document in geode. The geode-examples <https://github.com/apache/geode-examples/> are a good place to start if you want to see some working code.
>
> If you want to extract metadata from the document, geode doesn't provide anything to do that itself. But you could probably use a combination of something like apache tika to extract text from your document and geode to store and index the text.
>
> -Dan
>
> On Thu, Oct 26, 2017 at 11:05 PM, John Blum <jblum@pivotal.io <ma...@pivotal.io>> wrote:
> Alternatively, you can, and probably should, store the documents as a byte array (in value) instead.
>
> On Fri, Oct 27, 2017 at 1:16 AM, Akihiro Kitada <akitada@pivotal.io <ma...@pivotal.io>> wrote:
> Hello,
>
> I believe you can store any types of document files if you can convert them into some types of java objects.
>
>
>
>
> --
> Akihiro Kitada | Staff Customer Engineer | +81 80 3716 3736 <tel:+81%2080-3716-3736>
> Support.Pivotal.io <http://support.pivotal.io/> | Mon-Fri 9:00am to 5:30pm JST | 1-877-477-2269 <tel:(877)%20477-2269>
> <https://support.pivotal.io/> <https://twitter.com/pivotal> <https://www.linkedin.com/company/3048967> <https://www.facebook.com/pivotalsoftware> <https://plus.google.com/+Pivotal> <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
>
>
> 2017-10-27 13:29 GMT+09:00 Sneha George <geosneha@gmail.com <ma...@gmail.com>>:
> Hello ,
>
> Wanted to know if documents like ppts, pdfs and images can be stored in cache using Apache Geode. If , so could you provide me some direction on the implementation details.
>
> Thanks,
> Sneha
>
>
>
>
> --
> -John
> john.blum10101 (skype)
>
Re: Regarding storing documents in Apache Geode
Posted by Dan Smith <ds...@pivotal.io>.
Do you want to extract and query metadata in your documents, or are you
just trying to store them some where? As John pointed out, if you are just
trying to store them you can just to a region.put(document_name,
document_bytes) to store your document in geode. The geode-examples
<https://github.com/apache/geode-examples/> are a good place to start if
you want to see some working code.
If you want to extract metadata from the document, geode doesn't provide
anything to do that itself. But you could probably use a combination of
something like apache tika to extract text from your document and geode to
store and index the text.
-Dan
On Thu, Oct 26, 2017 at 11:05 PM, John Blum <jb...@pivotal.io> wrote:
> Alternatively, you can, and probably should, store the documents as a byte
> array (in value) instead.
>
> On Fri, Oct 27, 2017 at 1:16 AM, Akihiro Kitada <ak...@pivotal.io>
> wrote:
>
>> Hello,
>>
>> I believe you can store any types of document files if you can convert
>> them into some types of java objects.
>>
>>
>>
>>
>> --
>> Akihiro Kitada | Staff Customer Engineer | +81 80 3716 3736
>> <+81%2080-3716-3736>
>> Support.Pivotal.io <http://support.pivotal.io/> | Mon-Fri 9:00am to
>> 5:30pm JST | 1-877-477-2269 <(877)%20477-2269>
>> [image: support] <https://support.pivotal.io/> [image: twitter]
>> <https://twitter.com/pivotal> [image: linkedin]
>> <https://www.linkedin.com/company/3048967> [image: facebook]
>> <https://www.facebook.com/pivotalsoftware> [image: google plus]
>> <https://plus.google.com/+Pivotal> [image: youtube]
>> <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
>>
>>
>> 2017-10-27 13:29 GMT+09:00 Sneha George <ge...@gmail.com>:
>>
>>> Hello ,
>>>
>>> Wanted to know if documents like ppts, pdfs and images can be stored in
>>> cache using Apache Geode. If , so could you provide me some direction on
>>> the implementation details.
>>>
>>> Thanks,
>>> Sneha
>>>
>>
>>
>
>
> --
> -John
> john.blum10101 (skype)
>
Re: Regarding storing documents in Apache Geode
Posted by John Blum <jb...@pivotal.io>.
Alternatively, you can, and probably should, store the documents as a byte
array (in value) instead.
On Fri, Oct 27, 2017 at 1:16 AM, Akihiro Kitada <ak...@pivotal.io> wrote:
> Hello,
>
> I believe you can store any types of document files if you can convert
> them into some types of java objects.
>
>
>
>
> --
> Akihiro Kitada | Staff Customer Engineer | +81 80 3716 3736
> <+81%2080-3716-3736>
> Support.Pivotal.io <http://support.pivotal.io/> | Mon-Fri 9:00am to
> 5:30pm JST | 1-877-477-2269 <(877)%20477-2269>
> [image: support] <https://support.pivotal.io/> [image: twitter]
> <https://twitter.com/pivotal> [image: linkedin]
> <https://www.linkedin.com/company/3048967> [image: facebook]
> <https://www.facebook.com/pivotalsoftware> [image: google plus]
> <https://plus.google.com/+Pivotal> [image: youtube]
> <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
>
>
> 2017-10-27 13:29 GMT+09:00 Sneha George <ge...@gmail.com>:
>
>> Hello ,
>>
>> Wanted to know if documents like ppts, pdfs and images can be stored in
>> cache using Apache Geode. If , so could you provide me some direction on
>> the implementation details.
>>
>> Thanks,
>> Sneha
>>
>
>
--
-John
john.blum10101 (skype)
Re: Regarding storing documents in Apache Geode
Posted by Akihiro Kitada <ak...@pivotal.io>.
Hello,
I believe you can store any types of document files if you can convert them
into some types of java objects.
--
Akihiro Kitada | Staff Customer Engineer | +81 80 3716 3736
Support.Pivotal.io <http://support.pivotal.io/> | Mon-Fri 9:00am to
5:30pm JST | 1-877-477-2269
[image: support] <https://support.pivotal.io/> [image: twitter]
<https://twitter.com/pivotal> [image: linkedin]
<https://www.linkedin.com/company/3048967> [image: facebook]
<https://www.facebook.com/pivotalsoftware> [image: google plus]
<https://plus.google.com/+Pivotal> [image: youtube]
<https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
2017-10-27 13:29 GMT+09:00 Sneha George <ge...@gmail.com>:
> Hello ,
>
> Wanted to know if documents like ppts, pdfs and images can be stored in
> cache using Apache Geode. If , so could you provide me some direction on
> the implementation details.
>
> Thanks,
> Sneha
>