You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@poi.apache.org by jaehoon jeong <je...@gmail.com> on 2020/02/03 14:15:46 UTC
out of memory error when creating pptx file
Hello
I'm trying to generate a pptx file using the poi library.
The XMLSlideShow class contains about 600 XSLFSlides.
Each XSLFSlide object uses about 3mb of memory. 1.8gb of memory is required to create one pptx file.
It uses out too much memory, causing an out of memory error.
And I can't increase the heap memory size.
Is there a way to save slides in multiple times in one pptx file?
Is there a way to reduce memory usage?
Thanks in advance.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: out of memory error when creating pptx file
Posted by Rob Sargent <ro...@gmail.com>.
> On Feb 4, 2020, at 1:32 AM, jaehoon jeong <je...@gmail.com> wrote:
>
> Thank you for your reply.
>
> I can use about 4g of memory. But I have to handle multiple requests.
Memory is remarkably inexpensive compared to your time and your sanity
> For example, if three requests come in at the same time, an out of memory error occurs.
>
> One request takes about 30 seconds.
>
Are you automatically generating 600slides per online request. Is the slideshow a report?
> Saving each of the 100 slides and then merging them also uses a lot of memory.
>
> Can xerces merge individual pptx files without consuming too much memory?
Xerces is an XML tool. Perhaps you can stream edit a template to generate the final show.
>> On 2020/02/04 02:30:43, Rob Sargent <ro...@gmail.com> wrote:
>>
>>
>>> On 2/3/20 7:20 PM, jaehoon jeong wrote:
>>> Thank you for your reply.
>>>
>>> I am using oracle jdk1.8. But I do not want to increase heap memory.
>>> I want to find a solution in another way.
>> How little memory must this fit in?
>>>
>>> What does "chpaters" you mean?
>>> For example, does it mean to divide into 100 slides and save each file?
>>>
>>> I do not want to save as multiple files.
>>> I want to save all slides in one file.
>>>
>>> For example, I'm considering flushing slides to a file every 100 slides created.
>>> Is this possible using the poi library?
>> How far in to 600 do you get?
>> I'm not sure but to save each hundred slides would be to write a new
>> workbook. You would then have to merge those into a single workbook via
>> poi unless you are prepared to merge the underlying xml files? That
>> approach would certainly be more memory efficient but more a xerces
>> problem than a poi problem.
>>>
>>> On 2020/02/03 14:56:13, Rob Sargent <ro...@gmail.com> wrote:
>>>> Since no one really wants to sit through 600 slides, break the total up in “chapters” and see where that gets you?
>>>>
>>>> How much memory does you machine have, which version of java are you using.
>>>>
>>>>> On Feb 3, 2020, at 7:15 AM, jaehoon jeong <je...@gmail.com> wrote:
>>>>>
>>>>> Hello
>>>>>
>>>>> I'm trying to generate a pptx file using the poi library.
>>>>> The XMLSlideShow class contains about 600 XSLFSlides.
>>>>> Each XSLFSlide object uses about 3mb of memory. 1.8gb of memory is required to create one pptx file.
>>>>>
>>>>> It uses out too much memory, causing an out of memory error.
>>>>> And I can't increase the heap memory size.
>>>>>
>>>>> Is there a way to save slides in multiple times in one pptx file?
>>>>> Is there a way to reduce memory usage?
>>>>>
>>>>> Thanks in advance.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>>>>> For additional commands, e-mail: user-help@poi.apache.org
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>>>> For additional commands, e-mail: user-help@poi.apache.org
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>>> For additional commands, e-mail: user-help@poi.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>> For additional commands, e-mail: user-help@poi.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> For additional commands, e-mail: user-help@poi.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: out of memory error when creating pptx file
Posted by jaehoon jeong <je...@gmail.com>.
Thank you for your reply.
I can use about 4g of memory. But I have to handle multiple requests.
For example, if three requests come in at the same time, an out of memory error occurs.
One request takes about 30 seconds.
Saving each of the 100 slides and then merging them also uses a lot of memory.
Can xerces merge individual pptx files without consuming too much memory?
On 2020/02/04 02:30:43, Rob Sargent <ro...@gmail.com> wrote:
>
>
> On 2/3/20 7:20 PM, jaehoon jeong wrote:
> > Thank you for your reply.
> >
> > I am using oracle jdk1.8. But I do not want to increase heap memory.
> > I want to find a solution in another way.
> How little memory must this fit in?
> >
> > What does "chpaters" you mean?
> > For example, does it mean to divide into 100 slides and save each file?
> >
> > I do not want to save as multiple files.
> > I want to save all slides in one file.
> >
> > For example, I'm considering flushing slides to a file every 100 slides created.
> > Is this possible using the poi library?
> How far in to 600 do you get?
> I'm not sure but to save each hundred slides would be to write a new
> workbook. You would then have to merge those into a single workbook via
> poi unless you are prepared to merge the underlying xml files? That
> approach would certainly be more memory efficient but more a xerces
> problem than a poi problem.
> >
> > On 2020/02/03 14:56:13, Rob Sargent <ro...@gmail.com> wrote:
> >> Since no one really wants to sit through 600 slides, break the total up in “chapters” and see where that gets you?
> >>
> >> How much memory does you machine have, which version of java are you using.
> >>
> >>> On Feb 3, 2020, at 7:15 AM, jaehoon jeong <je...@gmail.com> wrote:
> >>>
> >>> Hello
> >>>
> >>> I'm trying to generate a pptx file using the poi library.
> >>> The XMLSlideShow class contains about 600 XSLFSlides.
> >>> Each XSLFSlide object uses about 3mb of memory. 1.8gb of memory is required to create one pptx file.
> >>>
> >>> It uses out too much memory, causing an out of memory error.
> >>> And I can't increase the heap memory size.
> >>>
> >>> Is there a way to save slides in multiple times in one pptx file?
> >>> Is there a way to reduce memory usage?
> >>>
> >>> Thanks in advance.
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> >>> For additional commands, e-mail: user-help@poi.apache.org
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> >> For additional commands, e-mail: user-help@poi.apache.org
> >>
> >>
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> > For additional commands, e-mail: user-help@poi.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> For additional commands, e-mail: user-help@poi.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: out of memory error when creating pptx file
Posted by Rob Sargent <ro...@gmail.com>.
On 2/3/20 7:20 PM, jaehoon jeong wrote:
> Thank you for your reply.
>
> I am using oracle jdk1.8. But I do not want to increase heap memory.
> I want to find a solution in another way.
How little memory must this fit in?
>
> What does "chpaters" you mean?
> For example, does it mean to divide into 100 slides and save each file?
>
> I do not want to save as multiple files.
> I want to save all slides in one file.
>
> For example, I'm considering flushing slides to a file every 100 slides created.
> Is this possible using the poi library?
How far in to 600 do you get?
I'm not sure but to save each hundred slides would be to write a new
workbook. You would then have to merge those into a single workbook via
poi unless you are prepared to merge the underlying xml files? That
approach would certainly be more memory efficient but more a xerces
problem than a poi problem.
>
> On 2020/02/03 14:56:13, Rob Sargent <ro...@gmail.com> wrote:
>> Since no one really wants to sit through 600 slides, break the total up in “chapters” and see where that gets you?
>>
>> How much memory does you machine have, which version of java are you using.
>>
>>> On Feb 3, 2020, at 7:15 AM, jaehoon jeong <je...@gmail.com> wrote:
>>>
>>> Hello
>>>
>>> I'm trying to generate a pptx file using the poi library.
>>> The XMLSlideShow class contains about 600 XSLFSlides.
>>> Each XSLFSlide object uses about 3mb of memory. 1.8gb of memory is required to create one pptx file.
>>>
>>> It uses out too much memory, causing an out of memory error.
>>> And I can't increase the heap memory size.
>>>
>>> Is there a way to save slides in multiple times in one pptx file?
>>> Is there a way to reduce memory usage?
>>>
>>> Thanks in advance.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>>> For additional commands, e-mail: user-help@poi.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>> For additional commands, e-mail: user-help@poi.apache.org
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> For additional commands, e-mail: user-help@poi.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: out of memory error when creating pptx file
Posted by jaehoon jeong <je...@gmail.com>.
Thank you for your reply.
I am using oracle jdk1.8. But I do not want to increase heap memory.
I want to find a solution in another way.
What does "chpaters" you mean?
For example, does it mean to divide into 100 slides and save each file?
I do not want to save as multiple files.
I want to save all slides in one file.
For example, I'm considering flushing slides to a file every 100 slides created.
Is this possible using the poi library?
On 2020/02/03 14:56:13, Rob Sargent <ro...@gmail.com> wrote:
> Since no one really wants to sit through 600 slides, break the total up in “chapters” and see where that gets you?
>
> How much memory does you machine have, which version of java are you using.
>
> > On Feb 3, 2020, at 7:15 AM, jaehoon jeong <je...@gmail.com> wrote:
> >
> > Hello
> >
> > I'm trying to generate a pptx file using the poi library.
> > The XMLSlideShow class contains about 600 XSLFSlides.
> > Each XSLFSlide object uses about 3mb of memory. 1.8gb of memory is required to create one pptx file.
> >
> > It uses out too much memory, causing an out of memory error.
> > And I can't increase the heap memory size.
> >
> > Is there a way to save slides in multiple times in one pptx file?
> > Is there a way to reduce memory usage?
> >
> > Thanks in advance.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> > For additional commands, e-mail: user-help@poi.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> For additional commands, e-mail: user-help@poi.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: out of memory error when creating pptx file
Posted by Rob Sargent <ro...@gmail.com>.
Since no one really wants to sit through 600 slides, break the total up in “chapters” and see where that gets you?
How much memory does you machine have, which version of java are you using.
> On Feb 3, 2020, at 7:15 AM, jaehoon jeong <je...@gmail.com> wrote:
>
> Hello
>
> I'm trying to generate a pptx file using the poi library.
> The XMLSlideShow class contains about 600 XSLFSlides.
> Each XSLFSlide object uses about 3mb of memory. 1.8gb of memory is required to create one pptx file.
>
> It uses out too much memory, causing an out of memory error.
> And I can't increase the heap memory size.
>
> Is there a way to save slides in multiple times in one pptx file?
> Is there a way to reduce memory usage?
>
> Thanks in advance.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> For additional commands, e-mail: user-help@poi.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: out of memory error when creating pptx file
Posted by kiwiwings <ki...@apache.org>.
With the current POI version, you'll end up with all slides in memory - I
think it could be possible to provide a lazy-loading interface where the
pages and shapes are parsed on the fly. But currently/as stated, I think the
solution is mostly without POI.
If you want you can send me an example (maybe just a few of the 600 slides)
privately and I'll have a look.
> Is there a way to make two xml files into one pptx file without using a
> lot of memory?
You need to do a few changes:
- renaming the slides
- adding the slides to presentation.xml + .refs
- in case you reference media files, update their filename and the slide
.refs
--
Sent from: http://apache-poi.1045710.n5.nabble.com/POI-User-f2280730.html
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: out of memory error when creating pptx file
Posted by jaehoon jeong <je...@gmail.com>.
Thank you for your reply.
The 600 slides have a similar structure.
Creating an xml file with template information and an xml file with data seems difficult.
But suppose I created each xml file.
How do I map the data?
I will parse the repetitive data xml and map it to a slides object. In the end, it will use a lot of memory.
Is there a way to make two xml files into one pptx file without using a lot of memory?
On 2020/02/03 19:21:51, Andreas Beeker <ki...@apache.org> wrote:
> Hi,
>
> XSLF and XmlBeans are undoubtedly memory hogs, but I can't solve this in a short period -
> I often thought about an internal model which we could use for XSLF / HSLF but lets get back to your issue.
>
> So when you say, you need to generate 600 slides ... does that mean they have similar structure?
> i.e. could you use a template mechanism?
>
> If true, I would generate one set of slides and then copy and fill those with the repetitive data outside of POI.
> So handling the two XML files (slide + .refs) and adding it to the zip file is no magic.
> Apart of the slides the presentation.xml and its .refs need also to be modified.
>
> Andi
>
>
>
> On 03.02.20 15:15, jaehoon jeong wrote:
> > Hello
> >
> > I'm trying to generate a pptx file using the poi library.
> > The XMLSlideShow class contains about 600 XSLFSlides.
> > Each XSLFSlide object uses about 3mb of memory. 1.8gb of memory is required to create one pptx file.
> >
> > It uses out too much memory, causing an out of memory error.
> > And I can't increase the heap memory size.
> >
> > Is there a way to save slides in multiple times in one pptx file?
> > Is there a way to reduce memory usage?
> >
> > Thanks in advance.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> > For additional commands, e-mail: user-help@poi.apache.org
> >
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: out of memory error when creating pptx file
Posted by Andreas Beeker <ki...@apache.org>.
Hi,
XSLF and XmlBeans are undoubtedly memory hogs, but I can't solve this in a short period -
I often thought about an internal model which we could use for XSLF / HSLF but lets get back to your issue.
So when you say, you need to generate 600 slides ... does that mean they have similar structure?
i.e. could you use a template mechanism?
If true, I would generate one set of slides and then copy and fill those with the repetitive data outside of POI.
So handling the two XML files (slide + .refs) and adding it to the zip file is no magic.
Apart of the slides the presentation.xml and its .refs need also to be modified.
Andi
On 03.02.20 15:15, jaehoon jeong wrote:
> Hello
>
> I'm trying to generate a pptx file using the poi library.
> The XMLSlideShow class contains about 600 XSLFSlides.
> Each XSLFSlide object uses about 3mb of memory. 1.8gb of memory is required to create one pptx file.
>
> It uses out too much memory, causing an out of memory error.
> And I can't increase the heap memory size.
>
> Is there a way to save slides in multiple times in one pptx file?
> Is there a way to reduce memory usage?
>
> Thanks in advance.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> For additional commands, e-mail: user-help@poi.apache.org
>