You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@xmlbeans.apache.org by Jason Berk <jb...@purdueefcu.com> on 2009/09/14 22:16:20 UTC
java.lang.OutOfMemoryError: Java heap space
I have an xml clob and account in a DB. I need to select the xml and
bind it with XML Beans. This works fine for the first 30,000 or so
accounts, but eventually I get "ava.lang.OutOfMemoryError: Java heap
space"
This is the line that is blowing up:
MemberStatementDocument doc = MemberStatementDocument.Factory.parse(xml,
validationOptions);
In this method to do the bind:
private MemberStatement bind(String xml) throws Exception {
ArrayList<XmlValidationError> validationErrors = new
ArrayList<XmlValidationError>();
XmlOptions validationOptions = new XmlOptions();
validationOptions.setErrorListener(validationErrors);
//validationOptions.setLoadLineNumbers();
//validationOptions.setLoadTrimTextBuffer();
//validationOptions.setUnsynchronized();
//validationOptions.setLoadUseXMLReader(
SAXParserFactory.newInstance().newSAXParser().getXMLReader() );
MemberStatementDocument doc =
MemberStatementDocument.Factory.parse(xml, validationOptions);
if (!doc.validate(validationOptions)) {
logger.error("Validation errors discovered
during binding:");
Iterator<XmlValidationError> iter =
validationErrors.iterator();
while (iter.hasNext()) {
logger.error(">> " + iter.next());
}
throw new XmlException("Validation errors
discovered during binding:");
}
MemberStatement memberStatement =
doc.getMemberStatement();
if (memberStatement.getUnknownShareList().size() > 0 ||
memberStatement.getUnknownLoanList().size() > 0) {
throw new XmlException("account contains unknown
shares or loans");
}
return memberStatement;
}
Is there a better way to be doing the binding? What about the parse
method is not playing well with the GC?
Jason
Now serving Boiler Spirit with every purchase. Switch to the new Purdue debit card today. Show your pride and earn Scorecard Rewards on the side. Just sign for your purchases and score valuable rewards points you can use for travel, electronics or even cash.
***This is a transmission from Purdue Employees Federal Credit
Union (PEFCU) and is intended solely for its authorized
recipient(s), and may contain information that is confidential
and or legally privileged. If you are not an addressee, or the
employee or agent responsible for delivering it to an addressee,
you are hereby notified that any use, dissemination,
distribution, publication or copying of the information
contained
in this email is strictly prohibited. If you have received this
transmission in error, please notify us by telephoning (765)
497-3328 or returning the email. You are then instructed to
delete the information from your computer. Thank you for your
cooperation.***
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: user-help@xmlbeans.apache.org
RE: java.lang.OutOfMemoryError: Java heap space
Posted by Jason Berk <jb...@purdueefcu.com>.
The size of the xml in the DB (per account) is < 1MB and usually around
30KB. I'm in a loop that basically does the following:
Accounts = getAccounts( )
For each account in Accounts{
Object boundObject = bind(account)
Object pdf = createPdf( boundObject )
savePdf( pdf );
}
After processing about 32K of my 55K accounts, my code throws the OOM
error
What I was wondering is, after I do a parse( ) to parse the string of
xml I get back from the query, and then bind it to get my object, do I
need to clear or close something out?
Jason
________________________________
From: Jacob Danner [mailto:jacob.danner@gmail.com]
Sent: Monday, September 14, 2009 4:27 PM
To: user@xmlbeans.apache.org
Subject: Re: java.lang.OutOfMemoryError: Java heap space
Hmm, are you putting 30K of xml into memory? or only parsing the results
as they are needed?
In other words, are there references left around that is causing the
heap to grow?
-jacobd
On Mon, Sep 14, 2009 at 1:16 PM, Jason Berk <jb...@purdueefcu.com>
wrote:
I have an xml clob and account in a DB. I need to select the xml and
bind it with XML Beans. This works fine for the first 30,000 or so
accounts, but eventually I get "ava.lang.OutOfMemoryError: Java heap
space"
This is the line that is blowing up:
MemberStatementDocument doc = MemberStatementDocument.Factory.parse(xml,
validationOptions);
In this method to do the bind:
private MemberStatement bind(String xml) throws Exception {
ArrayList<XmlValidationError> validationErrors = new
ArrayList<XmlValidationError>();
XmlOptions validationOptions = new XmlOptions();
validationOptions.setErrorListener(validationErrors);
//validationOptions.setLoadLineNumbers();
//validationOptions.setLoadTrimTextBuffer();
//validationOptions.setUnsynchronized();
//validationOptions.setLoadUseXMLReader(
SAXParserFactory.newInstance().newSAXParser().getXMLReader() );
MemberStatementDocument doc =
MemberStatementDocument.Factory.parse(xml, validationOptions);
if (!doc.validate(validationOptions)) {
logger.error("Validation errors discovered
during binding:");
Iterator<XmlValidationError> iter =
validationErrors.iterator();
while (iter.hasNext()) {
logger.error(">> " + iter.next());
}
throw new XmlException("Validation errors
discovered during binding:");
}
MemberStatement memberStatement =
doc.getMemberStatement();
if (memberStatement.getUnknownShareList().size() > 0 ||
memberStatement.getUnknownLoanList().size() > 0) {
throw new XmlException("account contains unknown
shares or loans");
}
return memberStatement;
}
Is there a better way to be doing the binding? What about the parse
method is not playing well with the GC?
Jason
Now serving Boiler Spirit with every purchase. Switch to the new Purdue
debit card today. Show your pride and earn Scorecard Rewards on the
side. Just sign for your purchases and score valuable rewards points you
can use for travel, electronics or even cash.
***This is a transmission from Purdue Employees Federal Credit
Union (PEFCU) and is intended solely for its authorized
recipient(s), and may contain information that is confidential
and or legally privileged. If you are not an addressee, or the
employee or agent responsible for delivering it to an addressee,
you are hereby notified that any use, dissemination,
distribution, publication or copying of the information
contained
in this email is strictly prohibited. If you have received this
transmission in error, please notify us by telephoning (765)
497-3328 or returning the email. You are then instructed to
delete the information from your computer. Thank you for your
cooperation.***
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: user-help@xmlbeans.apache.org
Now serving Boiler Spirit with every purchase. Switch to the new Purdue debit card today. Show your pride and earn Scorecard Rewards on the side. Just sign for your purchases and score valuable rewards points you can use for travel, electronics or even cash.
***This is a transmission from Purdue Employees Federal Credit
Union (PEFCU) and is intended solely for its authorized
recipient(s), and may contain information that is confidential
and or legally privileged. If you are not an addressee, or the
employee or agent responsible for delivering it to an addressee,
you are hereby notified that any use, dissemination,
distribution, publication or copying of the information
contained
in this email is strictly prohibited. If you have received this
transmission in error, please notify us by telephoning (765)
497-3328 or returning the email. You are then instructed to
delete the information from your computer. Thank you for your
cooperation.***
Re: java.lang.OutOfMemoryError: Java heap space
Posted by Jacob Danner <ja...@gmail.com>.
Hmm, are you putting 30K of xml into memory? or only parsing the results as
they are needed?
In other words, are there references left around that is causing the heap to
grow?
-jacobd
On Mon, Sep 14, 2009 at 1:16 PM, Jason Berk <jb...@purdueefcu.com> wrote:
> I have an xml clob and account in a DB. I need to select the xml and
> bind it with XML Beans. This works fine for the first 30,000 or so
> accounts, but eventually I get "ava.lang.OutOfMemoryError: Java heap
> space"
>
> This is the line that is blowing up:
> MemberStatementDocument doc = MemberStatementDocument.Factory.parse(xml,
> validationOptions);
>
> In this method to do the bind:
>
> private MemberStatement bind(String xml) throws Exception {
> ArrayList<XmlValidationError> validationErrors = new
> ArrayList<XmlValidationError>();
> XmlOptions validationOptions = new XmlOptions();
> validationOptions.setErrorListener(validationErrors);
> //validationOptions.setLoadLineNumbers();
> //validationOptions.setLoadTrimTextBuffer();
> //validationOptions.setUnsynchronized();
> //validationOptions.setLoadUseXMLReader(
> SAXParserFactory.newInstance().newSAXParser().getXMLReader() );
> MemberStatementDocument doc =
> MemberStatementDocument.Factory.parse(xml, validationOptions);
> if (!doc.validate(validationOptions)) {
> logger.error("Validation errors discovered
> during binding:");
> Iterator<XmlValidationError> iter =
> validationErrors.iterator();
> while (iter.hasNext()) {
> logger.error(">> " + iter.next());
> }
> throw new XmlException("Validation errors
> discovered during binding:");
> }
> MemberStatement memberStatement =
> doc.getMemberStatement();
> if (memberStatement.getUnknownShareList().size() > 0 ||
> memberStatement.getUnknownLoanList().size() > 0) {
> throw new XmlException("account contains unknown
> shares or loans");
> }
> return memberStatement;
> }
>
> Is there a better way to be doing the binding? What about the parse
> method is not playing well with the GC?
>
> Jason
> Now serving Boiler Spirit with every purchase. Switch to the new Purdue
> debit card today. Show your pride and earn Scorecard Rewards on the side.
> Just sign for your purchases and score valuable rewards points you can use
> for travel, electronics or even cash.
>
>
>
> ***This is a transmission from Purdue Employees Federal Credit
> Union (PEFCU) and is intended solely for its authorized
> recipient(s), and may contain information that is confidential
> and or legally privileged. If you are not an addressee, or the
> employee or agent responsible for delivering it to an addressee,
> you are hereby notified that any use, dissemination,
> distribution, publication or copying of the information
> contained
> in this email is strictly prohibited. If you have received this
> transmission in error, please notify us by telephoning (765)
> 497-3328 or returning the email. You are then instructed to
> delete the information from your computer. Thank you for your
> cooperation.***
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
> For additional commands, e-mail: user-help@xmlbeans.apache.org
>
>
RE: java.lang.OutOfMemoryError: Java heap space
Posted by Jason Berk <jb...@purdueefcu.com>.
FYI: the xml is < 1MB is size and is different for each account.
Jason
-----Original Message-----
From: Jason Berk [mailto:jberk@purdueefcu.com]
Sent: Monday, September 14, 2009 4:16 PM
To: user@xmlbeans.apache.org
Subject: java.lang.OutOfMemoryError: Java heap space
I have an xml clob and account in a DB. I need to select the xml and
bind it with XML Beans. This works fine for the first 30,000 or so
accounts, but eventually I get "ava.lang.OutOfMemoryError: Java heap
space"
This is the line that is blowing up:
MemberStatementDocument doc = MemberStatementDocument.Factory.parse(xml,
validationOptions);
In this method to do the bind:
private MemberStatement bind(String xml) throws Exception {
ArrayList<XmlValidationError> validationErrors = new
ArrayList<XmlValidationError>();
XmlOptions validationOptions = new XmlOptions();
validationOptions.setErrorListener(validationErrors);
//validationOptions.setLoadLineNumbers();
//validationOptions.setLoadTrimTextBuffer();
//validationOptions.setUnsynchronized();
//validationOptions.setLoadUseXMLReader(
SAXParserFactory.newInstance().newSAXParser().getXMLReader() );
MemberStatementDocument doc =
MemberStatementDocument.Factory.parse(xml, validationOptions);
if (!doc.validate(validationOptions)) {
logger.error("Validation errors discovered
during binding:");
Iterator<XmlValidationError> iter =
validationErrors.iterator();
while (iter.hasNext()) {
logger.error(">> " + iter.next());
}
throw new XmlException("Validation errors
discovered during binding:");
}
MemberStatement memberStatement =
doc.getMemberStatement();
if (memberStatement.getUnknownShareList().size() > 0 ||
memberStatement.getUnknownLoanList().size() > 0) {
throw new XmlException("account contains unknown
shares or loans");
}
return memberStatement;
}
Is there a better way to be doing the binding? What about the parse
method is not playing well with the GC?
Jason
Now serving Boiler Spirit with every purchase. Switch to the new Purdue
debit card today. Show your pride and earn Scorecard Rewards on the
side. Just sign for your purchases and score valuable rewards points you
can use for travel, electronics or even cash.
***This is a transmission from Purdue Employees Federal Credit
Union (PEFCU) and is intended solely for its authorized
recipient(s), and may contain information that is confidential
and or legally privileged. If you are not an addressee, or the
employee or agent responsible for delivering it to an addressee,
you are hereby notified that any use, dissemination,
distribution, publication or copying of the information
contained
in this email is strictly prohibited. If you have received this
transmission in error, please notify us by telephoning (765)
497-3328 or returning the email. You are then instructed to
delete the information from your computer. Thank you for your
cooperation.***
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: user-help@xmlbeans.apache.org
Now serving Boiler Spirit with every purchase. Switch to the new Purdue debit card today. Show your pride and earn Scorecard Rewards on the side. Just sign for your purchases and score valuable rewards points you can use for travel, electronics or even cash.
***This is a transmission from Purdue Employees Federal Credit
Union (PEFCU) and is intended solely for its authorized
recipient(s), and may contain information that is confidential
and or legally privileged. If you are not an addressee, or the
employee or agent responsible for delivering it to an addressee,
you are hereby notified that any use, dissemination,
distribution, publication or copying of the information
contained
in this email is strictly prohibited. If you have received this
transmission in error, please notify us by telephoning (765)
497-3328 or returning the email. You are then instructed to
delete the information from your computer. Thank you for your
cooperation.***
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: user-help@xmlbeans.apache.org