You are viewing a plain text version of this content. The canonical link for it is here.
Posted to j-users@xalan.apache.org by Enric Jaen <en...@nmg.upc.es> on 2006/01/13 08:52:03 UTC

XPATH outOfMemory while evaluating large XML files.

Hello, I got an OutOfMemory when I evaluate an XPATH expression a large XML file.

I am using this code:

	XPathFactory factory = XPathFactory.newInstance();
        XPath xpath = factory.newXPath();
        InputSource entities_is=new InputSource("file.xml");
        XPathExpression xpathExpr = xpath.compile(expr);
        return (NodeList)xpathExpr.evaluate(entities_is, XPathConstants.NODESET);

I am not an expert of XPATH development, therefore I'd appreciate if someone could give me an explanation of why is this error happening.
Is this because XPATH uses DOM internaly? If so, is there any implementation for XPATH for SAX?
Is there any other explanation/solution?

Thanks in advance for your help.
-Enric

Re: XPATH outOfMemory while evaluating large XML files.

Posted by Christine Li <jy...@ca.ibm.com>.
Hi, Enric

If you think there is bugs in the impl of XPath, please open a bug report 
at https://issues.apache.org/jira/secure/Dashboard.jspa
and attach a valid test case that can demonstrate the problem.

Thanks,

Christine Li
XSLT Development
IBM Toronto Lab
Tel: (905)413-2601
Email: jycli@ca.ibm.com



Enric Jaen <en...@nmg.upc.es> 
13/01/2006 02:52 AM

To
xalan-j-users@xml.apache.org
cc

Subject
XPATH outOfMemory while evaluating large XML files.






Hello, I got an OutOfMemory when I evaluate an XPATH expression a large 
XML file.

I am using this code:

                 XPathFactory factory = XPathFactory.newInstance();
        XPath xpath = factory.newXPath();
        InputSource entities_is=new InputSource("file.xml");
        XPathExpression xpathExpr = xpath.compile(expr);
        return (NodeList)xpathExpr.evaluate(entities_is, 
XPathConstants.NODESET);

I am not an expert of XPATH development, therefore I'd appreciate if 
someone could give me an explanation of why is this error happening.
Is this because XPATH uses DOM internaly? If so, is there any 
implementation for XPATH for SAX?
Is there any other explanation/solution?

Thanks in advance for your help.
-Enric

Re: XPATH outOfMemory while evaluating large XML files.

Posted by Enric Jaen <en...@nmg.upc.es>.
>If you think there is bugs in the impl of XPath, please open a bug report
>at https://issues.apache.org/jira/secure/Dashboard.jspa
>and attach a valid test case that can demonstrate the problem.

I don't think is a bug. I rather think that XPATH builds a DOM tree when returns a NodeSet (please correct me if I am wrong). When the file is about 6MB the java memory crashes. Two workarounds I have tried are to increase heap and divide the xml file. Both solutions bring the evaluation limit farther, but there is still a limit.

I think it would be possible an XPATH implementation for SAX, such as Sequential XPATH, but I haven't gone deelply into this.

-Enric



On Fri, 13 Jan 2006, Enric Jaen wrote:

> Hello, I got an OutOfMemory when I evaluate an XPATH expression a large XML file.
>
> I am using this code:
>
> 	XPathFactory factory = XPathFactory.newInstance();
>         XPath xpath = factory.newXPath();
>         InputSource entities_is=new InputSource("file.xml");
>         XPathExpression xpathExpr = xpath.compile(expr);
>         return (NodeList)xpathExpr.evaluate(entities_is, XPathConstants.NODESET);
>
> I am not an expert of XPATH development, therefore I'd appreciate if someone could give me an explanation of why is this error happening.
> Is this because XPATH uses DOM internaly? If so, is there any implementation for XPATH for SAX?
> Is there any other explanation/solution?
>
> Thanks in advance for your help.
> -Enric
>
>
>
>
>