Re: Any way to make XMLBeans output/flush XML to a file on the fly instead of building an entire object model in memory?

[Radu Preotiuc-Pietro <ra...@bea.com>]

No, what you are asking essentially is a paging mechanism.

In most very large documents scenarios, the following should work: Use
SAX or preferrably StAX (JDK 1.6's java.xml.stream) because with StAX
you can split the tree under a particular element in a separate
substream from which you can easily create an XMLBeans object model.
Process that subtree, then let go of the reference and move to the next
subtree.

Of course, with this approach you would need to handle any writing of
the new document to disk yourself, but this is probably the closest to
your ideal solution as you are going to get. Or, use XQuery.

Hope this helps,
Radu

On Wed, 2007-08-22 at 20:54 -0700, Wade Chandler wrote:
> The subject says it all basically. I have these data
> transfer services which use XML schemas. I have been
> using XMLBeans for a year or two with them. I have
> this one exchange using about 750MB or 1GB of RAM. I
> would like to be able to still use the provided
> interfaces for XMLBeans such as:
> 
> SomeTypeDocument doc = ...
> 
> SomeNewElement e = doc.addSomeNewElement();
> 
> ... so = e.addSomeOtherNewElement();
> 
> The only way I could see this working would be if we
> could tell a single element to hold a partial object
> model so it existed as an object long enough to add
> more data/objects to it until it was finished then it
> would be flushed out to disk and when it was cleaned
> up its partial model would go away. Basically only so
> many objects and RAM would actually be used as needed
> and data would be flushed to disk. If there is
> anything in XMLBeans to make something like this
> happen please let me know. If it just isn't designed
> in then no worries.
> 
> I would envision it working like:
> 
> * Create a new document object which will be flushed
> instead of completely held in memory. Basically a full
> DOM could not exist for such a thing.
> 
> * Add some top level element.
> 
> * Tell that element to retain itself as more data will
> be added to it.
> 
> * Other elements are added to it. They are set to not
> stay in memory or they are depending on whether they
> have other sub-elements. If not told to stick in the
> partial model their info is written to a buffer which
> is flushed when full. They are then just let go of
> when they are GCd.
> 
> Basically any top level elements would flush their
> opening tag and attributes when any sub-elements are
> added or when a method was called to make it happen.
> Then they could flush their end element tags when any
> siblings are added or told to do so. Hopefully it
> would let me validate a file against an internal
> schema (as they are held in the XMLBeans created JAR
> files now) as well...without having to link to some
> external resource. So, just wondering if anything
> exists in XMLBeans to make this happen or if anyone
> knows of any other projects which do such a thing.
> 
> Thanks in advance for any help or info,
> 
> Wade 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
> For additional commands, e-mail: user-help@xmlbeans.apache.org
> 

Notice:  This email message, together with any attachments, may contain information  of  BEA Systems,  Inc.,  its subsidiaries  and  affiliated entities,  that may be confidential,  proprietary,  copyrighted  and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: user-help@xmlbeans.apache.org

Re: Any way to make XMLBeans output/flush XML to a file on the fly instead of building an entire object model in memory?

[Wade Chandler <hw...@yahoo.com>]

Thanks Radu,

I'll look into it. I appreciate the answer.

Wade

--- Radu Preotiuc-Pietro <ra...@bea.com> wrote:

> No, what you are asking essentially is a paging
> mechanism.
> 
> In most very large documents scenarios, the
> following should work: Use
> SAX or preferrably StAX (JDK 1.6's java.xml.stream)
> because with StAX
> you can split the tree under a particular element in
> a separate
> substream from which you can easily create an
> XMLBeans object model.
> Process that subtree, then let go of the reference
> and move to the next
> subtree.
> 
> Of course, with this approach you would need to
> handle any writing of
> the new document to disk yourself, but this is
> probably the closest to
> your ideal solution as you are going to get. Or, use
> XQuery.
> 
> Hope this helps,
> Radu
> 
> On Wed, 2007-08-22 at 20:54 -0700, Wade Chandler
> wrote:
> > The subject says it all basically. I have these
> data
> > transfer services which use XML schemas. I have
> been
> > using XMLBeans for a year or two with them. I have
> > this one exchange using about 750MB or 1GB of RAM.
> I
> > would like to be able to still use the provided
> > interfaces for XMLBeans such as:
> > 
> > SomeTypeDocument doc = ...
> > 
> > SomeNewElement e = doc.addSomeNewElement();
> > 
> > ... so = e.addSomeOtherNewElement();
> > 
> > The only way I could see this working would be if
> we
> > could tell a single element to hold a partial
> object
> > model so it existed as an object long enough to
> add
> > more data/objects to it until it was finished then
> it
> > would be flushed out to disk and when it was
> cleaned
> > up its partial model would go away. Basically only
> so
> > many objects and RAM would actually be used as
> needed
> > and data would be flushed to disk. If there is
> > anything in XMLBeans to make something like this
> > happen please let me know. If it just isn't
> designed
> > in then no worries.
> > 
> > I would envision it working like:
> > 
> > * Create a new document object which will be
> flushed
> > instead of completely held in memory. Basically a
> full
> > DOM could not exist for such a thing.
> > 
> > * Add some top level element.
> > 
> > * Tell that element to retain itself as more data
> will
> > be added to it.
> > 
> > * Other elements are added to it. They are set to
> not
> > stay in memory or they are depending on whether
> they
> > have other sub-elements. If not told to stick in
> the
> > partial model their info is written to a buffer
> which
> > is flushed when full. They are then just let go of
> > when they are GCd.
> > 
> > Basically any top level elements would flush their
> > opening tag and attributes when any sub-elements
> are
> > added or when a method was called to make it
> happen.
> > Then they could flush their end element tags when
> any
> > siblings are added or told to do so. Hopefully it
> > would let me validate a file against an internal
> > schema (as they are held in the XMLBeans created
> JAR
> > files now) as well...without having to link to
> some
> > external resource. So, just wondering if anything
> > exists in XMLBeans to make this happen or if
> anyone
> > knows of any other projects which do such a thing.
> > 
> > Thanks in advance for any help or info,
> > 
> > Wade 
> > 
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> user-unsubscribe@xmlbeans.apache.org
> > For additional commands, e-mail:
> user-help@xmlbeans.apache.org
> > 
> 
> Notice:  This email message, together with any
> attachments, may contain information  of  BEA
> Systems,  Inc.,  its subsidiaries  and  affiliated
> entities,  that may be confidential,  proprietary, 
> copyrighted  and/or legally privileged, and is
> intended solely for the use of the individual or
> entity named in this message. If you are not the
> intended recipient, and have received this message
> in error, please immediately return this by email
> and then delete it.
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> user-unsubscribe@xmlbeans.apache.org
> For additional commands, e-mail:
> user-help@xmlbeans.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: user-help@xmlbeans.apache.org