You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chemistry.apache.org by Carl Osterwisch <co...@gmail.com> on 2015/03/25 14:51:53 UTC
Python CMISlib 0.5.1 with large documents
I'm getting a memory error when I call Folder.createDocument(), passing a
contentFile of 120MB. Smaller files are uploading without trouble.
Are there any tricks to stream the document so it doesn't need to consume
as much memory? Does the unreleased CMISlib 0.6 address this issue?
Thanks for any suggestions!
-Carl
File
"C:\Anaconda\lib\site-packages\cmislib-0.5.1-py2.7.egg\cmislib\model.py",
line 2756, in createDocument
contentEncoding)
File
"C:\Anaconda\lib\site-packages\cmislib-0.5.1-py2.7.egg\cmislib\model.py",
line 1185, in createDocument
result = self._cmisClient.post(postUrl.encode('utf-8'),
xmlDoc.toxml(encodin
g='utf-8'), ATOM_XML_ENTRY_TYPE)
File "C:\Anaconda\lib\xml\dom\minidom.py", line 46, in toxml
return self.toprettyxml("", "", encoding)
File "C:\Anaconda\lib\xml\dom\minidom.py", line 58, in toprettyxml
self.writexml(writer, "", indent, newl, encoding)
File "C:\Anaconda\lib\xml\dom\minidom.py", line 1749, in writexml
node.writexml(writer, indent, addindent, newl)
File "C:\Anaconda\lib\xml\dom\minidom.py", line 814, in writexml
node.writexml(writer, indent+addindent, addindent, newl)
File "C:\Anaconda\lib\xml\dom\minidom.py", line 814, in writexml
node.writexml(writer, indent+addindent, addindent, newl)
File "C:\Anaconda\lib\xml\dom\minidom.py", line 810, in writexml
self.childNodes[0].writexml(writer, '', '', '')
File "C:\Anaconda\lib\xml\dom\minidom.py", line 1038, in writexml
_write_data(writer, "%s%s%s" % (indent, self.data, newl))
File "C:\Anaconda\lib\xml\dom\minidom.py", line 298, in _write_data
writer.write(data)
File "C:\Anaconda\lib\codecs.py", line 351, in write
data, consumed = self.encode(object, self.errors)
MemoryError
Re: Python CMISlib 0.5.1 with large documents
Posted by Carl Osterwisch <co...@gmail.com>.
Thanks for the reply. I just added this as CMIS-904
<https://issues.apache.org/jira/browse/CMIS-904>.
On Wed, Mar 25, 2015 at 10:26 AM, Jeff Potts <je...@gmail.com> wrote:
> One thing that would help would be if the library implemented the
> appendContentStream method available in CMIS 1.1. Unfortunately, it is not
> in 0.6.
>
> Would you mind opening a Jira for this?
>
> Jeff
>
> On Wed, Mar 25, 2015 at 8:51 AM, Carl Osterwisch <co...@gmail.com>
> wrote:
>
> > I'm getting a memory error when I call Folder.createDocument(), passing a
> > contentFile of 120MB. Smaller files are uploading without trouble.
> >
> > Are there any tricks to stream the document so it doesn't need to consume
> > as much memory? Does the unreleased CMISlib 0.6 address this issue?
> >
> > Thanks for any suggestions!
> > -Carl
> >
> > File
> > "C:\Anaconda\lib\site-packages\cmislib-0.5.1-py2.7.egg\cmislib\model.py",
> > line 2756, in createDocument
> > contentEncoding)
> > File
> > "C:\Anaconda\lib\site-packages\cmislib-0.5.1-py2.7.egg\cmislib\model.py",
> > line 1185, in createDocument
> > result = self._cmisClient.post(postUrl.encode('utf-8'),
> > xmlDoc.toxml(encodin
> > g='utf-8'), ATOM_XML_ENTRY_TYPE)
> > File "C:\Anaconda\lib\xml\dom\minidom.py", line 46, in toxml
> > return self.toprettyxml("", "", encoding)
> > File "C:\Anaconda\lib\xml\dom\minidom.py", line 58, in toprettyxml
> > self.writexml(writer, "", indent, newl, encoding)
> > File "C:\Anaconda\lib\xml\dom\minidom.py", line 1749, in writexml
> > node.writexml(writer, indent, addindent, newl)
> > File "C:\Anaconda\lib\xml\dom\minidom.py", line 814, in writexml
> > node.writexml(writer, indent+addindent, addindent, newl)
> > File "C:\Anaconda\lib\xml\dom\minidom.py", line 814, in writexml
> > node.writexml(writer, indent+addindent, addindent, newl)
> > File "C:\Anaconda\lib\xml\dom\minidom.py", line 810, in writexml
> > self.childNodes[0].writexml(writer, '', '', '')
> > File "C:\Anaconda\lib\xml\dom\minidom.py", line 1038, in writexml
> > _write_data(writer, "%s%s%s" % (indent, self.data, newl))
> > File "C:\Anaconda\lib\xml\dom\minidom.py", line 298, in _write_data
> > writer.write(data)
> > File "C:\Anaconda\lib\codecs.py", line 351, in write
> > data, consumed = self.encode(object, self.errors)
> > MemoryError
> >
>
Re: Python CMISlib 0.5.1 with large documents
Posted by Jeff Potts <je...@gmail.com>.
One thing that would help would be if the library implemented the
appendContentStream method available in CMIS 1.1. Unfortunately, it is not
in 0.6.
Would you mind opening a Jira for this?
Jeff
On Wed, Mar 25, 2015 at 8:51 AM, Carl Osterwisch <co...@gmail.com> wrote:
> I'm getting a memory error when I call Folder.createDocument(), passing a
> contentFile of 120MB. Smaller files are uploading without trouble.
>
> Are there any tricks to stream the document so it doesn't need to consume
> as much memory? Does the unreleased CMISlib 0.6 address this issue?
>
> Thanks for any suggestions!
> -Carl
>
> File
> "C:\Anaconda\lib\site-packages\cmislib-0.5.1-py2.7.egg\cmislib\model.py",
> line 2756, in createDocument
> contentEncoding)
> File
> "C:\Anaconda\lib\site-packages\cmislib-0.5.1-py2.7.egg\cmislib\model.py",
> line 1185, in createDocument
> result = self._cmisClient.post(postUrl.encode('utf-8'),
> xmlDoc.toxml(encodin
> g='utf-8'), ATOM_XML_ENTRY_TYPE)
> File "C:\Anaconda\lib\xml\dom\minidom.py", line 46, in toxml
> return self.toprettyxml("", "", encoding)
> File "C:\Anaconda\lib\xml\dom\minidom.py", line 58, in toprettyxml
> self.writexml(writer, "", indent, newl, encoding)
> File "C:\Anaconda\lib\xml\dom\minidom.py", line 1749, in writexml
> node.writexml(writer, indent, addindent, newl)
> File "C:\Anaconda\lib\xml\dom\minidom.py", line 814, in writexml
> node.writexml(writer, indent+addindent, addindent, newl)
> File "C:\Anaconda\lib\xml\dom\minidom.py", line 814, in writexml
> node.writexml(writer, indent+addindent, addindent, newl)
> File "C:\Anaconda\lib\xml\dom\minidom.py", line 810, in writexml
> self.childNodes[0].writexml(writer, '', '', '')
> File "C:\Anaconda\lib\xml\dom\minidom.py", line 1038, in writexml
> _write_data(writer, "%s%s%s" % (indent, self.data, newl))
> File "C:\Anaconda\lib\xml\dom\minidom.py", line 298, in _write_data
> writer.write(data)
> File "C:\Anaconda\lib\codecs.py", line 351, in write
> data, consumed = self.encode(object, self.errors)
> MemoryError
>