You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Colvin Cowie <co...@gmail.com> on 2019/06/10 12:17:59 UTC
ContentStreamUpdateRequest no longer closes stream
Hello, I'm in the process of moving from Solr 6. to Solr 8.
We have a client application that streams CSV files to Solr using
ContentStreamUpdateRequest and then deletes the CSV file once the data is
indexed. That worked fine in Solr 6, but when using 8, the file is locked
and can't be deleted. (This is on Windows)
This seems to be because of the changes made in
https://issues.apache.org/jira/browse/SOLR-12142
--------------
*@Override public RequestWriter.ContentWriter getContentWriter(String
expectedType) { if (contentStreams == null || contentStreams.isEmpty()
|| contentStreams.size() > 1) return null; ContentStream stream =
contentStreams.get(0); return new RequestWriter.ContentWriter() {
@Override public void write(OutputStream os) throws IOException {
IOUtils.copy(stream.getStream(), os); } @Override public
String getContentType() { return stream.getContentType(); }
}; }*
--------------
As far as I know, IOUtils.copy will not close the stream.
Adding a close to it, is enough to "fix" it for me
* try { IOUtils.copy(innerStream, os); }
finally { IOUtils.closeQuietly(innerStream); }*
I've attached a simple test case. It passes with the change above and fails
without it.
So, is this a bug, or is there something I'm supposed to be doing elsewhere
to close the stream?
Thanks,
Colvin
Re: ContentStreamUpdateRequest no longer closes stream
Posted by Mikhail Khludnev <mk...@apache.org>.
FWIW, fixed in 8.2.
Thanks, Colvin!
On Wed, Jun 12, 2019 at 5:30 PM Colvin Cowie <co...@gmail.com>
wrote:
> I realize that attachments might not work on the mailing list, so here is
> the test case on Drive
>
> https://drive.google.com/file/d/0B7mypFpwbHptTE5nZE0weURFOExFSHphRFlUV0EyTElaOC0w/view?usp=sharing
>
> On Mon, 10 Jun 2019 at 13:17, Colvin Cowie <co...@gmail.com>
> wrote:
>
> > Hello, I'm in the process of moving from Solr 6. to Solr 8.
> > We have a client application that streams CSV files to Solr using
> > ContentStreamUpdateRequest and then deletes the CSV file once the data is
> > indexed. That worked fine in Solr 6, but when using 8, the file is locked
> > and can't be deleted. (This is on Windows)
> >
> > This seems to be because of the changes made in
> > https://issues.apache.org/jira/browse/SOLR-12142
> > --------------
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *@Override public RequestWriter.ContentWriter getContentWriter(String
> > expectedType) { if (contentStreams == null || contentStreams.isEmpty()
> > || contentStreams.size() > 1) return null; ContentStream stream =
> > contentStreams.get(0); return new RequestWriter.ContentWriter() {
> > @Override public void write(OutputStream os) throws IOException {
> > IOUtils.copy(stream.getStream(), os); } @Override public
> > String getContentType() { return stream.getContentType(); }
> > }; }*
> > --------------
> > As far as I know, IOUtils.copy will not close the stream.
> >
> > Adding a close to it, is enough to "fix" it for me
> >
> >
> >
> >
> > * try { IOUtils.copy(innerStream, os); }
> > finally { IOUtils.closeQuietly(innerStream); }*
> >
> > I've attached a simple test case. It passes with the change above and
> > fails without it.
> >
> > So, is this a bug, or is there something I'm supposed to be doing
> > elsewhere to close the stream?
> >
> > Thanks,
> > Colvin
> >
>
--
Sincerely yours
Mikhail Khludnev
Re: ContentStreamUpdateRequest no longer closes stream
Posted by Colvin Cowie <co...@gmail.com>.
I realize that attachments might not work on the mailing list, so here is
the test case on Drive
https://drive.google.com/file/d/0B7mypFpwbHptTE5nZE0weURFOExFSHphRFlUV0EyTElaOC0w/view?usp=sharing
On Mon, 10 Jun 2019 at 13:17, Colvin Cowie <co...@gmail.com>
wrote:
> Hello, I'm in the process of moving from Solr 6. to Solr 8.
> We have a client application that streams CSV files to Solr using
> ContentStreamUpdateRequest and then deletes the CSV file once the data is
> indexed. That worked fine in Solr 6, but when using 8, the file is locked
> and can't be deleted. (This is on Windows)
>
> This seems to be because of the changes made in
> https://issues.apache.org/jira/browse/SOLR-12142
> --------------
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *@Override public RequestWriter.ContentWriter getContentWriter(String
> expectedType) { if (contentStreams == null || contentStreams.isEmpty()
> || contentStreams.size() > 1) return null; ContentStream stream =
> contentStreams.get(0); return new RequestWriter.ContentWriter() {
> @Override public void write(OutputStream os) throws IOException {
> IOUtils.copy(stream.getStream(), os); } @Override public
> String getContentType() { return stream.getContentType(); }
> }; }*
> --------------
> As far as I know, IOUtils.copy will not close the stream.
>
> Adding a close to it, is enough to "fix" it for me
>
>
>
>
> * try { IOUtils.copy(innerStream, os); }
> finally { IOUtils.closeQuietly(innerStream); }*
>
> I've attached a simple test case. It passes with the change above and
> fails without it.
>
> So, is this a bug, or is there something I'm supposed to be doing
> elsewhere to close the stream?
>
> Thanks,
> Colvin
>