You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openjpa.apache.org by Mark Struberg <st...@yahoo.de.INVALID> on 2021/04/01 07:49:58 UTC
AbstractLobTest - test streaming from LOB
hi!
While working on SQLServer tests I came across a clarification in JDBC4 that reading a LOB via InputStream _is_ possible, but you have to consume that stream before you get the next attribute. ANY further get on the ResultSet will close the open InputStream per spec.
This has an effect on our 'extended' non-portable feature
@Persistent
private InputStream stream;
This is basically not possible anymore per JDBC4 spec is my interpretation.
Because if we get a List<InputStreamLobEntity> with 10.000 entries, then we would have 10.000 InputStreams open. That's of course a bad idea, and I fully understand that JDBC4 clarification.
Should we get rid of those tests and feature?
It was only avail on 4 databases anyway.
Tests are
TestInputStreamLob and
TestReaderLob
Wdyt?
LieGrue,
strub
Re: AbstractLobTest - test streaming from LOB
Posted by Maxim Solodovnik <so...@gmail.com>.
+1 to follow the spec and remove the tests
On Thu, 1 Apr 2021 at 14:50, Mark Struberg <st...@yahoo.de.invalid> wrote:
>
> hi!
>
> While working on SQLServer tests I came across a clarification in JDBC4 that reading a LOB via InputStream _is_ possible, but you have to consume that stream before you get the next attribute. ANY further get on the ResultSet will close the open InputStream per spec.
>
> This has an effect on our 'extended' non-portable feature
>
> @Persistent
> private InputStream stream;
>
> This is basically not possible anymore per JDBC4 spec is my interpretation.
> Because if we get a List<InputStreamLobEntity> with 10.000 entries, then we would have 10.000 InputStreams open. That's of course a bad idea, and I fully understand that JDBC4 clarification.
>
> Should we get rid of those tests and feature?
> It was only avail on 4 databases anyway.
> Tests are
> TestInputStreamLob and
> TestReaderLob
>
> Wdyt?
>
> LieGrue,
> strub
>
>
--
Best regards,
Maxim
Re: AbstractLobTest - test streaming from LOB
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Where I agree we don't need to make it supported where it is not supported,
I strongly think we must not break it where it currently works, in
particular for the 3.1.3 so plan can be to explicit it in the doc it is
"best effort" and will likely not be supported anymore.
Since when we have a connection we can test the JDBC version it is related
to we can log a warning when JDBC driver is v4 too.
This kind of solves all related issues for me to have this setup and avoids
to implement anything fancy.
wdyt?
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>
Le jeu. 1 avr. 2021 à 10:05, Mark Struberg <st...@yahoo.de.invalid> a
écrit :
> I've just checked the JPA-2.2 spec. It is not supported by the spec. So no
> need to invest a lot for a feature which is only supported on 4 databases
> (with different grade of quality) and not portable across other JPA
> providers.
>
> What works: if you do a select of just that Inputstream, then it might
> work well. Need to try out. The problem is really that with any newer JDBC
> driver you will get the InputStream or Reader closed by the JDBC driver
> when a further get to the ResultSet is performed.
>
> LieGrue,
> strub
>
> > Am 01.04.2021 um 09:54 schrieb Romain Manni-Bucau <rmannibucau@gmail.com
> >:
> >
> > Hmm, there are multiple solutions to that:
> >
> > 1. read the stream in mem and use a bytearrayinputstream if it fits
> > 2. store a pointer (id) to the record and reopen it if needed at read
> time
> > 3. something in between (like tolerate to load in mem 1M and rest is done
> > with the pointer impl
> >
> >
> > Romain Manni-Bucau
> > @rmannibucau <https://twitter.com/rmannibucau> | Blog
> > <https://rmannibucau.metawerx.net/> | Old Blog
> > <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> >
> >
> > Le jeu. 1 avr. 2021 à 09:50, Mark Struberg <st...@yahoo.de.invalid> a
> > écrit :
> >
> >> hi!
> >>
> >> While working on SQLServer tests I came across a clarification in JDBC4
> >> that reading a LOB via InputStream _is_ possible, but you have to
> consume
> >> that stream before you get the next attribute. ANY further get on the
> >> ResultSet will close the open InputStream per spec.
> >>
> >> This has an effect on our 'extended' non-portable feature
> >>
> >> @Persistent
> >> private InputStream stream;
> >>
> >> This is basically not possible anymore per JDBC4 spec is my
> interpretation.
> >> Because if we get a List<InputStreamLobEntity> with 10.000 entries, then
> >> we would have 10.000 InputStreams open. That's of course a bad idea,
> and I
> >> fully understand that JDBC4 clarification.
> >>
> >> Should we get rid of those tests and feature?
> >> It was only avail on 4 databases anyway.
> >> Tests are
> >> TestInputStreamLob and
> >> TestReaderLob
> >>
> >> Wdyt?
> >>
> >> LieGrue,
> >> strub
> >>
> >>
> >>
>
>
Re: AbstractLobTest - test streaming from LOB
Posted by Mark Struberg <st...@yahoo.de.INVALID>.
I've just checked the JPA-2.2 spec. It is not supported by the spec. So no need to invest a lot for a feature which is only supported on 4 databases (with different grade of quality) and not portable across other JPA providers.
What works: if you do a select of just that Inputstream, then it might work well. Need to try out. The problem is really that with any newer JDBC driver you will get the InputStream or Reader closed by the JDBC driver when a further get to the ResultSet is performed.
LieGrue,
strub
> Am 01.04.2021 um 09:54 schrieb Romain Manni-Bucau <rm...@gmail.com>:
>
> Hmm, there are multiple solutions to that:
>
> 1. read the stream in mem and use a bytearrayinputstream if it fits
> 2. store a pointer (id) to the record and reopen it if needed at read time
> 3. something in between (like tolerate to load in mem 1M and rest is done
> with the pointer impl
>
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>
>
> Le jeu. 1 avr. 2021 à 09:50, Mark Struberg <st...@yahoo.de.invalid> a
> écrit :
>
>> hi!
>>
>> While working on SQLServer tests I came across a clarification in JDBC4
>> that reading a LOB via InputStream _is_ possible, but you have to consume
>> that stream before you get the next attribute. ANY further get on the
>> ResultSet will close the open InputStream per spec.
>>
>> This has an effect on our 'extended' non-portable feature
>>
>> @Persistent
>> private InputStream stream;
>>
>> This is basically not possible anymore per JDBC4 spec is my interpretation.
>> Because if we get a List<InputStreamLobEntity> with 10.000 entries, then
>> we would have 10.000 InputStreams open. That's of course a bad idea, and I
>> fully understand that JDBC4 clarification.
>>
>> Should we get rid of those tests and feature?
>> It was only avail on 4 databases anyway.
>> Tests are
>> TestInputStreamLob and
>> TestReaderLob
>>
>> Wdyt?
>>
>> LieGrue,
>> strub
>>
>>
>>
Re: AbstractLobTest - test streaming from LOB
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hmm, there are multiple solutions to that:
1. read the stream in mem and use a bytearrayinputstream if it fits
2. store a pointer (id) to the record and reopen it if needed at read time
3. something in between (like tolerate to load in mem 1M and rest is done
with the pointer impl
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>
Le jeu. 1 avr. 2021 à 09:50, Mark Struberg <st...@yahoo.de.invalid> a
écrit :
> hi!
>
> While working on SQLServer tests I came across a clarification in JDBC4
> that reading a LOB via InputStream _is_ possible, but you have to consume
> that stream before you get the next attribute. ANY further get on the
> ResultSet will close the open InputStream per spec.
>
> This has an effect on our 'extended' non-portable feature
>
> @Persistent
> private InputStream stream;
>
> This is basically not possible anymore per JDBC4 spec is my interpretation.
> Because if we get a List<InputStreamLobEntity> with 10.000 entries, then
> we would have 10.000 InputStreams open. That's of course a bad idea, and I
> fully understand that JDBC4 clarification.
>
> Should we get rid of those tests and feature?
> It was only avail on 4 databases anyway.
> Tests are
> TestInputStreamLob and
> TestReaderLob
>
> Wdyt?
>
> LieGrue,
> strub
>
>
>