You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by "Hendrik Beck (camunda)" <he...@camunda.com> on 2007/09/12 15:22:29 UTC

[OCM] Binary values from byte[] field

Hi!

I am using JCR-OCM and have to store binaries in Jackrabbit. Therefore I
have a class BinaryDocument which has a property "byte[] data". The class is
being mapped to a node with node type "nt:resource" and the data property is
mapped to its field "jcr:property". Furthermore I have properties encoding,
lastModified and mimeType mapped to their counterparts in nt:resource.

Ok, I am accessing Jackrabbit via RMI which is the reason why I used byte[]
as data type for the actual binary data. What I do in my client is reading a
file (let's say a PDF) into a byte array, put that byte array into my
BinaryDocument object, send it over RMI to my stateless session bean, which
then persists it via Jackrabbit-OCM.

Now if I load this data again from my session bean and write it back into a
file I can't open it anymore. I assume there is some encoding / charset
mismatch somewhere on the way. As I only read and write byte arrays I
thought there's no operation in my client that actually changes the binary
stream in any way. Looking at the OCM sources I saw that
ByteArrayTypeConverterImpl creates the JCR value as follows:

  String value = new String(byte[]) propValue);
  valueFactory.createValue(value);

Is it possible that this operation is being done using the wrong charset for
my file?

Well, that's just a wild guess, I could honestly be totally wrong. I would
be thankful for any pointers. Are there people doing the same, like storing
binaries with OCM using byte[] for the jcr:data? Am I missing something?


Thanks in advance
Hendrik



RE: [OCM] Binary values from byte[] field

Posted by "Hendrik Beck (camunda)" <he...@camunda.com>.
Yeah, that works perfectly! ;-) Thanks a lot for fixing that immediately,
Christophe...


Regards
Hendrik


> -----Original Message-----
> From: Christophe Lombart [mailto:christophe.lombart@gmail.com]
> Sent: Thursday, September 13, 2007 4:48 AM
> To: users@jackrabbit.apache.org
> Subject: Re: [OCM] Binary values from byte[] field
> 
> I just commit a patch. Can you check if it is ok for you ?
> 
> Thanks,
> Christophe
> 
> 
> On 9/12/07, Christophe Lombart <ch...@gmail.com> wrote:
> >
> > huuuhh yes :-) we can modify the code like this. Probably a big mistake
> > when I write the code.
> >
> > On 9/12/07, Carsten Ziegeler < cziegeler@apache.org> wrote:
> > >
> > > Christophe Lombart wrote:
> > > > Hi Hendrik,
> > > >
> > > > I'm not 100 % sure that is related to a charset mismatch. This is
> not
> > > easy
> > > > to find the solution right now.
> > > > It seems that happens when you try open it after a second save right
> ?
> > > When
> > > > you open it after the first save, is it working ?
> > > > What do you mean by can't open it' ? Did you get an exception ?
> > > >
> > > >
> > > > Anyway, we should give the possibility to use  this constructor  :
> > > public
> > > > String(byte[] bytes, String charsetName)
> > > > But I'm just wondering how is possible to get the charset from this
> > > code
> > > > location (ByteArrayTypeConverterImpl).
> > > >
> > > I might be out of context, but wouldn't it be better to treat a binary
> > > values as a binary and not convert it into a String? Any conversion of
> > > arbitrary data to a String can lead to problems no matter what
> encoding
> > > you're using. So perhaps just using a byte array input stream would be
> > > better?
> > >
> > > Carsten
> > > --
> > > Carsten Ziegeler
> > > cziegeler@apache.org
> > >
> >
> >


Re: [OCM] Binary values from byte[] field

Posted by Christophe Lombart <ch...@gmail.com>.
I just commit a patch. Can you check if it is ok for you ?

Thanks,
Christophe


On 9/12/07, Christophe Lombart <ch...@gmail.com> wrote:
>
> huuuhh yes :-) we can modify the code like this. Probably a big mistake
> when I write the code.
>
> On 9/12/07, Carsten Ziegeler < cziegeler@apache.org> wrote:
> >
> > Christophe Lombart wrote:
> > > Hi Hendrik,
> > >
> > > I'm not 100 % sure that is related to a charset mismatch. This is not
> > easy
> > > to find the solution right now.
> > > It seems that happens when you try open it after a second save right ?
> > When
> > > you open it after the first save, is it working ?
> > > What do you mean by can't open it' ? Did you get an exception ?
> > >
> > >
> > > Anyway, we should give the possibility to use  this constructor  :
> > public
> > > String(byte[] bytes, String charsetName)
> > > But I'm just wondering how is possible to get the charset from this
> > code
> > > location (ByteArrayTypeConverterImpl).
> > >
> > I might be out of context, but wouldn't it be better to treat a binary
> > values as a binary and not convert it into a String? Any conversion of
> > arbitrary data to a String can lead to problems no matter what encoding
> > you're using. So perhaps just using a byte array input stream would be
> > better?
> >
> > Carsten
> > --
> > Carsten Ziegeler
> > cziegeler@apache.org
> >
>
>

Re: [OCM] Binary values from byte[] field

Posted by Christophe Lombart <ch...@gmail.com>.
huuuhh yes :-) we can modify the code like this. Probably a big mistake when
I write the code.

On 9/12/07, Carsten Ziegeler <cz...@apache.org> wrote:
>
> Christophe Lombart wrote:
> > Hi Hendrik,
> >
> > I'm not 100 % sure that is related to a charset mismatch. This is not
> easy
> > to find the solution right now.
> > It seems that happens when you try open it after a second save right ?
> When
> > you open it after the first save, is it working ?
> > What do you mean by can't open it' ? Did you get an exception ?
> >
> >
> > Anyway, we should give the possibility to use  this constructor  :
> public
> > String(byte[] bytes, String charsetName)
> > But I'm just wondering how is possible to get the charset from this code
> > location (ByteArrayTypeConverterImpl).
> >
> I might be out of context, but wouldn't it be better to treat a binary
> values as a binary and not convert it into a String? Any conversion of
> arbitrary data to a String can lead to problems no matter what encoding
> you're using. So perhaps just using a byte array input stream would be
> better?
>
> Carsten
> --
> Carsten Ziegeler
> cziegeler@apache.org
>

Re: [OCM] Binary values from byte[] field

Posted by Carsten Ziegeler <cz...@apache.org>.
Christophe Lombart wrote:
> Hi Hendrik,
> 
> I'm not 100 % sure that is related to a charset mismatch. This is not easy
> to find the solution right now.
> It seems that happens when you try open it after a second save right ? When
> you open it after the first save, is it working ?
> What do you mean by can't open it' ? Did you get an exception ?
> 
> 
> Anyway, we should give the possibility to use  this constructor  : public
> String(byte[] bytes, String charsetName)
> But I'm just wondering how is possible to get the charset from this code
> location (ByteArrayTypeConverterImpl).
> 
I might be out of context, but wouldn't it be better to treat a binary
values as a binary and not convert it into a String? Any conversion of
arbitrary data to a String can lead to problems no matter what encoding
you're using. So perhaps just using a byte array input stream would be
better?

Carsten
-- 
Carsten Ziegeler
cziegeler@apache.org

RE: [OCM] Binary values from byte[] field

Posted by "Hendrik Beck (camunda)" <he...@camunda.com>.
Hi Christope,

Thanks for your answer.

Just to clarify what exactly I am doing: I am retrieving that stored binary
value right after I stored it, both times from a remote client over RMI.
With "opening" it I meant that I just wrote a few lines that are writing the
byte array back into a file which I then try to open with an external
application (e.g. Acrobat Reader for PDFs).

I ended up in this approach after I wondered wether the document is really
stored correctly in JCR. I wanted to open it with my JCR-Explorer web
application, it didn't work. Also, full-text search using the TextExtractors
didn't work while it worked well with binary document stored in another way
(e.g. by adding it manually to the repository using JCR-Explorer). 

So, to answer your question: there is only one save() call involved, then
one after I persisted the data using the ObjectContentManager.

So at that point I am kind of stuck. I was also wondering wether it would be
possible to use the constructor you mentioned but also not sure how to
determine the right charset at that point. By the way, does this play
together with the "encoding" stored in nt:resource nodes? Or is that again
two different kinds of things?

Regards
Hendrik


> -----Original Message-----
> From: Christophe Lombart [mailto:christophe.lombart@gmail.com]
> Sent: Wednesday, September 12, 2007 8:46 PM
> To: users@jackrabbit.apache.org
> Subject: Re: [OCM] Binary values from byte[] field
> 
> Hi Hendrik,
> 
> I'm not 100 % sure that is related to a charset mismatch. This is not easy
> to find the solution right now.
> It seems that happens when you try open it after a second save right ?
> When
> you open it after the first save, is it working ?
> What do you mean by can't open it' ? Did you get an exception ?
> 
> 
> Anyway, we should give the possibility to use  this constructor  : public
> String(byte[] bytes, String charsetName)
> But I'm just wondering how is possible to get the charset from this code
> location (ByteArrayTypeConverterImpl).
> 
> 
> br
> Christophe
> 
> 
> On 9/12/07, Hendrik Beck (camunda) <he...@camunda.com> wrote:
> > Hi!
> >
> > I am using JCR-OCM and have to store binaries in Jackrabbit. Therefore I
> > have a class BinaryDocument which has a property "byte[] data". The
> class
> is
> > being mapped to a node with node type "nt:resource" and the data
> property
> is
> > mapped to its field "jcr:property". Furthermore I have properties
> encoding,
> > lastModified and mimeType mapped to their counterparts in nt:resource.
> >
> > Ok, I am accessing Jackrabbit via RMI which is the reason why I used
> byte[]
> > as data type for the actual binary data. What I do in my client is
> reading
> a
> > file (let's say a PDF) into a byte array, put that byte array into my
> > BinaryDocument object, send it over RMI to my stateless session bean,
> which
> > then persists it via Jackrabbit-OCM.
> >
> > Now if I load this data again from my session bean and write it back
> into
> a
> > file I can't open it anymore. I assume there is some encoding / charset
> > mismatch somewhere on the way. As I only read and write byte arrays I
> > thought there's no operation in my client that actually changes the
> binary
> > stream in any way. Looking at the OCM sources I saw that
> > ByteArrayTypeConverterImpl creates the JCR value as follows:
> >
> >   String value = new String(byte[]) propValue);
> >   valueFactory.createValue(value);
> >
> > Is it possible that this operation is being done using the wrong charset
> for
> > my file?
> >
> > Well, that's just a wild guess, I could honestly be totally wrong. I
> would
> > be thankful for any pointers. Are there people doing the same, like
> storing
> > binaries with OCM using byte[] for the jcr:data? Am I missing something?
> >
> >
> > Thanks in advance
> > Hendrik
> >
> >
> >


Re: [OCM] Binary values from byte[] field

Posted by Christophe Lombart <ch...@gmail.com>.
Hi Hendrik,

I'm not 100 % sure that is related to a charset mismatch. This is not easy
to find the solution right now.
It seems that happens when you try open it after a second save right ? When
you open it after the first save, is it working ?
What do you mean by can't open it' ? Did you get an exception ?


Anyway, we should give the possibility to use  this constructor  : public
String(byte[] bytes, String charsetName)
But I'm just wondering how is possible to get the charset from this code
location (ByteArrayTypeConverterImpl).


br
Christophe


On 9/12/07, Hendrik Beck (camunda) <he...@camunda.com> wrote:
> Hi!
>
> I am using JCR-OCM and have to store binaries in Jackrabbit. Therefore I
> have a class BinaryDocument which has a property "byte[] data". The class
is
> being mapped to a node with node type "nt:resource" and the data property
is
> mapped to its field "jcr:property". Furthermore I have properties
encoding,
> lastModified and mimeType mapped to their counterparts in nt:resource.
>
> Ok, I am accessing Jackrabbit via RMI which is the reason why I used
byte[]
> as data type for the actual binary data. What I do in my client is reading
a
> file (let's say a PDF) into a byte array, put that byte array into my
> BinaryDocument object, send it over RMI to my stateless session bean,
which
> then persists it via Jackrabbit-OCM.
>
> Now if I load this data again from my session bean and write it back into
a
> file I can't open it anymore. I assume there is some encoding / charset
> mismatch somewhere on the way. As I only read and write byte arrays I
> thought there's no operation in my client that actually changes the binary
> stream in any way. Looking at the OCM sources I saw that
> ByteArrayTypeConverterImpl creates the JCR value as follows:
>
>   String value = new String(byte[]) propValue);
>   valueFactory.createValue(value);
>
> Is it possible that this operation is being done using the wrong charset
for
> my file?
>
> Well, that's just a wild guess, I could honestly be totally wrong. I would
> be thankful for any pointers. Are there people doing the same, like
storing
> binaries with OCM using byte[] for the jcr:data? Am I missing something?
>
>
> Thanks in advance
> Hendrik
>
>
>

RE: [OCM] Binary values from byte[] field

Posted by "Hendrik Beck (camunda)" <he...@camunda.com>.
Hi,

I am just trying out that approach, it sounds promising to me. I would let
you know if I had any success as soon as my build is finished and I tested
it again, ok?

Regards and thanks again
Hendrik


> -----Original Message-----
> From: Felix Meschberger [mailto:fmeschbe@gmail.com]
> Sent: Wednesday, September 12, 2007 8:55 PM
> To: users@jackrabbit.apache.org
> Subject: Re: [OCM] Binary values from byte[] field
> 
> Hi,
> 
> Am Mittwoch, den 12.09.2007, 20:22 +0700 schrieb Hendrik Beck (camunda):
> >   String value = new String(byte[]) propValue);
> >   valueFactory.createValue(value);
> 
> Isn't this wrong anyway ? If we convert the binary data to String at
> all, we should use ISO-8859-1 as the charset to ensure identity
> conversion. But this seems somewhat strange as the data actually really
> is binary so, we should do
> 
>    InputStream ins = new ByteArrayInputStream((byte[]) propValue);
>    valueFactory.createValue(ins);
> 
> Likewise reading the value would consist of reading the value as a
> stream into a byte[] and return that array.
> 
> What do you think ?
> 
> Regards
> Felix
> 
> >
> > Is it possible that this operation is being done using the wrong charset
> for
> > my file?
> >
> > Well, that's just a wild guess, I could honestly be totally wrong. I
> would
> > be thankful for any pointers. Are there people doing the same, like
> storing
> > binaries with OCM using byte[] for the jcr:data? Am I missing something?
> >
> >
> > Thanks in advance
> > Hendrik
> >
> >



Re: [OCM] Binary values from byte[] field

Posted by Felix Meschberger <fm...@gmail.com>.
Hi,

Am Mittwoch, den 12.09.2007, 20:22 +0700 schrieb Hendrik Beck (camunda):
>   String value = new String(byte[]) propValue);
>   valueFactory.createValue(value);

Isn't this wrong anyway ? If we convert the binary data to String at
all, we should use ISO-8859-1 as the charset to ensure identity
conversion. But this seems somewhat strange as the data actually really
is binary so, we should do

   InputStream ins = new ByteArrayInputStream((byte[]) propValue);
   valueFactory.createValue(ins);

Likewise reading the value would consist of reading the value as a
stream into a byte[] and return that array.

What do you think ?

Regards
Felix

> 
> Is it possible that this operation is being done using the wrong charset for
> my file?
> 
> Well, that's just a wild guess, I could honestly be totally wrong. I would
> be thankful for any pointers. Are there people doing the same, like storing
> binaries with OCM using byte[] for the jcr:data? Am I missing something?
> 
> 
> Thanks in advance
> Hendrik
> 
>