You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by Daniel Hagen <dh...@h1-software.de> on 2006/04/18 19:54:25 UTC
SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Hi,
I noticed a problem during a test using the SimbpleDbPersistenceManager and
MySQL 5.0.
Strangely enough this problem occurred only after some nodes (~25000) where
created in the repository.
Now I receive the following error when creating a node:
ERROR org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager -
failed to write node state: ea7f3f8b-97fe-450d-8a8a-2b6d84c711c1
java.sql.SQLException: Data too long for column 'NODE_DATA' at row 1
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2921)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1570)
at
com.mysql.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement
.java:1085)
at
com.mysql.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStateme
nt.java:670)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1159)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1076)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1061)
at
org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager.store(SimpleD
bPersistenceManager.java:585)
at
org.apache.jackrabbit.core.state.AbstractPersistenceManager.store(AbstractPe
rsistenceManager.java:75)
at
org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager.store(SimpleD
bPersistenceManager.java:446)
at
org.apache.jackrabbit.core.state.SharedItemStateManager$Update.end(SharedIte
mStateManager.java:569)
at
org.apache.jackrabbit.core.state.SharedItemStateManager.update(SharedItemSta
teManager.java:692)
at
org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemState
Manager.java:315)
at
org.apache.jackrabbit.core.state.XAItemStateManager.update(XAItemStateManage
r.java:322)
at
org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemState
Manager.java:291)
at
org.apache.jackrabbit.core.state.SessionItemStateManager.update(SessionItemS
tateManager.java:257)
at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:1189)
at org.apache.jackrabbit.core.SessionImpl.save(SessionImpl.java:805)
Am I missing some MySQL Parameters I have to set?
The ddl I used to create the tables is
create table default_NODE (NODE_ID char(36) not null, NODE_DATA blob not
null) TYPE=InnoDB;
create unique index default_NODE_IDX on default_NODE (NODE_ID);
create table default_PROP (PROP_ID varchar(255) not null, PROP_DATA blob not
null) TYPE=InnoDB;
create unique index default_PROP_IDX on default_PROP (PROP_ID);
create table default_REFS (NODE_ID char(36) not null, REFS_DATA blob not
null) TYPE=InnoDB;
create unique index default_REFS_IDX on default_REFS (NODE_ID);
create table default_BINVAL (BINVAL_ID varchar(255) not null, BINVAL_DATA
longblob not null) TYPE=InnoDB;
create unique index default_BINVAL_IDX on default_BINVAL (BINVAL_ID);
Does anybody have a clue as to what is going wrong?
I will gladly provide more information (logs etc) if required.
Regards
Daniel
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Stefan Guggisberg <st...@gmail.com>.
hi florent
On 4/19/06, Florent Guillaume <fg...@nuxeo.com> wrote:
> Stefan Guggisberg wrote:
> > btw: you should in general try to avoid flat hierarchies in jackrabbit
> > as they come at a certain performance cost.
>
> Hi Stefan,
>
> Could you expand a bit on that? Aren't there any BTree-like structures used
> to store children on unordered nodes? Does this depend on the backend
> storage? Are any linear searches attempted in some places?
the NodeState object, jackrabbit's internal representation of a jcr Node,
maintains the collection of child node entries (name, subscript, uuid).
hash maps are used for accessing accessing an entry by name or uuid.
have a look at the NodeState and NodeState #ChildNodeEntries classes in the
o.a.j.core.state package. this should answer all your questions.
'fat' nodes take longer to load, copy and store. they obviously also use up
more memory and therefore may affect gc performance. manipulating a
large collection of same-name sibling entries (remove/insert) may take
more time than performing the same operation on a smaller collection.
cheers
stefan
>
> Thanks,
> Florent
>
> --
> Florent Guillaume, Nuxeo (Paris, France) Director of R&D
> +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
>
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Florent Guillaume <fg...@nuxeo.com>.
Stefan Guggisberg wrote:
> btw: you should in general try to avoid flat hierarchies in jackrabbit
> as they come at a certain performance cost.
Hi Stefan,
Could you expand a bit on that? Aren't there any BTree-like structures used
to store children on unordered nodes? Does this depend on the backend
storage? Are any linear searches attempted in some places?
Thanks,
Florent
--
Florent Guillaume, Nuxeo (Paris, France) Director of R&D
+33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Martin Perez <mp...@gmail.com>.
Alexandru,
I'm using Derby on my application and I had no problems uploading big files.
Martin
On 4/19/06, Alexandru Popescu <th...@gmail.com> wrote:
>
> Hi Stefan.
>
>
> I am wondering if this problem may occur with Derby too?
>
> tia,
>
> ./alex
> --
> .w( the_mindstorm )p.
>
> #: Stefan Guggisberg changed the world a bit at a time by saying (astral
> date: 4/19/2006 11:03 AM) :#
> > hi daniel
> > the default schema for mysql uses 'blob' columns for storing
> > serialized node states.
> > the blob data type has a maximum size limit of 65k. if your repository
> needs
> > to store large numbers of child nodes per node you can use the
> 'mediumblob'
> > data type instead.
> >
> > e.g.
> > create table default_NODE (NODE_ID char(36) not null, NODE_DATA
> > mediumblob not null);
> >
> > btw: you should in general try to avoid flat hierarchies in jackrabbit
> > as they come at
> > a certain performance cost.
> >
> > cheers
> > stefan
> >
> > On 4/18/06, Daniel Hagen <dh...@h1-software.de> wrote:
> >> Hi,
> >>
> >> I noticed a problem during a test using the SimbpleDbPersistenceManager
> and
> >> MySQL 5.0.
> >> Strangely enough this problem occurred only after some nodes (~25000)
> where
> >> created in the repository.
> >>
> >> Now I receive the following error when creating a node:
> >>
> >> ERROR org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager -
> >> failed to write node state: ea7f3f8b-97fe-450d-8a8a-2b6d84c711c1
> >> java.sql.SQLException: Data too long for column 'NODE_DATA' at row 1
> >> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2921)
> >> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1570)
> >> at
> >> com.mysql.jdbc.ServerPreparedStatement.serverExecute
> (ServerPreparedStatement
> >> .java:1085)
> >> at
> >> com.mysql.jdbc.ServerPreparedStatement.executeInternal
> (ServerPreparedStateme
> >> nt.java:670)
> >> at
> >> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java
> :1159)
> >> at
> >> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java
> :1076)
> >> at
> >> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java
> :1061)
> >> at
> >> org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager.store
> (SimpleD
> >> bPersistenceManager.java:585)
> >> at
> >> org.apache.jackrabbit.core.state.AbstractPersistenceManager.store
> (AbstractPe
> >> rsistenceManager.java:75)
> >> at
> >> org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager.store
> (SimpleD
> >> bPersistenceManager.java:446)
> >> at
> >> org.apache.jackrabbit.core.state.SharedItemStateManager$Update.end
> (SharedIte
> >> mStateManager.java:569)
> >> at
> >> org.apache.jackrabbit.core.state.SharedItemStateManager.update
> (SharedItemSta
> >> teManager.java:692)
> >> at
> >> org.apache.jackrabbit.core.state.LocalItemStateManager.update
> (LocalItemState
> >> Manager.java:315)
> >> at
> >> org.apache.jackrabbit.core.state.XAItemStateManager.update
> (XAItemStateManage
> >> r.java:322)
> >> at
> >> org.apache.jackrabbit.core.state.LocalItemStateManager.update
> (LocalItemState
> >> Manager.java:291)
> >> at
> >> org.apache.jackrabbit.core.state.SessionItemStateManager.update
> (SessionItemS
> >> tateManager.java:257)
> >> at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:1189)
> >> at org.apache.jackrabbit.core.SessionImpl.save(SessionImpl.java
> :805)
> >>
> >> Am I missing some MySQL Parameters I have to set?
> >>
> >> The ddl I used to create the tables is
> >>
> >> create table default_NODE (NODE_ID char(36) not null, NODE_DATA blob
> not
> >> null) TYPE=InnoDB;
> >> create unique index default_NODE_IDX on default_NODE (NODE_ID);
> >> create table default_PROP (PROP_ID varchar(255) not null, PROP_DATA
> blob not
> >> null) TYPE=InnoDB;
> >> create unique index default_PROP_IDX on default_PROP (PROP_ID);
> >> create table default_REFS (NODE_ID char(36) not null, REFS_DATA blob
> not
> >> null) TYPE=InnoDB;
> >> create unique index default_REFS_IDX on default_REFS (NODE_ID);
> >> create table default_BINVAL (BINVAL_ID varchar(255) not null,
> BINVAL_DATA
> >> longblob not null) TYPE=InnoDB;
> >> create unique index default_BINVAL_IDX on default_BINVAL (BINVAL_ID);
> >>
> >> Does anybody have a clue as to what is going wrong?
> >> I will gladly provide more information (logs etc) if required.
> >>
> >> Regards
> >>
> >> Daniel
> >>
> >>
> >
>
>
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Alexandru Popescu <th...@gmail.com>.
#: Brian Moseley changed the world a bit at a time by saying (astral date: 4/19/2006 7:07 PM) :#
> On 4/19/06, Alexandru Popescu <th...@gmail.com> wrote:
>
>> I am wondering if this problem may occur with Derby too?
>
> yes, it does. derby's default blob size is 1M, iirc.
>
I would really like to better understand what may go wrong here. Please stop me at what point I am
going wrong:
1/ The blob is used to persist a NodeState. A NodeState's heavy part is the ChildNodeEntries.
2/ ChildNodeEntries is mainly a double map of ChildNodeEntry
3/ a ChildNodeEntry is mainly NodeId.
4/ a NodeId is maily 2 long-s => 128bits
So, in order to meet the limitations of the DB schema, we should have a node that has as many
children as a serialized LinkedMap, HashMap of ChildNodeEntry will fill the length of the field.
Is this the correct computation?
./alex
--
.w( the_mindstorm )p.
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Martin Perez <mp...@gmail.com>.
Yes, maybe I'm wrong, but is there any difference on saving serialized node
states and file contents (externalBlobs disabled) on blob columns respect to
the allowed blob column size?
They should be handled on the same way, isn't it?
Please, correct me if I'm missing the point.
Martin
On 4/19/06, Brian Moseley <bc...@osafoundation.org> wrote:
>
> On 4/19/06, Martin Perez <mp...@gmail.com> wrote:
>
> > I have no problems uploading various megabyte files to derby without
> > configuring anything...
>
> that's not the issue. saving serialized node states with
> DerbyPersistenceManager is what we're talking about.
>
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Alexandru Popescu <th...@gmail.com>.
Thanks a lot Stefan. So, the main idea is to preserve the package and
to configure the schema parameter.
./alex
--
.w( the_mindstorm )p.
On 4/22/06, Stefan Guggisberg <st...@gmail.com> wrote:
> On 4/22/06, Alexandru Popescu <th...@gmail.com> wrote:
> > Hi!
> >
> > I am wondering how this customization of the DDL should be done... I
> > must go in the jar and modify the DDL in there? Or is it a way to
> > externalize it?
>
> you could e.g.
>
> - copy the derby.ddl, modify it and save it as e.g. derby_custom.ddl
> - put this .ddl in a jar file (package o.a.j.c.state.db) and add the
> this jar file
> to your classpath
> - make sure you configure your PersistenceManager to use the custom
> schema (i.e. <param name="schema" value="derby_custom"/>
>
> see
> http://jackrabbit.apache.org/apidocs/org/apache/jackrabbit/core/state/db/package-summary.html#package_description
> http://jackrabbit.apache.org/apidocs/org/apache/jackrabbit/core/state/db/DerbyPersistenceManager.html
>
> cheers
> stefan
> >
> > many thanks in advance,
> >
> > ./alex
> > --
> > .w( the_mindstorm )p.
> >
> >
> > On 4/20/06, Alexandru Popescu <th...@gmail.com> wrote:
> > > Hi Stefan and thanks for the explanations!
> > >
> > > There is no panic, but everybody should be able to understand this before going into production :-).
> > >
> > > I guess this is very good knowledge for everybody:
> > >
> > > [quote]
> > > 30k child nodes to one single node per normal blob entry. If you think that your structure will
> > > overpass this number (which is completely unrecommended) you can customize the blob size (at least
> > > with MySQL, Derby)
> > > [/quote]
> > >
> > >
> > > ./alex
> > > --
> > > .w( the_mindstorm )p.
> > >
> > >
> > > #: Stefan Guggisberg changed the world a bit at a time by saying (astral date: 4/20/2006 12:02 PM) :#
> > > > don't panic, folks! ;)
> > > >
> > > > the default schema for derby uses a 'blob' column for storing the
> > > > serialized node states. as brian pointed out correctly, the blob
> > > > column has a max. size of 1mb if no size has been specified.
> > > >
> > > > i ran a quick test: i was able to add ~37k child nodes before hitting
> > > > the 1mb size limit. i think that this is perfectly reasonable and
> > > > i guess that most repositories won't ever come near this limit.
> > > >
> > > > however, if you really think that you need to be able to add more
> > > > than ~30k child nodes to one single node you could easily change
> > > > your schema, e.g. by specifying 'blob(5M)' as the data type of
> > > > the NODE_DATA column..
> > > >
> > > > again: i would stronlgy discourage from this kind of 'flat' repository
> > > > model unless you have a very compelling reason. jcr's data model is
> > > > inherently hierarchical so i'd suggest to better make use of this powerfull
> > > > feature.
> > > >
> > > > cheers
> > > > stefan
> > > >
> > > >
> > > > On 4/19/06, Brian Moseley <bc...@osafoundation.org> wrote:
> > > >> On 4/19/06, Martin Perez <mp...@gmail.com> wrote:
> > > >>
> > > >> > I have no problems uploading various megabyte files to derby without
> > > >> > configuring anything...
> > > >>
> > > >> that's not the issue. saving serialized node states with
> > > >> DerbyPersistenceManager is what we're talking about.
> > > >>
> > > >
> > >
> > >
> >
>
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Stefan Guggisberg <st...@gmail.com>.
On 4/22/06, Alexandru Popescu <th...@gmail.com> wrote:
> Hi!
>
> I am wondering how this customization of the DDL should be done... I
> must go in the jar and modify the DDL in there? Or is it a way to
> externalize it?
you could e.g.
- copy the derby.ddl, modify it and save it as e.g. derby_custom.ddl
- put this .ddl in a jar file (package o.a.j.c.state.db) and add the
this jar file
to your classpath
- make sure you configure your PersistenceManager to use the custom
schema (i.e. <param name="schema" value="derby_custom"/>
see
http://jackrabbit.apache.org/apidocs/org/apache/jackrabbit/core/state/db/package-summary.html#package_description
http://jackrabbit.apache.org/apidocs/org/apache/jackrabbit/core/state/db/DerbyPersistenceManager.html
cheers
stefan
>
> many thanks in advance,
>
> ./alex
> --
> .w( the_mindstorm )p.
>
>
> On 4/20/06, Alexandru Popescu <th...@gmail.com> wrote:
> > Hi Stefan and thanks for the explanations!
> >
> > There is no panic, but everybody should be able to understand this before going into production :-).
> >
> > I guess this is very good knowledge for everybody:
> >
> > [quote]
> > 30k child nodes to one single node per normal blob entry. If you think that your structure will
> > overpass this number (which is completely unrecommended) you can customize the blob size (at least
> > with MySQL, Derby)
> > [/quote]
> >
> >
> > ./alex
> > --
> > .w( the_mindstorm )p.
> >
> >
> > #: Stefan Guggisberg changed the world a bit at a time by saying (astral date: 4/20/2006 12:02 PM) :#
> > > don't panic, folks! ;)
> > >
> > > the default schema for derby uses a 'blob' column for storing the
> > > serialized node states. as brian pointed out correctly, the blob
> > > column has a max. size of 1mb if no size has been specified.
> > >
> > > i ran a quick test: i was able to add ~37k child nodes before hitting
> > > the 1mb size limit. i think that this is perfectly reasonable and
> > > i guess that most repositories won't ever come near this limit.
> > >
> > > however, if you really think that you need to be able to add more
> > > than ~30k child nodes to one single node you could easily change
> > > your schema, e.g. by specifying 'blob(5M)' as the data type of
> > > the NODE_DATA column..
> > >
> > > again: i would stronlgy discourage from this kind of 'flat' repository
> > > model unless you have a very compelling reason. jcr's data model is
> > > inherently hierarchical so i'd suggest to better make use of this powerfull
> > > feature.
> > >
> > > cheers
> > > stefan
> > >
> > >
> > > On 4/19/06, Brian Moseley <bc...@osafoundation.org> wrote:
> > >> On 4/19/06, Martin Perez <mp...@gmail.com> wrote:
> > >>
> > >> > I have no problems uploading various megabyte files to derby without
> > >> > configuring anything...
> > >>
> > >> that's not the issue. saving serialized node states with
> > >> DerbyPersistenceManager is what we're talking about.
> > >>
> > >
> >
> >
>
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Alexandru Popescu <th...@gmail.com>.
Hi!
I am wondering how this customization of the DDL should be done... I
must go in the jar and modify the DDL in there? Or is it a way to
externalize it?
many thanks in advance,
./alex
--
.w( the_mindstorm )p.
On 4/20/06, Alexandru Popescu <th...@gmail.com> wrote:
> Hi Stefan and thanks for the explanations!
>
> There is no panic, but everybody should be able to understand this before going into production :-).
>
> I guess this is very good knowledge for everybody:
>
> [quote]
> 30k child nodes to one single node per normal blob entry. If you think that your structure will
> overpass this number (which is completely unrecommended) you can customize the blob size (at least
> with MySQL, Derby)
> [/quote]
>
>
> ./alex
> --
> .w( the_mindstorm )p.
>
>
> #: Stefan Guggisberg changed the world a bit at a time by saying (astral date: 4/20/2006 12:02 PM) :#
> > don't panic, folks! ;)
> >
> > the default schema for derby uses a 'blob' column for storing the
> > serialized node states. as brian pointed out correctly, the blob
> > column has a max. size of 1mb if no size has been specified.
> >
> > i ran a quick test: i was able to add ~37k child nodes before hitting
> > the 1mb size limit. i think that this is perfectly reasonable and
> > i guess that most repositories won't ever come near this limit.
> >
> > however, if you really think that you need to be able to add more
> > than ~30k child nodes to one single node you could easily change
> > your schema, e.g. by specifying 'blob(5M)' as the data type of
> > the NODE_DATA column..
> >
> > again: i would stronlgy discourage from this kind of 'flat' repository
> > model unless you have a very compelling reason. jcr's data model is
> > inherently hierarchical so i'd suggest to better make use of this powerfull
> > feature.
> >
> > cheers
> > stefan
> >
> >
> > On 4/19/06, Brian Moseley <bc...@osafoundation.org> wrote:
> >> On 4/19/06, Martin Perez <mp...@gmail.com> wrote:
> >>
> >> > I have no problems uploading various megabyte files to derby without
> >> > configuring anything...
> >>
> >> that's not the issue. saving serialized node states with
> >> DerbyPersistenceManager is what we're talking about.
> >>
> >
>
>
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Alexandru Popescu <th...@gmail.com>.
Hi Stefan and thanks for the explanations!
There is no panic, but everybody should be able to understand this before going into production :-).
I guess this is very good knowledge for everybody:
[quote]
30k child nodes to one single node per normal blob entry. If you think that your structure will
overpass this number (which is completely unrecommended) you can customize the blob size (at least
with MySQL, Derby)
[/quote]
./alex
--
.w( the_mindstorm )p.
#: Stefan Guggisberg changed the world a bit at a time by saying (astral date: 4/20/2006 12:02 PM) :#
> don't panic, folks! ;)
>
> the default schema for derby uses a 'blob' column for storing the
> serialized node states. as brian pointed out correctly, the blob
> column has a max. size of 1mb if no size has been specified.
>
> i ran a quick test: i was able to add ~37k child nodes before hitting
> the 1mb size limit. i think that this is perfectly reasonable and
> i guess that most repositories won't ever come near this limit.
>
> however, if you really think that you need to be able to add more
> than ~30k child nodes to one single node you could easily change
> your schema, e.g. by specifying 'blob(5M)' as the data type of
> the NODE_DATA column..
>
> again: i would stronlgy discourage from this kind of 'flat' repository
> model unless you have a very compelling reason. jcr's data model is
> inherently hierarchical so i'd suggest to better make use of this powerfull
> feature.
>
> cheers
> stefan
>
>
> On 4/19/06, Brian Moseley <bc...@osafoundation.org> wrote:
>> On 4/19/06, Martin Perez <mp...@gmail.com> wrote:
>>
>> > I have no problems uploading various megabyte files to derby without
>> > configuring anything...
>>
>> that's not the issue. saving serialized node states with
>> DerbyPersistenceManager is what we're talking about.
>>
>
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Stefan Guggisberg <st...@gmail.com>.
don't panic, folks! ;)
the default schema for derby uses a 'blob' column for storing the
serialized node states. as brian pointed out correctly, the blob
column has a max. size of 1mb if no size has been specified.
i ran a quick test: i was able to add ~37k child nodes before hitting
the 1mb size limit. i think that this is perfectly reasonable and
i guess that most repositories won't ever come near this limit.
however, if you really think that you need to be able to add more
than ~30k child nodes to one single node you could easily change
your schema, e.g. by specifying 'blob(5M)' as the data type of
the NODE_DATA column..
again: i would stronlgy discourage from this kind of 'flat' repository
model unless you have a very compelling reason. jcr's data model is
inherently hierarchical so i'd suggest to better make use of this powerfull
feature.
cheers
stefan
On 4/19/06, Brian Moseley <bc...@osafoundation.org> wrote:
> On 4/19/06, Martin Perez <mp...@gmail.com> wrote:
>
> > I have no problems uploading various megabyte files to derby without
> > configuring anything...
>
> that's not the issue. saving serialized node states with
> DerbyPersistenceManager is what we're talking about.
>
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Brian Moseley <bc...@osafoundation.org>.
On 4/19/06, Martin Perez <mp...@gmail.com> wrote:
> I have no problems uploading various megabyte files to derby without
> configuring anything...
that's not the issue. saving serialized node states with
DerbyPersistenceManager is what we're talking about.
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Martin Perez <mp...@gmail.com>.
mmm
I have no problems uploading various megabyte files to derby without
configuring anything...
On 4/19/06, Brian Moseley <bc...@osafoundation.org> wrote:
>
> On 4/19/06, Alexandru Popescu <th...@gmail.com> wrote:
>
> > I am wondering if this problem may occur with Derby too?
>
> yes, it does. derby's default blob size is 1M, iirc.
>
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Brian Moseley <bc...@osafoundation.org>.
On 4/19/06, Alexandru Popescu <th...@gmail.com> wrote:
> I am wondering if this problem may occur with Derby too?
yes, it does. derby's default blob size is 1M, iirc.
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Alexandru Popescu <th...@gmail.com>.
Hi Stefan.
I am wondering if this problem may occur with Derby too?
tia,
./alex
--
.w( the_mindstorm )p.
#: Stefan Guggisberg changed the world a bit at a time by saying (astral date: 4/19/2006 11:03 AM) :#
> hi daniel
> the default schema for mysql uses 'blob' columns for storing
> serialized node states.
> the blob data type has a maximum size limit of 65k. if your repository needs
> to store large numbers of child nodes per node you can use the 'mediumblob'
> data type instead.
>
> e.g.
> create table default_NODE (NODE_ID char(36) not null, NODE_DATA
> mediumblob not null);
>
> btw: you should in general try to avoid flat hierarchies in jackrabbit
> as they come at
> a certain performance cost.
>
> cheers
> stefan
>
> On 4/18/06, Daniel Hagen <dh...@h1-software.de> wrote:
>> Hi,
>>
>> I noticed a problem during a test using the SimbpleDbPersistenceManager and
>> MySQL 5.0.
>> Strangely enough this problem occurred only after some nodes (~25000) where
>> created in the repository.
>>
>> Now I receive the following error when creating a node:
>>
>> ERROR org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager -
>> failed to write node state: ea7f3f8b-97fe-450d-8a8a-2b6d84c711c1
>> java.sql.SQLException: Data too long for column 'NODE_DATA' at row 1
>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2921)
>> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1570)
>> at
>> com.mysql.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement
>> .java:1085)
>> at
>> com.mysql.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStateme
>> nt.java:670)
>> at
>> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1159)
>> at
>> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1076)
>> at
>> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1061)
>> at
>> org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager.store(SimpleD
>> bPersistenceManager.java:585)
>> at
>> org.apache.jackrabbit.core.state.AbstractPersistenceManager.store(AbstractPe
>> rsistenceManager.java:75)
>> at
>> org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager.store(SimpleD
>> bPersistenceManager.java:446)
>> at
>> org.apache.jackrabbit.core.state.SharedItemStateManager$Update.end(SharedIte
>> mStateManager.java:569)
>> at
>> org.apache.jackrabbit.core.state.SharedItemStateManager.update(SharedItemSta
>> teManager.java:692)
>> at
>> org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemState
>> Manager.java:315)
>> at
>> org.apache.jackrabbit.core.state.XAItemStateManager.update(XAItemStateManage
>> r.java:322)
>> at
>> org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemState
>> Manager.java:291)
>> at
>> org.apache.jackrabbit.core.state.SessionItemStateManager.update(SessionItemS
>> tateManager.java:257)
>> at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:1189)
>> at org.apache.jackrabbit.core.SessionImpl.save(SessionImpl.java:805)
>>
>> Am I missing some MySQL Parameters I have to set?
>>
>> The ddl I used to create the tables is
>>
>> create table default_NODE (NODE_ID char(36) not null, NODE_DATA blob not
>> null) TYPE=InnoDB;
>> create unique index default_NODE_IDX on default_NODE (NODE_ID);
>> create table default_PROP (PROP_ID varchar(255) not null, PROP_DATA blob not
>> null) TYPE=InnoDB;
>> create unique index default_PROP_IDX on default_PROP (PROP_ID);
>> create table default_REFS (NODE_ID char(36) not null, REFS_DATA blob not
>> null) TYPE=InnoDB;
>> create unique index default_REFS_IDX on default_REFS (NODE_ID);
>> create table default_BINVAL (BINVAL_ID varchar(255) not null, BINVAL_DATA
>> longblob not null) TYPE=InnoDB;
>> create unique index default_BINVAL_IDX on default_BINVAL (BINVAL_ID);
>>
>> Does anybody have a clue as to what is going wrong?
>> I will gladly provide more information (logs etc) if required.
>>
>> Regards
>>
>> Daniel
>>
>>
>
Re: SQLException "Data too long" using SimpleDbPersistenceManager/MySQL
Posted by Stefan Guggisberg <st...@gmail.com>.
hi daniel
the default schema for mysql uses 'blob' columns for storing
serialized node states.
the blob data type has a maximum size limit of 65k. if your repository needs
to store large numbers of child nodes per node you can use the 'mediumblob'
data type instead.
e.g.
create table default_NODE (NODE_ID char(36) not null, NODE_DATA
mediumblob not null);
btw: you should in general try to avoid flat hierarchies in jackrabbit
as they come at
a certain performance cost.
cheers
stefan
On 4/18/06, Daniel Hagen <dh...@h1-software.de> wrote:
> Hi,
>
> I noticed a problem during a test using the SimbpleDbPersistenceManager and
> MySQL 5.0.
> Strangely enough this problem occurred only after some nodes (~25000) where
> created in the repository.
>
> Now I receive the following error when creating a node:
>
> ERROR org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager -
> failed to write node state: ea7f3f8b-97fe-450d-8a8a-2b6d84c711c1
> java.sql.SQLException: Data too long for column 'NODE_DATA' at row 1
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2921)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1570)
> at
> com.mysql.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement
> .java:1085)
> at
> com.mysql.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStateme
> nt.java:670)
> at
> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1159)
> at
> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1076)
> at
> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1061)
> at
> org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager.store(SimpleD
> bPersistenceManager.java:585)
> at
> org.apache.jackrabbit.core.state.AbstractPersistenceManager.store(AbstractPe
> rsistenceManager.java:75)
> at
> org.apache.jackrabbit.core.state.db.SimpleDbPersistenceManager.store(SimpleD
> bPersistenceManager.java:446)
> at
> org.apache.jackrabbit.core.state.SharedItemStateManager$Update.end(SharedIte
> mStateManager.java:569)
> at
> org.apache.jackrabbit.core.state.SharedItemStateManager.update(SharedItemSta
> teManager.java:692)
> at
> org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemState
> Manager.java:315)
> at
> org.apache.jackrabbit.core.state.XAItemStateManager.update(XAItemStateManage
> r.java:322)
> at
> org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemState
> Manager.java:291)
> at
> org.apache.jackrabbit.core.state.SessionItemStateManager.update(SessionItemS
> tateManager.java:257)
> at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:1189)
> at org.apache.jackrabbit.core.SessionImpl.save(SessionImpl.java:805)
>
> Am I missing some MySQL Parameters I have to set?
>
> The ddl I used to create the tables is
>
> create table default_NODE (NODE_ID char(36) not null, NODE_DATA blob not
> null) TYPE=InnoDB;
> create unique index default_NODE_IDX on default_NODE (NODE_ID);
> create table default_PROP (PROP_ID varchar(255) not null, PROP_DATA blob not
> null) TYPE=InnoDB;
> create unique index default_PROP_IDX on default_PROP (PROP_ID);
> create table default_REFS (NODE_ID char(36) not null, REFS_DATA blob not
> null) TYPE=InnoDB;
> create unique index default_REFS_IDX on default_REFS (NODE_ID);
> create table default_BINVAL (BINVAL_ID varchar(255) not null, BINVAL_DATA
> longblob not null) TYPE=InnoDB;
> create unique index default_BINVAL_IDX on default_BINVAL (BINVAL_ID);
>
> Does anybody have a clue as to what is going wrong?
> I will gladly provide more information (logs etc) if required.
>
> Regards
>
> Daniel
>
>