You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@shindig.apache.org by David Primmer <da...@gmail.com> on 2008/05/20 03:15:29 UTC
updating the source dataset for shindig
I have some updated data people, social graph, activities, that I'd
like to add to shindig but I'm wondering the best way to store it and
work with it. Anyone have a suggestion? I'd like to replace the
state-basicfriendlist.xml file with something that's easier to edit
and work with that could be checked into shindig. Some suggestions
from the summit were http://www.hsqldb.org/,
http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
being c-based.
I'm primarily interested in supporting the java rest api so hsqldb
sounds best but I'm sure the other implementations would like to use
the same source data and may want something with more runtime
availability. Maybe we could store the seed data in multiple formats
for seeding a few different small database engines but we should start
with one.
I'd like to avoid non-embeded stuff like MySql.
There's been a request not to have the database engine included in the
shindig jar. We'd have to design the maven rules to generate alternate
binaries. Here's the maven rule to add hsqldb:
<dependency>
<groupId>hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>1.8.0.7</version>
<scope>test</scope>
</dependency>
davep
Re: updating the source dataset for shindig
Posted by Santiago Gala <sa...@gmail.com>.
On Tue, May 20, 2008 at 3:14 PM, Vincent Siveton
<vi...@gmail.com> wrote:
> Hi Santiago,
>
> 2008/5/20, Santiago Gala <sa...@gmail.com>:
>> The page you posted starts with: NOTE: By downloading the software
>> from this page, you acknowledge that the software available from here
>> is licensed under the GPL. We advise that you review the GPL before
>> downloading.
>>
>> So I guess it is impossible to be used in something to be released
>> under the Apache License.
>
> I was aware about mysql license, I just gave this pointer for information.
> Not sure if Shindig could only use this artifact for testing, ie
> without release test artifacts, but definitely Shindig *can't* release
> any artifacts with GPL artifacts.
In this case you should have warned about it. Licensing issues are
important for a lot of people, specially when we are doing a reference
implementation of a standard specification, and it is supposed to be
used in plenty of different contexts. Dropping a link without further
comment cannot be considered information, but rather adding confusion
to the mix.
Regards
Santiago
>
>> I love hsqldb, and have used it some time ago. What I loved most of it
>> is that the DB format we used was essentially an ASCII log of SQL
>> commands, so that it was amazing for debugging. This formats gets
>> collapsed to a set of INSERTs on close/open, so it is not as bad as it
>> looks.
>>
>> I wonder if derby has something similar, I always missed this for
>> debugging DB related issues when we moved off hsqldb.
>
> FYI here are some DB informations:
> http://db.apache.org/ddlutils/databases/hsqldb.html
> http://db.apache.org/ddlutils/databases/derby.html
PS) Not sure what is the purpose of those links either.
>
> Cheers,
>
> Vincent
>
>> Regards
>>
>> Santiago
>>
>>
>> On Tue, May 20, 2008 at 1:32 PM, Vincent Siveton
>> <vi...@gmail.com> wrote:
>> > 2008/5/19, David Primmer <da...@gmail.com>:
>> >> I have some updated data people, social graph, activities, that I'd
>> >> like to add to shindig but I'm wondering the best way to store it and
>> >> work with it. Anyone have a suggestion? I'd like to replace the
>> >> state-basicfriendlist.xml file with something that's easier to edit
>> >> and work with that could be checked into shindig. Some suggestions
>> >> from the summit were http://www.hsqldb.org/,
>> >> http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
>> >> being c-based.
>> >>
>> >> I'm primarily interested in supporting the java rest api so hsqldb
>> >> sounds best but I'm sure the other implementations would like to use
>> >> the same source data and may want something with more runtime
>> >> availability. Maybe we could store the seed data in multiple formats
>> >> for seeding a few different small database engines but we should start
>> >> with one.
>> >>
>> >> I'd like to avoid non-embeded stuff like MySql.
>> >
>> > FYI MySql provides an embedded DB using the mysql connector MXJ:
>> > http://dev.mysql.com/downloads/connector/mxj
>> >
>> > Cheers,
>> >
>> > Vincent
>> >
>> >>
>> >> There's been a request not to have the database engine included in the
>> >> shindig jar. We'd have to design the maven rules to generate alternate
>> >> binaries. Here's the maven rule to add hsqldb:
>> >>
>> >> <dependency>
>> >> <groupId>hsqldb</groupId>
>> >> <artifactId>hsqldb</artifactId>
>> >> <version>1.8.0.7</version>
>> >> <scope>test</scope>
>> >> </dependency>
>> >>
>> >> davep
>> >>
>> >
>>
>
Re: updating the source dataset for shindig
Posted by Vincent Siveton <vi...@gmail.com>.
Hi Santiago,
2008/5/20, Santiago Gala <sa...@gmail.com>:
> The page you posted starts with: NOTE: By downloading the software
> from this page, you acknowledge that the software available from here
> is licensed under the GPL. We advise that you review the GPL before
> downloading.
>
> So I guess it is impossible to be used in something to be released
> under the Apache License.
I was aware about mysql license, I just gave this pointer for information.
Not sure if Shindig could only use this artifact for testing, ie
without release test artifacts, but definitely Shindig *can't* release
any artifacts with GPL artifacts.
> I love hsqldb, and have used it some time ago. What I loved most of it
> is that the DB format we used was essentially an ASCII log of SQL
> commands, so that it was amazing for debugging. This formats gets
> collapsed to a set of INSERTs on close/open, so it is not as bad as it
> looks.
>
> I wonder if derby has something similar, I always missed this for
> debugging DB related issues when we moved off hsqldb.
FYI here are some DB informations:
http://db.apache.org/ddlutils/databases/hsqldb.html
http://db.apache.org/ddlutils/databases/derby.html
Cheers,
Vincent
> Regards
>
> Santiago
>
>
> On Tue, May 20, 2008 at 1:32 PM, Vincent Siveton
> <vi...@gmail.com> wrote:
> > 2008/5/19, David Primmer <da...@gmail.com>:
> >> I have some updated data people, social graph, activities, that I'd
> >> like to add to shindig but I'm wondering the best way to store it and
> >> work with it. Anyone have a suggestion? I'd like to replace the
> >> state-basicfriendlist.xml file with something that's easier to edit
> >> and work with that could be checked into shindig. Some suggestions
> >> from the summit were http://www.hsqldb.org/,
> >> http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
> >> being c-based.
> >>
> >> I'm primarily interested in supporting the java rest api so hsqldb
> >> sounds best but I'm sure the other implementations would like to use
> >> the same source data and may want something with more runtime
> >> availability. Maybe we could store the seed data in multiple formats
> >> for seeding a few different small database engines but we should start
> >> with one.
> >>
> >> I'd like to avoid non-embeded stuff like MySql.
> >
> > FYI MySql provides an embedded DB using the mysql connector MXJ:
> > http://dev.mysql.com/downloads/connector/mxj
> >
> > Cheers,
> >
> > Vincent
> >
> >>
> >> There's been a request not to have the database engine included in the
> >> shindig jar. We'd have to design the maven rules to generate alternate
> >> binaries. Here's the maven rule to add hsqldb:
> >>
> >> <dependency>
> >> <groupId>hsqldb</groupId>
> >> <artifactId>hsqldb</artifactId>
> >> <version>1.8.0.7</version>
> >> <scope>test</scope>
> >> </dependency>
> >>
> >> davep
> >>
> >
>
Re: updating the source dataset for shindig
Posted by Santiago Gala <sa...@gmail.com>.
The page you posted starts with: NOTE: By downloading the software
from this page, you acknowledge that the software available from here
is licensed under the GPL. We advise that you review the GPL before
downloading.
So I guess it is impossible to be used in something to be released
under the Apache License.
I love hsqldb, and have used it some time ago. What I loved most of it
is that the DB format we used was essentially an ASCII log of SQL
commands, so that it was amazing for debugging. This formats gets
collapsed to a set of INSERTs on close/open, so it is not as bad as it
looks.
I wonder if derby has something similar, I always missed this for
debugging DB related issues when we moved off hsqldb.
Regards
Santiago
On Tue, May 20, 2008 at 1:32 PM, Vincent Siveton
<vi...@gmail.com> wrote:
> 2008/5/19, David Primmer <da...@gmail.com>:
>> I have some updated data people, social graph, activities, that I'd
>> like to add to shindig but I'm wondering the best way to store it and
>> work with it. Anyone have a suggestion? I'd like to replace the
>> state-basicfriendlist.xml file with something that's easier to edit
>> and work with that could be checked into shindig. Some suggestions
>> from the summit were http://www.hsqldb.org/,
>> http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
>> being c-based.
>>
>> I'm primarily interested in supporting the java rest api so hsqldb
>> sounds best but I'm sure the other implementations would like to use
>> the same source data and may want something with more runtime
>> availability. Maybe we could store the seed data in multiple formats
>> for seeding a few different small database engines but we should start
>> with one.
>>
>> I'd like to avoid non-embeded stuff like MySql.
>
> FYI MySql provides an embedded DB using the mysql connector MXJ:
> http://dev.mysql.com/downloads/connector/mxj
>
> Cheers,
>
> Vincent
>
>>
>> There's been a request not to have the database engine included in the
>> shindig jar. We'd have to design the maven rules to generate alternate
>> binaries. Here's the maven rule to add hsqldb:
>>
>> <dependency>
>> <groupId>hsqldb</groupId>
>> <artifactId>hsqldb</artifactId>
>> <version>1.8.0.7</version>
>> <scope>test</scope>
>> </dependency>
>>
>> davep
>>
>
Re: updating the source dataset for shindig
Posted by Vincent Siveton <vi...@gmail.com>.
2008/5/19, David Primmer <da...@gmail.com>:
> I have some updated data people, social graph, activities, that I'd
> like to add to shindig but I'm wondering the best way to store it and
> work with it. Anyone have a suggestion? I'd like to replace the
> state-basicfriendlist.xml file with something that's easier to edit
> and work with that could be checked into shindig. Some suggestions
> from the summit were http://www.hsqldb.org/,
> http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
> being c-based.
>
> I'm primarily interested in supporting the java rest api so hsqldb
> sounds best but I'm sure the other implementations would like to use
> the same source data and may want something with more runtime
> availability. Maybe we could store the seed data in multiple formats
> for seeding a few different small database engines but we should start
> with one.
>
> I'd like to avoid non-embeded stuff like MySql.
FYI MySql provides an embedded DB using the mysql connector MXJ:
http://dev.mysql.com/downloads/connector/mxj
Cheers,
Vincent
>
> There's been a request not to have the database engine included in the
> shindig jar. We'd have to design the maven rules to generate alternate
> binaries. Here's the maven rule to add hsqldb:
>
> <dependency>
> <groupId>hsqldb</groupId>
> <artifactId>hsqldb</artifactId>
> <version>1.8.0.7</version>
> <scope>test</scope>
> </dependency>
>
> davep
>
Re: updating the source dataset for shindig
Posted by Kevin Brown <et...@google.com>.
On Mon, May 19, 2008 at 6:15 PM, David Primmer <da...@gmail.com>
wrote:
> I have some updated data people, social graph, activities, that I'd
> like to add to shindig but I'm wondering the best way to store it and
> work with it. Anyone have a suggestion? I'd like to replace the
> state-basicfriendlist.xml file with something that's easier to edit
> and work with that could be checked into shindig. Some suggestions
> from the summit were http://www.hsqldb.org/,
> http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
> being c-based.
>
> I'm primarily interested in supporting the java rest api so hsqldb
> sounds best but I'm sure the other implementations would like to use
> the same source data and may want something with more runtime
> availability. Maybe we could store the seed data in multiple formats
> for seeding a few different small database engines but we should start
> with one.
>
> I'd like to avoid non-embeded stuff like MySql.
>
> There's been a request not to have the database engine included in the
> shindig jar. We'd have to design the maven rules to generate alternate
> binaries. Here's the maven rule to add hsqldb:
>
> <dependency>
> <groupId>hsqldb</groupId>
> <artifactId>hsqldb</artifactId>
> <version>1.8.0.7</version>
> <scope>test</scope>
> </dependency>
As long as the scope is "compile", the jar itself won't be "included" in the
shindig artifacts -- you'll just get build errors if you try to instantiate
a class that depends on them if the appropriate dependencies aren't in your
classpath.
>
>
> davep
>
Re: updating the source dataset for shindig
Posted by Kevin Brown <et...@google.com>.
For PHP, the logical thing is to use http://us3.php.net/pdo, and then run
whatever as a separate process.
That's pretty standard LAMP stuff. You won't have it available for the
tests, though, so a mocked pdo would be appropriate.
On Tue, May 20, 2008 at 5:43 PM, Louis Ryan <lr...@google.com> wrote:
> Chris,
>
> If we were to choose Derby on the Java impl to provide the canonical
> implementation in Shindig what do you think an appropriate platform would
> be
> for PHP (sqlite, postgres, mysql ...?) It would obviously be highly
> desirable for the canonical Derby data-dump format to be importable into
> the
> PHP complement. It seems like there isnt a practical choice that isnt GPL
> so
> would it be OK for this to be a 'recipe' for PHP folks as opposed to
> comitted stuff in the Shindig repo?
>
> -Louis
>
> On Tue, May 20, 2008 at 3:58 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>
> > Good point, all it needs is clean interfaces in the core shindig code,
> and
> > then anyone could provide an implementation. Thats more distributed. As
> long
> > as its easy for someone downloading for the first time to get up and
> > running. Thats what was so good about Jackrabbit (for me at least), I
> could
> > download and evaluate it with real data OOTB after a simple build.
> > Ian
> >
> >
> >
> >
> > On 20 May 2008, at 22:29, Vasudeva Nori wrote:
> >
> > On Tue, May 20, 2008 at 2:15 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> >>
> >>> I agree, IMHO we only want 1 implementation to maintain and modify, if
> >>> that
> >>> impl can support multiple DB targets with no extra effort, then thats
> >>> good,
> >>> but as a RI shindig should not have to be editing DDL or SQL or Code
> >>> targeted at 10 or more databases, that would generate a
> >>> testing/development
> >>> nightmare as Cassie points out. IMHO OOTB, it should just run against
> >>> Derby,
> >>> perhaps with a simple pre-loaded dataset, or perhaps with some simple
> >>> update
> >>> API/Tool (maybe created using GWT and standard components)
> >>>
> >>>
> >> But the trick is to figure out WHICH db shindig will bless.
> >> and what would be the criteria?
> >>
> >> IMO, Instead of precluding people from contributing their favorite
> >> backend container, Shindig could accommodate any backend impl by
> >> including such code as an additional - and optionally downloadable &
> >> buildable - module.
> >> Who maintains these? people contributed them will, or some interested
> >> parties will all.
> >>
> >> Those taking shindig to production, with no existing backend would
> >>> probably
> >>> want to take the db layer and replace it with something more scalable.
> >>>
> >>> Anyway, that is what I am targeting.
> >>> Ian
> >>>
> >>>
> >>>
> >>> On 20 May 2008, at 21:50, Cassie wrote:
> >>>
> >>> I definitely don't think this last bit is a good idea. If we have
> >>>> implementations for all of them then we have to support all of them.
> >>>> Adding a new field would require updating tons of different backends -
> >>>> most of which wouldn't be used in prod. Let's just pick one, all
> >>>> agree, and go with it. And, as long as the db is easy enough for all
> >>>> of our users to run, then we should just delete the current xml state
> >>>> file stuff. One demo impl is enough :)
> >>>>
> >>>
> >>>
> >>>
> >
>
Re: updating the source dataset for shindig
Posted by Louis Ryan <lr...@google.com>.
Chris,
If we were to choose Derby on the Java impl to provide the canonical
implementation in Shindig what do you think an appropriate platform would be
for PHP (sqlite, postgres, mysql ...?) It would obviously be highly
desirable for the canonical Derby data-dump format to be importable into the
PHP complement. It seems like there isnt a practical choice that isnt GPL so
would it be OK for this to be a 'recipe' for PHP folks as opposed to
comitted stuff in the Shindig repo?
-Louis
On Tue, May 20, 2008 at 3:58 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> Good point, all it needs is clean interfaces in the core shindig code, and
> then anyone could provide an implementation. Thats more distributed. As long
> as its easy for someone downloading for the first time to get up and
> running. Thats what was so good about Jackrabbit (for me at least), I could
> download and evaluate it with real data OOTB after a simple build.
> Ian
>
>
>
>
> On 20 May 2008, at 22:29, Vasudeva Nori wrote:
>
> On Tue, May 20, 2008 at 2:15 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>>
>>> I agree, IMHO we only want 1 implementation to maintain and modify, if
>>> that
>>> impl can support multiple DB targets with no extra effort, then thats
>>> good,
>>> but as a RI shindig should not have to be editing DDL or SQL or Code
>>> targeted at 10 or more databases, that would generate a
>>> testing/development
>>> nightmare as Cassie points out. IMHO OOTB, it should just run against
>>> Derby,
>>> perhaps with a simple pre-loaded dataset, or perhaps with some simple
>>> update
>>> API/Tool (maybe created using GWT and standard components)
>>>
>>>
>> But the trick is to figure out WHICH db shindig will bless.
>> and what would be the criteria?
>>
>> IMO, Instead of precluding people from contributing their favorite
>> backend container, Shindig could accommodate any backend impl by
>> including such code as an additional - and optionally downloadable &
>> buildable - module.
>> Who maintains these? people contributed them will, or some interested
>> parties will all.
>>
>> Those taking shindig to production, with no existing backend would
>>> probably
>>> want to take the db layer and replace it with something more scalable.
>>>
>>> Anyway, that is what I am targeting.
>>> Ian
>>>
>>>
>>>
>>> On 20 May 2008, at 21:50, Cassie wrote:
>>>
>>> I definitely don't think this last bit is a good idea. If we have
>>>> implementations for all of them then we have to support all of them.
>>>> Adding a new field would require updating tons of different backends -
>>>> most of which wouldn't be used in prod. Let's just pick one, all
>>>> agree, and go with it. And, as long as the db is easy enough for all
>>>> of our users to run, then we should just delete the current xml state
>>>> file stuff. One demo impl is enough :)
>>>>
>>>
>>>
>>>
>
Re: updating the source dataset for shindig
Posted by Ian Boston <ie...@tfd.co.uk>.
Good point, all it needs is clean interfaces in the core shindig
code, and then anyone could provide an implementation. Thats more
distributed. As long as its easy for someone downloading for the
first time to get up and running. Thats what was so good about
Jackrabbit (for me at least), I could download and evaluate it with
real data OOTB after a simple build.
Ian
On 20 May 2008, at 22:29, Vasudeva Nori wrote:
> On Tue, May 20, 2008 at 2:15 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>> I agree, IMHO we only want 1 implementation to maintain and
>> modify, if that
>> impl can support multiple DB targets with no extra effort, then
>> thats good,
>> but as a RI shindig should not have to be editing DDL or SQL or Code
>> targeted at 10 or more databases, that would generate a testing/
>> development
>> nightmare as Cassie points out. IMHO OOTB, it should just run
>> against Derby,
>> perhaps with a simple pre-loaded dataset, or perhaps with some
>> simple update
>> API/Tool (maybe created using GWT and standard components)
>>
>
> But the trick is to figure out WHICH db shindig will bless.
> and what would be the criteria?
>
> IMO, Instead of precluding people from contributing their favorite
> backend container, Shindig could accommodate any backend impl by
> including such code as an additional - and optionally downloadable &
> buildable - module.
> Who maintains these? people contributed them will, or some interested
> parties will all.
>
>> Those taking shindig to production, with no existing backend would
>> probably
>> want to take the db layer and replace it with something more
>> scalable.
>>
>> Anyway, that is what I am targeting.
>> Ian
>>
>>
>>
>> On 20 May 2008, at 21:50, Cassie wrote:
>>
>>> I definitely don't think this last bit is a good idea. If we have
>>> implementations for all of them then we have to support all of them.
>>> Adding a new field would require updating tons of different
>>> backends -
>>> most of which wouldn't be used in prod. Let's just pick one, all
>>> agree, and go with it. And, as long as the db is easy enough for all
>>> of our users to run, then we should just delete the current xml
>>> state
>>> file stuff. One demo impl is enough :)
>>
>>
Re: updating the source dataset for shindig
Posted by Chris Chabot <ch...@xs4all.nl>.
I think the main point we made a while back already, was to try to
keep any DB bits out of shindig.
I went and made a side project partuza (see http://www.chabotc.com/partuza/
for more info), with a slight fix in the way the security token
works, this should work perfectly on the java version as well, so
might make a good 'live, db driven' test case? That was (one of the)
intention anyhow.
-- Chris
On May 21, 2008, at 12:10 AM, Kevin Brown wrote:
> On Tue, May 20, 2008 at 2:29 PM, Vasudeva Nori <va...@gmail.com>
> wrote:
>
>> On Tue, May 20, 2008 at 2:15 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>>> I agree, IMHO we only want 1 implementation to maintain and
>>> modify, if
>> that
>>> impl can support multiple DB targets with no extra effort, then
>>> thats
>> good,
>>> but as a RI shindig should not have to be editing DDL or SQL or Code
>>> targeted at 10 or more databases, that would generate a
>> testing/development
>>> nightmare as Cassie points out. IMHO OOTB, it should just run
>>> against
>> Derby,
>>> perhaps with a simple pre-loaded dataset, or perhaps with some
>>> simple
>> update
>>> API/Tool (maybe created using GWT and standard components)
>>>
>>
>> But the trick is to figure out WHICH db shindig will bless.
>> and what would be the criteria?
>>
>> IMO, Instead of precluding people from contributing their favorite
>> backend container, Shindig could accommodate any backend impl by
>> including such code as an additional - and optionally downloadable &
>> buildable - module.
>> Who maintains these? people contributed them will, or some interested
>> parties will all.
>
>
> It doesn't work that way. All committers are responsible for any
> piece of
> code in the Shindig repository. What you're suggesting is exactly
> what we
> were discussing a few days ago, which is the Jakarta model. Shindig
> can
> maintain at most a small set of very versatile implementations, not
> dozens
> of variants maintained by as many people (none of whom are working
> on the
> main code).
>
>
>>
>>
>>> Those taking shindig to production, with no existing backend would
>> probably
>>> want to take the db layer and replace it with something more
>>> scalable.
>>>
>>> Anyway, that is what I am targeting.
>>> Ian
>>>
>>>
>>>
>>> On 20 May 2008, at 21:50, Cassie wrote:
>>>
>>>> I definitely don't think this last bit is a good idea. If we have
>>>> implementations for all of them then we have to support all of
>>>> them.
>>>> Adding a new field would require updating tons of different
>>>> backends -
>>>> most of which wouldn't be used in prod. Let's just pick one, all
>>>> agree, and go with it. And, as long as the db is easy enough for
>>>> all
>>>> of our users to run, then we should just delete the current xml
>>>> state
>>>> file stuff. One demo impl is enough :)
>>>
>>>
>>
Re: updating the source dataset for shindig
Posted by Kevin Brown <et...@google.com>.
On Tue, May 20, 2008 at 2:29 PM, Vasudeva Nori <va...@gmail.com> wrote:
> On Tue, May 20, 2008 at 2:15 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> > I agree, IMHO we only want 1 implementation to maintain and modify, if
> that
> > impl can support multiple DB targets with no extra effort, then thats
> good,
> > but as a RI shindig should not have to be editing DDL or SQL or Code
> > targeted at 10 or more databases, that would generate a
> testing/development
> > nightmare as Cassie points out. IMHO OOTB, it should just run against
> Derby,
> > perhaps with a simple pre-loaded dataset, or perhaps with some simple
> update
> > API/Tool (maybe created using GWT and standard components)
> >
>
> But the trick is to figure out WHICH db shindig will bless.
> and what would be the criteria?
>
> IMO, Instead of precluding people from contributing their favorite
> backend container, Shindig could accommodate any backend impl by
> including such code as an additional - and optionally downloadable &
> buildable - module.
> Who maintains these? people contributed them will, or some interested
> parties will all.
It doesn't work that way. All committers are responsible for any piece of
code in the Shindig repository. What you're suggesting is exactly what we
were discussing a few days ago, which is the Jakarta model. Shindig can
maintain at most a small set of very versatile implementations, not dozens
of variants maintained by as many people (none of whom are working on the
main code).
>
>
> > Those taking shindig to production, with no existing backend would
> probably
> > want to take the db layer and replace it with something more scalable.
> >
> > Anyway, that is what I am targeting.
> > Ian
> >
> >
> >
> > On 20 May 2008, at 21:50, Cassie wrote:
> >
> >> I definitely don't think this last bit is a good idea. If we have
> >> implementations for all of them then we have to support all of them.
> >> Adding a new field would require updating tons of different backends -
> >> most of which wouldn't be used in prod. Let's just pick one, all
> >> agree, and go with it. And, as long as the db is easy enough for all
> >> of our users to run, then we should just delete the current xml state
> >> file stuff. One demo impl is enough :)
> >
> >
>
Re: updating the source dataset for shindig
Posted by Vasudeva Nori <va...@gmail.com>.
On Tue, May 20, 2008 at 2:15 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> I agree, IMHO we only want 1 implementation to maintain and modify, if that
> impl can support multiple DB targets with no extra effort, then thats good,
> but as a RI shindig should not have to be editing DDL or SQL or Code
> targeted at 10 or more databases, that would generate a testing/development
> nightmare as Cassie points out. IMHO OOTB, it should just run against Derby,
> perhaps with a simple pre-loaded dataset, or perhaps with some simple update
> API/Tool (maybe created using GWT and standard components)
>
But the trick is to figure out WHICH db shindig will bless.
and what would be the criteria?
IMO, Instead of precluding people from contributing their favorite
backend container, Shindig could accommodate any backend impl by
including such code as an additional - and optionally downloadable &
buildable - module.
Who maintains these? people contributed them will, or some interested
parties will all.
> Those taking shindig to production, with no existing backend would probably
> want to take the db layer and replace it with something more scalable.
>
> Anyway, that is what I am targeting.
> Ian
>
>
>
> On 20 May 2008, at 21:50, Cassie wrote:
>
>> I definitely don't think this last bit is a good idea. If we have
>> implementations for all of them then we have to support all of them.
>> Adding a new field would require updating tons of different backends -
>> most of which wouldn't be used in prod. Let's just pick one, all
>> agree, and go with it. And, as long as the db is easy enough for all
>> of our users to run, then we should just delete the current xml state
>> file stuff. One demo impl is enough :)
>
>
Re: updating the source dataset for shindig
Posted by Ian Boston <ie...@tfd.co.uk>.
I agree, IMHO we only want 1 implementation to maintain and modify,
if that impl can support multiple DB targets with no extra effort,
then thats good, but as a RI shindig should not have to be editing
DDL or SQL or Code targeted at 10 or more databases, that would
generate a testing/development nightmare as Cassie points out. IMHO
OOTB, it should just run against Derby, perhaps with a simple pre-
loaded dataset, or perhaps with some simple update API/Tool (maybe
created using GWT and standard components)
Those taking shindig to production, with no existing backend would
probably want to take the db layer and replace it with something more
scalable.
Anyway, that is what I am targeting.
Ian
On 20 May 2008, at 21:50, Cassie wrote:
> I definitely don't think this last bit is a good idea. If we have
> implementations for all of them then we have to support all of them.
> Adding a new field would require updating tons of different backends -
> most of which wouldn't be used in prod. Let's just pick one, all
> agree, and go with it. And, as long as the db is easy enough for all
> of our users to run, then we should just delete the current xml state
> file stuff. One demo impl is enough :)
Re: updating the source dataset for shindig
Posted by Cassie <do...@apache.org>.
2008/5/20 Vasudeva Nori <va...@gmail.com>:
> 2008/5/20 Louis Ryan <lr...@google.com>:
>> I think the initial goal is to have a canonical data set that can be
>> checked into SVN and that is easy for people to write unit tests against. I
>> believe most of the databases mentioned here can be dumped with common tools
>> to either CSV or a sequence of DDL and DML, the latter probably being more
>> generally useful across a broad range of DB platforms including non-embedded
>> ones as it should be WORA.
>> I think for in-process testing during feature development I don't think
>> transaction isolation should be an issue. I don't think any sane person
>> would suggest using an in-memory Java DB for production. Given that we are
>> Apache Derby would get my vote on principal.
> in-memory Java DB is not that useless though.
> I have worked on two big installations using TimesTen in memory
> database to maintain data in memory (in lieu of the rather slow EJB
> caching mechanism) and TimesTen interfaced with Oracle to persist and
> distribute data. Not sure if Derby is ready for primetime production
> usage but it is going to get there soon.
>
> does anyone seriously use the samplecontainer code in production? It
> must really be just for guideline. So, I would vote to have all
> database flavors added to Shindig as samplecontainers - CouchDB,
I definitely don't think this last bit is a good idea. If we have
implementations for all of them then we have to support all of them.
Adding a new field would require updating tons of different backends -
most of which wouldn't be used in prod. Let's just pick one, all
agree, and go with it. And, as long as the db is easy enough for all
of our users to run, then we should just delete the current xml state
file stuff. One demo impl is enough :)
> hsqldb, mysqldb and Derby of course. At the very least, having all
> databases as samplecontainers will be useful for anyone interested to
> pick the example samplecontainer and use it.
>>
>> Ian I think ALOT of people would love to have the essence of your work move
>> into Shindig. I will have a look at what you've got and get back to you.
>>
>> 2008/5/20 Ian Boston <ie...@tfd.co.uk>:
>>
>>> IMHO, I would not use HSQLDB for anything that gets anywhere close to
>>> production. It is not fully transactional, for instance it will not hold a
>>> lock correctly after update. (doesn't support read committed).
>>>
>>> from http://hsqldb.sourceforge.net/web/hsqlFAQ.html#FAQ (at the bottom)
>>> "HSQLDB support only READ UNCOMMITTED isolation level when queries are made
>>> from your Java program."
>>>
>>> If you want embedded, use Derby, which is transactional and does support
>>> read committed. We have been running a Continuum instance or Derby for 2
>>> years now, building about 20 projects with only a few maintenance problems,
>>> so its good for small scale production.
>>>
>>> So if you are thinking about a DB backend..... but your original question
>>> was about populating the contents of the store with some additional
>>> information ?
>>>
>>> If its a DB backend, then a SQL script would make sense, its easy to write
>>> a SQL loader ?
>>>
>>> If its still to be an XML file, then the simple thing is to edit the XML
>>> file (provided there isn't too much data).
>>>
>>>
>>> As a general ping, I am still working on a DB implementation of the social
>>> services that will support Derby and most other DB's, source code is at
>>> https://source.sakaiproject.org/contrib//tfd/trunk/social-db/ only because
>>> its somewhere I have commit to. The license if not stated is Apache2. The
>>> unit tests is contains fills the DB with a dataset of people and all other
>>> parts of the datamodel. I am also thinking (but not doing anything yet)
>>> about implementing a JPA version of this.
>>> Ian
>>>
>>>
>>>
>>>
>>> On 20 May 2008, at 02:52, David Primmer wrote:
>>>
>>> From scanning the docs quickly, HSQLDB has a native storage format of
>>>> csv that can directly attached without import or export
>>>> http://www.hsqldb.org/doc/guide/ch06.html there are some caveats with
>>>> this format and there are quite a few ways to run the database (memory
>>>> only, in process, cached etc.) and I'd defer to someone with more
>>>> experience.
>>>>
>>>> davep
>>>>
>>>> On Mon, May 19, 2008 at 6:24 PM, Jun Yang (杨骏) <jy...@google.com> wrote:
>>>>
>>>>> On Mon, May 19, 2008 at 6:18 PM, Dan Bentley <db...@google.com>
>>>>> wrote:
>>>>>
>>>>> Maybe we could keep the data in some plain text format and then just
>>>>>> have
>>>>>> scripts that can weave it into the different formats as required? This
>>>>>> might be easiest for ensuring that there is only ever one primary source
>>>>>> and
>>>>>> different versions don't have to be maintained?
>>>>>>
>>>>>>
>>>>> Agreed. How about CSV (with first line containing column/property
>>>>> names)?
>>>>>
>>>>>
>>>>> -Dan
>>>>>>
>>>>>>
>>>>> Jun
>>>>>
>>>>>
>>>>> On Mon, May 19, 2008 at 9:15 PM, David Primmer <david.primmer@gmail.com
>>>>>> >
>>>>>> wrote:
>>>>>>
>>>>>> I have some updated data people, social graph, activities, that I'd
>>>>>>> like to add to shindig but I'm wondering the best way to store it and
>>>>>>> work with it. Anyone have a suggestion? I'd like to replace the
>>>>>>> state-basicfriendlist.xml file with something that's easier to edit
>>>>>>> and work with that could be checked into shindig. Some suggestions
>>>>>>> from the summit were http://www.hsqldb.org/,
>>>>>>> http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
>>>>>>> being c-based.
>>>>>>>
>>>>>>> I'm primarily interested in supporting the java rest api so hsqldb
>>>>>>> sounds best but I'm sure the other implementations would like to use
>>>>>>> the same source data and may want something with more runtime
>>>>>>> availability. Maybe we could store the seed data in multiple formats
>>>>>>> for seeding a few different small database engines but we should start
>>>>>>> with one.
>>>>>>>
>>>>>>> I'd like to avoid non-embeded stuff like MySql.
>>>>>>>
>>>>>>> There's been a request not to have the database engine included in the
>>>>>>> shindig jar. We'd have to design the maven rules to generate alternate
>>>>>>> binaries. Here's the maven rule to add hsqldb:
>>>>>>>
>>>>>>> <dependency>
>>>>>>> <groupId>hsqldb</groupId>
>>>>>>> <artifactId>hsqldb</artifactId>
>>>>>>> <version>1.8.0.7</version>
>>>>>>> <scope>test</scope>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> davep
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>
>
Re: updating the source dataset for shindig
Posted by Vasudeva Nori <va...@gmail.com>.
2008/5/20 Louis Ryan <lr...@google.com>:
> I think the initial goal is to have a canonical data set that can be
> checked into SVN and that is easy for people to write unit tests against. I
> believe most of the databases mentioned here can be dumped with common tools
> to either CSV or a sequence of DDL and DML, the latter probably being more
> generally useful across a broad range of DB platforms including non-embedded
> ones as it should be WORA.
> I think for in-process testing during feature development I don't think
> transaction isolation should be an issue. I don't think any sane person
> would suggest using an in-memory Java DB for production. Given that we are
> Apache Derby would get my vote on principal.
in-memory Java DB is not that useless though.
I have worked on two big installations using TimesTen in memory
database to maintain data in memory (in lieu of the rather slow EJB
caching mechanism) and TimesTen interfaced with Oracle to persist and
distribute data. Not sure if Derby is ready for primetime production
usage but it is going to get there soon.
does anyone seriously use the samplecontainer code in production? It
must really be just for guideline. So, I would vote to have all
database flavors added to Shindig as samplecontainers - CouchDB,
hsqldb, mysqldb and Derby of course. At the very least, having all
databases as samplecontainers will be useful for anyone interested to
pick the example samplecontainer and use it.
>
> Ian I think ALOT of people would love to have the essence of your work move
> into Shindig. I will have a look at what you've got and get back to you.
>
> 2008/5/20 Ian Boston <ie...@tfd.co.uk>:
>
>> IMHO, I would not use HSQLDB for anything that gets anywhere close to
>> production. It is not fully transactional, for instance it will not hold a
>> lock correctly after update. (doesn't support read committed).
>>
>> from http://hsqldb.sourceforge.net/web/hsqlFAQ.html#FAQ (at the bottom)
>> "HSQLDB support only READ UNCOMMITTED isolation level when queries are made
>> from your Java program."
>>
>> If you want embedded, use Derby, which is transactional and does support
>> read committed. We have been running a Continuum instance or Derby for 2
>> years now, building about 20 projects with only a few maintenance problems,
>> so its good for small scale production.
>>
>> So if you are thinking about a DB backend..... but your original question
>> was about populating the contents of the store with some additional
>> information ?
>>
>> If its a DB backend, then a SQL script would make sense, its easy to write
>> a SQL loader ?
>>
>> If its still to be an XML file, then the simple thing is to edit the XML
>> file (provided there isn't too much data).
>>
>>
>> As a general ping, I am still working on a DB implementation of the social
>> services that will support Derby and most other DB's, source code is at
>> https://source.sakaiproject.org/contrib//tfd/trunk/social-db/ only because
>> its somewhere I have commit to. The license if not stated is Apache2. The
>> unit tests is contains fills the DB with a dataset of people and all other
>> parts of the datamodel. I am also thinking (but not doing anything yet)
>> about implementing a JPA version of this.
>> Ian
>>
>>
>>
>>
>> On 20 May 2008, at 02:52, David Primmer wrote:
>>
>> From scanning the docs quickly, HSQLDB has a native storage format of
>>> csv that can directly attached without import or export
>>> http://www.hsqldb.org/doc/guide/ch06.html there are some caveats with
>>> this format and there are quite a few ways to run the database (memory
>>> only, in process, cached etc.) and I'd defer to someone with more
>>> experience.
>>>
>>> davep
>>>
>>> On Mon, May 19, 2008 at 6:24 PM, Jun Yang (杨骏) <jy...@google.com> wrote:
>>>
>>>> On Mon, May 19, 2008 at 6:18 PM, Dan Bentley <db...@google.com>
>>>> wrote:
>>>>
>>>> Maybe we could keep the data in some plain text format and then just
>>>>> have
>>>>> scripts that can weave it into the different formats as required? This
>>>>> might be easiest for ensuring that there is only ever one primary source
>>>>> and
>>>>> different versions don't have to be maintained?
>>>>>
>>>>>
>>>> Agreed. How about CSV (with first line containing column/property
>>>> names)?
>>>>
>>>>
>>>> -Dan
>>>>>
>>>>>
>>>> Jun
>>>>
>>>>
>>>> On Mon, May 19, 2008 at 9:15 PM, David Primmer <david.primmer@gmail.com
>>>>> >
>>>>> wrote:
>>>>>
>>>>> I have some updated data people, social graph, activities, that I'd
>>>>>> like to add to shindig but I'm wondering the best way to store it and
>>>>>> work with it. Anyone have a suggestion? I'd like to replace the
>>>>>> state-basicfriendlist.xml file with something that's easier to edit
>>>>>> and work with that could be checked into shindig. Some suggestions
>>>>>> from the summit were http://www.hsqldb.org/,
>>>>>> http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
>>>>>> being c-based.
>>>>>>
>>>>>> I'm primarily interested in supporting the java rest api so hsqldb
>>>>>> sounds best but I'm sure the other implementations would like to use
>>>>>> the same source data and may want something with more runtime
>>>>>> availability. Maybe we could store the seed data in multiple formats
>>>>>> for seeding a few different small database engines but we should start
>>>>>> with one.
>>>>>>
>>>>>> I'd like to avoid non-embeded stuff like MySql.
>>>>>>
>>>>>> There's been a request not to have the database engine included in the
>>>>>> shindig jar. We'd have to design the maven rules to generate alternate
>>>>>> binaries. Here's the maven rule to add hsqldb:
>>>>>>
>>>>>> <dependency>
>>>>>> <groupId>hsqldb</groupId>
>>>>>> <artifactId>hsqldb</artifactId>
>>>>>> <version>1.8.0.7</version>
>>>>>> <scope>test</scope>
>>>>>> </dependency>
>>>>>>
>>>>>> davep
>>>>>>
>>>>>>
>>>>>
>>>>
>>
>
Re: updating the source dataset for shindig
Posted by Louis Ryan <lr...@google.com>.
I think the initial goal is to have a canonical data set that can be
checked into SVN and that is easy for people to write unit tests against. I
believe most of the databases mentioned here can be dumped with common tools
to either CSV or a sequence of DDL and DML, the latter probably being more
generally useful across a broad range of DB platforms including non-embedded
ones as it should be WORA.
I think for in-process testing during feature development I don't think
transaction isolation should be an issue. I don't think any sane person
would suggest using an in-memory Java DB for production. Given that we are
Apache Derby would get my vote on principal.
Ian I think ALOT of people would love to have the essence of your work move
into Shindig. I will have a look at what you've got and get back to you.
2008/5/20 Ian Boston <ie...@tfd.co.uk>:
> IMHO, I would not use HSQLDB for anything that gets anywhere close to
> production. It is not fully transactional, for instance it will not hold a
> lock correctly after update. (doesn't support read committed).
>
> from http://hsqldb.sourceforge.net/web/hsqlFAQ.html#FAQ (at the bottom)
> "HSQLDB support only READ UNCOMMITTED isolation level when queries are made
> from your Java program."
>
> If you want embedded, use Derby, which is transactional and does support
> read committed. We have been running a Continuum instance or Derby for 2
> years now, building about 20 projects with only a few maintenance problems,
> so its good for small scale production.
>
> So if you are thinking about a DB backend..... but your original question
> was about populating the contents of the store with some additional
> information ?
>
> If its a DB backend, then a SQL script would make sense, its easy to write
> a SQL loader ?
>
> If its still to be an XML file, then the simple thing is to edit the XML
> file (provided there isn't too much data).
>
>
> As a general ping, I am still working on a DB implementation of the social
> services that will support Derby and most other DB's, source code is at
> https://source.sakaiproject.org/contrib//tfd/trunk/social-db/ only because
> its somewhere I have commit to. The license if not stated is Apache2. The
> unit tests is contains fills the DB with a dataset of people and all other
> parts of the datamodel. I am also thinking (but not doing anything yet)
> about implementing a JPA version of this.
> Ian
>
>
>
>
> On 20 May 2008, at 02:52, David Primmer wrote:
>
> From scanning the docs quickly, HSQLDB has a native storage format of
>> csv that can directly attached without import or export
>> http://www.hsqldb.org/doc/guide/ch06.html there are some caveats with
>> this format and there are quite a few ways to run the database (memory
>> only, in process, cached etc.) and I'd defer to someone with more
>> experience.
>>
>> davep
>>
>> On Mon, May 19, 2008 at 6:24 PM, Jun Yang (杨骏) <jy...@google.com> wrote:
>>
>>> On Mon, May 19, 2008 at 6:18 PM, Dan Bentley <db...@google.com>
>>> wrote:
>>>
>>> Maybe we could keep the data in some plain text format and then just
>>>> have
>>>> scripts that can weave it into the different formats as required? This
>>>> might be easiest for ensuring that there is only ever one primary source
>>>> and
>>>> different versions don't have to be maintained?
>>>>
>>>>
>>> Agreed. How about CSV (with first line containing column/property
>>> names)?
>>>
>>>
>>> -Dan
>>>>
>>>>
>>> Jun
>>>
>>>
>>> On Mon, May 19, 2008 at 9:15 PM, David Primmer <david.primmer@gmail.com
>>>> >
>>>> wrote:
>>>>
>>>> I have some updated data people, social graph, activities, that I'd
>>>>> like to add to shindig but I'm wondering the best way to store it and
>>>>> work with it. Anyone have a suggestion? I'd like to replace the
>>>>> state-basicfriendlist.xml file with something that's easier to edit
>>>>> and work with that could be checked into shindig. Some suggestions
>>>>> from the summit were http://www.hsqldb.org/,
>>>>> http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
>>>>> being c-based.
>>>>>
>>>>> I'm primarily interested in supporting the java rest api so hsqldb
>>>>> sounds best but I'm sure the other implementations would like to use
>>>>> the same source data and may want something with more runtime
>>>>> availability. Maybe we could store the seed data in multiple formats
>>>>> for seeding a few different small database engines but we should start
>>>>> with one.
>>>>>
>>>>> I'd like to avoid non-embeded stuff like MySql.
>>>>>
>>>>> There's been a request not to have the database engine included in the
>>>>> shindig jar. We'd have to design the maven rules to generate alternate
>>>>> binaries. Here's the maven rule to add hsqldb:
>>>>>
>>>>> <dependency>
>>>>> <groupId>hsqldb</groupId>
>>>>> <artifactId>hsqldb</artifactId>
>>>>> <version>1.8.0.7</version>
>>>>> <scope>test</scope>
>>>>> </dependency>
>>>>>
>>>>> davep
>>>>>
>>>>>
>>>>
>>>
>
Re: updating the source dataset for shindig
Posted by Ian Boston <ie...@tfd.co.uk>.
IMHO, I would not use HSQLDB for anything that gets anywhere close to
production. It is not fully transactional, for instance it will not
hold a lock correctly after update. (doesn't support read committed).
from http://hsqldb.sourceforge.net/web/hsqlFAQ.html#FAQ (at the bottom)
"HSQLDB support only READ UNCOMMITTED isolation level when queries
are made from your Java program."
If you want embedded, use Derby, which is transactional and does
support read committed. We have been running a Continuum instance or
Derby for 2 years now, building about 20 projects with only a few
maintenance problems, so its good for small scale production.
So if you are thinking about a DB backend..... but your original
question was about populating the contents of the store with some
additional information ?
If its a DB backend, then a SQL script would make sense, its easy to
write a SQL loader ?
If its still to be an XML file, then the simple thing is to edit the
XML file (provided there isn't too much data).
As a general ping, I am still working on a DB implementation of the
social services that will support Derby and most other DB's, source
code is at https://source.sakaiproject.org/contrib//tfd/trunk/social-
db/ only because its somewhere I have commit to. The license if not
stated is Apache2. The unit tests is contains fills the DB with a
dataset of people and all other parts of the datamodel. I am also
thinking (but not doing anything yet) about implementing a JPA
version of this.
Ian
On 20 May 2008, at 02:52, David Primmer wrote:
> From scanning the docs quickly, HSQLDB has a native storage format of
> csv that can directly attached without import or export
> http://www.hsqldb.org/doc/guide/ch06.html there are some caveats with
> this format and there are quite a few ways to run the database (memory
> only, in process, cached etc.) and I'd defer to someone with more
> experience.
>
> davep
>
> On Mon, May 19, 2008 at 6:24 PM, Jun Yang (杨骏) <jy...@google.com>
> wrote:
>> On Mon, May 19, 2008 at 6:18 PM, Dan Bentley <db...@google.com>
>> wrote:
>>
>>> Maybe we could keep the data in some plain text format and then
>>> just have
>>> scripts that can weave it into the different formats as
>>> required? This
>>> might be easiest for ensuring that there is only ever one primary
>>> source
>>> and
>>> different versions don't have to be maintained?
>>>
>>
>> Agreed. How about CSV (with first line containing column/property
>> names)?
>>
>>
>>> -Dan
>>>
>>
>> Jun
>>
>>
>>> On Mon, May 19, 2008 at 9:15 PM, David Primmer
>>> <da...@gmail.com>
>>> wrote:
>>>
>>>> I have some updated data people, social graph, activities, that I'd
>>>> like to add to shindig but I'm wondering the best way to store
>>>> it and
>>>> work with it. Anyone have a suggestion? I'd like to replace the
>>>> state-basicfriendlist.xml file with something that's easier to edit
>>>> and work with that could be checked into shindig. Some suggestions
>>>> from the summit were http://www.hsqldb.org/,
>>>> http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
>>>> being c-based.
>>>>
>>>> I'm primarily interested in supporting the java rest api so hsqldb
>>>> sounds best but I'm sure the other implementations would like to
>>>> use
>>>> the same source data and may want something with more runtime
>>>> availability. Maybe we could store the seed data in multiple
>>>> formats
>>>> for seeding a few different small database engines but we should
>>>> start
>>>> with one.
>>>>
>>>> I'd like to avoid non-embeded stuff like MySql.
>>>>
>>>> There's been a request not to have the database engine included
>>>> in the
>>>> shindig jar. We'd have to design the maven rules to generate
>>>> alternate
>>>> binaries. Here's the maven rule to add hsqldb:
>>>>
>>>> <dependency>
>>>> <groupId>hsqldb</groupId>
>>>> <artifactId>hsqldb</artifactId>
>>>> <version>1.8.0.7</version>
>>>> <scope>test</scope>
>>>> </dependency>
>>>>
>>>> davep
>>>>
>>>
>>
Re: updating the source dataset for shindig
Posted by David Primmer <da...@gmail.com>.
From scanning the docs quickly, HSQLDB has a native storage format of
csv that can directly attached without import or export
http://www.hsqldb.org/doc/guide/ch06.html there are some caveats with
this format and there are quite a few ways to run the database (memory
only, in process, cached etc.) and I'd defer to someone with more
experience.
davep
On Mon, May 19, 2008 at 6:24 PM, Jun Yang (杨骏) <jy...@google.com> wrote:
> On Mon, May 19, 2008 at 6:18 PM, Dan Bentley <db...@google.com> wrote:
>
>> Maybe we could keep the data in some plain text format and then just have
>> scripts that can weave it into the different formats as required? This
>> might be easiest for ensuring that there is only ever one primary source
>> and
>> different versions don't have to be maintained?
>>
>
> Agreed. How about CSV (with first line containing column/property names)?
>
>
>> -Dan
>>
>
> Jun
>
>
>> On Mon, May 19, 2008 at 9:15 PM, David Primmer <da...@gmail.com>
>> wrote:
>>
>> > I have some updated data people, social graph, activities, that I'd
>> > like to add to shindig but I'm wondering the best way to store it and
>> > work with it. Anyone have a suggestion? I'd like to replace the
>> > state-basicfriendlist.xml file with something that's easier to edit
>> > and work with that could be checked into shindig. Some suggestions
>> > from the summit were http://www.hsqldb.org/,
>> > http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
>> > being c-based.
>> >
>> > I'm primarily interested in supporting the java rest api so hsqldb
>> > sounds best but I'm sure the other implementations would like to use
>> > the same source data and may want something with more runtime
>> > availability. Maybe we could store the seed data in multiple formats
>> > for seeding a few different small database engines but we should start
>> > with one.
>> >
>> > I'd like to avoid non-embeded stuff like MySql.
>> >
>> > There's been a request not to have the database engine included in the
>> > shindig jar. We'd have to design the maven rules to generate alternate
>> > binaries. Here's the maven rule to add hsqldb:
>> >
>> > <dependency>
>> > <groupId>hsqldb</groupId>
>> > <artifactId>hsqldb</artifactId>
>> > <version>1.8.0.7</version>
>> > <scope>test</scope>
>> > </dependency>
>> >
>> > davep
>> >
>>
>
Re: updating the source dataset for shindig
Posted by "Jun Yang (杨骏)" <jy...@google.com>.
On Mon, May 19, 2008 at 6:18 PM, Dan Bentley <db...@google.com> wrote:
> Maybe we could keep the data in some plain text format and then just have
> scripts that can weave it into the different formats as required? This
> might be easiest for ensuring that there is only ever one primary source
> and
> different versions don't have to be maintained?
>
Agreed. How about CSV (with first line containing column/property names)?
> -Dan
>
Jun
> On Mon, May 19, 2008 at 9:15 PM, David Primmer <da...@gmail.com>
> wrote:
>
> > I have some updated data people, social graph, activities, that I'd
> > like to add to shindig but I'm wondering the best way to store it and
> > work with it. Anyone have a suggestion? I'd like to replace the
> > state-basicfriendlist.xml file with something that's easier to edit
> > and work with that could be checked into shindig. Some suggestions
> > from the summit were http://www.hsqldb.org/,
> > http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
> > being c-based.
> >
> > I'm primarily interested in supporting the java rest api so hsqldb
> > sounds best but I'm sure the other implementations would like to use
> > the same source data and may want something with more runtime
> > availability. Maybe we could store the seed data in multiple formats
> > for seeding a few different small database engines but we should start
> > with one.
> >
> > I'd like to avoid non-embeded stuff like MySql.
> >
> > There's been a request not to have the database engine included in the
> > shindig jar. We'd have to design the maven rules to generate alternate
> > binaries. Here's the maven rule to add hsqldb:
> >
> > <dependency>
> > <groupId>hsqldb</groupId>
> > <artifactId>hsqldb</artifactId>
> > <version>1.8.0.7</version>
> > <scope>test</scope>
> > </dependency>
> >
> > davep
> >
>
Re: updating the source dataset for shindig
Posted by Dan Bentley <db...@google.com>.
Maybe we could keep the data in some plain text format and then just have
scripts that can weave it into the different formats as required? This
might be easiest for ensuring that there is only ever one primary source and
different versions don't have to be maintained?
-Dan
On Mon, May 19, 2008 at 9:15 PM, David Primmer <da...@gmail.com>
wrote:
> I have some updated data people, social graph, activities, that I'd
> like to add to shindig but I'm wondering the best way to store it and
> work with it. Anyone have a suggestion? I'd like to replace the
> state-basicfriendlist.xml file with something that's easier to edit
> and work with that could be checked into shindig. Some suggestions
> from the summit were http://www.hsqldb.org/,
> http://db.apache.org/derby/ and http://www.sqlite.org/, the latter
> being c-based.
>
> I'm primarily interested in supporting the java rest api so hsqldb
> sounds best but I'm sure the other implementations would like to use
> the same source data and may want something with more runtime
> availability. Maybe we could store the seed data in multiple formats
> for seeding a few different small database engines but we should start
> with one.
>
> I'd like to avoid non-embeded stuff like MySql.
>
> There's been a request not to have the database engine included in the
> shindig jar. We'd have to design the maven rules to generate alternate
> binaries. Here's the maven rule to add hsqldb:
>
> <dependency>
> <groupId>hsqldb</groupId>
> <artifactId>hsqldb</artifactId>
> <version>1.8.0.7</version>
> <scope>test</scope>
> </dependency>
>
> davep
>