Data model

["eberhard speer jr." <se...@ducis.net>]

The DeviceMap data-model has come up in passing in previous messages.
Here I would like to argue my view on the matter.
I'm hoping to get the debate on this matter going and also, seeing as
I'm rather exceptionally using HTML in email, that the formatting will
come out OK :-)
 

The key question is : what is the ideal technology and 'data-model' for
maintaining the DeviceMap device data.
The image below illustrates what I think is the best approach.
DeviceMap Data

It seems obvious to me that 'classic' SQL (A : RDBMS) as opposed to
'newfangled' NoSQL is the best solution in this case :

 1. the data model is quite simple, clearly defined and 'relational'
 2. the integrity constraints are well understood and easy to define and
    implement
 3. the data-set is small
 4. consistency and durability are essential
 5. extra-ordinary speed and massive scalability are not a requirement
 6. stability

Additionally, from the point of view of the application used to maintain
(B : Maintenance App) the data :

  * any (web-based) app on any stack can access any major RDBMS securely
    via a 'vanilla' DAL relying on parametrized stored-procedures
  * minimal code required to format and validate data
  * additional requirements such as user access, user rights (C : Users)
    can be easily accommodated
  * transaction, user activity and version (imported/exported data,
    generated sets) logs are a given
  * ad hoc queries to collect data (number of devices per brand, ...)
    and database (data-set versions,...) stats

This model : (A) RDBMS with (B) a 'classic' CRUD application front-end,
accessible by (C) various types of Users could then be used to :

 1. Import (validation/normalization) available relevant data sources :
    OpenDDR XML, UAProf RDFs,...
 2. Maintain and manage the main data-sets, users, data-set import and
    export versions and types,...
 3. Export or generate data-sets in different formats, technologies and
    flavors as required by current and future actual DeviceMap DDR 'clients'

Basically this means that the DeviceMap 'data-model' is 'nothing but' a
set of table-definitions with a collection of (parametrized)
stored-procedures which can be implemented on any sufficiently robust
RDBMS and to which any maintenance application on any stack can be
connected through a classic 'vanilla' data-access-layer.
This leaves the whole thing pretty much platform neutral yet allows
DeviceMap to maintain a consistent data-set and accommodate (export to)
any number of SQL, NoSQL, binary, flat-file or other DeviceMap DDR
'clients'.
I imagine a public 'face' to the maintenance app where any DeviceMap DDR
'subscriber'/user can freely download the data-set of their choice
(technology/platform/flavor, sub-set and even version) for their
DeviceMap DDR client implementation, while an 'authorized personnel
only' side is used to do the actual maintenance and management.



As you may have guessed, my OpenDDR fork uses such a data-model : import
OpenDDR and OMA RDF and export/generate Redis, JSON, SQL and Protobuf
data-sets for the actual OpenDDR code which, as discussed earlier, can
connect to these different data-sources.
The only thing missing, seeing as I'm currently the only user, is the
actual maintenance app (just a collection of tiny individual code-blocks
I now run from a 'command-line').

Looking forward to your comments etc.

Regards,

esjr

Re: Data model

[Werner Keil <we...@gmail.com>]

Hi,

Unless the underlying data structures, many defined by the W3C DDR standard
were to change drastically, I guess we will have to handle XML.

A plain RDBMS may not be the best foundation for that, what do you think?

Most NoSQL systems also use other formats, JSON especially, in case of
MongoDB at least. If a solid translation from XML to JSON works, those
could also be a better, more flexible option.

Beside XML databases:
http://stackoverflow.com/questions/3067205/best-xml-based-database

Also a nice overview and comparison by a Czech Java Bootcamp:
http://www.java-bootcamp.cz/cs/bootcamp4_valenta_xml-a-no-sql-d.pdf

Regards,

-- 

Werner Keil | JCP Executive Committee Member | Eclipse UOMo Lead, Babel
Language Champion | Java Godfather

Twitter @wernerkeil | #Java_Social | #EclipseUOMo | #OpenDDR
Skype werner.keil | Google+ gplus.to/wernerkeil

* Nordic Java NightHacking: January 31 2013, Copenhagen, Denmark. Werner
Keil, JCP Executive Committee Member welcomes Stephen Chin's "Nordic
NightHacking Tour" in Copenhagen

* Social Media Week: February 18 2013, Hamburg, Germany. Werner Keil, JCP
Executive Committee Member, Agorava Co-Founder will present "Enterprise
Social using Open Source Frameworks like Agorava"

On Wed, Jan 23, 2013 at 9:25 AM, eberhard speer jr. <se...@ducis.net>wrote:

> The DeviceMap data-model has come up in passing in previous messages.
> Here I would like to argue my view on the matter.
> I'm hoping to get the debate on this matter going and also, seeing as
> I'm rather exceptionally using HTML in email, that the formatting will
> come out OK :-)
>
>
> The key question is : what is the ideal technology and 'data-model' for
> maintaining the DeviceMap device data.
> The image below illustrates what I think is the best approach.
> DeviceMap Data
>
> It seems obvious to me that 'classic' SQL (A : RDBMS) as opposed to
> 'newfangled' NoSQL is the best solution in this case :
>
>  1. the data model is quite simple, clearly defined and 'relational'
>  2. the integrity constraints are well understood and easy to define and
>     implement
>  3. the data-set is small
>  4. consistency and durability are essential
>  5. extra-ordinary speed and massive scalability are not a requirement
>  6. stability
>
> Additionally, from the point of view of the application used to maintain
> (B : Maintenance App) the data :
>
>   * any (web-based) app on any stack can access any major RDBMS securely
>     via a 'vanilla' DAL relying on parametrized stored-procedures
>   * minimal code required to format and validate data
>   * additional requirements such as user access, user rights (C : Users)
>     can be easily accommodated
>   * transaction, user activity and version (imported/exported data,
>     generated sets) logs are a given
>   * ad hoc queries to collect data (number of devices per brand, ...)
>     and database (data-set versions,...) stats
>
> This model : (A) RDBMS with (B) a 'classic' CRUD application front-end,
> accessible by (C) various types of Users could then be used to :
>
>  1. Import (validation/normalization) available relevant data sources :
>     OpenDDR XML, UAProf RDFs,...
>  2. Maintain and manage the main data-sets, users, data-set import and
>     export versions and types,...
>  3. Export or generate data-sets in different formats, technologies and
>     flavors as required by current and future actual DeviceMap DDR
> 'clients'
>
> Basically this means that the DeviceMap 'data-model' is 'nothing but' a
> set of table-definitions with a collection of (parametrized)
> stored-procedures which can be implemented on any sufficiently robust
> RDBMS and to which any maintenance application on any stack can be
> connected through a classic 'vanilla' data-access-layer.
> This leaves the whole thing pretty much platform neutral yet allows
> DeviceMap to maintain a consistent data-set and accommodate (export to)
> any number of SQL, NoSQL, binary, flat-file or other DeviceMap DDR
> 'clients'.
> I imagine a public 'face' to the maintenance app where any DeviceMap DDR
> 'subscriber'/user can freely download the data-set of their choice
> (technology/platform/flavor, sub-set and even version) for their
> DeviceMap DDR client implementation, while an 'authorized personnel
> only' side is used to do the actual maintenance and management.
>
>
>
> As you may have guessed, my OpenDDR fork uses such a data-model : import
> OpenDDR and OMA RDF and export/generate Redis, JSON, SQL and Protobuf
> data-sets for the actual OpenDDR code which, as discussed earlier, can
> connect to these different data-sources.
> The only thing missing, seeing as I'm currently the only user, is the
> actual maintenance app (just a collection of tiny individual code-blocks
> I now run from a 'command-line').
>
> Looking forward to your comments etc.
>
> Regards,
>
> esjr
>
>