[CONF] Apache Tuscany: RDB DAS architecture guide (page edited)

[co...@apache.org]

RDB DAS architecture guide (TUSCANY) edited by Amita Vadhavkar
      Page: http://cwiki.apache.org/confluence/display/TUSCANY/RDB+DAS+architecture+guide
   Changes: http://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=56921&originalVersion=14&revisedVersion=15






Content:
---------------------------------------------------------------------

{section:border=false}
{column:width=15%}
{include: DAS Java Subproject Menu}
{column}
{column:width=85%}
{panel:title=Apache Tuscany RDB DAS Java Architecture Guide|borderStyle=solid|borderColor=#ECF4D1|titleBGColor=#ECF4D1|bgColor=#ffffff}
* [Overview|#Overview]
* [Function Description|#Function Description]
* [Architecture|#Architecture]
* [User Interface|#User Interface]
* [Class Diagram|#Class Diagram]
* [Design Details|#Design Details]
* [Configuration Schema(config.xsd)|#Configuration Schema(config.xsd)]
* [Scalability Aspects|#Scalability Aspects]
* [Future|#Future]
* [Samples|#Samples]
* [Terminology and Useful Links|#Terminology and Useful Links]
{panel}

h3. {anchor:Overview}Overview

Service Data Object (SDO) and Service Component Architecture (SCA) are the basic building blocks in Service Oriented Architecture (SOA). SDO is the way of representing data in SCA. As the applications using SOA have back-end data representation in different formats like Database, File System, XML Based Data, etc., there is a need to provide a mapping between these data representations and SDO. Java RDB DAS (Data Access Service for Relational Database) is the effort to suffice this mapping requirement for Relational Database. There is a C+\+ implementation coming up too.

h3. {anchor:Function Description} Function Description

RDB DAS provides 2 major capabilities:\-
1) Execute SQL Queries and return result as a graph of Data Objects.
2) Reflect changes made to a graph of Data Objects back to the database.
It also supports explicit CUD SQL statements execution against database.
!rdbDAS.gif!

h3. {anchor:Architecture} Architecture

This section describes the overall architecture of Java RDB DAS. The details are elaborated in [Design Details|#Design Details]. Also, for details of different DAS Configuration settings, refer to [Configuration Schema(config.xsd)|#Configuration Schema(config.xsd)].

DAS functions based on external Configuration. This configuration can be specified using file or can be programmatically build by the caller, using *ConfigHelper* interface.

Design follows a factory pattern to create DAS instance. DAS instance contains *Connection*, *Config* and *Commands*.

The *Connection* can either be passed from caller or can be created using the Configuration information. It is used for all the database operations done in that DAS instance and also for controlling transactions. Configuration parameter *managedtx* decides whether the transaction will be controlled by DAS or by caller.

*Commands* hold all the commands from Configuration file and also the ones that are created programmatically on the fly. The Command is implemented using different inherited classes like *Read/Insert/Update/Delete/StoredProcedure/OptimisticWrite* with root parent class *CommandImpl*. More details on different classes will be in [Design Details|#Design Details].

*Config* holds the reference for Configuration associated with the DAS instance and contains all details as mentioned in [Configuration Schema (config.xsd)|#Configuration Schema (config.xsd)].

Caller has 2 (and some other) main APIs available in DAS instance to use the Commands *getCommand(name)* and *applyChanges(Data Object)*
# *getCommand(name)* is typically used to retrieve the Command (SQL String) available from Config. SQL String, Connection and Parameters required for the SQL are contained in the Command instance. User also needs to call setParameter() on Command as required by the underlying SQL. User finally calls execute() /executeQuery() on this Command and based on whether it is CUD/Read operation. executeQuery() returns DataObject.
# *applyChanges(Data Object)* is the way to process a SDO DataObject and SDO ChangeSummary to create SQL UPDATE statement to be executed against database. More details on SDO integration are in [Design Details|#Design Details].

h3. {anchor:User Interface} User Interface

DAS exposes a set of interfaces to end user. Javadoc is available for all public APIs in binary distribution from [Downloads|http://cwiki.apache.org/confluence/display/TUSCANY/DAS+Downloads].
* *DASFactory* Different options to produce DAS instance using Factory pattern.
*Method Summary*
| DAS | createDAS(java.sql.Connection connection) | Creates a DAS based on the provided connection |
| DAS | createDAS(java.io.InputStream configStream) | Creates a DAS based on the provided config file stream |
| DAS | createDAS(java.io.InputStream configStream, java.sql.Connection connection) | Creates a DAS based on the provide config file stream and connection |

* *DAS Interface* to call APIs for DAS instance.
*Field Summary*
| static DASFactory | FACTORY |
*Method Summary*
| void | applyChanges(DataObject root) | The change history is scanned and modifications to the graph of data objects are flushed to the database. |
| Command | createCommand(java.lang.String sql) | Creates a Command based on the provided SQL statement |
| Command | getCommand(java.lang.String name) | Gets the named command from this factory's inventory |
| void | releaseResources() | If the CommandGroup is managing connections then this method must be called when the client is done with the instance. |

* *Command* Used to execute read / write (CRUD) on database.
*Method Summary*
| void | execute() | Performs the function defined by the command |
| DataObject | executeQuery() | Performs the function defined by the command and return the results in the root DataObject |
| int | getGeneratedKey() | Returns the value of the database-generated key. |
| java.lang.Object | getParameter(int index) | Returns the value of the associated Parameter |
| void | setParameter(int index, java.lang.Object value) | Sets the value of the associated Parameter |

* *ConfigHelper* It is used as an aid in programmatic construction of Config instances. This is an
alternative to providing needed configuration information in an XML file.
*Constructor Summary*
| ConfigHelper() |
| ConfigHelper(Config config) |
*Method Summary*
| Column | addColumn(Table table, java.lang.String columnName, java.lang.String propertyName) |
| void | addConnectionInfo(java.lang.String dataSourceName) |
| void | addConnectionInfo(java.lang.String dataSourceName, boolean managedtx) |
| void | addConnectionInfo(java.lang.String driverClass, java.lang.String databaseURL, java.lang.String user, java.lang.String password, int loginTimeout) |
| void | addCreateStatement(Table table, java.lang.String statement, java.lang.String parameters) |
| Command | addDeleteCommand(java.lang.String name, java.lang.String sql) |
| void | addDeleteStatement(Table table, java.lang.String statement, java.lang.String parameters) |
| Command | addInsertCommand(java.lang.String name, java.lang.String sql) |
| void | addPrimaryKey(java.lang.String columnName) |
| Relationship | addRelationship(java.lang.String parentName, java.lang.String childName) |
| Relationship | addRelationship(java.util.Vector parentNames, java.util.Vector childNames) |
| Command | addSelectCommand(java.lang.String name, java.lang.String sql) |
| Table | addTable(java.lang.String name, java.lang.String typeName) |
| Command | addUpdateCommand(java.lang.String name, java.lang.String sql) |
| void | addUpdateStatement(Table table, java.lang.String statement, java.lang.String parameters) |
| Config | getConfig() |
| void | setDataObjectModel(java.lang.String dataObjectModel) |

* *Converter* A lightweight Table-column <--> DataObject-property converter framework. Converters allow a user to insert a transformation between a column's value and its destination DataObject property's value. For example, by default, a VARCHAR column will be represented as a String in its corresponding DataObject property. A user could insert a converter that transforms the VARCHAR value to an Integer. If this is done then although the column returns character data, the DataObject property will be an Integer
*Method Summary*
| java.lang.Object | getColumnValue(java.lang.Object propertyData) | Transform the columnData object to a new value and possibly new type |
| java.lang.Object | getPropertyValue(java.lang.Object columnData) | Transform the columnData object to a new value and possibly new type. |

* *Pager* An iterator-like interface to conveniently move through chunks of data. The idea is that a Pager works with a read Command. The read command returns a large amount of data and the client wants to work with chunks of it at a time. So the Pager is created on the command and each call to next returns the next chunk of data.
*Method Summary*
| DataObject | getPage(int page) | Return a specific identified page |
| DataObject | next() | Get the next page of data |
| DataObject | previous() | Get the page prior to the last page returned |

* *GraphMerger* is utility class implementation (no interface), to provide functions for flexibly managing DataObjects.
*Constructor Summary*
| GraphMerger() |
*Method Summary*
| void | addPrimaryKey(java.lang.String key) |
| DataObject | emptyGraph(Config config) |
| DataObject | merge(DataObject primary, DataObject secondary) |
| DataObject | merge(java.util.List graphs) |

h3. {anchor:Class Diagram} Class Diagram

[Class Diagram|RDB DAS class diagram]

h3. {anchor:Design Details}Design Details

Major classes are described here at high level to show how these fit in the complete picture. For 
methods/members details please refer to javadoc from binary distribution at [Downloads|http://cwiki.apache.org/confluence/display/TUSCANY/DAS+Downloads].

*DASFactoryImpl* Expose methods to create DAS instance from Config or Connection or both.

*DASImpl* The sole instance of DAS existing for one DAS runtime. It contains MappingWrapper, JDBC Connection and a Map of all DAS Commands. To create appropriate instances of CommandImpl, DASImpl, scan's each Config Command for attribute kind, and based on that required instance of CommandImpl is added to the map. Connection from DataSource or Database Driver Manager both are supported. releaseResources() method allows external caller to close JDBC Connection when it is supplied by the caller.Provides methods to get/create DAS Commands and applyChanges(changed DataObject) to reflect DataObject changes to Database.

*MappingWrapper* Wrapper over DAS Config. Provides various get/set methods for all different elements in DAS Config.

*ConnectionImpl* Contains JDBC Connection, and whether it is managed by DAS/caller and also whether Database platform support auto-generated key column values.

*BaseCommandImpl* Parent class for all other CommandImpl classes. Contains MappingWrapper. setConnection() method to associate ConnectionImpl with each DAS Command.

*ApplyChangesCommandImpl* Extends BaseCommandImpl. Contains ChangeSummarizer. It is used to load DataObject's ChangeSummary and for each change, Generator classes (UpdateGenerator, DeleteGenerator) are called to form appropriate SQL Statement with parameters, which is executed against the Database.

*CommandImpl* Extends BaseCommandImpl and implements DAS Config's <Command>. So, in effect it is parent class for all explicit CRUD Commands. It contains Statement, Parameters and ResultSetShape.

*WriteCommandImpl* Extends CommandImpl. Parent for CUD. If SELECT(Read) is attempted, exception is thrown.

*DeleteCommandImpl* Extends WriteCommandImpl.

*InsertCommandImpl* Extends WriteCommandImpl. Supports Database generated keys.

*UpdateCommandImpl* Extends WriteCommandImpl.

*OptimisticWriteCommandImpl* Extends UpdateCommandImpl. Supports OCC feature. Here by design when a CollisionParameter's value is attempted to change more than once, the CollisionParameter does not allow its value change (as during first setValue(), flag indicating that value is changed is set). So, the attempt results in no database rows getting changes. On this condition exception is thrown as Update Collision is occurred.

*ReadCommandImpl* Extends CommandImpl. Used for explicit SELECT. Support's DAS feature, ResultDescriptor. If CUD is attempted, exception is thrown. Supports paging. When Query is executed (executeQuery()), the JDBC ResultSet is converted into DataObject Graph and returned to caller.

*SPCommandImpl* Extends ReadCommandImpl for StoredProcedure call. Method executeQuery() is used when the stored procedure returns ResultSet and method execute() is used when the stored procedure does not have any return ResultSet.
Parameters hold both IN and OUT parameters.

*PagerImpl* Holds ReadCommandImpl, page size and current page index. Implements next(), previous(), getPage(index) methods. In each based on current page index and page size, the ReadCommandImpl is executed with start and end boundaries set.

*ChangeFactory* This class is used during DAS.applyChanges(). With changed DataObject and ChangeSummary, required type of ChangeOperation instance is created (like DeleteOperation, UpdateOperation). This instance of ChangeOperation holds appropriate CommandImpl, which in itself holds the Parameters. Thus for a given DAS Config and Connection, ChangeFactory is used to propagate DataObject changes to Database via CommandImpl.

*ChangeSummarizer* Main function of this class is load Changes based on root DataObject of applyChanges() Command MappingWrapper is used to sort the ChangeOperations in Changes.

*Changes* Contains DeleteList, InsertList, Update List pertaining to one applyChanges() command.

*DeleteList* Maintains ordered/unordered list of DeleteOperations. If DAS Config defines any Relationships, MappingWrapper aids in forming Delete Order for all the Tables undergoing DeleteOperation, based on Parent-Child relationships. If Delete Order is available, DeleteList forms a List following appropriate Referential Integrity. Otherwise no order is maintained.

*InsertList* Maintains ordered/unordered list of CreateOperations. If DAS Config defines any Relationships, MappingWrapper aids in forming Insert Order for all the Tables undergoing InsertOperation, based on Parent-Child relationships. If Insert Order is available, InsertList forms a List following appropriate Referential Integrity. Otherwise no order is maintained.

*UpdateList* As UpdateOperation happens on single Table, there is no reason for sorting based on any Referntial Relationships. So, this class is just a wrapper on ArrayList.

*BaseGenerator* Parent class for Delete/Update/InsertGenerator. If in DAS Config, any Table Column specifies a ColumnConverter class, BaseGenerator, returns a new instance of this class.

*DeleteGenerator* Extends BaseGenerator. Based on Table and DAS Config, DELETE statement is generated with Primary Keys in WHERE clause. Using this, DeleteCommandImpl is instantiated. For Parameters in DeleteCommandImpl ColumnConverter is considered if present.

*InsertGenerator* Same as DeleteGenerator. Here Database Key value generation is also considered.

*UpdateGenerator* Extends BaseGenerator. In UPDATE statement WHERE clause Primary Keys are considered as well as Default OCC policy is considered. Also check is implemented for any violation for Foreign Key relationships. Parameters in UpdateCommandImpl consider ColumnConvert as appropriate. OCC attributes in DAS Config for <Column>, viz. managed and collision are considered in SET and WHERE clause formation respectively.

*ChangeOperation* Abstract parent class for Create/Delete/UpdateOperation. It contains WriteCommand, DatabaseObject and any generated ID.

*DeleteOperation* Extends ChangeOperation. Uses SDO's ChangeSummary based OldValues as in the DataGraph , deleted objects have lost their *settings*.

*CreateOperation* Extends ChangeOperation, makes use of Generated ID.

*UpdateOperation* Extends ChangeOperation and makes use of generated ID.

*DatabaseObject* Wraps DataObject. If a field is FK, it will return value from parent. It also sets the Database generated value in primary key property of DataObject.

*RowObjects* Holds a Map of TableName->DataObject. It uses relationship information to associate these DataObjects to form DataGraph .

*GraphBuilderMetadata* Class that form Metadata for forming DataObjects based on MetaData available in JDBC ResultSet. It uses DAS Config, ResultSet and ResultSetShape to form a Collection of ResultMetaData. It is used in ResultSetProcessor and thus further in RowObjects.

*ResultSetProcessor* This class is used to transform data in ResultSet into set of inter-related DataObjects. It contains TableRegistry, GraphBuilderMetadata and DataObjectMaker. Using GrapbBuilderMetadata and the JDBC ResultSet, it forms
ResultSetRow , then form RowObjects and connects related DataObjects using Relationship information from RowObjects to form the completed DataGraph.

*DataObjectMaker* Forms DataObject based on TableData. It is used in ResultSetProcessor for this.

*TableRegistry* Base Interface for Single/MultiTableRegistry.

*SingleTableRegistry* No functionality.

*MultiTableRegistry* Implements TableRegistry. Used when converting JDBC ResultSet of SQL SELECT into DataObjects.

*DefaultConverter* Implements Converter. In case of Blob database column type, returns byte\[\] for the column value when mapping from database data to property value.

*ResultMetaData* Holds complete ResultSet related metadata of the Query result\- ResultSet, ColumnConverters, MappingWrapper, SDO Type Names, SDO Property Names, ResultSetShape for the ResultSet, Map containing Table Type -> Column Property (equivalent to Database Table->Column). It provides various get methods on these members.

*ResultSetRow* A ResultSetRow is used to transform a single row of ResultSet into a set of DataObjects. It uses TableData as a intermediate place to keep Database Data for a Row , which is later used to form DataObject. It holds a List of all Table Data for the current row. This class is used in ResultSetProcessor.

*TableData* Holds single JDBC row for a Table and its Columns - tableName, list of PKs, Map of ColumnName->Data, if table has PK. Is used by DataObjectMaker to form DataObject based on Table Row.

*ResultSetTypeMap* Provides mapping from sql.Types to SDO Type. Used in ResultSetShape to populate SDO Type\[\] for all columns present in ResultSet.

*FactoryRegistry* Maintains registry of ChangeFactory. There is one FactoryRegistry for on DAS Config and Connection. It maintains a Map with entries of each SDO Type against ChangeFactory for current DAS Config+Connection.

*ParameterImpl* Holds values required for representing SQL statement/StoredProcedure parameter like name, index, value, SDO Type and columnConverter. Can have direction IN/OUT/IN-OUT. IN is default. OUT/IN-OUT is only in case of Stored Procedure.

*ManagedParameterImpl* Extends ParameterImpl. Expected to be integer or BigDecimal type, else exception is thrown. setValue() returns oldValue+1.

*CollisionParameter* Extends ParameterImpl. Design restricts setting value in CollisionParameter only once.

*Parameters* Maintains lists of parameters (used in SQL statements), IN/OUT parameters (used in stored procedures).

*ResultSetShape* This is an alternative to use ResultSetMetaData of the Query result. It can be used when some platforms like Oracle do not fully support MetaData. For a given SQL Query, this holds schemaName, tableName, columnName, SDO Column Type for each column in ResultSet. It has 2 constructors, one uses JDBC's ResultSetMetaData to form ResultSetShape and another uses DAS Config supplied ResultDescriptor to form ResultSetShape.

*Statement* Holds SQL String, JDBC connection, JDBC PreparedStatement and also state information whether the current statement is used by Pager.

*SDODataTypeHelper* Provides mapping between SDO's Type and JDBC's sql.Types.

*SDODataTypes* Uses SDO's TypeHelper to get SDO Type for given string.

*QualifiedColumn* Maintains schemaName, tableName and columnName for a Table's Column.

*RelationshipWrapper* Wrapper for DAS Config's <Relationship>. Only one method to return Collection of all FKColumns in a Relationship.

*TableWrapper* Wrapper on DAS Config's <Table>. Provides various get methods to get values of different elements and attributes within <Table>.

*ConfigUtil* Uses SDO's XMLHelper to load DAS Config Data Model in memory.

*DataObjectUtil* Provides utility methods for SDO DataObject like copying one DataObject into another, restoring Old Values of a DataObject using its SDO's ChangeSummary.

h3. {anchor:Configuration Schema(config.xsd)} Configuration Schema(config.xsd)

Complete DAS Configuration follows.
Convention +Element+ <attribute> _Child Element_
| +Config+\-> \\
<uri> Not used \\
<dataObjectModel> Needs a valid value when creating static DataObjects. \\
<databaseSchemaNameSupported> when true, multiple database schema under same database are supported \\
_Command_\-0..n \\
_Table_\-0..n \\
_Relationship_\-0..n \\
_ConnectionInfo_\-0..1 \\
+ConnectionInfo+\-> \\
<dataSource>complete URL of data source \\
<managedtx>if true, DAS manages database transaction \\
_ConnectionProperties_\-0..1 \\
+ConnectionProperties+\-> \\
<driverClass>complete qualified class name \\
<databaseURL>database URL \\
<username> authentic database user \\
<password> password for the user \\
<loginTimeout>in milliseconds \\
+Command+\-> \\
<name> \\
<SQL kind>Select/Update/Delete/Insert/procedure \\
_Parameter_\-0..n \\
_ResultDescriptor_\-0..n \\
+Parameter+\-> \\
<columnName> database table column name (no meaning when Command kind is procedure) \\
<direction>IN/OUT (OUT makes sense when Command kind is "procedure") \\
<index>used in set/getParameter() on Command based on direction (used when Command kind is procedure). \\
+ResultDescriptor+\-> Used as alternative to database result set metadata. \\
<columnName> database table column name \\
<tableName> database table name \\
<schemaName> used when databaseSchemaNameSupported=true \\
<columnType>SDO mapped data type \\
+Relationship+\-> \\
<name> \\
<primaryKeyTable> \\
<foreignKeyTable> \\
<many> =true allows for 1:n relationship. If it is false, it's 1:1 relationship. \\
<keyRestricted> In case of 1:1 relationship, if keyRestricted is not specified, the \\
link (relationship) between the parent and child rows can be broken, by setting the \\
parentKeyTable's row to null/deleting the row. Also, a different parent row can \\
be associated to the child row, which has one pre-existing parent row association. \\
If keyRestricted=true, this kind on change in existing relationship is not allowed, \\
user will get exception 'Can not modify a one to one relationship that is key \\
restricted'. \\
_KeyPair_\-0..n \\
+KeyPair+\-> \\
<PrimaryKeyColumn>database table column name \\
<ForeignKeyColumn>database table column name \\
+Table+\-> \\
<tableName>Database table name \\
<typeName>Data Object type name \\
<schemaName> used when databaseSchemaNameSupported=true \\
_Column_\-0..n \\
_Create_\-0..1 \\
_Update_\-0..1 \\
_Delete_\-0..1 \\
+Create+\-> \\
<sql> INSERT statement in PreparedStement format \\
<parameters>space separated column names used in explicit INSERT \\
+Update+\-> \\
<sql> UPDATE statement in PreparedStement format \\
<parameters> space separated column names used in explicit UPDATE \\
+Delete+\-> \\
<sql> DELETE statement in PreparedStement format \\
<parameters> space separated column names used in explicit DELETE \\
+Column+\-> \\
<columnName> database table column name \\
<propertyName> SDO data object's property name \\
<converterClassName> if converter is used, this needs a valid class name \\
<primaryKey>true if column is PK \\
<generated> true if column value is database generated \\
<collision> when true, during UPDATE statement generation, that column is \\
included in the WHERE clause.(OCC feature) \\
<managed> when true, during UPDATE statement generation, that column (needs \\
to be BigDecimal) value is incremented by 1 and used in the SET clause in \\
UPDATE. \\
managed=true, is effective only when collision=true, else managed attribute value \\
is ignored. |

h3. {anchor:Scalability Aspects} Scalability Aspects

Java RDB DAS has exposed [User Interface|#User Interface]for end user. Other than that, from design perspective, many major DAS classes do not have associated interfaces and by design are tightly coupled with SDO and Database. It needs further enhancement on this line, especially when more DAS flavors are coming up as mentioned in [Future|#Future]. As DAS is a bridge between SOA based applications and backend Data Sources, the interfaces with SOA and Data Source need to be flexible. E.g. SDO/Some Other mechanism should be pluggable when using DAS as well as different technologies based Data Sources can be supported like XML files, LDAP based file system and so forth. Work is in-progress on this front.

h3. {anchor:Future} Future

DAS is implemented in Java and C+\+ and both are integrated with SDO and support Relational Database based Data Source. There are other flavors of DAS coming up like LDAP DAS, XQuery DAS etc. These implementations will provide different ways to interface with the backend data source as their names suggest.

h3. {anchor:Samples} Samples

There are a couple of samples distributed to demonstrate working examples of using different DAS features. See User Guide/Samples for more details.

h3. {anchor:Terminology and Useful Links} Terminology and Useful Links

| 1 | User Guide | [Readme|http://cwiki.apache.org/confluence/display/TUSCANY/RDB+DAS+-+User+Guide] |
| 2 | Getting Started | [Readme|http://cwiki.apache.org/confluence/display/TUSCANY/Starting+with+DAS] |
| 3 | All Useful Links | [Links|http://cwiki.apache.org/confluence/display/TUSCANY/RDB+DAS] |
{column}
{section}

---------------------------------------------------------------------
CONFLUENCE INFORMATION
This message is automatically generated by Confluence

Unsubscribe or edit your notifications preferences
   http://cwiki.apache.org/confluence/users/viewnotifications.action

If you think it was sent incorrectly contact one of the administrators
   http://cwiki.apache.org/confluence/administrators.action

If you want more information on Confluence, or have a bug to report see
   http://www.atlassian.com/software/confluence



---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-commits-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-commits-help@ws.apache.org