You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@incubator.apache.org by Craig L Russell <Cr...@Sun.COM> on 2009/03/19 01:53:30 UTC
If you're not interested in Java and MySQL Cluster,
you can stop reading now...
I'm working (day job!) on a Java interface to the MySQL Cluster
database (a.k.a. NDB). NDB is a high performance, high availability
database used mostly in telecommunications applications. It can be the
backing store for a MySQL server (accessed via any of the MySQL client
APIs such as ODBC, JDBC) or can be used directly via primitive C++ or
Java APIs.
What I've been developing is a high performance, easy-to-use Java
interface to NDB that bypasses the MySQL Server and instead goes
directly to the native NDB API. I've used a Domain Object Model
similar to Hibernate, JDO, and JPA.
I'd like to work on this in Apache and would like to discuss the
opportunities with folks at ApacheCon. If this looks interesting,
please let me know. I'll post a proposal to the incubator wiki if
there is positive feedback.
All work is done through a Session instance, which is the primary
application interface between the user and the database.
The user acquires a Session instance from a SessionFactory, which in
turn is obtained via Services lookup. The factory is configured via
Properties, such as:
com.mysql.clusterj.connectstring=127.0.0.1:9311
com.mysql.clusterj.mysqld=localhost
com.mysql.clusterj.username=root
com.mysql.clusterj.password=
com.mysql.clusterj.testDatabase=test
The Session interface provides methods for insert, delete, query, and
update.
A separate Transaction interface provides methods for transaction
completion. The Transaction provides isActive(), begin(), commit(),
rollback(), setRollbackOnly(), and getRollbackOnly() methods.
Two modes of operation are implemented: autocommit on/off.
• With autocommit off:
• when a mutating operation is invoked on the Session, the
Transaction must be active, or an exception is thrown; and
• when a non-mutating operation is invoked when a Transaction is not
active, a transaction is begun and committed for the purpose of
executing the operation.
• With autocommit on:
• when a mutating operation is invoked on the Session, the
Transaction is begun and committed for the operation; and
• when a non-mutating operation is invoked, a transaction is begun
and committed for the purpose of executing the operation.
Deferred operation is proposed. By setting the flag
com.mysql.clusterj.defer.changes, mutating operations are deferred
until the Transaction is committed or the state is flushed via
Session.flush(). With defer changes, find() operations return a proxy
instance whose persistent properties are not initialized until the
user asks for one of them. Inserted instances are not actually sent to
the back end until the session is flushed. Deleted instances are not
deleted until the session is flushed. Queries return a Results
instance that is not instantiated until the user asks for a property
of an instance.
For ease of use, users can define JavaBeans-pattern interfaces to
represent data, with annotations to declare database metadata:
@PersistenceCapable(table="t_basic")
interface Employee {
@PrimaryKey
int getId();
void setId(int id);
String getName();
void setName(String name);
}
The Session.newInstance(Class cls) is a factory for instances that
implement the user interface as well as a high-performance data access
path for the implementation. For JPA/JDO StorageManager, the high-
performance data access path is used.
// Sample code to create a SessionFactory from a properties instance
protected void createSessionFactory() {
loadProperties();
if (sessionFactory == null) {
sessionFactory = ClusterJHelper.getSessionFactory(props);
}
loadSchema();
}
// Sample code to delete and create instances
public void localSetUp() {
createSessionFactory();
session = sessionFactory.getSession();
createEmployeeInstances(NUMBER_TO_INSERT);
tx = session.currentTransaction();
int count = 0;
for (int i = 0; i < NUMBER_TO_INSERT; ++i) {
tx.begin();
session.deletePersistent(employees.get(i));
try {
tx.commit();
++count;
} catch (Exception ex) {
// ignore exceptions -- might not be any instances to
delete
}
}
addTearDownClasses(Employee.class);
// System.out.println("Deleted " + count + " instances.");
}
public void testInsert() {
tx = session.currentTransaction();
tx.begin();
for (int i = 0; i < NUMBER_TO_INSERT; ++i) {
// must be done with an active transaction
session.makePersistent(employees.get(i));
}
tx.commit();
}
}
// Sample query code
public void primaryKeyGreaterThanQuery() {
tx.begin();
// QueryBuilder is the sessionFactory for queries
QueryBuilder builder = session.getQueryBuilder();
// DomainObject is the main interface
DomainObject dobj =
builder.createQueryDefinition(Employee.class);
// parameter name
PredicateOperand param = dobj.param("id");
// property name
PredicateOperand column = dobj.get("id");
// compare the column with the parameter
Predicate compare = column.greaterThan(param);
// set the where clause into the query
dobj.where(compare);
// create a query instance
Query query = session.createQuery(dobj);
// set the parameter values
query.setParameter("id", 6);
// get the results
List<Employee> results = query.getResultList();
// consistency check the results
consistencyCheck(results);
// verify we got the right instances
Set<Integer> expected = new HashSet<Integer>();
expected.add(7);
expected.add(8);
expected.add(9);
Set<Integer> actual = new HashSet<Integer>();
for (Employee emp: results) {
actual.add(emp.getId());
}
errorIfNotEqual("Wrong employee ids returned from query: ",
expected, actual);
tx.commit();
}
Craig L Russell
Architect, Sun Java Enterprise System http://db.apache.org/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!
Re: If you're not interested in Java and MySQL Cluster,
you can stop reading now...
Posted by Craig L Russell <Cr...@Sun.COM>.
Hi Andrus,
On Mar 19, 2009, at 2:06 AM, Andrus Adamchik wrote:
> Interesting. This is the first time I see a high-level Java design
> that is a diversion from DB independence. So how much performance
> improvement do you see by nixing JDBC?
Measurements that we've taken in some simple tests (insert, delete,
query by index, query by primary key) show response time improvements
of 70% to 90%. So a query against JDBC/MySQL/Cluster that takes 7
seconds takes just 1 second using the direct ClusterJ/Cluster interface.
But we didn't measure against JDBC/MySQL/InnoDB so it's hard to
extrapolate to other systems.
Craig
>
>
> Andrus
>
>
> On Mar 19, 2009, at 2:53 AM, Craig L Russell wrote:
>
>> I'm working (day job!) on a Java interface to the MySQL Cluster
>> database (a.k.a. NDB). NDB is a high performance, high availability
>> database used mostly in telecommunications applications. It can be
>> the backing store for a MySQL server (accessed via any of the MySQL
>> client APIs such as ODBC, JDBC) or can be used directly via
>> primitive C++ or Java APIs.
>>
>> What I've been developing is a high performance, easy-to-use Java
>> interface to NDB that bypasses the MySQL Server and instead goes
>> directly to the native NDB API. I've used a Domain Object Model
>> similar to Hibernate, JDO, and JPA.
>>
>> I'd like to work on this in Apache and would like to discuss the
>> opportunities with folks at ApacheCon. If this looks interesting,
>> please let me know. I'll post a proposal to the incubator wiki if
>> there is positive feedback.
>>
>> All work is done through a Session instance, which is the primary
>> application interface between the user and the database.
>> The user acquires a Session instance from a SessionFactory, which
>> in turn is obtained via Services lookup. The factory is configured
>> via Properties, such as:
>> com.mysql.clusterj.connectstring=127.0.0.1:9311
>> com.mysql.clusterj.mysqld=localhost
>> com.mysql.clusterj.username=root
>> com.mysql.clusterj.password=
>> com.mysql.clusterj.testDatabase=test
>> The Session interface provides methods for insert, delete, query,
>> and update.
>> A separate Transaction interface provides methods for transaction
>> completion. The Transaction provides isActive(), begin(), commit(),
>> rollback(), setRollbackOnly(), and getRollbackOnly() methods.
>> Two modes of operation are implemented: autocommit on/off.
>> • With autocommit off:
>> • when a mutating operation is invoked on the Session, the
>> Transaction must be active, or an exception is thrown; and
>> • when a non-mutating operation is invoked when a Transaction is
>> not active, a transaction is begun and committed for the purpose of
>> executing the operation.
>> • With autocommit on:
>> • when a mutating operation is invoked on the Session, the
>> Transaction is begun and committed for the operation; and
>> • when a non-mutating operation is invoked, a transaction is
>> begun and committed for the purpose of executing the operation.
>>
>> Deferred operation is proposed. By setting the flag
>> com.mysql.clusterj.defer.changes, mutating operations are deferred
>> until the Transaction is committed or the state is flushed via
>> Session.flush(). With defer changes, find() operations return a
>> proxy instance whose persistent properties are not initialized
>> until the user asks for one of them. Inserted instances are not
>> actually sent to the back end until the session is flushed. Deleted
>> instances are not deleted until the session is flushed. Queries
>> return a Results instance that is not instantiated until the user
>> asks for a property of an instance.
>>
>> For ease of use, users can define JavaBeans-pattern interfaces to
>> represent data, with annotations to declare database metadata:
>> @PersistenceCapable(table="t_basic")
>> interface Employee {
>> @PrimaryKey
>> int getId();
>> void setId(int id);
>> String getName();
>> void setName(String name);
>> }
>> The Session.newInstance(Class cls) is a factory for instances that
>> implement the user interface as well as a high-performance data
>> access path for the implementation. For JPA/JDO StorageManager, the
>> high-performance data access path is used.
>>
>>
>> // Sample code to create a SessionFactory from a properties instance
>> protected void createSessionFactory() {
>> loadProperties();
>> if (sessionFactory == null) {
>> sessionFactory = ClusterJHelper.getSessionFactory(props);
>> }
>> loadSchema();
>> }
>>
>> // Sample code to delete and create instances
>>
>> public void localSetUp() {
>> createSessionFactory();
>> session = sessionFactory.getSession();
>> createEmployeeInstances(NUMBER_TO_INSERT);
>> tx = session.currentTransaction();
>> int count = 0;
>> for (int i = 0; i < NUMBER_TO_INSERT; ++i) {
>> tx.begin();
>> session.deletePersistent(employees.get(i));
>> try {
>> tx.commit();
>> ++count;
>> } catch (Exception ex) {
>> // ignore exceptions -- might not be any instances to
>> delete
>> }
>> }
>> addTearDownClasses(Employee.class);
>> // System.out.println("Deleted " + count + " instances.");
>> }
>>
>> public void testInsert() {
>> tx = session.currentTransaction();
>> tx.begin();
>>
>> for (int i = 0; i < NUMBER_TO_INSERT; ++i) {
>> // must be done with an active transaction
>> session.makePersistent(employees.get(i));
>> }
>>
>> tx.commit();
>> }
>> }
>>
>> // Sample query code
>> public void primaryKeyGreaterThanQuery() {
>>
>> tx.begin();
>> // QueryBuilder is the sessionFactory for queries
>> QueryBuilder builder = session.getQueryBuilder();
>> // DomainObject is the main interface
>> DomainObject dobj =
>> builder.createQueryDefinition(Employee.class);
>>
>> // parameter name
>> PredicateOperand param = dobj.param("id");
>> // property name
>> PredicateOperand column = dobj.get("id");
>> // compare the column with the parameter
>> Predicate compare = column.greaterThan(param);
>> // set the where clause into the query
>> dobj.where(compare);
>> // create a query instance
>> Query query = session.createQuery(dobj);
>>
>> // set the parameter values
>> query.setParameter("id", 6);
>> // get the results
>> List<Employee> results = query.getResultList();
>> // consistency check the results
>> consistencyCheck(results);
>> // verify we got the right instances
>> Set<Integer> expected = new HashSet<Integer>();
>> expected.add(7);
>> expected.add(8);
>> expected.add(9);
>> Set<Integer> actual = new HashSet<Integer>();
>> for (Employee emp: results) {
>> actual.add(emp.getId());
>> }
>> errorIfNotEqual("Wrong employee ids returned from query: ",
>> expected, actual);
>> tx.commit();
>> }
>>
>> Craig L Russell
>> Architect, Sun Java Enterprise System http://db.apache.org/jdo
>> 408 276-5638 mailto:Craig.Russell@sun.com
>> P.S. A good JDO? O, Gasp!
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
Craig L Russell
Architect, Sun Java Enterprise System http://db.apache.org/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!
Re: If you're not interested in Java and MySQL Cluster, you can stop reading now...
Posted by Andrus Adamchik <an...@objectstyle.org>.
Interesting. This is the first time I see a high-level Java design
that is a diversion from DB independence. So how much performance
improvement do you see by nixing JDBC?
Andrus
On Mar 19, 2009, at 2:53 AM, Craig L Russell wrote:
> I'm working (day job!) on a Java interface to the MySQL Cluster
> database (a.k.a. NDB). NDB is a high performance, high availability
> database used mostly in telecommunications applications. It can be
> the backing store for a MySQL server (accessed via any of the MySQL
> client APIs such as ODBC, JDBC) or can be used directly via
> primitive C++ or Java APIs.
>
> What I've been developing is a high performance, easy-to-use Java
> interface to NDB that bypasses the MySQL Server and instead goes
> directly to the native NDB API. I've used a Domain Object Model
> similar to Hibernate, JDO, and JPA.
>
> I'd like to work on this in Apache and would like to discuss the
> opportunities with folks at ApacheCon. If this looks interesting,
> please let me know. I'll post a proposal to the incubator wiki if
> there is positive feedback.
>
> All work is done through a Session instance, which is the primary
> application interface between the user and the database.
> The user acquires a Session instance from a SessionFactory, which in
> turn is obtained via Services lookup. The factory is configured via
> Properties, such as:
> com.mysql.clusterj.connectstring=127.0.0.1:9311
> com.mysql.clusterj.mysqld=localhost
> com.mysql.clusterj.username=root
> com.mysql.clusterj.password=
> com.mysql.clusterj.testDatabase=test
> The Session interface provides methods for insert, delete, query,
> and update.
> A separate Transaction interface provides methods for transaction
> completion. The Transaction provides isActive(), begin(), commit(),
> rollback(), setRollbackOnly(), and getRollbackOnly() methods.
> Two modes of operation are implemented: autocommit on/off.
> • With autocommit off:
> • when a mutating operation is invoked on the Session, the
> Transaction must be active, or an exception is thrown; and
> • when a non-mutating operation is invoked when a Transaction is
> not active, a transaction is begun and committed for the purpose of
> executing the operation.
> • With autocommit on:
> • when a mutating operation is invoked on the Session, the
> Transaction is begun and committed for the operation; and
> • when a non-mutating operation is invoked, a transaction is begun
> and committed for the purpose of executing the operation.
>
> Deferred operation is proposed. By setting the flag
> com.mysql.clusterj.defer.changes, mutating operations are deferred
> until the Transaction is committed or the state is flushed via
> Session.flush(). With defer changes, find() operations return a
> proxy instance whose persistent properties are not initialized until
> the user asks for one of them. Inserted instances are not actually
> sent to the back end until the session is flushed. Deleted instances
> are not deleted until the session is flushed. Queries return a
> Results instance that is not instantiated until the user asks for a
> property of an instance.
>
> For ease of use, users can define JavaBeans-pattern interfaces to
> represent data, with annotations to declare database metadata:
> @PersistenceCapable(table="t_basic")
> interface Employee {
> @PrimaryKey
> int getId();
> void setId(int id);
> String getName();
> void setName(String name);
> }
> The Session.newInstance(Class cls) is a factory for instances that
> implement the user interface as well as a high-performance data
> access path for the implementation. For JPA/JDO StorageManager, the
> high-performance data access path is used.
>
>
> // Sample code to create a SessionFactory from a properties instance
> protected void createSessionFactory() {
> loadProperties();
> if (sessionFactory == null) {
> sessionFactory = ClusterJHelper.getSessionFactory(props);
> }
> loadSchema();
> }
>
> // Sample code to delete and create instances
>
> public void localSetUp() {
> createSessionFactory();
> session = sessionFactory.getSession();
> createEmployeeInstances(NUMBER_TO_INSERT);
> tx = session.currentTransaction();
> int count = 0;
> for (int i = 0; i < NUMBER_TO_INSERT; ++i) {
> tx.begin();
> session.deletePersistent(employees.get(i));
> try {
> tx.commit();
> ++count;
> } catch (Exception ex) {
> // ignore exceptions -- might not be any instances to
> delete
> }
> }
> addTearDownClasses(Employee.class);
> // System.out.println("Deleted " + count + " instances.");
> }
>
> public void testInsert() {
> tx = session.currentTransaction();
> tx.begin();
>
> for (int i = 0; i < NUMBER_TO_INSERT; ++i) {
> // must be done with an active transaction
> session.makePersistent(employees.get(i));
> }
>
> tx.commit();
> }
> }
>
> // Sample query code
> public void primaryKeyGreaterThanQuery() {
>
> tx.begin();
> // QueryBuilder is the sessionFactory for queries
> QueryBuilder builder = session.getQueryBuilder();
> // DomainObject is the main interface
> DomainObject dobj =
> builder.createQueryDefinition(Employee.class);
>
> // parameter name
> PredicateOperand param = dobj.param("id");
> // property name
> PredicateOperand column = dobj.get("id");
> // compare the column with the parameter
> Predicate compare = column.greaterThan(param);
> // set the where clause into the query
> dobj.where(compare);
> // create a query instance
> Query query = session.createQuery(dobj);
>
> // set the parameter values
> query.setParameter("id", 6);
> // get the results
> List<Employee> results = query.getResultList();
> // consistency check the results
> consistencyCheck(results);
> // verify we got the right instances
> Set<Integer> expected = new HashSet<Integer>();
> expected.add(7);
> expected.add(8);
> expected.add(9);
> Set<Integer> actual = new HashSet<Integer>();
> for (Employee emp: results) {
> actual.add(emp.getId());
> }
> errorIfNotEqual("Wrong employee ids returned from query: ",
> expected, actual);
> tx.commit();
> }
>
> Craig L Russell
> Architect, Sun Java Enterprise System http://db.apache.org/jdo
> 408 276-5638 mailto:Craig.Russell@sun.com
> P.S. A good JDO? O, Gasp!
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
RE: If you're not interested in Java and MySQL Cluster, you can stop reading now...
Posted by "Noel J. Bergman" <no...@devtech.com>.
Craig Russell wrote:
> the MySQL Cluster code is currently licensed strictly as GPL/
> Commercial. I'm working with the MySQL product folks on changing the
> license to be similar to the other MySQL connectors (waiving the GPL
> terms when used with approved open source licenses).
You should work with the Legal team. Background: when I had approached
our legal advisors (pre-Legal team) regarding the FOSS exception, I was
told (I can probably dig up the original e-mails if necessary) that it was
not sufficient, and thus we could not ship the GPL MySQL JDBC driver.
> The plan is to license all of the ClusterJ code under the Apache
> license except for the code that actually invokes the NDB Cluster APIs
> (which is a very small part).
What good is ClusterJ without that last bit?
To be clear, I'm interested, just concerned about the licensing issues.
--- Noel
Re: If you're not interested in Java and MySQL Cluster,
you can stop reading now...
Posted by Craig L Russell <Cr...@Sun.COM>.
Hi Gianugo,
On Mar 19, 2009, at 12:37 AM, Gianugo Rabellino wrote:
> On Thu, Mar 19, 2009 at 1:53 AM, Craig L Russell <Craig.Russell@sun.com
> > wrote:
>> I'm working (day job!) on a Java interface to the MySQL Cluster
>> database
>> (a.k.a. NDB). NDB is a high performance, high availability database
>> used
>> mostly in telecommunications applications. It can be the backing
>> store for a
>> MySQL server (accessed via any of the MySQL client APIs such as
>> ODBC, JDBC)
>> or can be used directly via primitive C++ or Java APIs.
>>
>> What I've been developing is a high performance, easy-to-use Java
>> interface
>> to NDB that bypasses the MySQL Server and instead goes directly to
>> the
>> native NDB API. I've used a Domain Object Model similar to
>> Hibernate, JDO,
>> and JPA.
>>
>> I'd like to work on this in Apache and would like to discuss the
>> opportunities with folks at ApacheCon. If this looks interesting,
>> please let
>> me know. I'll post a proposal to the incubator wiki if there is
>> positive
>> feedback.
>
> Interesting! However, I assume the MySQL side is all GPL. How does it
> mix 'n match with the Apache License?
Yes, the MySQL Cluster code is currently licensed strictly as GPL/
Commercial. I'm working with the MySQL product folks on changing the
license to be similar to the other MySQL connectors (waiving the GPL
terms when used with approved open source licenses).
The plan is to license all of the ClusterJ code under the Apache
license except for the code that actually invokes the NDB Cluster APIs
(which is a very small part). That part would no be hosted at Apache,
obviously.
Craig
>
>
> Thanks,
>
>
> --
> Gianugo Rabellino
> Sourcesense, making sense of Open Source: http://www.sourcesense.com
> (blogging at http://www.rabellino.it/blog/)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
Craig L Russell
Architect, Sun Java Enterprise System http://db.apache.org/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!
Re: If you're not interested in Java and MySQL Cluster, you can stop
reading now...
Posted by Gianugo Rabellino <gi...@gmail.com>.
On Thu, Mar 19, 2009 at 1:53 AM, Craig L Russell <Cr...@sun.com> wrote:
> I'm working (day job!) on a Java interface to the MySQL Cluster database
> (a.k.a. NDB). NDB is a high performance, high availability database used
> mostly in telecommunications applications. It can be the backing store for a
> MySQL server (accessed via any of the MySQL client APIs such as ODBC, JDBC)
> or can be used directly via primitive C++ or Java APIs.
>
> What I've been developing is a high performance, easy-to-use Java interface
> to NDB that bypasses the MySQL Server and instead goes directly to the
> native NDB API. I've used a Domain Object Model similar to Hibernate, JDO,
> and JPA.
>
> I'd like to work on this in Apache and would like to discuss the
> opportunities with folks at ApacheCon. If this looks interesting, please let
> me know. I'll post a proposal to the incubator wiki if there is positive
> feedback.
Interesting! However, I assume the MySQL side is all GPL. How does it
mix 'n match with the Apache License?
Thanks,
--
Gianugo Rabellino
Sourcesense, making sense of Open Source: http://www.sourcesense.com
(blogging at http://www.rabellino.it/blog/)
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org