You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-commits@db.apache.org by Apache Wiki <wi...@apache.org> on 2006/06/13 19:49:15 UTC
[Db-derby Wiki] Update of "Requirement Specifications for Monitoring &
Management Extensions using JMX" by SanketSharma
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Db-derby Wiki" for change notification.
The following page has been changed by SanketSharma:
http://wiki.apache.org/db-derby/Requirement_Specifications_for_Monitoring_%26_Management_Extensions_using_JMX
New page:
= Apache Derby =
== Requirement Specifications for Monitoring and Management Extensions using JMX ==
== Document Contents ==
1. [#Revision_History Revision History][[BR]]
2. [#Disclaimer Disclaimer and Legal Information][[BR]]
3. [#About About this document][[BR]]
3.1 [#Purpose Purpose][[BR]]
3.2 [#Audience Intended Audience][[BR]]
3.3 [#Notations Notations and Conventions][[BR]]
3.4 [#Structure Structure of this document][[BR]]
3.5 [#Note A note about ranking][[BR]]
4. [#Assumptions Assumptions and Dependencies][[BR]]
5. [#Analysis Requirement Analysis][[BR]]
5.1 [#Overview Overview][[BR]]
5.2 [#Current Current Scenario][[BR]]
5.2.1 [#Embedded Embedded Instance][[BR]]
5.2.2 [#Embedded_Server Embedded in Server Framework][[BR]]
5.2.3 [#Network_Server Network Server][[BR]]
5.3 [#Conclusion Conclusion][[BR]]
6. [#Identified Requirements Identified][[BR]]
6.1 [#Feature_Overview Overview][[BR]]
6.2 [#Feature_Identified Features Identified][[BR]]
6.3 [#RemoteLocal Remote vs. Local][[BR]]
6.4 [#Feature_Conclusion Conclusion][[BR]]
7. [#References References][[BR]]
== 1. Revision History ==
[[Anchor(Revision_History)]]
||<rowstyle="font-weight: bold; font-size: 12pt;color: white; text-align: center; background-color: #828DA6; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans, sanserif; padding: 6px; border: none;"> Version || Version Information || Date || Modified by || Comments ||
|| 1.0 || Document Created || Tuesday, June 06, 2006 || Sanket Sharma || Comments are welcome! ||
== 2. Disclaimer and Legal Information ==
[[Anchor(Disclaimer)]]
Copyright 2005 The Apache Software Foundation or its licensors, as applicable.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0.
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
== 3. About this Document ==
[[Anchor(About)]]
=== 3.1. Purpose ===
[[Anchor(Purpose)]]
This document identifies the requirements and design goals for adding Management extensions to Apache Derby as Google Summer of Code 2006 project. Consequently the document identifies a set of requirements, categorizes them and attempts to assign priority to the identified requirements for their implementation. However, the priorities assigned reflect my own personal views and are left open for the community to decide. The final priorities will be decided by the community as a whole. This document does not describe any high level architecture or implementation details. These will be described in a subsequent documents.
[[BR]]
=== 3.2. Intended Audience ===
[[Anchor(Audience)]]
The intended audience includes a wide community of those interested in developing monitoring and management applications for the Apache Derby. It also includes the end users of the database who will ultimately use these features and the developers who might use these features for fine tuning their applications.
[[BR]]
=== 3.3. Notations and Conventions ===
[[Anchor(Notations)]]
The documentation set uses the following conventions.
[[BR]]
||<rowstyle="font-weight: bold; font-size: 12pt;color: white; text-align: center; background-color: #828DA6; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans, sanserif; padding: 6px; border: none;"> Convention || Explanation || Example ||
||<|5> {{{ Monospace }}} || Filename || {{{ MyFile.java }}} ||
|| Directory Names and Path Names || {{{ \MyDir\MyDir2 }}} ||
|| Commands and Command line options || {{{ ecl-02 }}} ||
|| Method names, Classes, Interfaces in running text || Use the {{{ createService() }}} method to... ||
|| Parameters or other placeholders || {{{ doSomething(int val1, int val2) }}}||
|| ''' {{{ Monospace bold }}} ''' || User input || ''' {{{ [C:] dir }}} '''||
|| '' Italics '' || Emphasis; Introducing or defining terms || The term '' access '' takes as its subject ||
|| [ ] || Optional Items || {{{ -Fa[c] }}} Indicates these command-line options: {{{ -Fa }}} and {{{ -Fac }}} ||
||<|4> {|} ||<|4> Choice of one item from a selection of two or more items. || -aX{K | W | P} Indicates these command-line options: ||
|| {{{ -aXK }}} ||
|| {{{ -aXW }}} ||
|| {{{ -aXP }}} ||
[[BR]]
=== 3.4. Structure of this document ===
[[Anchor(Structure)]]
This section is divided into seven sections. Section one to four provide basic information about legal stuff and notational conventions used in this document. Section five describes what features currently exist in Derby to monitor and tune the performance of the system and why we need a better monitoring and management system. Section six then tries to identify features that might be useful to developers and administrators. While this section enumerates a whole bunch of features, this section also tries to assign a priority to their implementation. Section seven enlists the sources of information for this document.
[[BR]]
=== 3.5. A Note about Ranking ===
[[Anchor(Note)]]
Developers and Derby users who read this document are requested to assign their priorities to the features enlisted in section six. I have provided an initial version of priorities based on my personal opinion. Developers and Users are requested add a column to the table, add their names as coloumn heading and assign priorities to the features. This will help us in deciding which features have to be implemented and in which order.
[[BR]]
== 4. Assumptions and Dependencies ==
[[Anchor(Assumptions)]]
[[BR]]
This document follows Java Coding conventions defined at http://java.sun.com/docs/codeconv/html/CodeConvTOC.doc.html
[[BR]]
== 5. Requirement Analysis ==
[[Anchor(Analysis)]]
=== 5.1. Overview ===
[[Anchor(Overview)]]
Apache Derby is a relational, light weight, embedded database developed entirely in Java. Being an embedded database it is very flexible in its use and can be used either embedded inside an application sharing the same JVM as the application or can be embedded inside a server framework (such as Geronimo) or can be deployed as a standalone server (with the Network server framework) in a traditional client server environment. However, currently derby does not support any mature monitoring and management features which lets a developer or an administrator view statistics about the health of a system and fine tune system parameters via a management console or application. Such functionality is indispensable for any database system of any scale for a multitude of reasons. This section tries to identify the requirements for such a monitoring and management framework.
[[BR]]
=== 5.2. Current Scenario ===
[[Anchor(Current)]]
The following section identifies the what is currently supported in derby with reference to:
[[BR]]
''' 1. Monitoring the state the system ''' [[BR]]
[[BR]]
''' 2. Tuning system parameters ''' [[BR]]
[[BR]]
Although current features can not be termed as monitoring or management features, but they do provide some primitive functionality. The following section describes these in details with respect to the three ways Derby can be used - embedded and as network server.
[[BR]]
==== 5.2.1. Embedded Instance ====
[[Anchor(Embedded)]]
When Derby runs as an embedded database engine sharing the same JVM there are quite a few ways for setting the properties for a Derby instance, both system wide and database. Such properties can be set either in the derby.properties file or can be at run time by: [[BR]]
[[BR]]
* Using parameters while starting application e.g. java -Dderby.xxx=something YourApplication. [[BR]]
[[BR]]
* Using API's provided. [[BR]]
[[BR]]
While these features let an application read or update system properties, there is no interactive way of doing it. Further, these can not be termed as monitoring and management features.
What is needed is a functionality which a developer can use to tune the system while examining its state from outside the application. A typical usage scenario would be when a user runs his application and then starts the management console to view the current state of system or to debug his application. Without writing any code in his application he is able to change the system state and view how the system responds. Such functionality should be available to all applications without any coding. The developer/administrator should be able to tune the system at runtime without changing his code.
[[BR]]
==== 5.2.2. Embedded inside a server framework ====
[[Anchor(Embedded_Server)]]
[[BR]]Essentially, this is same as 5.2.1 where the application server or some other server framework becomes client for Derby. Most application servers come with a management console that lets administrators and developers configure various system wide settings locally or remotely. Since Derby does not provide any management framework that integrates into application server frameworks, developers of such systems using Derby have to write wrappers around Derby API calls to provide any management functionality, if at all.
These servers usually generate a lot of runtime statistics and building monitoring extensions right into Derby will make integration much more transparent. The need for a standard management framework is accentuated in this case. Currently, the only way a developer can change system settings is via API calls or derby.properties file.
[[BR]]
==== 5.2.3. Network Server ====
[[Anchor(Network_Server)]]
In this mode Derby runs in embedded mode, wrapped inside a server framework that multiple remote clients connect to. Derby can run as full fledged relational database server with the help of bundled server framework. Multiple remote clients can connect to database server simultaneously while the server takes care of concurrency issues and ACID properties.
Again, .properties file and API calls are the only mechanisms to tune the system. However, in this mode Derby provides a Servlet that can be used for remote administration of Server. The servlet is first deployed in a servlet container like Tomcat which enables administration of the server over HTTP. However the servlet interface only provides management support without any support for monitoring the server.
The proposed extensions will enable a server framework using derby to provide live monitoring of system along with the "regular" administration features found in most commercial database systems. Some of these features have been identified in the following sections.
[[BR]]
=== 5.3. Conclusion ===
[[Anchor(Conclusion)]]
Whether running in embedded mode or as server, the developer or user has very limited choices for controlling the behavior of system. In light of the facts above it is desirable to have a monitoring and management extension for derby for that enables developers and administrators to view the performance of a Derby instance without writing any extra code. The exact list of features to be supported by such a extension are discussed below
[[BR]]
== 6. Requirements Identified ==
[[Anchor(Identified)]]
=== 6.1. Overview ===
[[Anchor(Feature_Overview)]]
This section describes the requirements of the new monitoring and management system for Apache Derby. The requirements are classified in three categories: [[BR]]
''' 1. Requirements for the Monitoring System: ''' Requirements identified under this category are critical for assessing the health of the system. These are either counters or variables that provide insight into the state of the system. [[BR]]
''' 2. Requirements for Management System: ''' Requirements identified under this category are essential to fine tuning the state of the system. These operations provide the user ability to change system properties. [[BR]]
''' 3. Requirements for Notification System: ''' Requirements identified under this category are not essential for a management or monitoring system but provide extended functionality to enterprise applications that embed derby. [[BR]]
The requirements identified are for the core Derby engine and do not take into consideration any special requirements for a specific case.
=== 6.2. Proposed Features ===
[[Anchor(Feature_Identified)]]
||<rowstyle="font-weight: bold; font-size: 12pt;color: white; text-align: center; background-color: #828DA6; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans, sanserif; padding: 6px; border: none;">Cateogory/Code || Features || Priority (Sanket) || Priority (David) ||
||<rowstyle="font-weight: bold; font-size: 12pt;color: white; text-align: center; BACKGROUND: #00ccff; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans, sanserif; padding: 6px; border: none;"> a) Monitoring || || || ||
|| MON1 || View derby system/database properties (derby.xxx) || 0 ||0||
|| MON2 || Sys info e.g. version, jvm version etc.|| 0 ||0||
|| MON3 || Locks held in the system || 1 ||2||
|| MON4 || Number of commits/sec || 0 ||1||
|| MON5 || Number of roll backs/sec || 0 ||1||
|| MON6 || Time since last save point || 2 ||2||
|| MON7 || Number of system threads || 0 ||1||
|| MON8 || Number of users connected || 0 ||1||
|| MON9 || Number of open connections || 0 ||1||
|| MON10 || Transaction Granularity || 2 ||2||
|| MON11 || Open cursors || 1 ||1||
|| MON12 || Databases || || ||
|| MON12.1 || Database properties (derby.xxx)|| 0 ||0||
|| MON12.2 || Database Objects || 2 ||2||
|| MON12.3 || Database Size || 1 ||2||
|| MON13 || Memory usage stats || 1 ||1||
|| MON14 || CPU usage stats || 1 ||1||
|| MON15 || Stats about Garbage collection || 1 ||2||
|| MON16 || View number of cached objects (Query)|| 2 ||2||
|| MON17 || Deadlocks detected/trace || 1 ||2||
|| MON18 || Stats about disk I/O || 0 ||1||
|| MON19 || Query plan || 2 ||2||
||<rowstyle="font-weight: bold; font-size: 12pt;color: white; text-align: center; BACKGROUND: #00ccff; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans, sanserif; padding: 6px; border: none;"> b) Management || || || ||
|| MGMT1 || Modify derby/database properties (derby.xxx) || 0||0||
|| MGMT2 || Boot a database || 1 ||2||
|| MGMT3 || Shutdown a database || 0 ||0||
|| MGMT4 || Drop a database || 2 ||2||
|| MGMT5 || Create a database || 2 ||3||
|| MGMT6 || Create database objects || 2 ||3||
|| MGMT7 || Backup a database || 1 ||1||
|| MGMT8 || Restore a database from backup || 1 ||1||
|| MGMT9 || Recovery || 1 ||? Not sure if this is needed - we do auto-recovery||
|| MGMT10 || Force a checkpoint || 2 ||3||
|| MGMT11 || Rebuild indexes || 2 ||2||
|| MGMT12 || Security || || ||
|| MGMT12.1|| Modify Authorization || 1 ||2||
|| MGMT12.2|| Modify Authentication scheme || 1 ||2||
|| MGMT13 || Replacement of management servlet|| 2 ||3||
||<rowstyle="font-weight: bold; font-size: 12pt;color: white; text-align: center; BACKGROUND: #00ccff; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans, sanserif; padding: 6px; border: none;"> c) Notifications || || || ||
|| NOT1 || When a transaction fails || 0 ||1||
|| NOT2 || When a database is booted || 1 ||2||
|| NOT3 || When a database is shutdown || 1 ||2||
|| NOT4 || When a user connects || 2 ||4||
|| NOT5 || When a user disconnects || 2 ||4||
|| NOT6 || Every time a query is fired || 2 ||4||
|| NOT7 || Every time a stored procedure/function is run || 2 ||3||
|| NOT8 || All database operations like creating/deleting/modifying etc. || 2 ||4||
This is not an exhaustive list and is open to further discussions by the community. In particular, I would like to see some more suggestions on notifications.
Some of these attributes or features may lack sufficient detail at the moment, but these details will be worked out as the work progresses.
=== 6.3. Remote VS. Local ===
[[Anchor(RemoteLocal)]]
In my opinion, remote access to these features should not be provided. My opinion is based on the following observations:
* ''' Embedded Nature of derby: ''' The scope of the database can not extend beyond that of application. If the application as a whole does not provide any remote administration feature, it does not make much sense for only a part of the application to be remotely accessible.[[BR]]
* ''' Overhead involved: ''' I think adding remote manageability will incur more overhead on the hosting machine. This will be against Derby’s design philosophy. [[BR]]
* ''' Design Considerations: ''' Let us think why such functionality will be needed:
''' a) Embedded inside application: ''' If the developer really needs to provide remote management functionality for just the database, I think it is a design flaw. The database should be removed from the application and deployed as a server which can be administered more easily. In case he wants to make the entire application remotely accessible, it will not be very difficult. [[BR]]
''' b) Embedded inside a server framework: ''' In this case I think it is the server’s responsibility to expose this functionality to the management application. How would feel if your car’s speed-o-meter were on its tires, fuel gauges on the fuel tank, torque meter on the engine etc. [[BR]]
''' c) As a database server: ''' Continuing what I said in 5.2.3 and (2) above, I think it is the responsibility of the server framework to expose this functionality to outside world, not the database engine. [[BR]]
However, keeping in mind the pressing demand by users of Derby, a middle path can be providing remote administration as "Optional Services" that can be enabled and disabled via some command line option when Derby is stared. [[BR]]
=== 6.4. Conclusion ===
[[Anchor(Feature_Conclusion)]]
[[BR]]
The priorities assigned reflect my own personal views and are open for community discussion. The final rakings will be decided by the community.
== 7.0 References ==
[[Anchor(References)]]
*Java Platform Profiling Architecture, JSR 163 at http://www.jcp.org/en/jsr/overview
*Java 2 Standard Edition, http://java.sun.com/j2se/1.5.0/docs/api/
*Apache Foundation’ Derby project http://db.apache.org/derby
*IBM’s Cloudscape developer page: http://www-128.ibm.com/developerworks/db2/zones/cloudscape/
*Java Management Extensions (JMX): http://java.sun.com/j2se/1.5.0/docs/guide/jmx/index.html
*Java Specification Request for JMX: http://www.jcp.org/en/jsr/detail?id=160
*Java Specification Request for JMX: http://www.jcp.org/en/jsr/detail?id=03