[Db-derby Wiki] Update of "High Level Design" by SanketSharma

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Db-derby Wiki" for change notification.

The following page has been changed by SanketSharma:
http://wiki.apache.org/db-derby/High_Level_Design

New page:
= THIS PAGE IS BEING UPDATED =
= Apache Derby =

== Draft Proposal ==
== High - Level Design for JMX Extensions==

== Document Contents ==

1. [#Revision_History Revision History][[BR]]
2. [#Disclaimer Disclaimer and Legal Information][[BR]]
3. [#About About this document][[BR]]
	3.1 [#Purpose Purpose][[BR]]
	3.2 [#Audience Intended Audience][[BR]]
	3.3 [#Notations Notations and Conventions][[BR]]
	3.4 [#Structure Structure of this document][[BR]]
4. [#Assumptions Assumptions and Dependencies][[BR]]
5. [#Analysis High Level Design][[BR]]
	5.1 [#Overview JMX Overview][[BR]]
	5.2 [#Current Component Structure][[BR]]
		5.2.1 [#Embedded Monitoring Service][[BR]]
		5.2.2 [#Embedded_Server Monitor MBean][[BR]]

== 1. Revision History ==
[[Anchor(Revision_History)]]

||<rowstyle="font-weight: bold;	font-size: 12pt;color: white; text-align: center; background-color: #828DA6; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans, sanserif; padding: 6px; border: none;"> Version || Version Information || Date || Modified by || Comments ||
|| 1.0 || Document Created || Tuesday, June 27, 2006 || Sanket Sharma || Comments are welcome! ||

== 2. Disclaimer and Legal Information ==
[[Anchor(Disclaimer)]]
Copyright 2005 The Apache Software Foundation or its licensors, as applicable. 

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0. 

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 

== 3. About this Document ==
[[Anchor(About)]]
=== 3.1. Purpose ===
[[Anchor(Purpose)]]
         This document identifies the components and the interfaces required for implementing management extensions to Apache Derby in a modular fashion. The document is not intended to describe detailed implementation associated with a specific module.

[[BR]]
=== 3.2. Intended Audience ===
[[Anchor(Audience)]]
          The intended audience includes a wide community of those interested in developing monitoring and management modules for the Apache Derby. 
[[BR]]
=== 3.3. Notation Conventions ===
[[Anchor(Notations)]]
          The documentation set uses the following conventions.
[[BR]]
          ||<rowstyle="font-weight: bold;	font-size: 12pt;color: white; text-align: center; background-color: #828DA6; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans, sanserif; padding: 6px; border: none;"> Convention || Explanation || Example ||
          ||<|5> {{{ Monospace }}} || Filename || {{{ MyFile.java }}} ||
          || Directory Names and Path Names || {{{ \MyDir\MyDir2 }}} ||
          || Commands and Command line options || {{{ ecl-02 }}} ||
          || Method names, Classes, Interfaces in running text || Use the {{{ createService() }}} method to... ||
          || Parameters or other placeholders || {{{ doSomething(int val1, int val2) }}}||
          || ''' {{{ Monospace bold }}} ''' || User input || ''' {{{ [C:] dir }}} '''||
          || '' Italics '' || Emphasis; Introducing or defining terms || The term '' access '' takes as its subject ||
          || [ ]     || Optional Items || {{{ -Fa[c] }}} Indicates these command-line options: {{{ -Fa }}} and {{{ -Fac }}} ||
          ||<|4> {|} ||<|4> Choice of one item from a selection of two or more items. || -aX{K | W | P} Indicates these command-line options: ||
          || {{{ -aXK }}} ||
          || {{{ -aXW }}} ||
          || {{{ -aXP }}} ||
[[BR]]
=== 3.4. Structure of this document ===
[[Anchor(Structure)]]
          This section is divided into seven sections. Section one to four provide basic information about legal stuff and notational conventions used in this document and can be skipped over without loss of any details. Section five begins with a very high level architectural overview of JMX and JMX enabled applications followed by architectural overview of Derby. This section also identified how JMX components will be plugged into existing architecture and describes indivisual JMX components in greater detail. 
Section 6 describes the communication scenario between these components. Section seven discusses the issues that might aries following a JMX implementation. Section eight enlists the sources of information for this document. 
[[BR]]
=== 3.5. A Note  ===
[[Anchor(Note)]]
           This document is still evolving! If you feel this document lacks some detail which ought to have been here, please feel free to drop me mail !
 
[[BR]]
== 4. Assumptions and Dependencies ==
[[Anchor(Assumptions)]]
[[BR]]
	  It is assumed that the reader is fairly experienced with Java and know what classes, interfaces, attributes, events, constructors are. A basic understanding of how introspection works will be helpful in understanding of JMX MBeans. It is assumed that the reader is familiar with JMX although, for those who are not, a brief introduction has been included with this document.
	    This document follows Java Coding conventions defined at http://java.sun.com/docs/codeconv/html/CodeConvTOC.doc.html
[[BR]]
[[Anchor(Analysis)]]
== 5. High Level Design ==


	  As the project is about adding JMX features to Derby, I thought it would greatly benifit the reader if the High Level Design is accompanied by a brief overview of JMX architecture. So this section gives the reader a bird's eye view of the JMX world without getting into any deatails. For a detailed reading of JMX, please refer to the resources and links mentioned in [#References References]

[[Anchor(Overview)]]
=== 5.1. JMX Overview ===
	The Java Management extensions (also called the JMX specification) define an architecture, design patterns, APIs, and the services for applications and network management & monitoring in the Java programming language. Java Management Extensions, also known as JMX, were developed as Java Community Process Java Specification Request (JSR) 3 to address management needs of applications written in Java while being compatible with existing management technologies like SNMP etc.
	JMX specification provides Java developers across all industries means to instrument Java code, create smart Java agents, implement distributed management middleware and managers, and smoothly integrate these solutions into existing management and monitoring systems.

	The JMX specification describes the JMX architecture in three distinct layers, each with well defined functions and interfaces to communicate with other layers. A JMX application can be viewed as the following stack:

	Each of these three layers are described below:

	''' 1) The Instrumentation Layer: ''' This is the layer that is closest to the resource being managed. A resource is managable if it is writtent in Java or provides Java wrappers and has been instrumented using one or more MBeans. A MBean is a normal Java Bean except for the fact that it exposes the management interface of a resource according in accordance with the JMX specification.  Each MBean exposes a piece of management configuration or functionality of an underlying resource. A MBean can be classified as standard or dynamic. To a management application, they appear all the same but to a developer, they are quite different both in terms of the way they are developed and the flexibility they offer. Standard MBeans are developed using design patterns as specified in JMX specification e.g. the management interfaces must have same name as the classname with MBean appended to it, get() and set() methods for attributes etc. Dynamic MBeans on the other hand, expose their mana
 gement interface via specific interfaces e.g. by implementing DynamicMBean interfaces and by creating MBeanInfo objects.

The specification identifies three key components at this level:

	a.) MBeans: The specification defines a MBean as a concrete class that is instrumented by either implementing its own management interface or by implementing DynamicMBean interface. It may also include an implementation of NotificationBroadcaster interface. There are four types of MBeans:
		a.1) Standard MBeans : Define and implement their own management interface. Simplest to design and implement.[[BR]]
		a.2) Dynamic MBeans: Implement a predefined interface but expose their management interface at run time. These are the most flexible.[[BR]]
		a.3) Model MBeans: Are the most powerful, configurable and flexible. They only provide a generic dynamic MBean which can be customized to expose any management interface and instrument any resource at runtime.[[BR]]
		a.4) Open MBeans: Are dynamic MBeans that use basic data types and are self described.[[BR]]

	b.) Notification Model: The specification provides a notification model based on the Java event model that lets applications register and recieve notifications of all events that occur in a MBean of its interest.[[BR]]

	c.) Metadata classes: The specification define classes that are used for both introspection of standard MBeans and self description of all dynamic MBeans. Together, these classes describe the management interface of a MBean in terms of Attributes, Constructors, Operations and their Parameters. [[BR]]

	For a detailed discussion of these topics, please refer to the resources mentioned at the end of this section.

	'''2. Agent Level: ''' The agent level provides specifications for implementing Agents. Agents are JMX components that expose the functionality of MBeans to outside applications. This level builds upon instrumentation level to define a standardized agent to manage JMX managable resources. Agents are usally located on the same machines as the MBeans they control, although, this is not a requirement. The JMX agent consists of following to components:[[BR]]
		
		a.) MBean Server: Acts a registry for all MBeans. All management operations applied to resource have to go through the MBean server. The MBean server provides a standardized interface for querying, instantiating and accessing MBeans. Any resource that needs to be managed from outside the JVM, needs to be registered with the a MBean Server.[[BR]]
		b.) Agent Services: The JMX specification defines the following services that any compliant agent must provide the the registered MBeans:

		b.1.) Dynamic loading services allow agent to instantiate MBeans using Java classes and native libraries dynamically loaded from the network.
		b.2.) Monitoring capabilities for attribute values in MBeans notifies the listeners upon detecting certain conditions.
		b.3.) A Timer service that can send notifications at predetermined intervals
		b.4) A relation service that defines relations between MBeans and maintains the relations.

	'''3. Distributed Level: ''' The Distributed services level provides interfaces for implementing JMX managers. As defined by the specification, this level defines management interfaces and components that can operate on agents. Typicall, these services can expose the management view of a JMX agent and its MBeans by mapping their semantic meaning into constructs of data rich protocol e.g. HTML or SNMP.

=== 5.2 Component Structure ===
	The basic design principle followed while designing this architecture was to have a MBean, a Model MBean per service instance that is loaded into the monitor. The monitor being a service itself, is managed via System manager MBean.

	The diagram below gives a high level component structure of Derby along with JMX components loaded.



	
	Blue components in the diagram services that are started by the monitor. The monitor itself,along with its utility services is represented by shade of green. MBeans, (Model MBeans) are represented by shades of pink and are labeled as "service name" MBean. The arrows represents call backs and returns from JMX Agent to MBean and from MBean to managed resource. 
	As I said earlier, the basic design idea for adding extensions to Derby is to have an Model MBean per service that is loaded in Derby. Each MBean will manage a logical aspect of the system. Please note that the diagram is only intended to give the user a overview of the architecture. Several details, including modules and services have been intentionally left out to highlight the architecture. 

	The choice of this architecture was based upon following factors:

	1.) Miniumum changes to exisiting code base are required due to the use of ModelMBeans. Model MBeans keep a reference of the object they are managing and implement all operations as callbacks. While reviewing the code, I felt that quite a lot existing functionality can very easily be exposed through Model MBeans.[[BR]]
	2.) Extensible: The design is inherantly extensible. If new services are added by developers, they can be simply plugged into existing architecture with minimal changes. The architecture neither restricts the development of new services or burdens the developer with writing of extra code for exposing to management client. Infact most of the code required for a Model MBean can be generated with help of tools.

=== 5.3 Indivisual Components ===
	The main components to be delivered as a result of this project are going to be MBeans, therefore I will focus this discussion on MBeans. JMX agent will be discussed only when needed.
	The following components are identified for the JMX extensions:[[BR]]

	1.) System Management Service[[BR]]
	2.) Monitor MBean[[BR]]
	3.) JDBC(Driver) MBean[[BR]]
	4.) Authentication MBean[[BR]]
	5.) Database MBean[[BR]]
	(More MBeans to be added)[[BR]]


	''' 5.3.1. System Management Service:''' This service is nothing but a JMX Agent that is booted up, which in turn will register MBeans. The Agent performs two functions - Provides the MBean server which acts as central repository & factory interfaces to create our Model MBeans and provides the agent that lets us perform remote administration. The Management service will be not loaded in a normal derby startup. The management modules will be started only when Derby is supplied with an argument like derby.management=true. If you are worried about security and local vs. remote access or cases when derby is started in a server framework, please see the issues section for details. The 

	'''5.3.2. Monitor MBean: ''' The Monitor MBean will be a Model MBean which will manage the Derby Monitor. This MBean will be an instance of javax.management.modelmbean.RequiredModelMBean class and will be customized at run time to provide the management functionality of the requried MBean. By calling a setManagedResource method on the RequiredModelMBean object and passing the reference of the Monitor as an argument, the MBean will be associated with the monitor. The actual task of mapping attribute and operation calls to Monitor will be done by MBeanInfoSupport object, which will also be created at runtime. <<Insert actual code with comments>>

			// Code after the Agent has been started	//[[BR]]

	//Step 1: Obtain an instance of resource [[BR]]
	
	Monitor myMon= Monitor.getMonitor();[[BR]]

	//Step 2: Create a RequiredModeMBeanInfo object [[BR]]

	ModelMBean monitorMBean= new RequiredModelMBean();
[[BR]]
	//Step 3: Create necessary Metadata and descriptor class[[BR]]

	ModelMBeanAttributeInfo attributeInfo = new ModelMBeanAttributeInfo()
		attributeInfo = new ModelMBeanAttributeInfo(  "EngineVersion",
  								"java.lang.String",
  								"Version of database engine",
  								true,
  								false,
  								false,
							);
	DescriptorSupport desc = attributeInfo.getDescriptor(  );[[BR]]
	desc.setField("getMethod", "getEngineVersion");[[BR]]
	.........[[BR]]
	.......[[BR]]

	//Step 4: Create metadata resource[[BR]]

	ModelMBeanInfo mbeanInfo = new ModelMBeanInfoSupport(Monitor.getClass().getName(  ),
    								"Derby Monitor MBean",
    								attributeInfo,
    								null,
    								null,
    								null
  								);[[BR]]

	//Step 5: Set the metadata of RequiredModelMBean to the metadata for the resource (from Step 3) and the resource to be managed through the ModelMBean interface. [[BR]]


  			monitorMBean.setModelMBeanInfo(mbeanInfo);[[BR]]
  			monitorMBean.setManagedResource(monitor, "ObjectReference");[[BR]]
  			MBeanServer mbeanServer = MBeanServerFactory.createMBeanServer(  );[[BR]]
  			mbeanServer.registerMBean(queueModelMBean, objName);[[BR]]
	// . . .[[BR]]

	[[BR]]
'''5.3.3. JDBC(Driver)/Authentication/Database/Other MBeans:'''  All other MBeans will be implemented using the same procedure as described for the Monitor MBean above.