working of HICC

[Shreya Chakravarty <sh...@persistent.co.in>]

Hi eric,



I am interested in finding out how HICC extracts data from the MySQL database and does analysis on that (Which part of the code does that).

Theoretically I have read the concept of SALSA and JCDF and what are swimlanes and heatmaps used for.



Please let me know what part of code should I look at to get an idea about the analysis stuff done by HICC.



Regards,

Shreya



Below is our mail conversation:



Hi Shreya,



I think you are on the right track.  The HICC war file contains a script

called single-series-chart-javascript.jsp.  This jsp script handles all

mechanism of database to JSON graphing string.  However, the database part

is slowly phasing out, and will be replaced with chukwaRecord to JSON

graphing string.

=20

Regards,

Eric





Hi Guys,

I am new to chukwa and I am trying to understand how the analytics part work=
s in chukwa.

My understanding is:

*         DB tables are created using database_cretae_tables.sql script

*         MetricDataLoader class is invoked either from dbLoader.sh or mdl.s=
h script which in turns loads data to MySql.

*         HICC pulls data from MySql and does the analysis part

I have gone through the MetricDataLoader class and understood the code flow=
 but I am not able to find out how HICC draws data from the database for ana=
lysis.
Please let me know if my understanding is correct, some pointers in this reg=
ard would be very helpful.
Could not find much documentation regarding working of HICC.

Regards,
Shreya



Shreya Chakravarty | Module Lead - IBM BU | Persistent Systems Limited
shreya_chakravarty@persistent.co.in<ma...@persistent.co.in> | Cell: +91 97663 - 10680 | Tel: +91 (20) 391 77809
Persistent Systems - Innovation in software product design, development and delivery -  www.persistentsys.com<http://www.persistentsys.com/>


DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

Re: working of HICC

[Eric Yang <ey...@yahoo-inc.com>]

Shreya,

Hicc is a viewer of the analyzed data for data visualization only.  It does
not do data analysis.  The part that does data analysis is in
src/java/org/apache/hadoop/chukwa/analysis/salsa/fsm.  This part of the code
is triggered by ³chukwa.post.demux.data.loader² in chukwa-demux-conf.xml.
Data in the database are already analyzed prior to loading to the database.
There was a hack to aggregate data in the database, it¹s called
src/java/org/apache/hadoop/chukwa/database/Aggregator.java, and the SQL
driven by this program is feed through conf/aggregator.sql.  This was built
for prototyping purpose to show the final product of Chukwa is going to be
like.  It is not intended to run the database part in a large production
environment.
Hope this helps.

Regards,
Eric

On 5/20/10 9:03 PM, "Shreya Chakravarty"
<sh...@persistent.co.in> wrote:

> Hi eric,
>  
> I am interested in finding out how HICC extracts data from the MySQL database
> and does analysis on that (Which part of the code does that).
> Theoretically I have read the concept of SALSA and JCDF and what are swimlanes
> and heatmaps used for.
>  
> Please let me know what part of code should I look at to get an idea about the
> analysis stuff done by HICC.
>  
> Regards,
> Shreya 
>  
> Below is our mail conversation:
>  
> Hi Shreya,
>  
> I think you are on the right track.  The HICC war file contains a script
> called single-series-chart-javascript.jsp.  This jsp script handles all
> mechanism of database to JSON graphing string.  However, the database part
> is slowly phasing out, and will be replaced with chukwaRecord to JSON
> graphing string.
> =20
> Regards,
> Eric
>  
>  
>  
>  
>  
> Hi Guys,
>  
> I am new to chukwa and I am trying to understand how the analytics part work=
> s in chukwa.
>  
> My understanding is:
>  
> *        DB tables are created using database_cretae_tables.sql script
>  
> *        MetricDataLoader class is invoked either from dbLoader.sh or mdl.s=
> h script which in turns loads data to MySql.
>  
> *        HICC pulls data from MySql and does the analysis part
>  
> I have gone through the MetricDataLoader class and understood the code flow=
>  but I am not able to find out how HICC draws data from the database for ana=
> lysis.
> Please let me know if my understanding is correct, some pointers in this reg=
> ard would be very helpful.
> Could not find much documentation regarding working of HICC.
>  
> Regards,
> Shreya
>  
>  
>  
> Shreya Chakravarty | Module Lead ­ IBM BU | Persistent Systems Limited
> shreya_chakravarty@persistent.co.in <ma...@persistent.co.in>
> | Cell: +91 97663 - 10680 | Tel: +91 (20) 391 77809
> 
> Persistent Systems - Innovation in software product design, development and
> delivery -  www.persistentsys.com <http://www.persistentsys.com/>
>  
> DISCLAIMER ========== This e-mail may contain privileged and confidential
> information which is the property of Persistent Systems Ltd. It is intended
> only for the use of the individual or entity to which it is addressed. If you
> are not the intended recipient, you are not authorized to read, retain, copy,
> print, distribute or use this message. If you have received this communication
> in error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>