You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Anirban Roy (Updated) (JIRA)" <ji...@apache.org> on 2012/04/16 14:36:19 UTC
[jira] [Updated] (ZOOKEEPER-1218) zktreeutil tool enhancement
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anirban Roy updated ZOOKEEPER-1218:
-----------------------------------
Description:
============================================
zktreeutil - Zookeeper Tree Data Utility
Author: Anirban Roy (r_anirban at yahoo.com)
Organization: Yahoo Inc.
============================================
zktreeutil program is intended to manage and manipulate zk-tree data quickly,
efficiently and with ease. The utility operates on free-form ZK-tree and hence
can be used for any cluster managed by Zookeeper. Here are the basic
functionalities -
EXPORT: The whole/partial ZK-tree is exported into a XML file. This helps in
capturing a current snapshot of the data for backup/analysis. For a subtree
export, one need to specify the path to the ZK-subtree with proper option.
Since Zookeeper store binary data against znode, the data dumped on xml file
is based64 encoded with an attribute "encode=true". Optionally one may specify
not to encode data with --noencode option if data stored on zookeeper is
guaranteed to be text data.
IMPORT: The ZK (sub)tree can be imported from XML into ZK cluster. This helps in
priming the new ZK cluster with static configuration. The import can be
non-intrusive by making only additions and modifications in the existing data.
One may optionally delete existing (sub)tree before importing the new data
with --force option. The znodes which carries an attribute "encode=true" will be
decoded and written to zookeeper.
DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can
ignore some ZK-tree branches (possibly dynamic data) on reading the optional
ignore flag from XML file. Taking diff on a ZK-subtree achieved by providing
path to ZK-subtree with diff command.
UPDATE: Make the incremental changes into the live ZK-tree from saved XML,
essentially after running the diff.
DUMP: Dumps the ZK (sub)tree on the standard output device reading either from
live ZK server or XML file.
The exported ZK data into XML file can be shortened by only keeping the static
ZK nodes which are required to prime an application. The dynamic zk nodes
(created on-the-fly) can be ignored by setting a 'ignore' attribute at the root
node of the dynamic subtree (see tests/zk_sample.xml), possibly deleting all
inner ZK nodes under that. Once ignored, the whole subtree is ignored during
DIFF, UPDATE and WRITE.
Pre-requisites
--------------
1. Linux system with 2.6.X kernel.
2. Zookeeper C client library (locally built at ../../c/.libs) >= 3.X.X
3. Development build libraries (rpm packages):
a. boost-devel >= 1.32.0
b. libxml2-devel >= 2.6.26
c. log4cxx-devel >= 0.9.7-7
d. openssl-devel >= 0.9.7a
e. cppunit >= 1.12.0-2
Build instructions
------------------
1. cd into this directory
2. autoreconf -if
3. ./configure # Configure the build env
4. make # Build the tool
5. make check # Run unit-tests
6. ./src/zktreeutil --help # Usage help
Testing and usage of zktreeutil
--------------------------------
1. Run Zookeeper server locally on port 2181
2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/
3. ./src/zktreeutil --help # show help
4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zkdata_test.xml 2>/dev/null # import sample ZK tree
5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --path=/myapp/version-1.0 2>/dev/null # dump Zk subtree
5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --depth=3 2>/dev/null # dump Zk tree till certain depth
6. ./src/zktreeutil --xmlfile=zkdata_test.xml -D 2>/dev/null # dump the xml data
7. Change zkdata_test.xml with adding/deleting/chaging some nodes
8. ./src/zktreeutil -z localhost:2181 -F -x zkdata_test.xml -p /myapp/version-1.0/configuration 2>/dev/null # take a diff of changes
9. ./src/zktreeutil -z localhost:2181 -E --noencode 2>/dev/null > zk_sample2.xml # export the mofied ZK tree
10. ./src/zktreeutil -z localhost:2181 -U -x zkdata_test.xml -p /myapp/version-1.0/distributions 2>/dev/null # update with incr. changes
11. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree
For more details of usage, please see the unit tests. Hope this helps. Please
reach out to me for any bugs, comments or suggestions.
was:
==========================================
zktreeutil - Zookeeper Tree Data Utility
Author: Anirban Roy (r_anirban@yahoo.com)
Organization: Yahoo Inc.
==========================================
zktreeutil program is intended to manage and manipulate zk-tree data quickly,
efficiently and with ease. The utility operates on free-form ZK-tree and hence
can be used for any cluster managed by Zookeeper. Here are the basic
functionalities -
EXPORT: The whole/partial ZK-tree is exported into a XML file. This helps in
capturing a current snapshot of the data for backup/analysis. For a subtree
export, one need to specify the path to the ZK-subtree with proper option.
Since Zookeeper store binary data against znode, the data dumped on xml file
is based64 encoded with an attribute "encode=true". Optionally one may specify
not to encode data with --noencode option if data stored on zookeeper is
guaranteed to be text data.
IMPORT: The ZK (sub)tree can be imported from XML into ZK cluster. This helps in
priming the new ZK cluster with static configuration. The import can be
non-intrusive by making only additions and modifications in the existing data.
One may optionally delete existing (sub)tree before importing the new data
with --force option. The znodes which carries an attribute "encode=true" will be
decoded and written to zookeeper.
DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can
ignore some ZK-tree branches (possibly dynamic data) on reading the optional
ignore flag from XML file. Taking diff on a ZK-subtree achieved by providing
path to ZK-subtree with diff command.
UPDATE: Make the incremental changes into the live ZK-tree from saved XML,
essentially after running the diff.
DUMP: Dumps the ZK (sub)tree on the standard output device reading either from
live ZK server or XML file.
The exported ZK data into XML file can be shortened by only keeping the static
ZK nodes which are required to prime an application. The dynamic zk nodes
(created on-the-fly) can be ignored by setting a 'ignore' attribute at the root
node of the dynamic subtree (see tests/zk_sample.xml), possibly deleting all
inner ZK nodes under that. Once ignored, the whole subtree is ignored during
DIFF, UPDATE and WRITE.
Pre-requisites
--------------
1. Linux system with 2.6.X kernel.
2. Zookeeper C client library (locally built at ../../c/.libs) >= 3.X.X
3. Development build libraries (rpm packages):
a. boost-devel >= 1.32.0
b. libxml2-devel >= 2.7.3
c. log4cxx0100-devel >= 0.9.0
d. openssl-devel >= 0.9.7a
Build instructions
------------------
1. cd into this directory
2. autoreconf -if
3. ./configure
4. make
5. 'zktreeutil' binary created under src directory
Testing and usage of zktreeutil
--------------------------------
1. Run Zookeeper server locally on port 2181
2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/
3. ./src/zktreeutil --help # show help
4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zk_sample.xml 2>/dev/null # import sample ZK tree
5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --path=/myapp/version-1.0 2>/dev/null # dump Zk subtree
6. ./src/zktreeutil --zookeeper=localhost:2181 --dump --depth=3 2>/dev/null # dump Zk tree till certain depth
7. ./src/zktreeutil --xmlfile=zk_sample.xml -D 2>/dev/null # dump the xml data
8. Change zk_sample.xml with adding/deleting/chaging some nodes
9. ./src/zktreeutil -z localhost:2181 -F -x zk_sample.xml -p /myapp/version-1.0/configuration 2>/dev/null # take a diff of changes
10. ./src/zktreeutil -z localhost:2181 -E --noencode 2>/dev/null > zk_sample2.xml # export the mofied ZK tree
11. ./src/zktreeutil -z localhost:2181 -U -x zk_sample.xml -p /myapp/version-1.0/distributions 2>/dev/null # update with incr. changes
12. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree
Affects Version/s: (was: 3.3.3)
3.4.0
Fix Version/s: (was: 3.5.0)
3.4.0
Release Note:
1. Export/import capability of binary data
2. Null data handing
3. Export/import/dump/diff capability of subtree
4. Efficient subtree handling
5. Improved logging
6. Improved testability with unittests
7. Option to dump/export on file
8. Fix to handle new state introduced in ZOOKEEPER-1108
was:
1. Export/import capability of binary data
2. Null data handing
3. Export/import/dump/diff capability of subtree
4. Efficient subtree handling
> zktreeutil tool enhancement
> ---------------------------
>
> Key: ZOOKEEPER-1218
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1218
> Project: ZooKeeper
> Issue Type: Improvement
> Components: contrib
> Affects Versions: 3.4.0
> Environment: GNU/Linux i386/i686/x64_84
> Reporter: Anirban Roy
> Assignee: Anirban Roy
> Labels: patch
> Fix For: 3.4.0
>
> Attachments: zktreeutil.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> ============================================
> zktreeutil - Zookeeper Tree Data Utility
> Author: Anirban Roy (r_anirban at yahoo.com)
> Organization: Yahoo Inc.
> ============================================
> zktreeutil program is intended to manage and manipulate zk-tree data quickly,
> efficiently and with ease. The utility operates on free-form ZK-tree and hence
> can be used for any cluster managed by Zookeeper. Here are the basic
> functionalities -
> EXPORT: The whole/partial ZK-tree is exported into a XML file. This helps in
> capturing a current snapshot of the data for backup/analysis. For a subtree
> export, one need to specify the path to the ZK-subtree with proper option.
> Since Zookeeper store binary data against znode, the data dumped on xml file
> is based64 encoded with an attribute "encode=true". Optionally one may specify
> not to encode data with --noencode option if data stored on zookeeper is
> guaranteed to be text data.
> IMPORT: The ZK (sub)tree can be imported from XML into ZK cluster. This helps in
> priming the new ZK cluster with static configuration. The import can be
> non-intrusive by making only additions and modifications in the existing data.
> One may optionally delete existing (sub)tree before importing the new data
> with --force option. The znodes which carries an attribute "encode=true" will be
> decoded and written to zookeeper.
> DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can
> ignore some ZK-tree branches (possibly dynamic data) on reading the optional
> ignore flag from XML file. Taking diff on a ZK-subtree achieved by providing
> path to ZK-subtree with diff command.
> UPDATE: Make the incremental changes into the live ZK-tree from saved XML,
> essentially after running the diff.
> DUMP: Dumps the ZK (sub)tree on the standard output device reading either from
> live ZK server or XML file.
> The exported ZK data into XML file can be shortened by only keeping the static
> ZK nodes which are required to prime an application. The dynamic zk nodes
> (created on-the-fly) can be ignored by setting a 'ignore' attribute at the root
> node of the dynamic subtree (see tests/zk_sample.xml), possibly deleting all
> inner ZK nodes under that. Once ignored, the whole subtree is ignored during
> DIFF, UPDATE and WRITE.
> Pre-requisites
> --------------
> 1. Linux system with 2.6.X kernel.
> 2. Zookeeper C client library (locally built at ../../c/.libs) >= 3.X.X
> 3. Development build libraries (rpm packages):
> a. boost-devel >= 1.32.0
> b. libxml2-devel >= 2.6.26
> c. log4cxx-devel >= 0.9.7-7
> d. openssl-devel >= 0.9.7a
> e. cppunit >= 1.12.0-2
> Build instructions
> ------------------
> 1. cd into this directory
> 2. autoreconf -if
> 3. ./configure # Configure the build env
> 4. make # Build the tool
> 5. make check # Run unit-tests
> 6. ./src/zktreeutil --help # Usage help
> Testing and usage of zktreeutil
> --------------------------------
> 1. Run Zookeeper server locally on port 2181
> 2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/
> 3. ./src/zktreeutil --help # show help
> 4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zkdata_test.xml 2>/dev/null # import sample ZK tree
> 5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --path=/myapp/version-1.0 2>/dev/null # dump Zk subtree
> 5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --depth=3 2>/dev/null # dump Zk tree till certain depth
> 6. ./src/zktreeutil --xmlfile=zkdata_test.xml -D 2>/dev/null # dump the xml data
> 7. Change zkdata_test.xml with adding/deleting/chaging some nodes
> 8. ./src/zktreeutil -z localhost:2181 -F -x zkdata_test.xml -p /myapp/version-1.0/configuration 2>/dev/null # take a diff of changes
> 9. ./src/zktreeutil -z localhost:2181 -E --noencode 2>/dev/null > zk_sample2.xml # export the mofied ZK tree
> 10. ./src/zktreeutil -z localhost:2181 -U -x zkdata_test.xml -p /myapp/version-1.0/distributions 2>/dev/null # update with incr. changes
> 11. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree
> For more details of usage, please see the unit tests. Hope this helps. Please
> reach out to me for any bugs, comments or suggestions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira