You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by Cameron Goodale <si...@gmail.com> on 2011/07/23 03:43:57 UTC
Re: [jira] [Updated] (OODT-306) File Manager Tools for BASH and TCSH
So I have been mulling this over since I wrote this JIRA issue and I think I
will break fmquery into:
lucene_query and sql_query since the query_tool that is being called under
the code is able to query both, and prefixing fm seems a bit redundant.
-Cameron
On Fri, Jul 22, 2011 at 6:03 PM, Cameron Goodale (JIRA) <ji...@apache.org>wrote:
>
> [
> https://issues.apache.org/jira/browse/OODT-306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>
> Cameron Goodale updated OODT-306:
> ---------------------------------
>
> Description:
> *Chris Mattmann* has been writing some really useful shell alias commands
> to help with some of the really repetitive tasks that tend to appear when
> using the File Manager component. This issue will be used to share and
> commit 2 files that contain environment settings that can be easily sourced,
> or just copied into a users .bashrc or .tcshrc startup file.
>
> The files will contain the scripting commands and some basic documentation.
>
> I plan to drop the files into oodt/filemgr/src/main/resources since they
> really only apply to the filemanager.
>
> Here is my initial working Docu-Code (for BASH) I plan to clean up and
> commit.
>
> {code}
> ##### Copy and Paste this Block into the .bashrc of your deployment user
> account ##########
> #
> # The following aliases must be used within a filemgr installation's
> # bin directory since relative pathing is being used. This block also
> # assumes that the filemgr is running on port 9000 (the default port of
> filemgr)
> #
> alias fmquery="java
> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
> -Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.QueryTool --url
> http://localhost:9000 --lucene -query "
> #
> alias fmdel="java
> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
> -Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.DeleteProduct
> --fileManagerUrl http://localhost:9000 --read"
> #
> alias metdump="java -Djava.ext.dirs=../lib
> org.apache.oodt.cas.filemgr.tools.MetadataDumper --url $FILEMGR_URL --out .
> --productId "
> #
> ######## END OF BLOCK #######
> {code}
> h2. Previous Commands Explained
>
> h3. fmquery
> This will execute a Lucene Query against the filemanager catalog and return
> a list of ProductIDs. These Product IDs can then be piped into the fmdel
> alias to remove all products that match the query criteria.
>
> *Usage:*
> - Return the ProductIds to the terminal window -
> fmquery "ProductType:ISMRawData"
>
> - Pipe the ProductIds to a file -
> fmquery "ProductType:ISMRawData" > ProductID_List.txt
>
>
> h3. fmdel
> Use this command to delete a single product from the catalog. This will
> remove the lucene index entry from the catalog and will remove the archived
> product from disk. (If you just want to see what products will be removed
> from the archive, then you will need to add the --nocommit parameter to the
> fmdel command alias after the --read param)
>
> *Usage:*
> - Remove one product from the catalog
> fmdel 0691ee17-96ec-11e0-8556-3b3ca12e4f64
>
> - Remove a set of products from the catalog using the fmquery command
> [fmquery Metadata_Key:"Matching_Key_Value" | fmdel]
> For example: To remove all Products with a GenericFile ProductType use
> fmquery ProductType:"GenericFile" | fmdel
>
> h3. metdump
> Just added to the list. This will take in a product_id and dump out the
> product's metadata into the current directory. *This one needs a little
> more testing and docs.*
>
> Comments are welcome.
>
> was:
> Chris Mattmann has been writing some really useful shell alias commands to
> help with some of the really repetitive tasks that tend to appear when using
> the File Manager component. This issue will be used to share and commit 2
> files that contain environment settings that can be easily sourced, or just
> copied into a users .bashrc or .tcshrc startup file.
>
> The files will contain the scripting commands and some basic documentation.
>
> I plan to drop the files into oodt/filemgr/src/main/resources since they
> really only apply to the filemanager.
>
> Here is my initial working Docu-Code (for BASH) I plan to clean up and
> commit.
>
>
> ##### Copy and Paste this Block into the .bashrc of your deployment user
> account ##########
> #
> # The following aliases must be used within a filemgr installation's
> # bin directory since relative pathing is being used. This block also
> # assumes that the filemgr is running on port 9000 (the default port of
> filemgr)
> #
> alias fmquery="java
> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
> -Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.QueryTool --url
> http://localhost:9000 --lucene -query "
> #
> alias fmdel="java
> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
> -Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.DeleteProduct
> --fileManagerUrl http://localhost:9000 --read"
> #
> alias metdump="java -Djava.ext.dirs=../lib
> org.apache.oodt.cas.filemgr.tools.MetadataDumper --url $FILEMGR_URL -
> -out . --productId "
> #
> ######## END OF BLOCK #######
> Previous Commands Explained
>
> fmquery - This will execute a Lucene Query against the filemanager catalog
> and return a list of ProductIDs. These Product IDs can then be piped into
> the fmdel alias to remove all products that match the query criteria.
>
> Usage:
> - Return the ProductIds to the terminal window -
> fmquery "ProductType:ISMRawData"
>
> - Pipe the ProductIds to a file -
> fmquery "ProductType:ISMRawData" > ProductID_List.txt
>
>
> fmdel - Use this command to delete a single product from the catalog. This
> will remove the lucene index entry from the catalog and will remove the
> archived product from disk. (If you just want to see what products will be
> removed from the archive, then you will need to add the --nocommit parameter
> to the fmdel command alias after the --read param)
>
> Usage:
> - Remove one product from the catalog
> fmdel 0691ee17-96ec-11e0-8556-3b3ca12e4f64
>
> - Remove a set of products from the catalog using the fmquery command
> [fmquery Metadata_Key:"Matching_Key_Value" | fmdel]
> For example: To remove all Products with a GenericFile ProductType use
> fmquery ProductType:"GenericFile" | fmdel
>
> metdump - Just added to the list. This will take in a product_id and dump
> out the product's metadata into the current directory. This one needs a
> little more testing and docs.
>
> Comments are welcome.
>
>
> Cleaned up the formatting and wrapped the example in a proper code block
>
> > File Manager Tools for BASH and TCSH
> > ------------------------------------
> >
> > Key: OODT-306
> > URL: https://issues.apache.org/jira/browse/OODT-306
> > Project: OODT
> > Issue Type: Task
> > Components: file manager
> > Environment: *nix OS
> > Reporter: Cameron Goodale
> > Assignee: Cameron Goodale
> > Priority: Trivial
> >
> > *Chris Mattmann* has been writing some really useful shell alias commands
> to help with some of the really repetitive tasks that tend to appear when
> using the File Manager component. This issue will be used to share and
> commit 2 files that contain environment settings that can be easily sourced,
> or just copied into a users .bashrc or .tcshrc startup file.
> > The files will contain the scripting commands and some basic
> documentation.
> > I plan to drop the files into oodt/filemgr/src/main/resources since
> they really only apply to the filemanager.
> > Here is my initial working Docu-Code (for BASH) I plan to clean up and
> commit.
> > {code}
> > ##### Copy and Paste this Block into the .bashrc of your deployment user
> account ##########
> > #
> > # The following aliases must be used within a filemgr installation's
> > # bin directory since relative pathing is being used. This block also
> > # assumes that the filemgr is running on port 9000 (the default port of
> filemgr)
> > #
> > alias fmquery="java
> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
> -Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.QueryTool --url
> http://localhost:9000 --lucene -query "
> > #
> > alias fmdel="java
> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
> -Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.DeleteProduct
> --fileManagerUrl http://localhost:9000 --read"
> > #
> > alias metdump="java -Djava.ext.dirs=../lib
> org.apache.oodt.cas.filemgr.tools.MetadataDumper --url $FILEMGR_URL --out .
> --productId "
> > #
> > ######## END OF BLOCK #######
> > {code}
> > h2. Previous Commands Explained
> > h3. fmquery
> > This will execute a Lucene Query against the filemanager catalog and
> return a list of ProductIDs. These Product IDs can then be piped into the
> fmdel alias to remove all products that match the query criteria.
> > *Usage:*
> > - Return the ProductIds to the terminal window -
> > fmquery "ProductType:ISMRawData"
> > - Pipe the ProductIds to a file -
> > fmquery "ProductType:ISMRawData" > ProductID_List.txt
> > h3. fmdel
> > Use this command to delete a single product from the catalog. This will
> remove the lucene index entry from the catalog and will remove the archived
> product from disk. (If you just want to see what products will be removed
> from the archive, then you will need to add the --nocommit parameter to the
> fmdel command alias after the --read param)
> > *Usage:*
> > - Remove one product from the catalog
> > fmdel 0691ee17-96ec-11e0-8556-3b3ca12e4f64
> > - Remove a set of products from the catalog using the fmquery command
> > [fmquery Metadata_Key:"Matching_Key_Value" | fmdel]
> > For example: To remove all Products with a GenericFile ProductType
> use
> > fmquery ProductType:"GenericFile" | fmdel
> > h3. metdump
> > Just added to the list. This will take in a product_id and dump out the
> product's metadata into the current directory. *This one needs a little
> more testing and docs.*
> > Comments are welcome.
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
--
Sent from a Tin Can attached to a String
Re: [jira] [Updated] (OODT-306) File Manager Tools for BASH and TCSH
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
+1 from me...
Cheers,
Chris
On Jul 22, 2011, at 6:43 PM, Cameron Goodale wrote:
> So I have been mulling this over since I wrote this JIRA issue and I think I
> will break fmquery into:
>
> lucene_query and sql_query since the query_tool that is being called under
> the code is able to query both, and prefixing fm seems a bit redundant.
>
>
> -Cameron
>
>
> On Fri, Jul 22, 2011 at 6:03 PM, Cameron Goodale (JIRA) <ji...@apache.org>wrote:
>
>>
>> [
>> https://issues.apache.org/jira/browse/OODT-306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>>
>> Cameron Goodale updated OODT-306:
>> ---------------------------------
>>
>> Description:
>> *Chris Mattmann* has been writing some really useful shell alias commands
>> to help with some of the really repetitive tasks that tend to appear when
>> using the File Manager component. This issue will be used to share and
>> commit 2 files that contain environment settings that can be easily sourced,
>> or just copied into a users .bashrc or .tcshrc startup file.
>>
>> The files will contain the scripting commands and some basic documentation.
>>
>> I plan to drop the files into oodt/filemgr/src/main/resources since they
>> really only apply to the filemanager.
>>
>> Here is my initial working Docu-Code (for BASH) I plan to clean up and
>> commit.
>>
>> {code}
>> ##### Copy and Paste this Block into the .bashrc of your deployment user
>> account ##########
>> #
>> # The following aliases must be used within a filemgr installation's
>> # bin directory since relative pathing is being used. This block also
>> # assumes that the filemgr is running on port 9000 (the default port of
>> filemgr)
>> #
>> alias fmquery="java
>> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
>> -Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.QueryTool --url
>> http://localhost:9000 --lucene -query "
>> #
>> alias fmdel="java
>> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
>> -Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.DeleteProduct
>> --fileManagerUrl http://localhost:9000 --read"
>> #
>> alias metdump="java -Djava.ext.dirs=../lib
>> org.apache.oodt.cas.filemgr.tools.MetadataDumper --url $FILEMGR_URL --out .
>> --productId "
>> #
>> ######## END OF BLOCK #######
>> {code}
>> h2. Previous Commands Explained
>>
>> h3. fmquery
>> This will execute a Lucene Query against the filemanager catalog and return
>> a list of ProductIDs. These Product IDs can then be piped into the fmdel
>> alias to remove all products that match the query criteria.
>>
>> *Usage:*
>> - Return the ProductIds to the terminal window -
>> fmquery "ProductType:ISMRawData"
>>
>> - Pipe the ProductIds to a file -
>> fmquery "ProductType:ISMRawData" > ProductID_List.txt
>>
>>
>> h3. fmdel
>> Use this command to delete a single product from the catalog. This will
>> remove the lucene index entry from the catalog and will remove the archived
>> product from disk. (If you just want to see what products will be removed
>> from the archive, then you will need to add the --nocommit parameter to the
>> fmdel command alias after the --read param)
>>
>> *Usage:*
>> - Remove one product from the catalog
>> fmdel 0691ee17-96ec-11e0-8556-3b3ca12e4f64
>>
>> - Remove a set of products from the catalog using the fmquery command
>> [fmquery Metadata_Key:"Matching_Key_Value" | fmdel]
>> For example: To remove all Products with a GenericFile ProductType use
>> fmquery ProductType:"GenericFile" | fmdel
>>
>> h3. metdump
>> Just added to the list. This will take in a product_id and dump out the
>> product's metadata into the current directory. *This one needs a little
>> more testing and docs.*
>>
>> Comments are welcome.
>>
>> was:
>> Chris Mattmann has been writing some really useful shell alias commands to
>> help with some of the really repetitive tasks that tend to appear when using
>> the File Manager component. This issue will be used to share and commit 2
>> files that contain environment settings that can be easily sourced, or just
>> copied into a users .bashrc or .tcshrc startup file.
>>
>> The files will contain the scripting commands and some basic documentation.
>>
>> I plan to drop the files into oodt/filemgr/src/main/resources since they
>> really only apply to the filemanager.
>>
>> Here is my initial working Docu-Code (for BASH) I plan to clean up and
>> commit.
>>
>>
>> ##### Copy and Paste this Block into the .bashrc of your deployment user
>> account ##########
>> #
>> # The following aliases must be used within a filemgr installation's
>> # bin directory since relative pathing is being used. This block also
>> # assumes that the filemgr is running on port 9000 (the default port of
>> filemgr)
>> #
>> alias fmquery="java
>> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
>> -Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.QueryTool --url
>> http://localhost:9000 --lucene -query "
>> #
>> alias fmdel="java
>> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
>> -Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.DeleteProduct
>> --fileManagerUrl http://localhost:9000 --read"
>> #
>> alias metdump="java -Djava.ext.dirs=../lib
>> org.apache.oodt.cas.filemgr.tools.MetadataDumper --url $FILEMGR_URL -
>> -out . --productId "
>> #
>> ######## END OF BLOCK #######
>> Previous Commands Explained
>>
>> fmquery - This will execute a Lucene Query against the filemanager catalog
>> and return a list of ProductIDs. These Product IDs can then be piped into
>> the fmdel alias to remove all products that match the query criteria.
>>
>> Usage:
>> - Return the ProductIds to the terminal window -
>> fmquery "ProductType:ISMRawData"
>>
>> - Pipe the ProductIds to a file -
>> fmquery "ProductType:ISMRawData" > ProductID_List.txt
>>
>>
>> fmdel - Use this command to delete a single product from the catalog. This
>> will remove the lucene index entry from the catalog and will remove the
>> archived product from disk. (If you just want to see what products will be
>> removed from the archive, then you will need to add the --nocommit parameter
>> to the fmdel command alias after the --read param)
>>
>> Usage:
>> - Remove one product from the catalog
>> fmdel 0691ee17-96ec-11e0-8556-3b3ca12e4f64
>>
>> - Remove a set of products from the catalog using the fmquery command
>> [fmquery Metadata_Key:"Matching_Key_Value" | fmdel]
>> For example: To remove all Products with a GenericFile ProductType use
>> fmquery ProductType:"GenericFile" | fmdel
>>
>> metdump - Just added to the list. This will take in a product_id and dump
>> out the product's metadata into the current directory. This one needs a
>> little more testing and docs.
>>
>> Comments are welcome.
>>
>>
>> Cleaned up the formatting and wrapped the example in a proper code block
>>
>>> File Manager Tools for BASH and TCSH
>>> ------------------------------------
>>>
>>> Key: OODT-306
>>> URL: https://issues.apache.org/jira/browse/OODT-306
>>> Project: OODT
>>> Issue Type: Task
>>> Components: file manager
>>> Environment: *nix OS
>>> Reporter: Cameron Goodale
>>> Assignee: Cameron Goodale
>>> Priority: Trivial
>>>
>>> *Chris Mattmann* has been writing some really useful shell alias commands
>> to help with some of the really repetitive tasks that tend to appear when
>> using the File Manager component. This issue will be used to share and
>> commit 2 files that contain environment settings that can be easily sourced,
>> or just copied into a users .bashrc or .tcshrc startup file.
>>> The files will contain the scripting commands and some basic
>> documentation.
>>> I plan to drop the files into oodt/filemgr/src/main/resources since
>> they really only apply to the filemanager.
>>> Here is my initial working Docu-Code (for BASH) I plan to clean up and
>> commit.
>>> {code}
>>> ##### Copy and Paste this Block into the .bashrc of your deployment user
>> account ##########
>>> #
>>> # The following aliases must be used within a filemgr installation's
>>> # bin directory since relative pathing is being used. This block also
>>> # assumes that the filemgr is running on port 9000 (the default port of
>> filemgr)
>>> #
>>> alias fmquery="java
>> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
>> -Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.QueryTool --url
>> http://localhost:9000 --lucene -query "
>>> #
>>> alias fmdel="java
>> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
>> -Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.DeleteProduct
>> --fileManagerUrl http://localhost:9000 --read"
>>> #
>>> alias metdump="java -Djava.ext.dirs=../lib
>> org.apache.oodt.cas.filemgr.tools.MetadataDumper --url $FILEMGR_URL --out .
>> --productId "
>>> #
>>> ######## END OF BLOCK #######
>>> {code}
>>> h2. Previous Commands Explained
>>> h3. fmquery
>>> This will execute a Lucene Query against the filemanager catalog and
>> return a list of ProductIDs. These Product IDs can then be piped into the
>> fmdel alias to remove all products that match the query criteria.
>>> *Usage:*
>>> - Return the ProductIds to the terminal window -
>>> fmquery "ProductType:ISMRawData"
>>> - Pipe the ProductIds to a file -
>>> fmquery "ProductType:ISMRawData" > ProductID_List.txt
>>> h3. fmdel
>>> Use this command to delete a single product from the catalog. This will
>> remove the lucene index entry from the catalog and will remove the archived
>> product from disk. (If you just want to see what products will be removed
>> from the archive, then you will need to add the --nocommit parameter to the
>> fmdel command alias after the --read param)
>>> *Usage:*
>>> - Remove one product from the catalog
>>> fmdel 0691ee17-96ec-11e0-8556-3b3ca12e4f64
>>> - Remove a set of products from the catalog using the fmquery command
>>> [fmquery Metadata_Key:"Matching_Key_Value" | fmdel]
>>> For example: To remove all Products with a GenericFile ProductType
>> use
>>> fmquery ProductType:"GenericFile" | fmdel
>>> h3. metdump
>>> Just added to the list. This will take in a product_id and dump out the
>> product's metadata into the current directory. *This one needs a little
>> more testing and docs.*
>>> Comments are welcome.
>>
>> --
>> This message is automatically generated by JIRA.
>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>
>>
>>
>
>
> --
>
> Sent from a Tin Can attached to a String
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++