You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oodt.apache.org by niloofar montazeri <mo...@gmail.com> on 2011/03/18 02:23:16 UTC
FileManager and RepositoryManager
Hi!
I am having trouble with trackimg a design decision about the FileManager
component back into the code.
The decision is: "The system Interface uses the information about Product
Type policy made available by the Repository Manager in order to understand
whether or not the product should be transferred, where its root repository
path should be, and so on ."
Looking at the code of ingestProduct method and examining all the methods
that access the FileManager's repositoryManager instance, I found that the
only role the repositoryManager plays in ingestProduct method is to assign
meta-data to the product. After that, there is no such decision point for
sending or not sending the product (the product is always transferred):
public String ingestProduct(Hashtable<String, Object> productHash,
Hashtable<String, String> metadata, boolean clientTransfer)
throws VersioningException, RepositoryManagerException,
DataTransferException, CatalogException {
Product p = XmlRpcStructFactory.getProductFromXmlRpc(productHash);
try {
// first, create the product
p.setTransferStatus(Product.STATUS_TRANSFER);
catalogProduct(p);
// now add the metadata
Metadata m = new Metadata();
m.addMetadata((Hashtable)metadata);
*addMetadata(p, m);*
if (!clientTransfer) {
LOG.log(Level.FINEST,
"File Manager: ingest: no client transfer enabled, "
+ "server transfering product: [" + p.getProductName() +
"]");
// version the product
Versioner versioner = null;
try {
versioner = GenericFileManagerObjectFactory
.getVersionerFromClassName(p.getProductType().getVersioner());
versioner.createDataStoreReferences(p, m);
} catch (Exception e) {
LOG.log(Level.SEVERE,
"ingestProduct: VersioningException when versioning Product: "
+ p.getProductName() + " with Versioner "
+ p.getProductType().getVersioner() + ": Message: "
+ e.getMessage());
throw new VersioningException(e);
}
// add the newly versioned references to the data store
addProductReferences(p);
// now transfer the product
try {
dataTransfer.transferProduct(p);
// now update the product's transfer status in the data store
p.setTransferStatus(Product.STATUS_RECEIVED);
try {
catalog.setProductTransferStatus(p);
} catch (CatalogException e) {
LOG.log(Level.SEVERE, "ingestProduct: CatalogException "
+ "when updating product transfer status for Product: "
+ p.getProductName() + " Message: " + e.getMessage());
throw e;
}
} catch (Exception e) {
LOG.log(Level.SEVERE,
"ingestProduct: DataTransferException when transfering
Product: "
+ p.getProductName() + ": Message: " + e.getMessage());
throw new DataTransferException(e);
}
}
// that's it!
return p.getProductId();
} catch (Exception e) {
e.printStackTrace();
throw new CatalogException("Error ingesting product [" + p + "] : "
+ e.getMessage());
}
Am I missing something?
Thanks,
Niloofar
Re: FileManager and RepositoryManager
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hi Niloofar,
One key aspect is the ProductType. Filling it requires knowledge and information that the repository manager provides. Check out the way that the product type gets filled on ingest, and that will lead you down a path for figuring this out.
Thanks,
Chris
On Mar 17, 2011, at 6:23 PM, niloofar montazeri wrote:
> Hi!
>
> I am having trouble with trackimg a design decision about the FileManager component back into the code.
>
> The decision is: "The system Interface uses the information about Product Type policy made available by the Repository Manager in order to understand whether or not the product should be transferred, where its root repository path should be, and so on ."
>
> Looking at the code of ingestProduct method and examining all the methods that access the FileManager's repositoryManager instance, I found that the only role the repositoryManager plays in ingestProduct method is to assign meta-data to the product. After that, there is no such decision point for sending or not sending the product (the product is always transferred):
>
> public String ingestProduct(Hashtable<String, Object> productHash,
> Hashtable<String, String> metadata, boolean clientTransfer)
> throws VersioningException, RepositoryManagerException,
> DataTransferException, CatalogException {
>
> Product p = XmlRpcStructFactory.getProductFromXmlRpc(productHash);
>
> try {
> // first, create the product
> p.setTransferStatus(Product.STATUS_TRANSFER);
> catalogProduct(p);
>
> // now add the metadata
> Metadata m = new Metadata();
> m.addMetadata((Hashtable)metadata);
>
> addMetadata(p, m);
>
> if (!clientTransfer) {
> LOG.log(Level.FINEST,
> "File Manager: ingest: no client transfer enabled, "
> + "server transfering product: [" + p.getProductName() + "]");
>
> // version the product
> Versioner versioner = null;
> try {
> versioner = GenericFileManagerObjectFactory
> .getVersionerFromClassName(p.getProductType().getVersioner());
> versioner.createDataStoreReferences(p, m);
> } catch (Exception e) {
> LOG.log(Level.SEVERE,
> "ingestProduct: VersioningException when versioning Product: "
> + p.getProductName() + " with Versioner "
> + p.getProductType().getVersioner() + ": Message: "
> + e.getMessage());
> throw new VersioningException(e);
> }
>
> // add the newly versioned references to the data store
> addProductReferences(p);
>
> // now transfer the product
> try {
> dataTransfer.transferProduct(p);
> // now update the product's transfer status in the data store
> p.setTransferStatus(Product.STATUS_RECEIVED);
>
> try {
> catalog.setProductTransferStatus(p);
> } catch (CatalogException e) {
> LOG.log(Level.SEVERE, "ingestProduct: CatalogException "
> + "when updating product transfer status for Product: "
> + p.getProductName() + " Message: " + e.getMessage());
> throw e;
> }
> } catch (Exception e) {
> LOG.log(Level.SEVERE,
> "ingestProduct: DataTransferException when transfering Product: "
> + p.getProductName() + ": Message: " + e.getMessage());
> throw new DataTransferException(e);
> }
> }
>
> // that's it!
> return p.getProductId();
> } catch (Exception e) {
> e.printStackTrace();
> throw new CatalogException("Error ingesting product [" + p + "] : "
> + e.getMessage());
> }
>
>
> Am I missing something?
>
> Thanks,
> Niloofar
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: FileManager and RepositoryManager
Posted by niloofar montazeri <mo...@gmail.com>.
Thanks Cameron,
I am confused now.
>From this statement: "The system Interface uses the information about
Product Type policy made available by the Repository Manager in order to
understand whether or not the product should be transferred, where its root
repository path should be, and so on ."
Can you explain for me what this part means?
"in order to understand whether or not the product should be transferred"
I expected there to be a check on the product type and the based on the
type, either the product will be transferred or not.
But I suspect that the versioner decides whether the product should be
transferred or not. What I see in the ingestProduct method is that the
product will be transferred regardless of its type.(?)
Thanks
Niloofar
On Thu, Mar 17, 2011 at 8:30 PM, Cameron Goodale <si...@gmail.com> wrote:
> Niloofar,
> I think you are looking for a versioner. A versioner determines where a
> product is archived after ingestion. The versioner is declared in the
> product-types.xml file.
> Hope that helps.
>
> Cameron
> On Mar 17, 2011 6:24 PM, "niloofar montazeri" <
> montazeri.niloofar@gmail.com>
> wrote:
> > Hi!
> >
> > I am having trouble with trackimg a design decision about the FileManager
> > component back into the code.
> >
> > The decision is: "The system Interface uses the information about Product
> > Type policy made available by the Repository Manager in order to
> understand
> > whether or not the product should be transferred, where its root
> repository
> > path should be, and so on ."
> >
> > Looking at the code of ingestProduct method and examining all the methods
> > that access the FileManager's repositoryManager instance, I found that
> the
> > only role the repositoryManager plays in ingestProduct method is to
> assign
> > meta-data to the product. After that, there is no such decision point for
> > sending or not sending the product (the product is always transferred):
> >
> > public String ingestProduct(Hashtable<String, Object> productHash,
> > Hashtable<String, String> metadata, boolean clientTransfer)
> > throws VersioningException, RepositoryManagerException,
> > DataTransferException, CatalogException {
> >
> > Product p = XmlRpcStructFactory.getProductFromXmlRpc(productHash);
> >
> > try {
> > // first, create the product
> > p.setTransferStatus(Product.STATUS_TRANSFER);
> > catalogProduct(p);
> >
> > // now add the metadata
> > Metadata m = new Metadata();
> > m.addMetadata((Hashtable)metadata);
> >
> > *addMetadata(p, m);*
> >
> > if (!clientTransfer) {
> > LOG.log(Level.FINEST,
> > "File Manager: ingest: no client transfer enabled, "
> > + "server transfering product: [" + p.getProductName() +
> > "]");
> >
> > // version the product
> > Versioner versioner = null;
> > try {
> > versioner = GenericFileManagerObjectFactory
> > .getVersionerFromClassName(p.getProductType().getVersioner());
> > versioner.createDataStoreReferences(p, m);
> > } catch (Exception e) {
> > LOG.log(Level.SEVERE,
> > "ingestProduct: VersioningException when versioning Product: "
> > + p.getProductName() + " with Versioner "
> > + p.getProductType().getVersioner() + ": Message: "
> > + e.getMessage());
> > throw new VersioningException(e);
> > }
> >
> > // add the newly versioned references to the data store
> > addProductReferences(p);
> >
> > // now transfer the product
> > try {
> > dataTransfer.transferProduct(p);
> > // now update the product's transfer status in the data store
> > p.setTransferStatus(Product.STATUS_RECEIVED);
> >
> > try {
> > catalog.setProductTransferStatus(p);
> > } catch (CatalogException e) {
> > LOG.log(Level.SEVERE, "ingestProduct: CatalogException "
> > + "when updating product transfer status for Product: "
> > + p.getProductName() + " Message: " + e.getMessage());
> > throw e;
> > }
> > } catch (Exception e) {
> > LOG.log(Level.SEVERE,
> > "ingestProduct: DataTransferException when transfering
> > Product: "
> > + p.getProductName() + ": Message: " + e.getMessage());
> > throw new DataTransferException(e);
> > }
> > }
> >
> > // that's it!
> > return p.getProductId();
> > } catch (Exception e) {
> > e.printStackTrace();
> > throw new CatalogException("Error ingesting product [" + p + "] : "
> > + e.getMessage());
> > }
> >
> >
> > Am I missing something?
> >
> > Thanks,
> > Niloofar
>
Re: FileManager and RepositoryManager
Posted by Cameron Goodale <si...@gmail.com>.
Niloofar,
I think you are looking for a versioner. A versioner determines where a
product is archived after ingestion. The versioner is declared in the
product-types.xml file.
Hope that helps.
Cameron
On Mar 17, 2011 6:24 PM, "niloofar montazeri" <mo...@gmail.com>
wrote:
> Hi!
>
> I am having trouble with trackimg a design decision about the FileManager
> component back into the code.
>
> The decision is: "The system Interface uses the information about Product
> Type policy made available by the Repository Manager in order to
understand
> whether or not the product should be transferred, where its root
repository
> path should be, and so on ."
>
> Looking at the code of ingestProduct method and examining all the methods
> that access the FileManager's repositoryManager instance, I found that the
> only role the repositoryManager plays in ingestProduct method is to assign
> meta-data to the product. After that, there is no such decision point for
> sending or not sending the product (the product is always transferred):
>
> public String ingestProduct(Hashtable<String, Object> productHash,
> Hashtable<String, String> metadata, boolean clientTransfer)
> throws VersioningException, RepositoryManagerException,
> DataTransferException, CatalogException {
>
> Product p = XmlRpcStructFactory.getProductFromXmlRpc(productHash);
>
> try {
> // first, create the product
> p.setTransferStatus(Product.STATUS_TRANSFER);
> catalogProduct(p);
>
> // now add the metadata
> Metadata m = new Metadata();
> m.addMetadata((Hashtable)metadata);
>
> *addMetadata(p, m);*
>
> if (!clientTransfer) {
> LOG.log(Level.FINEST,
> "File Manager: ingest: no client transfer enabled, "
> + "server transfering product: [" + p.getProductName() +
> "]");
>
> // version the product
> Versioner versioner = null;
> try {
> versioner = GenericFileManagerObjectFactory
> .getVersionerFromClassName(p.getProductType().getVersioner());
> versioner.createDataStoreReferences(p, m);
> } catch (Exception e) {
> LOG.log(Level.SEVERE,
> "ingestProduct: VersioningException when versioning Product: "
> + p.getProductName() + " with Versioner "
> + p.getProductType().getVersioner() + ": Message: "
> + e.getMessage());
> throw new VersioningException(e);
> }
>
> // add the newly versioned references to the data store
> addProductReferences(p);
>
> // now transfer the product
> try {
> dataTransfer.transferProduct(p);
> // now update the product's transfer status in the data store
> p.setTransferStatus(Product.STATUS_RECEIVED);
>
> try {
> catalog.setProductTransferStatus(p);
> } catch (CatalogException e) {
> LOG.log(Level.SEVERE, "ingestProduct: CatalogException "
> + "when updating product transfer status for Product: "
> + p.getProductName() + " Message: " + e.getMessage());
> throw e;
> }
> } catch (Exception e) {
> LOG.log(Level.SEVERE,
> "ingestProduct: DataTransferException when transfering
> Product: "
> + p.getProductName() + ": Message: " + e.getMessage());
> throw new DataTransferException(e);
> }
> }
>
> // that's it!
> return p.getProductId();
> } catch (Exception e) {
> e.printStackTrace();
> throw new CatalogException("Error ingesting product [" + p + "] : "
> + e.getMessage());
> }
>
>
> Am I missing something?
>
> Thanks,
> Niloofar
Re: FileManager and RepositoryManager
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hi Niloofar,
One key aspect is the ProductType. Filling it requires knowledge and information that the repository manager provides. Check out the way that the product type gets filled on ingest, and that will lead you down a path for figuring this out.
Thanks,
Chris
On Mar 17, 2011, at 6:23 PM, niloofar montazeri wrote:
> Hi!
>
> I am having trouble with trackimg a design decision about the FileManager component back into the code.
>
> The decision is: "The system Interface uses the information about Product Type policy made available by the Repository Manager in order to understand whether or not the product should be transferred, where its root repository path should be, and so on ."
>
> Looking at the code of ingestProduct method and examining all the methods that access the FileManager's repositoryManager instance, I found that the only role the repositoryManager plays in ingestProduct method is to assign meta-data to the product. After that, there is no such decision point for sending or not sending the product (the product is always transferred):
>
> public String ingestProduct(Hashtable<String, Object> productHash,
> Hashtable<String, String> metadata, boolean clientTransfer)
> throws VersioningException, RepositoryManagerException,
> DataTransferException, CatalogException {
>
> Product p = XmlRpcStructFactory.getProductFromXmlRpc(productHash);
>
> try {
> // first, create the product
> p.setTransferStatus(Product.STATUS_TRANSFER);
> catalogProduct(p);
>
> // now add the metadata
> Metadata m = new Metadata();
> m.addMetadata((Hashtable)metadata);
>
> addMetadata(p, m);
>
> if (!clientTransfer) {
> LOG.log(Level.FINEST,
> "File Manager: ingest: no client transfer enabled, "
> + "server transfering product: [" + p.getProductName() + "]");
>
> // version the product
> Versioner versioner = null;
> try {
> versioner = GenericFileManagerObjectFactory
> .getVersionerFromClassName(p.getProductType().getVersioner());
> versioner.createDataStoreReferences(p, m);
> } catch (Exception e) {
> LOG.log(Level.SEVERE,
> "ingestProduct: VersioningException when versioning Product: "
> + p.getProductName() + " with Versioner "
> + p.getProductType().getVersioner() + ": Message: "
> + e.getMessage());
> throw new VersioningException(e);
> }
>
> // add the newly versioned references to the data store
> addProductReferences(p);
>
> // now transfer the product
> try {
> dataTransfer.transferProduct(p);
> // now update the product's transfer status in the data store
> p.setTransferStatus(Product.STATUS_RECEIVED);
>
> try {
> catalog.setProductTransferStatus(p);
> } catch (CatalogException e) {
> LOG.log(Level.SEVERE, "ingestProduct: CatalogException "
> + "when updating product transfer status for Product: "
> + p.getProductName() + " Message: " + e.getMessage());
> throw e;
> }
> } catch (Exception e) {
> LOG.log(Level.SEVERE,
> "ingestProduct: DataTransferException when transfering Product: "
> + p.getProductName() + ": Message: " + e.getMessage());
> throw new DataTransferException(e);
> }
> }
>
> // that's it!
> return p.getProductId();
> } catch (Exception e) {
> e.printStackTrace();
> throw new CatalogException("Error ingesting product [" + p + "] : "
> + e.getMessage());
> }
>
>
> Am I missing something?
>
> Thanks,
> Niloofar
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++