You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by "Starch, Michael D (388L)" <Mi...@jpl.nasa.gov> on 2011/05/24 01:14:23 UTC
Re: Changes to Elements.xml file
Chris,
Is there some sort of coding standards document that I should be abiding by, or does oodt just follow standard java "best practices"? This is the kind of information that seems like it would be easier for me to look up rather than ask on the developer list.
Finally, how can I access the archived dev-list mailings?
Thanks,
Michael
On May 19, 2011, at 11:06 PM, Mattmann, Chris A (388J) wrote:
> Hi Michael,
>
> I'd be happy to reply below, but yes I'd suggest initially CC'ing dev@oodt.apache.org. Giving this information is great, but I prefer the community to benefit from these types of discussions (and it means that I or you or Albert or Cecilia can point others to this question if it is asked again rather than typing the same answer again or fwd'ing something private). Also no worries about researching it -- you can ask questions on that list -- researched or not. We welcome feedback from the community.
>
> Answers inline below:
>
> On May 19, 2011, at 2:44 PM, Starch, Michael D (388L) wrote:
>
>> Chris,
>>
>> In order to create tables in the new database architecture that Brian Foster set up for use, we need the ability to specify some information beyond that which is stored in elements.xml. We wish to store this information as part of PCS, rather than continually haggling back and forth with our DBA in order to get this set in an ad-hoc manner.
>>
>> The data we need to store is:
>>
>> -Is it a vector type?
>> -Data-type
>> -Max data size
>> and potentially a few more items.
>
> Gotcha.
>
>>
>> As it stands now, Brian has used the DCElement tag in elements.xml file in order to store some of this information but not all of it. Thus it seems natural that the rest of this information gets stored along side it.
>>
>> There seem to be three ways to accomplish this:
>>
>> 1. Add more information to the DCElement tag,
>> 2. Add additional tags to the element file containing this information
>> 3. Create a sperate file to store this information (Not part of PCS)
>
> I've got a 4th option.
>
> 4. Write a new ValidationLayer implementation, that extends XMLValidationLayer, but adds your desired information to elements.xml. For that extra information, I'd favor your proposed option #2 -- I think it's cleaner.
>
>>
>> As head of oodt, we were looking for you input as to which option to choose, or if you wish this question to be brought to dev@oodt.apache, where can I research this information before asking the question on that list?
>>
>> Below is a brief analysis of each option, from our perspective.
>>
>> Thanks,
>>
>> Michael Starch
>>
>> 1. Add more information to the DCElements tag.
>> -I believe Brain said this tag was unused, so he "borrowed" it to suit this purpose.
>> -Adding more information to the tag would "hijack" this tag to a further extent.
>> -The changes could be made locally to our ColumnBasedDataSourceCatalog in the manner Brian already used.
>> -If the DCElement is used ouside of the new use Brian invented, this change could affect the intended (original) use
>
> I don't like this one (nor do I think others will) since it hijacks dcElement (which is supposed to be a mapping to a Dublin Core element name, rather than used for other information).
>
>>
>> 2. Add more tags to elements.xml
>> -Most elegant solution, as each data field has a tag specified to hold exactly it
>> -No need to use tags which were originally intended for some other purpose
>> -It is an architectural change, changing the format of the elements.xml file
>> -Must be done carefully to prevent us from deviating from apache-oodt when it is not necessary to do so.
>
> +1 for this option.
>
>>
>> 3. A separate file
>> -Creates data/configuration duplication, as some of these fields must stay (in some form) in elements.xml
>> -Separates similar pieces of data
>> -Doesn't alter PCS as it exists now
>
> This is fine too, but I prefer #2.
>
>>
>> We favor number 1 or number 2 as it keeps similar data together, and feels like it is the most logical place to store this information.
>>
>> I personally favor number 2, as it does not use tags for unintended purposes, but I do not wish to make a change that affects our compatibility with apache code. Hence your input.
>>
>
> +1 to your preference. I think the way to accomplish it is to write a YourExtendedValidationLayer extends XMLValidationLayer and that provides a customized:
> 1. YourElementClass extends Element
> - provides extra props (getters+setters) that you want to leverage
> 2. implementations of the SerDe for your specialized elements.xml that includes those extra tags
> 3. provides the ability to access this ValidationLayer information and Element information in ColumnBasedDataSourceCatalog.
>
> BTW, I think it would be great to bake up patches for Apache OODT for what you are working on, including ColumnBasedDataSourceCatalog (already there but in a branch, would be nice to forward port to trunk, and your proposed ValidationLayer and Element extensions). Thoughts? Do you have the time? I think the community at Apache would sincerely appreciate the effort.
>
> Cheers,
> Chris
>
> P.S. Would you be OK with me forwarding this thread to dev@oodt.a.o? I think there's some good info here that I'd hate to be lost in the ether...
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> Phone: +1 (818) 354-8810
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
Re: Changes to Elements.xml file
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Michael,
> TypeHandling sounds like it could fix the issue, but I can't really say until I find out how it works (sounds like there is little documentation on the subject).
Yep I plan on trying to fix that along with OODT-147.
>
> One major concern of mine is: Even if TypeHandling enforce the element to be formatted a certain way, does it allow me to access to the child fields of a complex element (no longer a simple type) from within the ValidationLayer?
Hmm, I don't think it's the solution to to your ValidationLayer that you're writing. There's still a need for that.
>
> The cost of even reading in these types into PCS is paid for by the fact that, with my current solution, I can access that data and use that inside PCS to develop tools and other things.
+1.
> If we switch to TypeHandling, and I lose that ability, then the solution is no better off that the unparsed tag solution Brian suggested (which is indeed far quicker to implement).
Yeah I wasn't proposing TypeHandling as a replacement for what you are doing, at least at the moment. I think there is a lot of overlap though with what I intend to do with OODT-147, which is:
1. understand element types more precisely.
2. map that understanding to classes that both handle how elements go in (as queries), and come out (as metadata).
3. validation layer updates needed to handle that.
>
> I am open to any new ideas, but I might need some more information.
No probs, I don't have anything concrete yet, but as I develop it, I hope to collaborate with you on what you're doing and however far along you are then.
Cheers,
Chris
> On May 25, 2011, at 11:45 PM, Mattmann, Chris A (388J) wrote:
>
>> Hey Guys,
>>
>> This made me think of the following issue I filed:
>>
>> https://issues.apache.org/jira/browse/OODT-147
>>
>> I think we can deal with a lot of this there. What do you think of my proposal, Michael? Brian, I know what you think :-) Because you and I talked about it on the issue.
>>
>> Cheers,
>> Chris
>>
>> On May 24, 2011, at 7:08 AM, holenoter wrote:
>>
>>>
>>> hey michael,
>>>
>>> if you just need to store this information in the elements.xml file then just add new tags . . . you don't need to change any code:
>>>
>>> <element id="some-id" name="some-name">
>>> (current elements here)
>>> ....
>>> <!-- unparsed tags -->
>>> <maxSize>10</maxSize>
>>> </element>
>>>
>>> Also . . . the dublin core stuff is fine for peate . . . if you want to create a patch for the trunk at apache that is a different story, but peate will never need the dublin core feature, so the patch as it currently stands is perfectly okay
>>>
>>> -brian
>>>
>>> On May 24, 2011, at 07:18 AM, "Starch, Michael D (388L)" <Mi...@jpl.nasa.gov> wrote:
>>>
>>>> Brian,
>>>>
>>>> PEATE is busy as always, but doing well.
>>>>
>>>> We need to be able to store this information somewhere so we can just tell the DBA to use these values. As it stands now we do a lot of back-and-forth to get the right values to support the database, and we need to stream-line the process by having this information pre-selected by the project. Due to the data's similarity to the other data in the elements file we selected this as the place to put it and Chris agreed.
>>>>
>>>> Does this answer your questions?
>>>>
>>>> -Michael
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On May 24, 2011, at 1:47 AM, holenoter wrote:
>>>>
>>>>> hey michael,
>>>>>
>>>>> hope everything is going well for peate . . .
>>>>>
>>>>> yup dublin core element was used because of the time crunch before i left . . . being it hasn't been used up to this point anywhere, good odds it probably won't be used anytime in the "peate future" . . . however, dublin core is the wrong place to put this feature because dublin core is used to describe the element itself, not its value's type . . . pretty much chris's recommendation is the way to go, however if you are going to be working towards a patch to the trunk, instead of extending Element as chris recommended just add a "type" member variable to the existing Element class and then modify the XMLValidationLayer and have it read a <type> tag into that "type" member variable in Element . . . i think there is a strong case for having the Element understand to some degree its value's type . . . not sure why you need more information in your elements.xml file than i already added? . . . the database should be doing this validation (otherwise your code will have to become double, float, long, int, etc... aware) . . . the current ColumnBasedDataSourceCatalog handles all these types already via Oracle (just set any double, float, long, int, etc... to a <dcElement>NUMBER</dcElement> in the elements.xml file. . .you have to set Oracle double precision anyway even if you where to treat it as a double in the filemgr) . . . if you want to just keep the information in one spot (i.e. elements.xml) than just add unused tags to the elements.xml file with the information you want (XMLValidationLayer will just ignore these tags) . . . for example you can add a <maxSize> tag to elements.xml without affecting the filemgr. . .as far as validating max size and the like, this really should be happening in your metadata extractors.
>>>>>
>>>>> -brian
>>>>>
>>>>> On May 24, 2011, at 12:59 AM, "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> wrote:
>>>>>
>>>>>> Hi Michael,
>>>>>>
>>>>>> Thanks for your email!
>>>>>>
>>>>>> You can find the OODT Apache archives here:
>>>>>>
>>>>>> http://mail-archives.apache.org/mod_mbox/oodt-dev/
>>>>>>
>>>>>> As for standards, best practices are certainly part of them but we do have a few standards that we try and stick to including Dublin Core, ISO-11179, RDF, etc., when we are defining metadata elements. For your purposes in elements.xml in CAS, the biggest approach is to use unique, namespaced identifiers for element IDs, product type IDs, etc.
>>>>>>
>>>>>> HTH!
>>>>>>
>>>>>> Cheers,
>>>>>> Chris
>>>>>>
>>>>>> On May 23, 2011, at 1:14 PM, Starch, Michael D (388L) wrote:
>>>>>>
>>>>>>> Chris,
>>>>>>>
>>>>>>> Is there some sort of coding standards document that I should be abiding by, or does oodt just follow standard java "best practices"? This is the kind of information that seems like it would be easier for me to look up rather than ask on the developer list.
>>>>>>>
>>>>>>> Finally, how can I access the archived dev-list mailings?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Michael
>>>>>>>
>>>>>>>
>>>>>>> On May 19, 2011, at 11:06 PM, Mattmann, Chris A (388J) wrote:
>>>>>>>
>>>>>>>> Hi Michael,
>>>>>>>>
>>>>>>>> I'd be happy to reply below, but yes I'd suggest initially CC'ing dev@oodt.apache.org. Giving this information is great, but I prefer the community to benefit from these types of discussions (and it means that I or you or Albert or Cecilia can point others to this question if it is asked again rather than typing the same answer again or fwd'ing something private). Also no worries about researching it -- you can ask questions on that list -- researched or not. We welcome feedback from the community.
>>>>>>>>
>>>>>>>> Answers inline below:
>>>>>>>>
>>>>>>>> On May 19, 2011, at 2:44 PM, Starch, Michael D (388L) wrote:
>>>>>>>>
>>>>>>>>> Chris,
>>>>>>>>>
>>>>>>>>> In order to create tables in the new database architecture that Brian Foster set up for use, we need the ability to specify some information beyond that which is stored in elements.xml. We wish to store this information as part of PCS, rather than continually haggling back and forth with our DBA in order to get this set in an ad-hoc manner.
>>>>>>>>>
>>>>>>>>> The data we need to store is:
>>>>>>>>>
>>>>>>>>> -Is it a vector type?
>>>>>>>>> -Data-type
>>>>>>>>> -Max data size
>>>>>>>>> and potentially a few more items.
>>>>>>>>
>>>>>>>> Gotcha.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> As it stands now, Brian has used the DCElement tag in elements.xml file in order to store some of this information but not all of it. Thus it seems natural that the rest of this information gets stored along side it.
>>>>>>>>>
>>>>>>>>> There seem to be three ways to accomplish this:
>>>>>>>>>
>>>>>>>>> 1. Add more information to the DCElement tag,
>>>>>>>>> 2. Add additional tags to the element file containing this information
>>>>>>>>> 3. Create a sperate file to store this information (Not part of PCS)
>>>>>>>>
>>>>>>>> I've got a 4th option.
>>>>>>>>
>>>>>>>> 4. Write a new ValidationLayer implementation, that extends XMLValidationLayer, but adds your desired information to elements.xml. For that extra information, I'd favor your proposed option #2 -- I think it's cleaner.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> As head of oodt, we were looking for you input as to which option to choose, or if you wish this question to be brought to dev@oodt.apache, where can I research this information before asking the question on that list?
>>>>>>>>>
>>>>>>>>> Below is a brief analysis of each option, from our perspective.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Michael Starch
>>>>>>>>>
>>>>>>>>> 1. Add more information to the DCElements tag.
>>>>>>>>> -I believe Brain said this tag was unused, so he "borrowed" it to suit this purpose.
>>>>>>>>> -Adding more information to the tag would "hijack" this tag to a further extent.
>>>>>>>>> -The changes could be made locally to our ColumnBasedDataSourceCatalog in the manner Brian already used.
>>>>>>>>> -If the DCElement is used ouside of the new use Brian invented, this change could affect the intended (original) use
>>>>>>>>
>>>>>>>> I don't like this one (nor do I think others will) since it hijacks dcElement (which is supposed to be a mapping to a Dublin Core element name, rather than used for other information).
>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2. Add more tags to elements.xml
>>>>>>>>> -Most elegant solution, as each data field has a tag specified to hold exactly it
>>>>>>>>> -No need to use tags which were originally intended for some other purpose
>>>>>>>>> -It is an architectural change, changing the format of the elements.xml file
>>>>>>>>> -Must be done carefully to prevent us from deviating from apache-oodt when it is not necessary to do so.
>>>>>>>>
>>>>>>>> +1 for this option.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> 3. A separate file
>>>>>>>>> -Creates data/configuration duplication, as some of these fields must stay (in some form) in elements.xml
>>>>>>>>> -Separates similar pieces of data
>>>>>>>>> -Doesn't alter PCS as it exists now
>>>>>>>>
>>>>>>>> This is fine too, but I prefer #2.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> We favor number 1 or number 2 as it keeps similar data together, and feels like it is the most logical place to store this information.
>>>>>>>>>
>>>>>>>>> I personally favor number 2, as it does not use tags for unintended purposes, but I do not wish to make a change that affects our compatibility with apache code. Hence your input.
>>>>>>>>>
>>>>>>>>
>>>>>>>> +1 to your preference. I think the way to accomplish it is to write a YourExtendedValidationLayer extends XMLValidationLayer and that provides a customized:
>>>>>>>> 1. YourElementClass extends Element
>>>>>>>> - provides extra props (getters+setters) that you want to leverage
>>>>>>>> 2. implementations of the SerDe for your specialized elements.xml that includes those extra tags
>>>>>>>> 3. provides the ability to access this ValidationLayer information and Element information in ColumnBasedDataSourceCatalog.
>>>>>>>>
>>>>>>>> BTW, I think it would be great to bake up patches for Apache OODT for what you are working on, including ColumnBasedDataSourceCatalog (already there but in a branch, would be nice to forward port to trunk, and your proposed ValidationLayer and Element extensions). Thoughts? Do you have the time? I think the community at Apache would sincerely appreciate the effort.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Chris
>>>>>>>>
>>>>>>>> P.S. Would you be OK with me forwarding this thread to dev@oodt.a.o? I think there's some good info here that I'd hate to be lost in the ether...
>>>>>>>>
>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>> Chris Mattmann, Ph.D.
>>>>>>>> Senior Computer Scientist
>>>>>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>>>>>> Office: 171-266B, Mailstop: 171-246
>>>>>>>> Email: chris.a.mattmann@nasa.gov
>>>>>>>> WWW: http://sunset.usc.edu/~mattmann/
>>>>>>>> Phone: +1 (818) 354-8810
>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>> Adjunct Assistant Professor, Computer Science Department
>>>>>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>> Chris Mattmann, Ph.D.
>>>>>> Senior Computer Scientist
>>>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>>>> Office: 171-266B, Mailstop: 171-246
>>>>>> Email: chris.a.mattmann@nasa.gov
>>>>>> WWW: http://sunset.usc.edu/~mattmann/
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>> Adjunct Assistant Professor, Computer Science Department
>>>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>
>>>>
>>
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Changes to Elements.xml file
Posted by "Starch, Michael D (388L)" <Mi...@jpl.nasa.gov>.
Chris,
TypeHandling sounds like it could fix the issue, but I can't really say until I find out how it works (sounds like there is little documentation on the subject).
One major concern of mine is: Even if TypeHandling enforce the element to be formatted a certain way, does it allow me to access to the child fields of a complex element (no longer a simple type) from within the ValidationLayer?
The cost of even reading in these types into PCS is paid for by the fact that, with my current solution, I can access that data and use that inside PCS to develop tools and other things. If we switch to TypeHandling, and I lose that ability, then the solution is no better off that the unparsed tag solution Brian suggested (which is indeed far quicker to implement).
I am open to any new ideas, but I might need some more information.
-Michael
On May 25, 2011, at 11:45 PM, Mattmann, Chris A (388J) wrote:
> Hey Guys,
>
> This made me think of the following issue I filed:
>
> https://issues.apache.org/jira/browse/OODT-147
>
> I think we can deal with a lot of this there. What do you think of my proposal, Michael? Brian, I know what you think :-) Because you and I talked about it on the issue.
>
> Cheers,
> Chris
>
> On May 24, 2011, at 7:08 AM, holenoter wrote:
>
>>
>> hey michael,
>>
>> if you just need to store this information in the elements.xml file then just add new tags . . . you don't need to change any code:
>>
>> <element id="some-id" name="some-name">
>> (current elements here)
>> ....
>> <!-- unparsed tags -->
>> <maxSize>10</maxSize>
>> </element>
>>
>> Also . . . the dublin core stuff is fine for peate . . . if you want to create a patch for the trunk at apache that is a different story, but peate will never need the dublin core feature, so the patch as it currently stands is perfectly okay
>>
>> -brian
>>
>> On May 24, 2011, at 07:18 AM, "Starch, Michael D (388L)" <Mi...@jpl.nasa.gov> wrote:
>>
>>> Brian,
>>>
>>> PEATE is busy as always, but doing well.
>>>
>>> We need to be able to store this information somewhere so we can just tell the DBA to use these values. As it stands now we do a lot of back-and-forth to get the right values to support the database, and we need to stream-line the process by having this information pre-selected by the project. Due to the data's similarity to the other data in the elements file we selected this as the place to put it and Chris agreed.
>>>
>>> Does this answer your questions?
>>>
>>> -Michael
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On May 24, 2011, at 1:47 AM, holenoter wrote:
>>>
>>>> hey michael,
>>>>
>>>> hope everything is going well for peate . . .
>>>>
>>>> yup dublin core element was used because of the time crunch before i left . . . being it hasn't been used up to this point anywhere, good odds it probably won't be used anytime in the "peate future" . . . however, dublin core is the wrong place to put this feature because dublin core is used to describe the element itself, not its value's type . . . pretty much chris's recommendation is the way to go, however if you are going to be working towards a patch to the trunk, instead of extending Element as chris recommended just add a "type" member variable to the existing Element class and then modify the XMLValidationLayer and have it read a <type> tag into that "type" member variable in Element . . . i think there is a strong case for having the Element understand to some degree its value's type . . . not sure why you need more information in your elements.xml file than i already added? . . . the database should be doing this validation (otherwise your code will have to become double, float, long, int, etc... aware) . . . the current ColumnBasedDataSourceCatalog handles all these types already via Oracle (just set any double, float, long, int, etc... to a <dcElement>NUMBER</dcElement> in the elements.xml file. . .you have to set Oracle double precision anyway even if you where to treat it as a double in the filemgr) . . . if you want to just keep the information in one spot (i.e. elements.xml) than just add unused tags to the elements.xml file with the information you want (XMLValidationLayer will just ignore these tags) . . . for example you can add a <maxSize> tag to elements.xml without affecting the filemgr. . .as far as validating max size and the like, this really should be happening in your metadata extractors.
>>>>
>>>> -brian
>>>>
>>>> On May 24, 2011, at 12:59 AM, "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> wrote:
>>>>
>>>>> Hi Michael,
>>>>>
>>>>> Thanks for your email!
>>>>>
>>>>> You can find the OODT Apache archives here:
>>>>>
>>>>> http://mail-archives.apache.org/mod_mbox/oodt-dev/
>>>>>
>>>>> As for standards, best practices are certainly part of them but we do have a few standards that we try and stick to including Dublin Core, ISO-11179, RDF, etc., when we are defining metadata elements. For your purposes in elements.xml in CAS, the biggest approach is to use unique, namespaced identifiers for element IDs, product type IDs, etc.
>>>>>
>>>>> HTH!
>>>>>
>>>>> Cheers,
>>>>> Chris
>>>>>
>>>>> On May 23, 2011, at 1:14 PM, Starch, Michael D (388L) wrote:
>>>>>
>>>>>> Chris,
>>>>>>
>>>>>> Is there some sort of coding standards document that I should be abiding by, or does oodt just follow standard java "best practices"? This is the kind of information that seems like it would be easier for me to look up rather than ask on the developer list.
>>>>>>
>>>>>> Finally, how can I access the archived dev-list mailings?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Michael
>>>>>>
>>>>>>
>>>>>> On May 19, 2011, at 11:06 PM, Mattmann, Chris A (388J) wrote:
>>>>>>
>>>>>>> Hi Michael,
>>>>>>>
>>>>>>> I'd be happy to reply below, but yes I'd suggest initially CC'ing dev@oodt.apache.org. Giving this information is great, but I prefer the community to benefit from these types of discussions (and it means that I or you or Albert or Cecilia can point others to this question if it is asked again rather than typing the same answer again or fwd'ing something private). Also no worries about researching it -- you can ask questions on that list -- researched or not. We welcome feedback from the community.
>>>>>>>
>>>>>>> Answers inline below:
>>>>>>>
>>>>>>> On May 19, 2011, at 2:44 PM, Starch, Michael D (388L) wrote:
>>>>>>>
>>>>>>>> Chris,
>>>>>>>>
>>>>>>>> In order to create tables in the new database architecture that Brian Foster set up for use, we need the ability to specify some information beyond that which is stored in elements.xml. We wish to store this information as part of PCS, rather than continually haggling back and forth with our DBA in order to get this set in an ad-hoc manner.
>>>>>>>>
>>>>>>>> The data we need to store is:
>>>>>>>>
>>>>>>>> -Is it a vector type?
>>>>>>>> -Data-type
>>>>>>>> -Max data size
>>>>>>>> and potentially a few more items.
>>>>>>>
>>>>>>> Gotcha.
>>>>>>>
>>>>>>>>
>>>>>>>> As it stands now, Brian has used the DCElement tag in elements.xml file in order to store some of this information but not all of it. Thus it seems natural that the rest of this information gets stored along side it.
>>>>>>>>
>>>>>>>> There seem to be three ways to accomplish this:
>>>>>>>>
>>>>>>>> 1. Add more information to the DCElement tag,
>>>>>>>> 2. Add additional tags to the element file containing this information
>>>>>>>> 3. Create a sperate file to store this information (Not part of PCS)
>>>>>>>
>>>>>>> I've got a 4th option.
>>>>>>>
>>>>>>> 4. Write a new ValidationLayer implementation, that extends XMLValidationLayer, but adds your desired information to elements.xml. For that extra information, I'd favor your proposed option #2 -- I think it's cleaner.
>>>>>>>
>>>>>>>>
>>>>>>>> As head of oodt, we were looking for you input as to which option to choose, or if you wish this question to be brought to dev@oodt.apache, where can I research this information before asking the question on that list?
>>>>>>>>
>>>>>>>> Below is a brief analysis of each option, from our perspective.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Michael Starch
>>>>>>>>
>>>>>>>> 1. Add more information to the DCElements tag.
>>>>>>>> -I believe Brain said this tag was unused, so he "borrowed" it to suit this purpose.
>>>>>>>> -Adding more information to the tag would "hijack" this tag to a further extent.
>>>>>>>> -The changes could be made locally to our ColumnBasedDataSourceCatalog in the manner Brian already used.
>>>>>>>> -If the DCElement is used ouside of the new use Brian invented, this change could affect the intended (original) use
>>>>>>>
>>>>>>> I don't like this one (nor do I think others will) since it hijacks dcElement (which is supposed to be a mapping to a Dublin Core element name, rather than used for other information).
>>>>>>>
>>>>>>>>
>>>>>>>> 2. Add more tags to elements.xml
>>>>>>>> -Most elegant solution, as each data field has a tag specified to hold exactly it
>>>>>>>> -No need to use tags which were originally intended for some other purpose
>>>>>>>> -It is an architectural change, changing the format of the elements.xml file
>>>>>>>> -Must be done carefully to prevent us from deviating from apache-oodt when it is not necessary to do so.
>>>>>>>
>>>>>>> +1 for this option.
>>>>>>>
>>>>>>>>
>>>>>>>> 3. A separate file
>>>>>>>> -Creates data/configuration duplication, as some of these fields must stay (in some form) in elements.xml
>>>>>>>> -Separates similar pieces of data
>>>>>>>> -Doesn't alter PCS as it exists now
>>>>>>>
>>>>>>> This is fine too, but I prefer #2.
>>>>>>>
>>>>>>>>
>>>>>>>> We favor number 1 or number 2 as it keeps similar data together, and feels like it is the most logical place to store this information.
>>>>>>>>
>>>>>>>> I personally favor number 2, as it does not use tags for unintended purposes, but I do not wish to make a change that affects our compatibility with apache code. Hence your input.
>>>>>>>>
>>>>>>>
>>>>>>> +1 to your preference. I think the way to accomplish it is to write a YourExtendedValidationLayer extends XMLValidationLayer and that provides a customized:
>>>>>>> 1. YourElementClass extends Element
>>>>>>> - provides extra props (getters+setters) that you want to leverage
>>>>>>> 2. implementations of the SerDe for your specialized elements.xml that includes those extra tags
>>>>>>> 3. provides the ability to access this ValidationLayer information and Element information in ColumnBasedDataSourceCatalog.
>>>>>>>
>>>>>>> BTW, I think it would be great to bake up patches for Apache OODT for what you are working on, including ColumnBasedDataSourceCatalog (already there but in a branch, would be nice to forward port to trunk, and your proposed ValidationLayer and Element extensions). Thoughts? Do you have the time? I think the community at Apache would sincerely appreciate the effort.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Chris
>>>>>>>
>>>>>>> P.S. Would you be OK with me forwarding this thread to dev@oodt.a.o? I think there's some good info here that I'd hate to be lost in the ether...
>>>>>>>
>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>> Chris Mattmann, Ph.D.
>>>>>>> Senior Computer Scientist
>>>>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>>>>> Office: 171-266B, Mailstop: 171-246
>>>>>>> Email: chris.a.mattmann@nasa.gov
>>>>>>> WWW: http://sunset.usc.edu/~mattmann/
>>>>>>> Phone: +1 (818) 354-8810
>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>> Adjunct Assistant Professor, Computer Science Department
>>>>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>> Chris Mattmann, Ph.D.
>>>>> Senior Computer Scientist
>>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>>> Office: 171-266B, Mailstop: 171-246
>>>>> Email: chris.a.mattmann@nasa.gov
>>>>> WWW: http://sunset.usc.edu/~mattmann/
>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>> Adjunct Assistant Professor, Computer Science Department
>>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>
>>>
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
Re: Changes to Elements.xml file
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Guys,
This made me think of the following issue I filed:
https://issues.apache.org/jira/browse/OODT-147
I think we can deal with a lot of this there. What do you think of my proposal, Michael? Brian, I know what you think :-) Because you and I talked about it on the issue.
Cheers,
Chris
On May 24, 2011, at 7:08 AM, holenoter wrote:
>
> hey michael,
>
> if you just need to store this information in the elements.xml file then just add new tags . . . you don't need to change any code:
>
> <element id="some-id" name="some-name">
> (current elements here)
> ....
> <!-- unparsed tags -->
> <maxSize>10</maxSize>
> </element>
>
> Also . . . the dublin core stuff is fine for peate . . . if you want to create a patch for the trunk at apache that is a different story, but peate will never need the dublin core feature, so the patch as it currently stands is perfectly okay
>
> -brian
>
> On May 24, 2011, at 07:18 AM, "Starch, Michael D (388L)" <Mi...@jpl.nasa.gov> wrote:
>
>> Brian,
>>
>> PEATE is busy as always, but doing well.
>>
>> We need to be able to store this information somewhere so we can just tell the DBA to use these values. As it stands now we do a lot of back-and-forth to get the right values to support the database, and we need to stream-line the process by having this information pre-selected by the project. Due to the data's similarity to the other data in the elements file we selected this as the place to put it and Chris agreed.
>>
>> Does this answer your questions?
>>
>> -Michael
>>
>>
>>
>>
>>
>>
>>
>> On May 24, 2011, at 1:47 AM, holenoter wrote:
>>
>>> hey michael,
>>>
>>> hope everything is going well for peate . . .
>>>
>>> yup dublin core element was used because of the time crunch before i left . . . being it hasn't been used up to this point anywhere, good odds it probably won't be used anytime in the "peate future" . . . however, dublin core is the wrong place to put this feature because dublin core is used to describe the element itself, not its value's type . . . pretty much chris's recommendation is the way to go, however if you are going to be working towards a patch to the trunk, instead of extending Element as chris recommended just add a "type" member variable to the existing Element class and then modify the XMLValidationLayer and have it read a <type> tag into that "type" member variable in Element . . . i think there is a strong case for having the Element understand to some degree its value's type . . . not sure why you need more information in your elements.xml file than i already added? . . . the database should be doing this validation (otherwise your code will have to become double, float, long, int, etc... aware) . . . the current ColumnBasedDataSourceCatalog handles all these types already via Oracle (just set any double, float, long, int, etc... to a <dcElement>NUMBER</dcElement> in the elements.xml file. . .you have to set Oracle double precision anyway even if you where to treat it as a double in the filemgr) . . . if you want to just keep the information in one spot (i.e. elements.xml) than just add unused tags to the elements.xml file with the information you want (XMLValidationLayer will just ignore these tags) . . . for example you can add a <maxSize> tag to elements.xml without affecting the filemgr. . .as far as validating max size and the like, this really should be happening in your metadata extractors.
>>>
>>> -brian
>>>
>>> On May 24, 2011, at 12:59 AM, "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> wrote:
>>>
>>>> Hi Michael,
>>>>
>>>> Thanks for your email!
>>>>
>>>> You can find the OODT Apache archives here:
>>>>
>>>> http://mail-archives.apache.org/mod_mbox/oodt-dev/
>>>>
>>>> As for standards, best practices are certainly part of them but we do have a few standards that we try and stick to including Dublin Core, ISO-11179, RDF, etc., when we are defining metadata elements. For your purposes in elements.xml in CAS, the biggest approach is to use unique, namespaced identifiers for element IDs, product type IDs, etc.
>>>>
>>>> HTH!
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>> On May 23, 2011, at 1:14 PM, Starch, Michael D (388L) wrote:
>>>>
>>>> > Chris,
>>>> >
>>>> > Is there some sort of coding standards document that I should be abiding by, or does oodt just follow standard java "best practices"? This is the kind of information that seems like it would be easier for me to look up rather than ask on the developer list.
>>>> >
>>>> > Finally, how can I access the archived dev-list mailings?
>>>> >
>>>> > Thanks,
>>>> >
>>>> > Michael
>>>> >
>>>> >
>>>> > On May 19, 2011, at 11:06 PM, Mattmann, Chris A (388J) wrote:
>>>> >
>>>> >> Hi Michael,
>>>> >>
>>>> >> I'd be happy to reply below, but yes I'd suggest initially CC'ing dev@oodt.apache.org. Giving this information is great, but I prefer the community to benefit from these types of discussions (and it means that I or you or Albert or Cecilia can point others to this question if it is asked again rather than typing the same answer again or fwd'ing something private). Also no worries about researching it -- you can ask questions on that list -- researched or not. We welcome feedback from the community.
>>>> >>
>>>> >> Answers inline below:
>>>> >>
>>>> >> On May 19, 2011, at 2:44 PM, Starch, Michael D (388L) wrote:
>>>> >>
>>>> >>> Chris,
>>>> >>>
>>>> >>> In order to create tables in the new database architecture that Brian Foster set up for use, we need the ability to specify some information beyond that which is stored in elements.xml. We wish to store this information as part of PCS, rather than continually haggling back and forth with our DBA in order to get this set in an ad-hoc manner.
>>>> >>>
>>>> >>> The data we need to store is:
>>>> >>>
>>>> >>> -Is it a vector type?
>>>> >>> -Data-type
>>>> >>> -Max data size
>>>> >>> and potentially a few more items.
>>>> >>
>>>> >> Gotcha.
>>>> >>
>>>> >>>
>>>> >>> As it stands now, Brian has used the DCElement tag in elements.xml file in order to store some of this information but not all of it. Thus it seems natural that the rest of this information gets stored along side it.
>>>> >>>
>>>> >>> There seem to be three ways to accomplish this:
>>>> >>>
>>>> >>> 1. Add more information to the DCElement tag,
>>>> >>> 2. Add additional tags to the element file containing this information
>>>> >>> 3. Create a sperate file to store this information (Not part of PCS)
>>>> >>
>>>> >> I've got a 4th option.
>>>> >>
>>>> >> 4. Write a new ValidationLayer implementation, that extends XMLValidationLayer, but adds your desired information to elements.xml. For that extra information, I'd favor your proposed option #2 -- I think it's cleaner.
>>>> >>
>>>> >>>
>>>> >>> As head of oodt, we were looking for you input as to which option to choose, or if you wish this question to be brought to dev@oodt.apache, where can I research this information before asking the question on that list?
>>>> >>>
>>>> >>> Below is a brief analysis of each option, from our perspective.
>>>> >>>
>>>> >>> Thanks,
>>>> >>>
>>>> >>> Michael Starch
>>>> >>>
>>>> >>> 1. Add more information to the DCElements tag.
>>>> >>> -I believe Brain said this tag was unused, so he "borrowed" it to suit this purpose.
>>>> >>> -Adding more information to the tag would "hijack" this tag to a further extent.
>>>> >>> -The changes could be made locally to our ColumnBasedDataSourceCatalog in the manner Brian already used.
>>>> >>> -If the DCElement is used ouside of the new use Brian invented, this change could affect the intended (original) use
>>>> >>
>>>> >> I don't like this one (nor do I think others will) since it hijacks dcElement (which is supposed to be a mapping to a Dublin Core element name, rather than used for other information).
>>>> >>
>>>> >>>
>>>> >>> 2. Add more tags to elements.xml
>>>> >>> -Most elegant solution, as each data field has a tag specified to hold exactly it
>>>> >>> -No need to use tags which were originally intended for some other purpose
>>>> >>> -It is an architectural change, changing the format of the elements.xml file
>>>> >>> -Must be done carefully to prevent us from deviating from apache-oodt when it is not necessary to do so.
>>>> >>
>>>> >> +1 for this option.
>>>> >>
>>>> >>>
>>>> >>> 3. A separate file
>>>> >>> -Creates data/configuration duplication, as some of these fields must stay (in some form) in elements.xml
>>>> >>> -Separates similar pieces of data
>>>> >>> -Doesn't alter PCS as it exists now
>>>> >>
>>>> >> This is fine too, but I prefer #2.
>>>> >>
>>>> >>>
>>>> >>> We favor number 1 or number 2 as it keeps similar data together, and feels like it is the most logical place to store this information.
>>>> >>>
>>>> >>> I personally favor number 2, as it does not use tags for unintended purposes, but I do not wish to make a change that affects our compatibility with apache code. Hence your input.
>>>> >>>
>>>> >>
>>>> >> +1 to your preference. I think the way to accomplish it is to write a YourExtendedValidationLayer extends XMLValidationLayer and that provides a customized:
>>>> >> 1. YourElementClass extends Element
>>>> >> - provides extra props (getters+setters) that you want to leverage
>>>> >> 2. implementations of the SerDe for your specialized elements.xml that includes those extra tags
>>>> >> 3. provides the ability to access this ValidationLayer information and Element information in ColumnBasedDataSourceCatalog.
>>>> >>
>>>> >> BTW, I think it would be great to bake up patches for Apache OODT for what you are working on, including ColumnBasedDataSourceCatalog (already there but in a branch, would be nice to forward port to trunk, and your proposed ValidationLayer and Element extensions). Thoughts? Do you have the time? I think the community at Apache would sincerely appreciate the effort.
>>>> >>
>>>> >> Cheers,
>>>> >> Chris
>>>> >>
>>>> >> P.S. Would you be OK with me forwarding this thread to dev@oodt.a.o? I think there's some good info here that I'd hate to be lost in the ether...
>>>> >>
>>>> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> >> Chris Mattmann, Ph.D.
>>>> >> Senior Computer Scientist
>>>> >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>> >> Office: 171-266B, Mailstop: 171-246
>>>> >> Email: chris.a.mattmann@nasa.gov
>>>> >> WWW: http://sunset.usc.edu/~mattmann/
>>>> >> Phone: +1 (818) 354-8810
>>>> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> >> Adjunct Assistant Professor, Computer Science Department
>>>> >> University of Southern California, Los Angeles, CA 90089 USA
>>>> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> >>
>>>> >
>>>>
>>>>
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> Chris Mattmann, Ph.D.
>>>> Senior Computer Scientist
>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>> Office: 171-266B, Mailstop: 171-246
>>>> Email: chris.a.mattmann@nasa.gov
>>>> WWW: http://sunset.usc.edu/~mattmann/
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> Adjunct Assistant Professor, Computer Science Department
>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>
>>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Changes to Elements.xml file
Posted by holenoter <ho...@me.com>.
hey michael,�
if you just need to store this information in the elements.xml file then just add new tags . . . you don't need to change any code:
<element id="some-id" name="some-name">
(current elements here)
....
<!-- unparsed tags -->
<maxSize>10</maxSize>
</element>
Also�. . . the dublin core stuff is fine for peate . . . if you want to create a patch for the trunk at apache that is a different story, but peate will never need the dublin core feature, so the patch as it currently stands is perfectly okay
-brian
On May 24, 2011, at 07:18 AM, "Starch, Michael D (388L)" <Mi...@jpl.nasa.gov> wrote:
Brian,
PEATE is busy as always, but doing well.
We need to be able to store this information somewhere so we can just tell the DBA to use these values. �As it stands now we do a lot of back-and-forth to get the right values to support the database, and we need to stream-line the process by having this information pre-selected by the project. �Due to the data's similarity to the other data in the elements file we selected this as the place to put it and Chris agreed.
Does this answer your questions?
-Michael
On May 24, 2011, at 1:47 AM, holenoter wrote:
hey michael,
hope everything is going well for peate . . .
yup dublin core element was used because of the time crunch before i left . . . being it hasn't been used up to this point anywhere, good odds it probably won't be used anytime in the "peate future" . . however, dublin core is the wrong place to put this feature because dublin core is used to describe the element itself, not its value's type . . . pretty much chris's recommendation is the way to go, however if you are going to be working towards a patch to the trunk, instead of extending Element as chris recommended just add a "type" member variable to the existing Element class and then modify the XMLValidationLayer and have it read a <type> tag into that "type" member variable in Element . . . i think there is a strong case for having the Element understand to some degree its value's type . . . not sure why you need more information in your elements.xml file than i already added? . . . the database should be doing this validation (otherwise your code will have to become double, float, long, int, etc... aware) . . . the current ColumnBasedDataSourceCatalog handles all these types already via Oracle (just set any double, float, long, int, etc.. to a <dcElement>NUMBER</dcElement> in the elements.xml file. . .you have to set Oracle double precision anyway even if you where to treat it as a double in the filemgr) . . . if you want to just keep the information in one spot (i.e. elements.xml) than just add unused tags to the elements.xml file with the information you want (XMLValidationLayer will just ignore these tags) . . . for example you can add a <maxSize> tag to elements.xml without affecting the filemgr. . .as far as validating max size and the like, this really should be happening in your metadata extractors.
-brian
On May 24, 2011, at 12:59 AM, "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> wrote:
Hi Michael,
Thanks for your email!
You can find the OODT Apache archives here:
http://mail-archives.apache.org/mod_mbox/oodt-dev/
As for standards, best practices are certainly part of them but we do have a few standards that we try and stick to including Dublin Core, ISO-11179, RDF, etc., when we are defining metadata elements. For your purposes in elements.xml in CAS, the biggest approach is to use unique, namespaced identifiers for element IDs, product type IDs, etc.
HTH!
Cheers,
Chris
On May 23, 2011, at 1:14 PM, Starch, Michael D (388L) wrote:
> Chris,
>
> Is there some sort of coding standards document that I should be abiding by, or does oodt just follow standard java "best practices"? This is the kind of information that seems like it would be easier for me to look up rather than ask on the developer list.
>
> Finally, how can I access the archived dev-list mailings?
>
> Thanks,
>
> Michael
>
>
> On May 19, 2011, at 11:06 PM, Mattmann, Chris A (388J) wrote:
>
>> Hi Michael,
>>
>> I'd be happy to reply below, but yes I'd suggest initially CC'ing dev@oodt.apache.org. Giving this information is great, but I prefer the community to benefit from these types of discussions (and it means that I or you or Albert or Cecilia can point others to this question if it is asked again rather than typing the same answer again or fwd'ing something private). Also no worries about researching it -- you can ask questions on that list -- researched or not. We welcome feedback from the community.
>>
>> Answers inline below:
>>
>> On May 19, 2011, at 2:44 PM, Starch, Michael D (388L) wrote:
>>
>>> Chris,
>>>
>>> In order to create tables in the new database architecture that Brian Foster set up for use, we need the ability to specify some information beyond that which is stored in elements.xml. We wish to store this information as part of PCS, rather than continually haggling back and forth with our DBA in order to get this set in an ad-hoc manner.
>>>
>>> The data we need to store is:
>>>
>>> -Is it a vector type?
>>> -Data-type
>>> -Max data size
>>> and potentially a few more items.
>>
>> Gotcha.
>>
>>>
>>> As it stands now, Brian has used the DCElement tag in elements.xml file in order to store some of this information but not all of it. Thus it seems natural that the rest of this information gets stored along side it.
>>>
>>> There seem to be three ways to accomplish this:
>>>
>>> 1. Add more information to the DCElement tag,
>>> 2. Add additional tags to the element file containing this information
>>> 3. Create a sperate file to store this information (Not part of PCS)
>>
>> I've got a 4th option.
>>
>> 4. Write a new ValidationLayer implementation, that extends XMLValidationLayer, but adds your desired information to elements.xml. For that extra information, I'd favor your proposed option #2 -- I think it's cleaner.
>>
>>>
>>> As head of oodt, we were looking for you input as to which option to choose, or if you wish this question to be brought to dev@oodt.apache, where can I research this information before asking the question on that list?
>>>
>>> Below is a brief analysis of each option, from our perspective.
>>>
>>> Thanks,
>>>
>>> Michael Starch
>>>
>>> 1. Add more information to the DCElements tag.
>>> -I believe Brain said this tag was unused, so he "borrowed" it to suit this purpose.
>>> -Adding more information to the tag would "hijack" this tag to a further extent.
>>> -The changes could be made locally to our ColumnBasedDataSourceCatalog in the manner Brian already used.
>>> -If the DCElement is used ouside of the new use Brian invented, this change could affect the intended (original) use
>>
>> I don't like this one (nor do I think others will) since it hijacks dcElement (which is supposed to be a mapping to a Dublin Core element name, rather than used for other information).
>>
>>>
>>> 2. Add more tags to elements.xml
>>> -Most elegant solution, as each data field has a tag specified to hold exactly it
>>> -No need to use tags which were originally intended for some other purpose
>>> -It is an architectural change, changing the format of the elements.xml file
>>> -Must be done carefully to prevent us from deviating from apache-oodt when it is not necessary to do so.
>>
>> +1 for this option.
>>
>>>
>>> 3. A separate file
>>> -Creates data/configuration duplication, as some of these fields must stay (in some form) in elements.xml
>>> -Separates similar pieces of data
>>> -Doesn't alter PCS as it exists now
>>
>> This is fine too, but I prefer #2.
>>
>>>
>>> We favor number 1 or number 2 as it keeps similar data together, and feels like it is the most logical place to store this information.
>>>
>>> I personally favor number 2, as it does not use tags for unintended purposes, but I do not wish to make a change that affects our compatibility with apache code. Hence your input.
>>>
>>
>> +1 to your preference. I think the way to accomplish it is to write a YourExtendedValidationLayer extends XMLValidationLayer and that provides a customized:
>> 1. YourElementClass extends Element
>> - provides extra props (getters+setters) that you want to leverage
>> 2. implementations of the SerDe for your specialized elements.xml that includes those extra tags
>> 3. provides the ability to access this ValidationLayer information and Element information in ColumnBasedDataSourceCatalog.
>>
>> BTW, I think it would be great to bake up patches for Apache OODT for what you are working on, including ColumnBasedDataSourceCatalog (already there but in a branch, would be nice to forward port to trunk, and your proposed ValidationLayer and Element extensions). Thoughts? Do you have the time? I think the community at Apache would sincerely appreciate the effort.
>>
>> Cheers,
>> Chris
>>
>> P.S. Would you be OK with me forwarding this thread to dev@oodt.a.o? I think there's some good info here that I'd hate to be lost in the ether...
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> Phone: +1 (818) 354-8810
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Changes to Elements.xml file
Posted by "Starch, Michael D (388L)" <Mi...@jpl.nasa.gov>.
Brian,
PEATE is busy as always, but doing well.
We need to be able to store this information somewhere so we can just tell the DBA to use these values. As it stands now we do a lot of back-and-forth to get the right values to support the database, and we need to stream-line the process by having this information pre-selected by the project. Due to the data's similarity to the other data in the elements file we selected this as the place to put it and Chris agreed.
Does this answer your questions?
-Michael
On May 24, 2011, at 1:47 AM, holenoter wrote:
hey michael,
hope everything is going well for peate . . .
yup dublin core element was used because of the time crunch before i left . . . being it hasn't been used up to this point anywhere, good odds it probably won't be used anytime in the "peate future" . . . however, dublin core is the wrong place to put this feature because dublin core is used to describe the element itself, not its value's type . . . pretty much chris's recommendation is the way to go, however if you are going to be working towards a patch to the trunk, instead of extending Element as chris recommended just add a "type" member variable to the existing Element class and then modify the XMLValidationLayer and have it read a <type> tag into that "type" member variable in Element . . . i think there is a strong case for having the Element understand to some degree its value's type . . . not sure why you need more information in your elements.xml file than i already added? . . . the database should be doing this validation (otherwise your code will have to become double, float, long, int, etc... aware) . . . the current ColumnBasedDataSourceCatalog handles all these types already via Oracle (just set any double, float, long, int, etc... to a <dcElement>NUMBER</dcElement> in the elements.xml file. . .you have to set Oracle double precision anyway even if you where to treat it as a double in the filemgr) . . . if you want to just keep the information in one spot (i.e. elements.xml) than just add unused tags to the elements.xml file with the information you want (XMLValidationLayer will just ignore these tags) . . . for example you can add a <maxSize> tag to elements.xml without affecting the filemgr. . .as far as validating max size and the like, this really should be happening in your metadata extractors.
-brian
On May 24, 2011, at 12:59 AM, "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>> wrote:
Hi Michael,
Thanks for your email!
You can find the OODT Apache archives here:
http://mail-archives.apache.org/mod_mbox/oodt-dev/
As for standards, best practices are certainly part of them but we do have a few standards that we try and stick to including Dublin Core, ISO-11179, RDF, etc., when we are defining metadata elements. For your purposes in elements.xml in CAS, the biggest approach is to use unique, namespaced identifiers for element IDs, product type IDs, etc.
HTH!
Cheers,
Chris
On May 23, 2011, at 1:14 PM, Starch, Michael D (388L) wrote:
> Chris,
>
> Is there some sort of coding standards document that I should be abiding by, or does oodt just follow standard java "best practices"? This is the kind of information that seems like it would be easier for me to look up rather than ask on the developer list.
>
> Finally, how can I access the archived dev-list mailings?
>
> Thanks,
>
> Michael
>
>
> On May 19, 2011, at 11:06 PM, Mattmann, Chris A (388J) wrote:
>
>> Hi Michael,
>>
>> I'd be happy to reply below, but yes I'd suggest initially CC'ing dev@oodt.apache.org<ma...@oodt.apache.org>. Giving this information is great, but I prefer the community to benefit from these types of discussions (and it means that I or you or Albert or Cecilia can point others to this question if it is asked again rather than typing the same answer again or fwd'ing something private). Also no worries about researching it -- you can ask questions on that list -- researched or not. We welcome feedback from the community.
>>
>> Answers inline below:
>>
>> On May 19, 2011, at 2:44 PM, Starch, Michael D (388L) wrote:
>>
>>> Chris,
>>>
>>> In order to create tables in the new database architecture that Brian Foster set up for use, we need the ability to specify some information beyond that which is stored in elements.xml. We wish to store this information as part of PCS, rather than continually haggling back and forth with our DBA in order to get this set in an ad-hoc manner.
>>>
>>> The data we need to store is:
>>>
>>> -Is it a vector type?
>>> -Data-type
>>> -Max data size
>>> and potentially a few more items.
>>
>> Gotcha.
>>
>>>
>>> As it stands now, Brian has used the DCElement tag in elements.xml file in order to store some of this information but not all of it. Thus it seems natural that the rest of this information gets stored along side it.
>>>
>>> There seem to be three ways to accomplish this:
>>>
>>> 1. Add more information to the DCElement tag,
>>> 2. Add additional tags to the element file containing this information
>>> 3. Create a sperate file to store this information (Not part of PCS)
>>
>> I've got a 4th option.
>>
>> 4. Write a new ValidationLayer implementation, that extends XMLValidationLayer, but adds your desired information to elements.xml. For that extra information, I'd favor your proposed option #2 -- I think it's cleaner.
>>
>>>
>>> As head of oodt, we were looking for you input as to which option to choose, or if you wish this question to be brought to dev@oodt.apache<ma...@oodt.apache>, where can I research this information before asking the question on that list?
>>>
>>> Below is a brief analysis of each option, from our perspective.
>>>
>>> Thanks,
>>>
>>> Michael Starch
>>>
>>> 1. Add more information to the DCElements tag.
>>> -I believe Brain said this tag was unused, so he "borrowed" it to suit this purpose.
>>> -Adding more information to the tag would "hijack" this tag to a further extent.
>>> -The changes could be made locally to our ColumnBasedDataSourceCatalog in the manner Brian already used.
>>> -If the DCElement is used ouside of the new use Brian invented, this change could affect the intended (original) use
>>
>> I don't like this one (nor do I think others will) since it hijacks dcElement (which is supposed to be a mapping to a Dublin Core element name, rather than used for other information).
>>
>>>
>>> 2. Add more tags to elements.xml
>>> -Most elegant solution, as each data field has a tag specified to hold exactly it
>>> -No need to use tags which were originally intended for some other purpose
>>> -It is an architectural change, changing the format of the elements.xml file
>>> -Must be done carefully to prevent us from deviating from apache-oodt when it is not necessary to do so.
>>
>> +1 for this option.
>>
>>>
>>> 3. A separate file
>>> -Creates data/configuration duplication, as some of these fields must stay (in some form) in elements.xml
>>> -Separates similar pieces of data
>>> -Doesn't alter PCS as it exists now
>>
>> This is fine too, but I prefer #2.
>>
>>>
>>> We favor number 1 or number 2 as it keeps similar data together, and feels like it is the most logical place to store this information.
>>>
>>> I personally favor number 2, as it does not use tags for unintended purposes, but I do not wish to make a change that affects our compatibility with apache code. Hence your input.
>>>
>>
>> +1 to your preference. I think the way to accomplish it is to write a YourExtendedValidationLayer extends XMLValidationLayer and that provides a customized:
>> 1. YourElementClass extends Element
>> - provides extra props (getters+setters) that you want to leverage
>> 2. implementations of the SerDe for your specialized elements.xml that includes those extra tags
>> 3. provides the ability to access this ValidationLayer information and Element information in ColumnBasedDataSourceCatalog.
>>
>> BTW, I think it would be great to bake up patches for Apache OODT for what you are working on, including ColumnBasedDataSourceCatalog (already there but in a branch, would be nice to forward port to trunk, and your proposed ValidationLayer and Element extensions). Thoughts? Do you have the time? I think the community at Apache would sincerely appreciate the effort.
>>
>> Cheers,
>> Chris
>>
>> P.S. Would you be OK with me forwarding this thread to dev@oodt.a.o<ma...@oodt.a.o>? I think there's some good info here that I'd hate to be lost in the ether...
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov<ma...@nasa.gov>
>> WWW: http://sunset.usc.edu/~mattmann/
>> Phone: +1 (818) 354-8810
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov<ma...@nasa.gov>
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Changes to Elements.xml file
Posted by holenoter <ho...@me.com>.
hey michael,
hope everything is going well for peate . . .
yup dublin core element was used because of the time crunch before i left . . . being it hasn't been used up to this point anywhere, good odds it probably won't be used anytime in the "peate future" . . . however, dublin core is the wrong place to put this feature because dublin core is used to describe the element itself, not its value's type . . . pretty much chris's recommendation is the way to go, however if you are going to be working towards a patch to the trunk, instead of extending Element as chris recommended just add a "type" member variable to the existing Element class and then modify the XMLValidationLayer and have it read a <type> tag into that "type" member variable in Element . . . i think there is a strong case for having the Element understand to some degree its value's type . . . not sure why you need more information in your elements.xml file than i already added? . . . the database should be doing this validation (otherwise your code will have to become double, float, long, int, etc... aware) . . . the current ColumnBasedDataSourceCatalog handles all these types already via Oracle (just set any double, float, long, int, etc... to a <dcElement>NUMBER</dcElement> in the elements.xml file. . .you have to set Oracle double precision anyway even if you where to treat it as a double in the filemgr) . . if you want to just keep the information in one spot (i.e. elements.xml) than just add unused tags to the elements.xml file with the information you want (XMLValidationLayer will just ignore these tags) . . . for example you can add a <maxSize> tag to elements.xml without affecting the filemgr. . .as far as validating max size and the like, this really should be happening in your metadata extractors.
-brian
On May 24, 2011, at 12:59 AM, "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> wrote:
Hi Michael,
Thanks for your email!
You can find the OODT Apache archives here:
http://mail-archives.apache.org/mod_mbox/oodt-dev/
As for standards, best practices are certainly part of them but we do have a few standards that we try and stick to including Dublin Core, ISO-11179, RDF, etc., when we are defining metadata elements. For your purposes in elements.xml in CAS, the biggest approach is to use unique, namespaced identifiers for element IDs, product type IDs, etc.
HTH!
Cheers,
Chris
On May 23, 2011, at 1:14 PM, Starch, Michael D (388L) wrote:
> Chris,
>
> Is there some sort of coding standards document that I should be abiding by, or does oodt just follow standard java "best practices"? This is the kind of information that seems like it would be easier for me to look up rather than ask on the developer list.
>
> Finally, how can I access the archived dev-list mailings?
>
> Thanks,
>
> Michael
>
>
> On May 19, 2011, at 11:06 PM, Mattmann, Chris A (388J) wrote:
>
>> Hi Michael,
>>
>> I'd be happy to reply below, but yes I'd suggest initially CC'ing dev@oodt.apache.org. Giving this information is great, but I prefer the community to benefit from these types of discussions (and it means that I or you or Albert or Cecilia can point others to this question if it is asked again rather than typing the same answer again or fwd'ing something private). Also no worries about researching it -- you can ask questions on that list -- researched or not. We welcome feedback from the community.
>>
>> Answers inline below:
>>
>> On May 19, 2011, at 2:44 PM, Starch, Michael D (388L) wrote:
>>
>>> Chris,
>>>
>>> In order to create tables in the new database architecture that Brian Foster set up for use, we need the ability to specify some information beyond that which is stored in elements.xml. We wish to store this information as part of PCS, rather than continually haggling back and forth with our DBA in order to get this set in an ad-hoc manner.
>>>
>>> The data we need to store is:
>>>
>>> -Is it a vector type?
>>> -Data-type
>>> -Max data size
>>> and potentially a few more items.
>>
>> Gotcha.
>>
>>>
>>> As it stands now, Brian has used the DCElement tag in elements.xml file in order to store some of this information but not all of it. Thus it seems natural that the rest of this information gets stored along side it.
>>>
>>> There seem to be three ways to accomplish this:
>>>
>>> 1. Add more information to the DCElement tag,
>>> 2. Add additional tags to the element file containing this information
>>> 3. Create a sperate file to store this information (Not part of PCS)
>>
>> I've got a 4th option.
>>
>> 4. Write a new ValidationLayer implementation, that extends XMLValidationLayer, but adds your desired information to elements.xml. For that extra information, I'd favor your proposed option #2 -- I think it's cleaner.
>>
>>>
>>> As head of oodt, we were looking for you input as to which option to choose, or if you wish this question to be brought to dev@oodt.apache, where can I research this information before asking the question on that list?
>>>
>>> Below is a brief analysis of each option, from our perspective.
>>>
>>> Thanks,
>>>
>>> Michael Starch
>>>
>>> 1. Add more information to the DCElements tag.
>>> -I believe Brain said this tag was unused, so he "borrowed" it to suit this purpose.
>>> -Adding more information to the tag would "hijack" this tag to a further extent.
>>> -The changes could be made locally to our ColumnBasedDataSourceCatalog in the manner Brian already used.
>>> -If the DCElement is used ouside of the new use Brian invented, this change could affect the intended (original) use
>>
>> I don't like this one (nor do I think others will) since it hijacks dcElement (which is supposed to be a mapping to a Dublin Core element name, rather than used for other information).
>>
>>>
>>> 2. Add more tags to elements.xml
>>> -Most elegant solution, as each data field has a tag specified to hold exactly it
>>> -No need to use tags which were originally intended for some other purpose
>>> -It is an architectural change, changing the format of the elements.xml file
>>> -Must be done carefully to prevent us from deviating from apache-oodt when it is not necessary to do so.
>>
>> +1 for this option.
>>
>>>
>>> 3. A separate file
>>> -Creates data/configuration duplication, as some of these fields must stay (in some form) in elements.xml
>>> -Separates similar pieces of data
>>> -Doesn't alter PCS as it exists now
>>
>> This is fine too, but I prefer #2.
>>
>>>
>>> We favor number 1 or number 2 as it keeps similar data together, and feels like it is the most logical place to store this information.
>>>
>>> I personally favor number 2, as it does not use tags for unintended purposes, but I do not wish to make a change that affects our compatibility with apache code. Hence your input.
>>>
>>
>> +1 to your preference. I think the way to accomplish it is to write a YourExtendedValidationLayer extends XMLValidationLayer and that provides a customized:
>> 1. YourElementClass extends Element
>> - provides extra props (getters+setters) that you want to leverage
>> 2. implementations of the SerDe for your specialized elements.xml that includes those extra tags
>> 3. provides the ability to access this ValidationLayer information and Element information in ColumnBasedDataSourceCatalog.
>>
>> BTW, I think it would be great to bake up patches for Apache OODT for what you are working on, including ColumnBasedDataSourceCatalog (already there but in a branch, would be nice to forward port to trunk, and your proposed ValidationLayer and Element extensions). Thoughts? Do you have the time? I think the community at Apache would sincerely appreciate the effort.
>>
>> Cheers,
>> Chris
>>
>> P.S Would you be OK with me forwarding this thread to dev@oodt.a.o? I think there's some good info here that I'd hate to be lost in the ether...
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> Phone: +1 (818) 354-8810
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Changes to Elements.xml file
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hi Michael,
Thanks for your email!
You can find the OODT Apache archives here:
http://mail-archives.apache.org/mod_mbox/oodt-dev/
As for standards, best practices are certainly part of them but we do have a few standards that we try and stick to including Dublin Core, ISO-11179, RDF, etc., when we are defining metadata elements. For your purposes in elements.xml in CAS, the biggest approach is to use unique, namespaced identifiers for element IDs, product type IDs, etc.
HTH!
Cheers,
Chris
On May 23, 2011, at 1:14 PM, Starch, Michael D (388L) wrote:
> Chris,
>
> Is there some sort of coding standards document that I should be abiding by, or does oodt just follow standard java "best practices"? This is the kind of information that seems like it would be easier for me to look up rather than ask on the developer list.
>
> Finally, how can I access the archived dev-list mailings?
>
> Thanks,
>
> Michael
>
>
> On May 19, 2011, at 11:06 PM, Mattmann, Chris A (388J) wrote:
>
>> Hi Michael,
>>
>> I'd be happy to reply below, but yes I'd suggest initially CC'ing dev@oodt.apache.org. Giving this information is great, but I prefer the community to benefit from these types of discussions (and it means that I or you or Albert or Cecilia can point others to this question if it is asked again rather than typing the same answer again or fwd'ing something private). Also no worries about researching it -- you can ask questions on that list -- researched or not. We welcome feedback from the community.
>>
>> Answers inline below:
>>
>> On May 19, 2011, at 2:44 PM, Starch, Michael D (388L) wrote:
>>
>>> Chris,
>>>
>>> In order to create tables in the new database architecture that Brian Foster set up for use, we need the ability to specify some information beyond that which is stored in elements.xml. We wish to store this information as part of PCS, rather than continually haggling back and forth with our DBA in order to get this set in an ad-hoc manner.
>>>
>>> The data we need to store is:
>>>
>>> -Is it a vector type?
>>> -Data-type
>>> -Max data size
>>> and potentially a few more items.
>>
>> Gotcha.
>>
>>>
>>> As it stands now, Brian has used the DCElement tag in elements.xml file in order to store some of this information but not all of it. Thus it seems natural that the rest of this information gets stored along side it.
>>>
>>> There seem to be three ways to accomplish this:
>>>
>>> 1. Add more information to the DCElement tag,
>>> 2. Add additional tags to the element file containing this information
>>> 3. Create a sperate file to store this information (Not part of PCS)
>>
>> I've got a 4th option.
>>
>> 4. Write a new ValidationLayer implementation, that extends XMLValidationLayer, but adds your desired information to elements.xml. For that extra information, I'd favor your proposed option #2 -- I think it's cleaner.
>>
>>>
>>> As head of oodt, we were looking for you input as to which option to choose, or if you wish this question to be brought to dev@oodt.apache, where can I research this information before asking the question on that list?
>>>
>>> Below is a brief analysis of each option, from our perspective.
>>>
>>> Thanks,
>>>
>>> Michael Starch
>>>
>>> 1. Add more information to the DCElements tag.
>>> -I believe Brain said this tag was unused, so he "borrowed" it to suit this purpose.
>>> -Adding more information to the tag would "hijack" this tag to a further extent.
>>> -The changes could be made locally to our ColumnBasedDataSourceCatalog in the manner Brian already used.
>>> -If the DCElement is used ouside of the new use Brian invented, this change could affect the intended (original) use
>>
>> I don't like this one (nor do I think others will) since it hijacks dcElement (which is supposed to be a mapping to a Dublin Core element name, rather than used for other information).
>>
>>>
>>> 2. Add more tags to elements.xml
>>> -Most elegant solution, as each data field has a tag specified to hold exactly it
>>> -No need to use tags which were originally intended for some other purpose
>>> -It is an architectural change, changing the format of the elements.xml file
>>> -Must be done carefully to prevent us from deviating from apache-oodt when it is not necessary to do so.
>>
>> +1 for this option.
>>
>>>
>>> 3. A separate file
>>> -Creates data/configuration duplication, as some of these fields must stay (in some form) in elements.xml
>>> -Separates similar pieces of data
>>> -Doesn't alter PCS as it exists now
>>
>> This is fine too, but I prefer #2.
>>
>>>
>>> We favor number 1 or number 2 as it keeps similar data together, and feels like it is the most logical place to store this information.
>>>
>>> I personally favor number 2, as it does not use tags for unintended purposes, but I do not wish to make a change that affects our compatibility with apache code. Hence your input.
>>>
>>
>> +1 to your preference. I think the way to accomplish it is to write a YourExtendedValidationLayer extends XMLValidationLayer and that provides a customized:
>> 1. YourElementClass extends Element
>> - provides extra props (getters+setters) that you want to leverage
>> 2. implementations of the SerDe for your specialized elements.xml that includes those extra tags
>> 3. provides the ability to access this ValidationLayer information and Element information in ColumnBasedDataSourceCatalog.
>>
>> BTW, I think it would be great to bake up patches for Apache OODT for what you are working on, including ColumnBasedDataSourceCatalog (already there but in a branch, would be nice to forward port to trunk, and your proposed ValidationLayer and Element extensions). Thoughts? Do you have the time? I think the community at Apache would sincerely appreciate the effort.
>>
>> Cheers,
>> Chris
>>
>> P.S. Would you be OK with me forwarding this thread to dev@oodt.a.o? I think there's some good info here that I'd hate to be lost in the ether...
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> Phone: +1 (818) 354-8810
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Changes to Elements.xml file
Posted by "Ramirez, Paul M (388J)" <pa...@jpl.nasa.gov>.
Hey Michael,
The archives to the mailing lists are linked here:
http://mail-archives.apache.org/mod_mbox/
You will also see the ones that we had for the OODT project as we were in incubation.
Hope this helps,
Paul Ramirez
On May 23, 2011, at 4:14 PM, Starch, Michael D (388L) wrote:
> Chris,
>
> Is there some sort of coding standards document that I should be abiding by, or does oodt just follow standard java "best practices"? This is the kind of information that seems like it would be easier for me to look up rather than ask on the developer list.
>
> Finally, how can I access the archived dev-list mailings?
>
> Thanks,
>
> Michael
>
>
> On May 19, 2011, at 11:06 PM, Mattmann, Chris A (388J) wrote:
>
>> Hi Michael,
>>
>> I'd be happy to reply below, but yes I'd suggest initially CC'ing dev@oodt.apache.org. Giving this information is great, but I prefer the community to benefit from these types of discussions (and it means that I or you or Albert or Cecilia can point others to this question if it is asked again rather than typing the same answer again or fwd'ing something private). Also no worries about researching it -- you can ask questions on that list -- researched or not. We welcome feedback from the community.
>>
>> Answers inline below:
>>
>> On May 19, 2011, at 2:44 PM, Starch, Michael D (388L) wrote:
>>
>>> Chris,
>>>
>>> In order to create tables in the new database architecture that Brian Foster set up for use, we need the ability to specify some information beyond that which is stored in elements.xml. We wish to store this information as part of PCS, rather than continually haggling back and forth with our DBA in order to get this set in an ad-hoc manner.
>>>
>>> The data we need to store is:
>>>
>>> -Is it a vector type?
>>> -Data-type
>>> -Max data size
>>> and potentially a few more items.
>>
>> Gotcha.
>>
>>>
>>> As it stands now, Brian has used the DCElement tag in elements.xml file in order to store some of this information but not all of it. Thus it seems natural that the rest of this information gets stored along side it.
>>>
>>> There seem to be three ways to accomplish this:
>>>
>>> 1. Add more information to the DCElement tag,
>>> 2. Add additional tags to the element file containing this information
>>> 3. Create a sperate file to store this information (Not part of PCS)
>>
>> I've got a 4th option.
>>
>> 4. Write a new ValidationLayer implementation, that extends XMLValidationLayer, but adds your desired information to elements.xml. For that extra information, I'd favor your proposed option #2 -- I think it's cleaner.
>>
>>>
>>> As head of oodt, we were looking for you input as to which option to choose, or if you wish this question to be brought to dev@oodt.apache, where can I research this information before asking the question on that list?
>>>
>>> Below is a brief analysis of each option, from our perspective.
>>>
>>> Thanks,
>>>
>>> Michael Starch
>>>
>>> 1. Add more information to the DCElements tag.
>>> -I believe Brain said this tag was unused, so he "borrowed" it to suit this purpose.
>>> -Adding more information to the tag would "hijack" this tag to a further extent.
>>> -The changes could be made locally to our ColumnBasedDataSourceCatalog in the manner Brian already used.
>>> -If the DCElement is used ouside of the new use Brian invented, this change could affect the intended (original) use
>>
>> I don't like this one (nor do I think others will) since it hijacks dcElement (which is supposed to be a mapping to a Dublin Core element name, rather than used for other information).
>>
>>>
>>> 2. Add more tags to elements.xml
>>> -Most elegant solution, as each data field has a tag specified to hold exactly it
>>> -No need to use tags which were originally intended for some other purpose
>>> -It is an architectural change, changing the format of the elements.xml file
>>> -Must be done carefully to prevent us from deviating from apache-oodt when it is not necessary to do so.
>>
>> +1 for this option.
>>
>>>
>>> 3. A separate file
>>> -Creates data/configuration duplication, as some of these fields must stay (in some form) in elements.xml
>>> -Separates similar pieces of data
>>> -Doesn't alter PCS as it exists now
>>
>> This is fine too, but I prefer #2.
>>
>>>
>>> We favor number 1 or number 2 as it keeps similar data together, and feels like it is the most logical place to store this information.
>>>
>>> I personally favor number 2, as it does not use tags for unintended purposes, but I do not wish to make a change that affects our compatibility with apache code. Hence your input.
>>>
>>
>> +1 to your preference. I think the way to accomplish it is to write a YourExtendedValidationLayer extends XMLValidationLayer and that provides a customized:
>> 1. YourElementClass extends Element
>> - provides extra props (getters+setters) that you want to leverage
>> 2. implementations of the SerDe for your specialized elements.xml that includes those extra tags
>> 3. provides the ability to access this ValidationLayer information and Element information in ColumnBasedDataSourceCatalog.
>>
>> BTW, I think it would be great to bake up patches for Apache OODT for what you are working on, including ColumnBasedDataSourceCatalog (already there but in a branch, would be nice to forward port to trunk, and your proposed ValidationLayer and Element extensions). Thoughts? Do you have the time? I think the community at Apache would sincerely appreciate the effort.
>>
>> Cheers,
>> Chris
>>
>> P.S. Would you be OK with me forwarding this thread to dev@oodt.a.o? I think there's some good info here that I'd hate to be lost in the ether...
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> Phone: +1 (818) 354-8810
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>