You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by Chris Mattmann <ch...@jpl.nasa.gov> on 2005/12/13 18:07:16 UTC
Standard metadata property names in the ParseData metadata
Hi Folks,
I was just thinking about the ParseData java.util.Properties metaata object
and thinking about the way that we store names in there. Currently, people
are free to name their string-based properties anything that they want, such
as having names of "Content-type", "content-TyPe", "CONTENT_TYPE" all having
the same meaning. Stefan G. I believe proposed a solution in which all
property names be converted to lower case, but in essence this really only
fixes half the problem right (the case of identifying that "CONTENT_TYPE"
and "conTeNT-TyPE" and all the permutations are really the same). What about
if named it "Content Type", or "ContentType"?
I propose that a way to correct this would be to create a standard set of
named Strings in the ParseData class that the protocol framework and the
parsing framework could use to identify common properties such as
"Content-type", "Creator", "Language", etc.
The properties would be defined at the top of the ParseData class,
something like:
public class ParseData{
.....
public static final String CONTENT_TYPE = "content-type";
public static final String CREATOR = "creator";
....
}
In this fashion, users could at least know what the name of the standard
properties that they can obtain from the ParseData are, for example by
making a call to ParseData.getMetadata().get(ParseData.CONTENT_TYPE) to get
the content type or a call to
ParseData.getMetadata().set(ParseData.CONTENT_TYPE, "text/xml"); Of course,
this wouldn't preclude users from doing what they are currently doing, it
would just provide a standard method of obtaining some of the more common,
critical metadata without pouring over the code base to figure out what they
are named.
What do you all think? If you guys think that this is a good solution, I'll
create an issue in JIRA about it and contribute a patch near the end of the
week.
Cheers,
Chris
______________________________________________
Chris A. Mattmann
Chris.Mattmann@jpl.nasa.gov
Staff Member
Modeling and Data Management Systems Section (387)
Data Management Systems and Technologies Group
_________________________________________________
Jet Propulsion Laboratory Pasadena, CA
Office: 171-266B Mailstop: 171-246
_______________________________________________________
Disclaimer: The opinions presented within are my own and do not reflect
those of either NASA, JPL, or the California Institute of Technology.
Re: Standard metadata property names in the ParseData metadata
Posted by Chris Mattmann <ch...@jpl.nasa.gov>.
Hi Guys,
Okay, that makes sense then. I will create an issue in JIRA later today
describing the update, and then begin working on this over the next few
days.
Thanks for your responses and reviews.
Cheers,
Chris
On 12/13/05 12:45 PM, "Jérôme Charron" <je...@gmail.com> wrote:
>> I agree, too. Perhaps we should use the names as they appear in the
>> Dublin Core for those properties that are defined there
>
> A big YES!
>
>
>> - just prepended
>> them with "X-nutch-" in order to avoid name-clashes with other
>> properties (e.g. blindly copied from the protocol headers).
>
> Another big YES!
>
>
> --
> http://motrech.free.fr/
> http://www.frutch.org/
______________________________________________
Chris A. Mattmann
Chris.Mattmann@jpl.nasa.gov
Staff Member
Modeling and Data Management Systems Section (387)
Data Management Systems and Technologies Group
_________________________________________________
Jet Propulsion Laboratory Pasadena, CA
Office: 171-266B Mailstop: 171-246
_______________________________________________________
Disclaimer: The opinions presented within are my own and do not reflect
those of either NASA, JPL, or the California Institute of Technology.
Re: Standard metadata property names in the ParseData metadata
Posted by Jérôme Charron <je...@gmail.com>.
> I agree, too. Perhaps we should use the names as they appear in the
> Dublin Core for those properties that are defined there
A big YES!
> - just prepended
> them with "X-nutch-" in order to avoid name-clashes with other
> properties (e.g. blindly copied from the protocol headers).
Another big YES!
--
http://motrech.free.fr/
http://www.frutch.org/
Re: Standard metadata property names in the ParseData metadata
Posted by Andrzej Bialecki <ab...@getopt.org>.
Stefan Groschupf wrote:
> +1!
> BTW, did you notice that Jerome committed a patch that makes Content
> meta data now case insensitive?
I agree, too. Perhaps we should use the names as they appear in the
Dublin Core for those properties that are defined there - just prepended
them with "X-nutch-" in order to avoid name-clashes with other
properties (e.g. blindly copied from the protocol headers).
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
Re: Standard metadata property names in the ParseData metadata
Posted by Jérôme Charron <je...@gmail.com>.
+1
A simple solution that provides a standard way to access common meta data.
Great!
--
http://motrech.free.fr/
http://www.frutch.org/
Re: Standard metadata property names in the ParseData metadata
Posted by Chris Mattmann <ch...@jpl.nasa.gov>.
Hi Stefan,
Thanks. Yup, I noticed it and I think it will really help out a lot. Great
job to the both of you :-)
Cheers,
Chris
On 12/13/05 10:59 AM, "Stefan Groschupf" <sg...@media-style.com> wrote:
> +1!
> BTW, did you notice that Jerome committed a patch that makes Content
> meta data now case insensitive?
>
> Stefan
>
> Am 13.12.2005 um 18:07 schrieb Chris Mattmann:
>
>> Hi Folks,
>>
>> I was just thinking about the ParseData java.util.Properties
>> metaata object
>> and thinking about the way that we store names in there. Currently,
>> people
>> are free to name their string-based properties anything that they
>> want, such
>> as having names of "Content-type", "content-TyPe", "CONTENT_TYPE"
>> all having
>> the same meaning. Stefan G. I believe proposed a solution in which all
>> property names be converted to lower case, but in essence this
>> really only
>> fixes half the problem right (the case of identifying that
>> "CONTENT_TYPE"
>> and "conTeNT-TyPE" and all the permutations are really the same).
>> What about
>> if named it "Content Type", or "ContentType"?
>>
>> I propose that a way to correct this would be to create a standard
>> set of
>> named Strings in the ParseData class that the protocol framework
>> and the
>> parsing framework could use to identify common properties such as
>> "Content-type", "Creator", "Language", etc.
>>
>> The properties would be defined at the top of the ParseData class,
>> something like:
>>
>> public class ParseData{
>>
>> .....
>>
>> public static final String CONTENT_TYPE = "content-type";
>> public static final String CREATOR = "creator";
>>
>> ....
>>
>> }
>>
>>
>> In this fashion, users could at least know what the name of the
>> standard
>> properties that they can obtain from the ParseData are, for example by
>> making a call to ParseData.getMetadata().get
>> (ParseData.CONTENT_TYPE) to get
>> the content type or a call to
>> ParseData.getMetadata().set(ParseData.CONTENT_TYPE, "text/xml"); Of
>> course,
>> this wouldn't preclude users from doing what they are currently
>> doing, it
>> would just provide a standard method of obtaining some of the more
>> common,
>> critical metadata without pouring over the code base to figure out
>> what they
>> are named.
>>
>> What do you all think? If you guys think that this is a good
>> solution, I'll
>> create an issue in JIRA about it and contribute a patch near the
>> end of the
>> week.
>>
>> Cheers,
>> Chris
>>
>> ______________________________________________
>> Chris A. Mattmann
>> Chris.Mattmann@jpl.nasa.gov
>> Staff Member
>> Modeling and Data Management Systems Section (387)
>> Data Management Systems and Technologies Group
>>
>> _________________________________________________
>> Jet Propulsion Laboratory Pasadena, CA
>> Office: 171-266B Mailstop: 171-246
>> _______________________________________________________
>>
>> Disclaimer: The opinions presented within are my own and do not
>> reflect
>> those of either NASA, JPL, or the California Institute of Technology.
>>
>>
>>
>
> ---------------------------------------------------------------
> company: http://www.media-style.com
> forum: http://www.text-mining.org
> blog: http://www.find23.net
>
>
______________________________________________
Chris A. Mattmann
Chris.Mattmann@jpl.nasa.gov
Staff Member
Modeling and Data Management Systems Section (387)
Data Management Systems and Technologies Group
_________________________________________________
Jet Propulsion Laboratory Pasadena, CA
Office: 171-266B Mailstop: 171-246
_______________________________________________________
Disclaimer: The opinions presented within are my own and do not reflect
those of either NASA, JPL, or the California Institute of Technology.
Re: Standard metadata property names in the ParseData metadata
Posted by Stefan Groschupf <sg...@media-style.com>.
+1!
BTW, did you notice that Jerome committed a patch that makes Content
meta data now case insensitive?
Stefan
Am 13.12.2005 um 18:07 schrieb Chris Mattmann:
> Hi Folks,
>
> I was just thinking about the ParseData java.util.Properties
> metaata object
> and thinking about the way that we store names in there. Currently,
> people
> are free to name their string-based properties anything that they
> want, such
> as having names of "Content-type", "content-TyPe", "CONTENT_TYPE"
> all having
> the same meaning. Stefan G. I believe proposed a solution in which all
> property names be converted to lower case, but in essence this
> really only
> fixes half the problem right (the case of identifying that
> "CONTENT_TYPE"
> and "conTeNT-TyPE" and all the permutations are really the same).
> What about
> if named it "Content Type", or "ContentType"?
>
> I propose that a way to correct this would be to create a standard
> set of
> named Strings in the ParseData class that the protocol framework
> and the
> parsing framework could use to identify common properties such as
> "Content-type", "Creator", "Language", etc.
>
> The properties would be defined at the top of the ParseData class,
> something like:
>
> public class ParseData{
>
> .....
>
> public static final String CONTENT_TYPE = "content-type";
> public static final String CREATOR = "creator";
>
> ....
>
> }
>
>
> In this fashion, users could at least know what the name of the
> standard
> properties that they can obtain from the ParseData are, for example by
> making a call to ParseData.getMetadata().get
> (ParseData.CONTENT_TYPE) to get
> the content type or a call to
> ParseData.getMetadata().set(ParseData.CONTENT_TYPE, "text/xml"); Of
> course,
> this wouldn't preclude users from doing what they are currently
> doing, it
> would just provide a standard method of obtaining some of the more
> common,
> critical metadata without pouring over the code base to figure out
> what they
> are named.
>
> What do you all think? If you guys think that this is a good
> solution, I'll
> create an issue in JIRA about it and contribute a patch near the
> end of the
> week.
>
> Cheers,
> Chris
>
> ______________________________________________
> Chris A. Mattmann
> Chris.Mattmann@jpl.nasa.gov
> Staff Member
> Modeling and Data Management Systems Section (387)
> Data Management Systems and Technologies Group
>
> _________________________________________________
> Jet Propulsion Laboratory Pasadena, CA
> Office: 171-266B Mailstop: 171-246
> _______________________________________________________
>
> Disclaimer: The opinions presented within are my own and do not
> reflect
> those of either NASA, JPL, or the California Institute of Technology.
>
>
>
---------------------------------------------------------------
company: http://www.media-style.com
forum: http://www.text-mining.org
blog: http://www.find23.net