You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@vxquery.apache.org by Menaka Madushanka <me...@gmail.com> on 2016/06/22 21:04:35 UTC
Update Index Query completed.
Hello,
This is the summary of the implementation. (Included in Pull Request
message as well)
*Update Index Query*
The update-index query takes two arguments, collection directory and index
directory.
It shares some of the functionalities from build-index-on-collection query
so, some changes were done to the following classes in order to use them in
updating index process and to maximize code reuse
1. IndexConstructorUtil.java : Created a new function to get an instance
of IndexDocumentBuilder which can be used in IndexUpdater.java class.
2. IndexDocumentBuilder.java : Added a new string filed containing the
corresponding file path which is needed to retrieve a document related to
an XML file.
*Metadata handling*
Here a POJO is created to properly manage the metadata for a file.
(XmlMetadata.java)
Currently it contains following fields.
1. File path
2. File Name (Not used)
3. Checksum String
When storing metadata, a HashMap is created with file path as the key and
XmlMetadata object. This map is then serialized and written to a file named
metadata.file and stored in the same directory where the index is stored.
*Update Index process*
- If a file is detected as modified, the current index document related
to that file is deleted and newly created index document is added.
- If a new file is detected, a new index document will be created and
added to the existing index.
- If the file is deleted, delete the index document related to that file.
- After every task, update the metadata object and after all processes
completed, write the new metadata map to the file.
Please review the pull request and merge.
https://github.com/apache/vxquery/pull/62
Thank you
Menaka
--
*Menaka Madushanka Jayawardena*
Faculty of Engineering, <http://www.pdn.ac.lk/eng>
University of Peradeniyaya.
LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
TP:- 071 885 1183/ 071 350 5470
Re: Update Index Query completed.
Posted by Menaka Madushanka <me...@gmail.com>.
Thank you very much Preston.
I'll update the wiki.
On 25 June 2016 at 01:41, Preston Carman <pr...@apache.org> wrote:
> done.
>
> On Fri, Jun 24, 2016 at 12:04 PM, Menaka Madushanka
> <me...@gmail.com> wrote:
> > Hi Preston,
> >
> > My username is menaka123
> >
> > On 25 June 2016 at 00:29, Preston Carman <pr...@apache.org> wrote:
> >
> >> We can give him access. I just need to know his username.
> >>
> >> On Fri, Jun 24, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu>
> wrote:
> >> > Is it possible to give Menaka edit access for that page? I think this
> >> would
> >> > be helpful throughout the summer. Or would he need to be a committee?
> >> > Steven
> >> >
> >> > On Friday, June 24, 2016, Menaka Madushanka <me...@gmail.com>
> >> wrote:
> >> >
> >> >> Thank you very much Preston. :-)
> >> >>
> >> >> I can do that.
> >> >>
> >> >> On 24 June 2016 at 21:24, Preston Carman <prestonc@apache.org
> >> >> <javascript:_e(%7B%7D,'cvml','prestonc@apache.org');>> wrote:
> >> >>
> >> >>> What do you think about adding your description of the update
> process
> >> >>> to the wiki [1]? We can use this as the start of documenting the
> >> >>> indexing functionality. You have written a nice description and it
> >> >>> would be nice to have the in a place that others can see it to learn
> >> >>> about our indexing process.
> >> >>>
> >> >>> [1]
> >> >>>
> >>
> https://cwiki.apache.org/confluence/display/VXQUERY/Lucene+Indexing+Project+2016
> >> >>>
> >> >>> On Thu, Jun 23, 2016 at 3:39 PM, Menaka Madushanka
> >> >>> <menaka12350@gmail.com
> >> >>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
> >> >>> > Hello,
> >> >>> >
> >> >>> > I modified the implementation to use only one argument for update
> >> index
> >> >>> > query.
> >> >>> >
> >> >>> > So the new query structure would be,
> >> >>> >
> >> >>> > *update-index(index_folder)*
> >> >>> >
> >> >>> > Collection information is stored when creating the index for the
> >> first
> >> >>> time
> >> >>> > in build-index-on-collection query and stored as metadata.
> >> >>> >
> >> >>> > Thank you very much
> >> >>> > Menaka
> >> >>> >
> >> >>> > On 24 June 2016 at 03:42, Menaka Madushanka <
> menaka12350@gmail.com
> >> >>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
> >> >>> >
> >> >>> >> Hello Steven,
> >> >>> >>
> >> >>> >> Almost done. :-)
> >> >>> >>
> >> >>> >> On 24 June 2016 at 03:16, Steven Jacobs <sjaco002@ucr.edu
> >> >>> <javascript:_e(%7B%7D,'cvml','sjaco002@ucr.edu');>> wrote:
> >> >>> >>
> >> >>> >>> Auto-correct is always changing your name when I don't pay
> >> attention,
> >> >>> I
> >> >>> >>> apologize Menaka.
> >> >>> >>> Steven
> >> >>> >>>
> >> >>> >>> On Thu, Jun 23, 2016 at 2:45 PM, Steven Jacobs <
> sjaco002@ucr.edu
> >> >>> <javascript:_e(%7B%7D,'cvml','sjaco002@ucr.edu');>> wrote:
> >> >>> >>>
> >> >>> >>>> Melaka- One high level comment. I think it will be better to
> have
> >> >>> >>>> update-index take a single argument as we discussed (just the
> >> index
> >> >>> >>>> folder). The collection location can be saved as part of the
> >> metadata
> >> >>> >>>> information in the collection folder.
> >> >>> >>>> Steven
> >> >>> >>>>
> >> >>> >>>> On Wed, Jun 22, 2016 at 2:04 PM, Menaka Madushanka <
> >> >>> >>>> menaka12350@gmail.com
> >> >>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
> >> >>> >>>>
> >> >>> >>>>> Hello,
> >> >>> >>>>>
> >> >>> >>>>> This is the summary of the implementation. (Included in Pull
> >> Request
> >> >>> >>>>> message as well)
> >> >>> >>>>>
> >> >>> >>>>> *Update Index Query*
> >> >>> >>>>> The update-index query takes two arguments, collection
> directory
> >> and
> >> >>> >>>>> index directory.
> >> >>> >>>>> It shares some of the functionalities from
> >> build-index-on-collection
> >> >>> >>>>> query so, some changes were done to the following classes in
> >> order
> >> >>> to use
> >> >>> >>>>> them in updating index process and to maximize code reuse
> >> >>> >>>>>
> >> >>> >>>>>
> >> >>> >>>>> 1. IndexConstructorUtil.java : Created a new function to
> get
> >> an
> >> >>> >>>>> instance of IndexDocumentBuilder which can be used in
> >> >>> IndexUpdater.java
> >> >>> >>>>> class.
> >> >>> >>>>> 2. IndexDocumentBuilder.java : Added a new string filed
> >> >>> containing
> >> >>> >>>>> the corresponding file path which is needed to retrieve a
> >> >>> document related
> >> >>> >>>>> to an XML file.
> >> >>> >>>>>
> >> >>> >>>>>
> >> >>> >>>>> *Metadata handling*
> >> >>> >>>>> Here a POJO is created to properly manage the metadata for a
> >> file.
> >> >>> >>>>> (XmlMetadata.java)
> >> >>> >>>>> Currently it contains following fields.
> >> >>> >>>>>
> >> >>> >>>>>
> >> >>> >>>>> 1. File path
> >> >>> >>>>> 2. File Name (Not used)
> >> >>> >>>>> 3. Checksum String
> >> >>> >>>>>
> >> >>> >>>>> When storing metadata, a HashMap is created with file path as
> the
> >> >>> key
> >> >>> >>>>> and XmlMetadata object. This map is then serialized and
> written
> >> to
> >> >>> a file
> >> >>> >>>>> named metadata.file and stored in the same directory where the
> >> >>> index is
> >> >>> >>>>> stored.
> >> >>> >>>>>
> >> >>> >>>>> *Update Index process*
> >> >>> >>>>>
> >> >>> >>>>> - If a file is detected as modified, the current index
> >> document
> >> >>> >>>>> related to that file is deleted and newly created index
> >> document
> >> >>> is added.
> >> >>> >>>>> - If a new file is detected, a new index document will be
> >> created
> >> >>> >>>>> and added to the existing index.
> >> >>> >>>>> - If the file is deleted, delete the index document
> related to
> >> >>> that
> >> >>> >>>>> file.
> >> >>> >>>>> - After every task, update the metadata object and after
> all
> >> >>> >>>>> processes completed, write the new metadata map to the
> file.
> >> >>> >>>>>
> >> >>> >>>>> Please review the pull request and merge.
> >> >>> >>>>>
> >> >>> >>>>> https://github.com/apache/vxquery/pull/62
> >> >>> >>>>>
> >> >>> >>>>> Thank you
> >> >>> >>>>> Menaka
> >> >>> >>>>>
> >> >>> >>>>>
> >> >>> >>>>> --
> >> >>> >>>>> *Menaka Madushanka Jayawardena*
> >> >>> >>>>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> >> >>> >>>>> University of Peradeniyaya.
> >> >>> >>>>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> >> >>> >>>>> TP:- 071 885 1183/ 071 350 5470
> >> >>> >>>>>
> >> >>> >>>>
> >> >>> >>>>
> >> >>> >>>
> >> >>> >>
> >> >>> >>
> >> >>> >> --
> >> >>> >> *Menaka Madushanka Jayawardena*
> >> >>> >> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> >> >>> >> University of Peradeniyaya.
> >> >>> >> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> >> >>> >> TP:- 071 885 1183/ 071 350 5470
> >> >>> >>
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > --
> >> >>> > *Menaka Madushanka Jayawardena*
> >> >>> > Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> >> >>> > University of Peradeniyaya.
> >> >>> > LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> >> >>> > TP:- 071 885 1183/ 071 350 5470
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> *Menaka Madushanka Jayawardena*
> >> >> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> >> >> University of Peradeniyaya.
> >> >> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> >> >> TP:- 071 885 1183/ 071 350 5470
> >> >>
> >>
> >
> >
> >
> > --
> > *Menaka Madushanka Jayawardena*
> > Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> > University of Peradeniyaya.
> > LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> > TP:- 071 885 1183/ 071 350 5470
>
--
*Menaka Madushanka Jayawardena*
Faculty of Engineering, <http://www.pdn.ac.lk/eng>
University of Peradeniyaya.
LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
TP:- 071 885 1183/ 071 350 5470
Re: Update Index Query completed.
Posted by Preston Carman <pr...@apache.org>.
done.
On Fri, Jun 24, 2016 at 12:04 PM, Menaka Madushanka
<me...@gmail.com> wrote:
> Hi Preston,
>
> My username is menaka123
>
> On 25 June 2016 at 00:29, Preston Carman <pr...@apache.org> wrote:
>
>> We can give him access. I just need to know his username.
>>
>> On Fri, Jun 24, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu> wrote:
>> > Is it possible to give Menaka edit access for that page? I think this
>> would
>> > be helpful throughout the summer. Or would he need to be a committee?
>> > Steven
>> >
>> > On Friday, June 24, 2016, Menaka Madushanka <me...@gmail.com>
>> wrote:
>> >
>> >> Thank you very much Preston. :-)
>> >>
>> >> I can do that.
>> >>
>> >> On 24 June 2016 at 21:24, Preston Carman <prestonc@apache.org
>> >> <javascript:_e(%7B%7D,'cvml','prestonc@apache.org');>> wrote:
>> >>
>> >>> What do you think about adding your description of the update process
>> >>> to the wiki [1]? We can use this as the start of documenting the
>> >>> indexing functionality. You have written a nice description and it
>> >>> would be nice to have the in a place that others can see it to learn
>> >>> about our indexing process.
>> >>>
>> >>> [1]
>> >>>
>> https://cwiki.apache.org/confluence/display/VXQUERY/Lucene+Indexing+Project+2016
>> >>>
>> >>> On Thu, Jun 23, 2016 at 3:39 PM, Menaka Madushanka
>> >>> <menaka12350@gmail.com
>> >>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
>> >>> > Hello,
>> >>> >
>> >>> > I modified the implementation to use only one argument for update
>> index
>> >>> > query.
>> >>> >
>> >>> > So the new query structure would be,
>> >>> >
>> >>> > *update-index(index_folder)*
>> >>> >
>> >>> > Collection information is stored when creating the index for the
>> first
>> >>> time
>> >>> > in build-index-on-collection query and stored as metadata.
>> >>> >
>> >>> > Thank you very much
>> >>> > Menaka
>> >>> >
>> >>> > On 24 June 2016 at 03:42, Menaka Madushanka <menaka12350@gmail.com
>> >>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
>> >>> >
>> >>> >> Hello Steven,
>> >>> >>
>> >>> >> Almost done. :-)
>> >>> >>
>> >>> >> On 24 June 2016 at 03:16, Steven Jacobs <sjaco002@ucr.edu
>> >>> <javascript:_e(%7B%7D,'cvml','sjaco002@ucr.edu');>> wrote:
>> >>> >>
>> >>> >>> Auto-correct is always changing your name when I don't pay
>> attention,
>> >>> I
>> >>> >>> apologize Menaka.
>> >>> >>> Steven
>> >>> >>>
>> >>> >>> On Thu, Jun 23, 2016 at 2:45 PM, Steven Jacobs <sjaco002@ucr.edu
>> >>> <javascript:_e(%7B%7D,'cvml','sjaco002@ucr.edu');>> wrote:
>> >>> >>>
>> >>> >>>> Melaka- One high level comment. I think it will be better to have
>> >>> >>>> update-index take a single argument as we discussed (just the
>> index
>> >>> >>>> folder). The collection location can be saved as part of the
>> metadata
>> >>> >>>> information in the collection folder.
>> >>> >>>> Steven
>> >>> >>>>
>> >>> >>>> On Wed, Jun 22, 2016 at 2:04 PM, Menaka Madushanka <
>> >>> >>>> menaka12350@gmail.com
>> >>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
>> >>> >>>>
>> >>> >>>>> Hello,
>> >>> >>>>>
>> >>> >>>>> This is the summary of the implementation. (Included in Pull
>> Request
>> >>> >>>>> message as well)
>> >>> >>>>>
>> >>> >>>>> *Update Index Query*
>> >>> >>>>> The update-index query takes two arguments, collection directory
>> and
>> >>> >>>>> index directory.
>> >>> >>>>> It shares some of the functionalities from
>> build-index-on-collection
>> >>> >>>>> query so, some changes were done to the following classes in
>> order
>> >>> to use
>> >>> >>>>> them in updating index process and to maximize code reuse
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> 1. IndexConstructorUtil.java : Created a new function to get
>> an
>> >>> >>>>> instance of IndexDocumentBuilder which can be used in
>> >>> IndexUpdater.java
>> >>> >>>>> class.
>> >>> >>>>> 2. IndexDocumentBuilder.java : Added a new string filed
>> >>> containing
>> >>> >>>>> the corresponding file path which is needed to retrieve a
>> >>> document related
>> >>> >>>>> to an XML file.
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> *Metadata handling*
>> >>> >>>>> Here a POJO is created to properly manage the metadata for a
>> file.
>> >>> >>>>> (XmlMetadata.java)
>> >>> >>>>> Currently it contains following fields.
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> 1. File path
>> >>> >>>>> 2. File Name (Not used)
>> >>> >>>>> 3. Checksum String
>> >>> >>>>>
>> >>> >>>>> When storing metadata, a HashMap is created with file path as the
>> >>> key
>> >>> >>>>> and XmlMetadata object. This map is then serialized and written
>> to
>> >>> a file
>> >>> >>>>> named metadata.file and stored in the same directory where the
>> >>> index is
>> >>> >>>>> stored.
>> >>> >>>>>
>> >>> >>>>> *Update Index process*
>> >>> >>>>>
>> >>> >>>>> - If a file is detected as modified, the current index
>> document
>> >>> >>>>> related to that file is deleted and newly created index
>> document
>> >>> is added.
>> >>> >>>>> - If a new file is detected, a new index document will be
>> created
>> >>> >>>>> and added to the existing index.
>> >>> >>>>> - If the file is deleted, delete the index document related to
>> >>> that
>> >>> >>>>> file.
>> >>> >>>>> - After every task, update the metadata object and after all
>> >>> >>>>> processes completed, write the new metadata map to the file.
>> >>> >>>>>
>> >>> >>>>> Please review the pull request and merge.
>> >>> >>>>>
>> >>> >>>>> https://github.com/apache/vxquery/pull/62
>> >>> >>>>>
>> >>> >>>>> Thank you
>> >>> >>>>> Menaka
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> --
>> >>> >>>>> *Menaka Madushanka Jayawardena*
>> >>> >>>>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>> >>> >>>>> University of Peradeniyaya.
>> >>> >>>>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>> >>> >>>>> TP:- 071 885 1183/ 071 350 5470
>> >>> >>>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> *Menaka Madushanka Jayawardena*
>> >>> >> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>> >>> >> University of Peradeniyaya.
>> >>> >> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>> >>> >> TP:- 071 885 1183/ 071 350 5470
>> >>> >>
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > *Menaka Madushanka Jayawardena*
>> >>> > Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>> >>> > University of Peradeniyaya.
>> >>> > LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>> >>> > TP:- 071 885 1183/ 071 350 5470
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> *Menaka Madushanka Jayawardena*
>> >> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>> >> University of Peradeniyaya.
>> >> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>> >> TP:- 071 885 1183/ 071 350 5470
>> >>
>>
>
>
>
> --
> *Menaka Madushanka Jayawardena*
> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> University of Peradeniyaya.
> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> TP:- 071 885 1183/ 071 350 5470
Re: Update Index Query completed.
Posted by Menaka Madushanka <me...@gmail.com>.
Hi Preston,
My username is menaka123
On 25 June 2016 at 00:29, Preston Carman <pr...@apache.org> wrote:
> We can give him access. I just need to know his username.
>
> On Fri, Jun 24, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu> wrote:
> > Is it possible to give Menaka edit access for that page? I think this
> would
> > be helpful throughout the summer. Or would he need to be a committee?
> > Steven
> >
> > On Friday, June 24, 2016, Menaka Madushanka <me...@gmail.com>
> wrote:
> >
> >> Thank you very much Preston. :-)
> >>
> >> I can do that.
> >>
> >> On 24 June 2016 at 21:24, Preston Carman <prestonc@apache.org
> >> <javascript:_e(%7B%7D,'cvml','prestonc@apache.org');>> wrote:
> >>
> >>> What do you think about adding your description of the update process
> >>> to the wiki [1]? We can use this as the start of documenting the
> >>> indexing functionality. You have written a nice description and it
> >>> would be nice to have the in a place that others can see it to learn
> >>> about our indexing process.
> >>>
> >>> [1]
> >>>
> https://cwiki.apache.org/confluence/display/VXQUERY/Lucene+Indexing+Project+2016
> >>>
> >>> On Thu, Jun 23, 2016 at 3:39 PM, Menaka Madushanka
> >>> <menaka12350@gmail.com
> >>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
> >>> > Hello,
> >>> >
> >>> > I modified the implementation to use only one argument for update
> index
> >>> > query.
> >>> >
> >>> > So the new query structure would be,
> >>> >
> >>> > *update-index(index_folder)*
> >>> >
> >>> > Collection information is stored when creating the index for the
> first
> >>> time
> >>> > in build-index-on-collection query and stored as metadata.
> >>> >
> >>> > Thank you very much
> >>> > Menaka
> >>> >
> >>> > On 24 June 2016 at 03:42, Menaka Madushanka <menaka12350@gmail.com
> >>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
> >>> >
> >>> >> Hello Steven,
> >>> >>
> >>> >> Almost done. :-)
> >>> >>
> >>> >> On 24 June 2016 at 03:16, Steven Jacobs <sjaco002@ucr.edu
> >>> <javascript:_e(%7B%7D,'cvml','sjaco002@ucr.edu');>> wrote:
> >>> >>
> >>> >>> Auto-correct is always changing your name when I don't pay
> attention,
> >>> I
> >>> >>> apologize Menaka.
> >>> >>> Steven
> >>> >>>
> >>> >>> On Thu, Jun 23, 2016 at 2:45 PM, Steven Jacobs <sjaco002@ucr.edu
> >>> <javascript:_e(%7B%7D,'cvml','sjaco002@ucr.edu');>> wrote:
> >>> >>>
> >>> >>>> Melaka- One high level comment. I think it will be better to have
> >>> >>>> update-index take a single argument as we discussed (just the
> index
> >>> >>>> folder). The collection location can be saved as part of the
> metadata
> >>> >>>> information in the collection folder.
> >>> >>>> Steven
> >>> >>>>
> >>> >>>> On Wed, Jun 22, 2016 at 2:04 PM, Menaka Madushanka <
> >>> >>>> menaka12350@gmail.com
> >>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
> >>> >>>>
> >>> >>>>> Hello,
> >>> >>>>>
> >>> >>>>> This is the summary of the implementation. (Included in Pull
> Request
> >>> >>>>> message as well)
> >>> >>>>>
> >>> >>>>> *Update Index Query*
> >>> >>>>> The update-index query takes two arguments, collection directory
> and
> >>> >>>>> index directory.
> >>> >>>>> It shares some of the functionalities from
> build-index-on-collection
> >>> >>>>> query so, some changes were done to the following classes in
> order
> >>> to use
> >>> >>>>> them in updating index process and to maximize code reuse
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> 1. IndexConstructorUtil.java : Created a new function to get
> an
> >>> >>>>> instance of IndexDocumentBuilder which can be used in
> >>> IndexUpdater.java
> >>> >>>>> class.
> >>> >>>>> 2. IndexDocumentBuilder.java : Added a new string filed
> >>> containing
> >>> >>>>> the corresponding file path which is needed to retrieve a
> >>> document related
> >>> >>>>> to an XML file.
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> *Metadata handling*
> >>> >>>>> Here a POJO is created to properly manage the metadata for a
> file.
> >>> >>>>> (XmlMetadata.java)
> >>> >>>>> Currently it contains following fields.
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> 1. File path
> >>> >>>>> 2. File Name (Not used)
> >>> >>>>> 3. Checksum String
> >>> >>>>>
> >>> >>>>> When storing metadata, a HashMap is created with file path as the
> >>> key
> >>> >>>>> and XmlMetadata object. This map is then serialized and written
> to
> >>> a file
> >>> >>>>> named metadata.file and stored in the same directory where the
> >>> index is
> >>> >>>>> stored.
> >>> >>>>>
> >>> >>>>> *Update Index process*
> >>> >>>>>
> >>> >>>>> - If a file is detected as modified, the current index
> document
> >>> >>>>> related to that file is deleted and newly created index
> document
> >>> is added.
> >>> >>>>> - If a new file is detected, a new index document will be
> created
> >>> >>>>> and added to the existing index.
> >>> >>>>> - If the file is deleted, delete the index document related to
> >>> that
> >>> >>>>> file.
> >>> >>>>> - After every task, update the metadata object and after all
> >>> >>>>> processes completed, write the new metadata map to the file.
> >>> >>>>>
> >>> >>>>> Please review the pull request and merge.
> >>> >>>>>
> >>> >>>>> https://github.com/apache/vxquery/pull/62
> >>> >>>>>
> >>> >>>>> Thank you
> >>> >>>>> Menaka
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> --
> >>> >>>>> *Menaka Madushanka Jayawardena*
> >>> >>>>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> >>> >>>>> University of Peradeniyaya.
> >>> >>>>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> >>> >>>>> TP:- 071 885 1183/ 071 350 5470
> >>> >>>>>
> >>> >>>>
> >>> >>>>
> >>> >>>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> *Menaka Madushanka Jayawardena*
> >>> >> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> >>> >> University of Peradeniyaya.
> >>> >> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> >>> >> TP:- 071 885 1183/ 071 350 5470
> >>> >>
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > *Menaka Madushanka Jayawardena*
> >>> > Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> >>> > University of Peradeniyaya.
> >>> > LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> >>> > TP:- 071 885 1183/ 071 350 5470
> >>>
> >>
> >>
> >>
> >> --
> >> *Menaka Madushanka Jayawardena*
> >> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> >> University of Peradeniyaya.
> >> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> >> TP:- 071 885 1183/ 071 350 5470
> >>
>
--
*Menaka Madushanka Jayawardena*
Faculty of Engineering, <http://www.pdn.ac.lk/eng>
University of Peradeniyaya.
LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
TP:- 071 885 1183/ 071 350 5470
Re: Update Index Query completed.
Posted by Preston Carman <pr...@apache.org>.
We can give him access. I just need to know his username.
On Fri, Jun 24, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu> wrote:
> Is it possible to give Menaka edit access for that page? I think this would
> be helpful throughout the summer. Or would he need to be a committee?
> Steven
>
> On Friday, June 24, 2016, Menaka Madushanka <me...@gmail.com> wrote:
>
>> Thank you very much Preston. :-)
>>
>> I can do that.
>>
>> On 24 June 2016 at 21:24, Preston Carman <prestonc@apache.org
>> <javascript:_e(%7B%7D,'cvml','prestonc@apache.org');>> wrote:
>>
>>> What do you think about adding your description of the update process
>>> to the wiki [1]? We can use this as the start of documenting the
>>> indexing functionality. You have written a nice description and it
>>> would be nice to have the in a place that others can see it to learn
>>> about our indexing process.
>>>
>>> [1]
>>> https://cwiki.apache.org/confluence/display/VXQUERY/Lucene+Indexing+Project+2016
>>>
>>> On Thu, Jun 23, 2016 at 3:39 PM, Menaka Madushanka
>>> <menaka12350@gmail.com
>>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
>>> > Hello,
>>> >
>>> > I modified the implementation to use only one argument for update index
>>> > query.
>>> >
>>> > So the new query structure would be,
>>> >
>>> > *update-index(index_folder)*
>>> >
>>> > Collection information is stored when creating the index for the first
>>> time
>>> > in build-index-on-collection query and stored as metadata.
>>> >
>>> > Thank you very much
>>> > Menaka
>>> >
>>> > On 24 June 2016 at 03:42, Menaka Madushanka <menaka12350@gmail.com
>>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
>>> >
>>> >> Hello Steven,
>>> >>
>>> >> Almost done. :-)
>>> >>
>>> >> On 24 June 2016 at 03:16, Steven Jacobs <sjaco002@ucr.edu
>>> <javascript:_e(%7B%7D,'cvml','sjaco002@ucr.edu');>> wrote:
>>> >>
>>> >>> Auto-correct is always changing your name when I don't pay attention,
>>> I
>>> >>> apologize Menaka.
>>> >>> Steven
>>> >>>
>>> >>> On Thu, Jun 23, 2016 at 2:45 PM, Steven Jacobs <sjaco002@ucr.edu
>>> <javascript:_e(%7B%7D,'cvml','sjaco002@ucr.edu');>> wrote:
>>> >>>
>>> >>>> Melaka- One high level comment. I think it will be better to have
>>> >>>> update-index take a single argument as we discussed (just the index
>>> >>>> folder). The collection location can be saved as part of the metadata
>>> >>>> information in the collection folder.
>>> >>>> Steven
>>> >>>>
>>> >>>> On Wed, Jun 22, 2016 at 2:04 PM, Menaka Madushanka <
>>> >>>> menaka12350@gmail.com
>>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
>>> >>>>
>>> >>>>> Hello,
>>> >>>>>
>>> >>>>> This is the summary of the implementation. (Included in Pull Request
>>> >>>>> message as well)
>>> >>>>>
>>> >>>>> *Update Index Query*
>>> >>>>> The update-index query takes two arguments, collection directory and
>>> >>>>> index directory.
>>> >>>>> It shares some of the functionalities from build-index-on-collection
>>> >>>>> query so, some changes were done to the following classes in order
>>> to use
>>> >>>>> them in updating index process and to maximize code reuse
>>> >>>>>
>>> >>>>>
>>> >>>>> 1. IndexConstructorUtil.java : Created a new function to get an
>>> >>>>> instance of IndexDocumentBuilder which can be used in
>>> IndexUpdater.java
>>> >>>>> class.
>>> >>>>> 2. IndexDocumentBuilder.java : Added a new string filed
>>> containing
>>> >>>>> the corresponding file path which is needed to retrieve a
>>> document related
>>> >>>>> to an XML file.
>>> >>>>>
>>> >>>>>
>>> >>>>> *Metadata handling*
>>> >>>>> Here a POJO is created to properly manage the metadata for a file.
>>> >>>>> (XmlMetadata.java)
>>> >>>>> Currently it contains following fields.
>>> >>>>>
>>> >>>>>
>>> >>>>> 1. File path
>>> >>>>> 2. File Name (Not used)
>>> >>>>> 3. Checksum String
>>> >>>>>
>>> >>>>> When storing metadata, a HashMap is created with file path as the
>>> key
>>> >>>>> and XmlMetadata object. This map is then serialized and written to
>>> a file
>>> >>>>> named metadata.file and stored in the same directory where the
>>> index is
>>> >>>>> stored.
>>> >>>>>
>>> >>>>> *Update Index process*
>>> >>>>>
>>> >>>>> - If a file is detected as modified, the current index document
>>> >>>>> related to that file is deleted and newly created index document
>>> is added.
>>> >>>>> - If a new file is detected, a new index document will be created
>>> >>>>> and added to the existing index.
>>> >>>>> - If the file is deleted, delete the index document related to
>>> that
>>> >>>>> file.
>>> >>>>> - After every task, update the metadata object and after all
>>> >>>>> processes completed, write the new metadata map to the file.
>>> >>>>>
>>> >>>>> Please review the pull request and merge.
>>> >>>>>
>>> >>>>> https://github.com/apache/vxquery/pull/62
>>> >>>>>
>>> >>>>> Thank you
>>> >>>>> Menaka
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> *Menaka Madushanka Jayawardena*
>>> >>>>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>>> >>>>> University of Peradeniyaya.
>>> >>>>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>>> >>>>> TP:- 071 885 1183/ 071 350 5470
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>
>>> >>
>>> >>
>>> >> --
>>> >> *Menaka Madushanka Jayawardena*
>>> >> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>>> >> University of Peradeniyaya.
>>> >> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>>> >> TP:- 071 885 1183/ 071 350 5470
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > *Menaka Madushanka Jayawardena*
>>> > Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>>> > University of Peradeniyaya.
>>> > LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>>> > TP:- 071 885 1183/ 071 350 5470
>>>
>>
>>
>>
>> --
>> *Menaka Madushanka Jayawardena*
>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>> University of Peradeniyaya.
>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>> TP:- 071 885 1183/ 071 350 5470
>>
Re: Update Index Query completed.
Posted by Steven Jacobs <sj...@ucr.edu>.
Is it possible to give Menaka edit access for that page? I think this would
be helpful throughout the summer. Or would he need to be a committee?
Steven
On Friday, June 24, 2016, Menaka Madushanka <me...@gmail.com> wrote:
> Thank you very much Preston. :-)
>
> I can do that.
>
> On 24 June 2016 at 21:24, Preston Carman <prestonc@apache.org
> <javascript:_e(%7B%7D,'cvml','prestonc@apache.org');>> wrote:
>
>> What do you think about adding your description of the update process
>> to the wiki [1]? We can use this as the start of documenting the
>> indexing functionality. You have written a nice description and it
>> would be nice to have the in a place that others can see it to learn
>> about our indexing process.
>>
>> [1]
>> https://cwiki.apache.org/confluence/display/VXQUERY/Lucene+Indexing+Project+2016
>>
>> On Thu, Jun 23, 2016 at 3:39 PM, Menaka Madushanka
>> <menaka12350@gmail.com
>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
>> > Hello,
>> >
>> > I modified the implementation to use only one argument for update index
>> > query.
>> >
>> > So the new query structure would be,
>> >
>> > *update-index(index_folder)*
>> >
>> > Collection information is stored when creating the index for the first
>> time
>> > in build-index-on-collection query and stored as metadata.
>> >
>> > Thank you very much
>> > Menaka
>> >
>> > On 24 June 2016 at 03:42, Menaka Madushanka <menaka12350@gmail.com
>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
>> >
>> >> Hello Steven,
>> >>
>> >> Almost done. :-)
>> >>
>> >> On 24 June 2016 at 03:16, Steven Jacobs <sjaco002@ucr.edu
>> <javascript:_e(%7B%7D,'cvml','sjaco002@ucr.edu');>> wrote:
>> >>
>> >>> Auto-correct is always changing your name when I don't pay attention,
>> I
>> >>> apologize Menaka.
>> >>> Steven
>> >>>
>> >>> On Thu, Jun 23, 2016 at 2:45 PM, Steven Jacobs <sjaco002@ucr.edu
>> <javascript:_e(%7B%7D,'cvml','sjaco002@ucr.edu');>> wrote:
>> >>>
>> >>>> Melaka- One high level comment. I think it will be better to have
>> >>>> update-index take a single argument as we discussed (just the index
>> >>>> folder). The collection location can be saved as part of the metadata
>> >>>> information in the collection folder.
>> >>>> Steven
>> >>>>
>> >>>> On Wed, Jun 22, 2016 at 2:04 PM, Menaka Madushanka <
>> >>>> menaka12350@gmail.com
>> <javascript:_e(%7B%7D,'cvml','menaka12350@gmail.com');>> wrote:
>> >>>>
>> >>>>> Hello,
>> >>>>>
>> >>>>> This is the summary of the implementation. (Included in Pull Request
>> >>>>> message as well)
>> >>>>>
>> >>>>> *Update Index Query*
>> >>>>> The update-index query takes two arguments, collection directory and
>> >>>>> index directory.
>> >>>>> It shares some of the functionalities from build-index-on-collection
>> >>>>> query so, some changes were done to the following classes in order
>> to use
>> >>>>> them in updating index process and to maximize code reuse
>> >>>>>
>> >>>>>
>> >>>>> 1. IndexConstructorUtil.java : Created a new function to get an
>> >>>>> instance of IndexDocumentBuilder which can be used in
>> IndexUpdater.java
>> >>>>> class.
>> >>>>> 2. IndexDocumentBuilder.java : Added a new string filed
>> containing
>> >>>>> the corresponding file path which is needed to retrieve a
>> document related
>> >>>>> to an XML file.
>> >>>>>
>> >>>>>
>> >>>>> *Metadata handling*
>> >>>>> Here a POJO is created to properly manage the metadata for a file.
>> >>>>> (XmlMetadata.java)
>> >>>>> Currently it contains following fields.
>> >>>>>
>> >>>>>
>> >>>>> 1. File path
>> >>>>> 2. File Name (Not used)
>> >>>>> 3. Checksum String
>> >>>>>
>> >>>>> When storing metadata, a HashMap is created with file path as the
>> key
>> >>>>> and XmlMetadata object. This map is then serialized and written to
>> a file
>> >>>>> named metadata.file and stored in the same directory where the
>> index is
>> >>>>> stored.
>> >>>>>
>> >>>>> *Update Index process*
>> >>>>>
>> >>>>> - If a file is detected as modified, the current index document
>> >>>>> related to that file is deleted and newly created index document
>> is added.
>> >>>>> - If a new file is detected, a new index document will be created
>> >>>>> and added to the existing index.
>> >>>>> - If the file is deleted, delete the index document related to
>> that
>> >>>>> file.
>> >>>>> - After every task, update the metadata object and after all
>> >>>>> processes completed, write the new metadata map to the file.
>> >>>>>
>> >>>>> Please review the pull request and merge.
>> >>>>>
>> >>>>> https://github.com/apache/vxquery/pull/62
>> >>>>>
>> >>>>> Thank you
>> >>>>> Menaka
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Menaka Madushanka Jayawardena*
>> >>>>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>> >>>>> University of Peradeniyaya.
>> >>>>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>> >>>>> TP:- 071 885 1183/ 071 350 5470
>> >>>>>
>> >>>>
>> >>>>
>> >>>
>> >>
>> >>
>> >> --
>> >> *Menaka Madushanka Jayawardena*
>> >> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>> >> University of Peradeniyaya.
>> >> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>> >> TP:- 071 885 1183/ 071 350 5470
>> >>
>> >
>> >
>> >
>> > --
>> > *Menaka Madushanka Jayawardena*
>> > Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>> > University of Peradeniyaya.
>> > LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>> > TP:- 071 885 1183/ 071 350 5470
>>
>
>
>
> --
> *Menaka Madushanka Jayawardena*
> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> University of Peradeniyaya.
> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> TP:- 071 885 1183/ 071 350 5470
>
Re: Update Index Query completed.
Posted by Menaka Madushanka <me...@gmail.com>.
Thank you very much Preston. :-)
I can do that.
On 24 June 2016 at 21:24, Preston Carman <pr...@apache.org> wrote:
> What do you think about adding your description of the update process
> to the wiki [1]? We can use this as the start of documenting the
> indexing functionality. You have written a nice description and it
> would be nice to have the in a place that others can see it to learn
> about our indexing process.
>
> [1]
> https://cwiki.apache.org/confluence/display/VXQUERY/Lucene+Indexing+Project+2016
>
> On Thu, Jun 23, 2016 at 3:39 PM, Menaka Madushanka
> <me...@gmail.com> wrote:
> > Hello,
> >
> > I modified the implementation to use only one argument for update index
> > query.
> >
> > So the new query structure would be,
> >
> > *update-index(index_folder)*
> >
> > Collection information is stored when creating the index for the first
> time
> > in build-index-on-collection query and stored as metadata.
> >
> > Thank you very much
> > Menaka
> >
> > On 24 June 2016 at 03:42, Menaka Madushanka <me...@gmail.com>
> wrote:
> >
> >> Hello Steven,
> >>
> >> Almost done. :-)
> >>
> >> On 24 June 2016 at 03:16, Steven Jacobs <sj...@ucr.edu> wrote:
> >>
> >>> Auto-correct is always changing your name when I don't pay attention, I
> >>> apologize Menaka.
> >>> Steven
> >>>
> >>> On Thu, Jun 23, 2016 at 2:45 PM, Steven Jacobs <sj...@ucr.edu>
> wrote:
> >>>
> >>>> Melaka- One high level comment. I think it will be better to have
> >>>> update-index take a single argument as we discussed (just the index
> >>>> folder). The collection location can be saved as part of the metadata
> >>>> information in the collection folder.
> >>>> Steven
> >>>>
> >>>> On Wed, Jun 22, 2016 at 2:04 PM, Menaka Madushanka <
> >>>> menaka12350@gmail.com> wrote:
> >>>>
> >>>>> Hello,
> >>>>>
> >>>>> This is the summary of the implementation. (Included in Pull Request
> >>>>> message as well)
> >>>>>
> >>>>> *Update Index Query*
> >>>>> The update-index query takes two arguments, collection directory and
> >>>>> index directory.
> >>>>> It shares some of the functionalities from build-index-on-collection
> >>>>> query so, some changes were done to the following classes in order
> to use
> >>>>> them in updating index process and to maximize code reuse
> >>>>>
> >>>>>
> >>>>> 1. IndexConstructorUtil.java : Created a new function to get an
> >>>>> instance of IndexDocumentBuilder which can be used in
> IndexUpdater.java
> >>>>> class.
> >>>>> 2. IndexDocumentBuilder.java : Added a new string filed containing
> >>>>> the corresponding file path which is needed to retrieve a
> document related
> >>>>> to an XML file.
> >>>>>
> >>>>>
> >>>>> *Metadata handling*
> >>>>> Here a POJO is created to properly manage the metadata for a file.
> >>>>> (XmlMetadata.java)
> >>>>> Currently it contains following fields.
> >>>>>
> >>>>>
> >>>>> 1. File path
> >>>>> 2. File Name (Not used)
> >>>>> 3. Checksum String
> >>>>>
> >>>>> When storing metadata, a HashMap is created with file path as the key
> >>>>> and XmlMetadata object. This map is then serialized and written to a
> file
> >>>>> named metadata.file and stored in the same directory where the index
> is
> >>>>> stored.
> >>>>>
> >>>>> *Update Index process*
> >>>>>
> >>>>> - If a file is detected as modified, the current index document
> >>>>> related to that file is deleted and newly created index document
> is added.
> >>>>> - If a new file is detected, a new index document will be created
> >>>>> and added to the existing index.
> >>>>> - If the file is deleted, delete the index document related to
> that
> >>>>> file.
> >>>>> - After every task, update the metadata object and after all
> >>>>> processes completed, write the new metadata map to the file.
> >>>>>
> >>>>> Please review the pull request and merge.
> >>>>>
> >>>>> https://github.com/apache/vxquery/pull/62
> >>>>>
> >>>>> Thank you
> >>>>> Menaka
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Menaka Madushanka Jayawardena*
> >>>>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> >>>>> University of Peradeniyaya.
> >>>>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> >>>>> TP:- 071 885 1183/ 071 350 5470
> >>>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >> --
> >> *Menaka Madushanka Jayawardena*
> >> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> >> University of Peradeniyaya.
> >> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> >> TP:- 071 885 1183/ 071 350 5470
> >>
> >
> >
> >
> > --
> > *Menaka Madushanka Jayawardena*
> > Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> > University of Peradeniyaya.
> > LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> > TP:- 071 885 1183/ 071 350 5470
>
--
*Menaka Madushanka Jayawardena*
Faculty of Engineering, <http://www.pdn.ac.lk/eng>
University of Peradeniyaya.
LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
TP:- 071 885 1183/ 071 350 5470
Re: Update Index Query completed.
Posted by Preston Carman <pr...@apache.org>.
What do you think about adding your description of the update process
to the wiki [1]? We can use this as the start of documenting the
indexing functionality. You have written a nice description and it
would be nice to have the in a place that others can see it to learn
about our indexing process.
[1] https://cwiki.apache.org/confluence/display/VXQUERY/Lucene+Indexing+Project+2016
On Thu, Jun 23, 2016 at 3:39 PM, Menaka Madushanka
<me...@gmail.com> wrote:
> Hello,
>
> I modified the implementation to use only one argument for update index
> query.
>
> So the new query structure would be,
>
> *update-index(index_folder)*
>
> Collection information is stored when creating the index for the first time
> in build-index-on-collection query and stored as metadata.
>
> Thank you very much
> Menaka
>
> On 24 June 2016 at 03:42, Menaka Madushanka <me...@gmail.com> wrote:
>
>> Hello Steven,
>>
>> Almost done. :-)
>>
>> On 24 June 2016 at 03:16, Steven Jacobs <sj...@ucr.edu> wrote:
>>
>>> Auto-correct is always changing your name when I don't pay attention, I
>>> apologize Menaka.
>>> Steven
>>>
>>> On Thu, Jun 23, 2016 at 2:45 PM, Steven Jacobs <sj...@ucr.edu> wrote:
>>>
>>>> Melaka- One high level comment. I think it will be better to have
>>>> update-index take a single argument as we discussed (just the index
>>>> folder). The collection location can be saved as part of the metadata
>>>> information in the collection folder.
>>>> Steven
>>>>
>>>> On Wed, Jun 22, 2016 at 2:04 PM, Menaka Madushanka <
>>>> menaka12350@gmail.com> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> This is the summary of the implementation. (Included in Pull Request
>>>>> message as well)
>>>>>
>>>>> *Update Index Query*
>>>>> The update-index query takes two arguments, collection directory and
>>>>> index directory.
>>>>> It shares some of the functionalities from build-index-on-collection
>>>>> query so, some changes were done to the following classes in order to use
>>>>> them in updating index process and to maximize code reuse
>>>>>
>>>>>
>>>>> 1. IndexConstructorUtil.java : Created a new function to get an
>>>>> instance of IndexDocumentBuilder which can be used in IndexUpdater.java
>>>>> class.
>>>>> 2. IndexDocumentBuilder.java : Added a new string filed containing
>>>>> the corresponding file path which is needed to retrieve a document related
>>>>> to an XML file.
>>>>>
>>>>>
>>>>> *Metadata handling*
>>>>> Here a POJO is created to properly manage the metadata for a file.
>>>>> (XmlMetadata.java)
>>>>> Currently it contains following fields.
>>>>>
>>>>>
>>>>> 1. File path
>>>>> 2. File Name (Not used)
>>>>> 3. Checksum String
>>>>>
>>>>> When storing metadata, a HashMap is created with file path as the key
>>>>> and XmlMetadata object. This map is then serialized and written to a file
>>>>> named metadata.file and stored in the same directory where the index is
>>>>> stored.
>>>>>
>>>>> *Update Index process*
>>>>>
>>>>> - If a file is detected as modified, the current index document
>>>>> related to that file is deleted and newly created index document is added.
>>>>> - If a new file is detected, a new index document will be created
>>>>> and added to the existing index.
>>>>> - If the file is deleted, delete the index document related to that
>>>>> file.
>>>>> - After every task, update the metadata object and after all
>>>>> processes completed, write the new metadata map to the file.
>>>>>
>>>>> Please review the pull request and merge.
>>>>>
>>>>> https://github.com/apache/vxquery/pull/62
>>>>>
>>>>> Thank you
>>>>> Menaka
>>>>>
>>>>>
>>>>> --
>>>>> *Menaka Madushanka Jayawardena*
>>>>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>>>>> University of Peradeniyaya.
>>>>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>>>>> TP:- 071 885 1183/ 071 350 5470
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> *Menaka Madushanka Jayawardena*
>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>> University of Peradeniyaya.
>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>> TP:- 071 885 1183/ 071 350 5470
>>
>
>
>
> --
> *Menaka Madushanka Jayawardena*
> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> University of Peradeniyaya.
> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> TP:- 071 885 1183/ 071 350 5470
Re: Update Index Query completed.
Posted by Menaka Madushanka <me...@gmail.com>.
Hello,
I modified the implementation to use only one argument for update index
query.
So the new query structure would be,
*update-index(index_folder)*
Collection information is stored when creating the index for the first time
in build-index-on-collection query and stored as metadata.
Thank you very much
Menaka
On 24 June 2016 at 03:42, Menaka Madushanka <me...@gmail.com> wrote:
> Hello Steven,
>
> Almost done. :-)
>
> On 24 June 2016 at 03:16, Steven Jacobs <sj...@ucr.edu> wrote:
>
>> Auto-correct is always changing your name when I don't pay attention, I
>> apologize Menaka.
>> Steven
>>
>> On Thu, Jun 23, 2016 at 2:45 PM, Steven Jacobs <sj...@ucr.edu> wrote:
>>
>>> Melaka- One high level comment. I think it will be better to have
>>> update-index take a single argument as we discussed (just the index
>>> folder). The collection location can be saved as part of the metadata
>>> information in the collection folder.
>>> Steven
>>>
>>> On Wed, Jun 22, 2016 at 2:04 PM, Menaka Madushanka <
>>> menaka12350@gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> This is the summary of the implementation. (Included in Pull Request
>>>> message as well)
>>>>
>>>> *Update Index Query*
>>>> The update-index query takes two arguments, collection directory and
>>>> index directory.
>>>> It shares some of the functionalities from build-index-on-collection
>>>> query so, some changes were done to the following classes in order to use
>>>> them in updating index process and to maximize code reuse
>>>>
>>>>
>>>> 1. IndexConstructorUtil.java : Created a new function to get an
>>>> instance of IndexDocumentBuilder which can be used in IndexUpdater.java
>>>> class.
>>>> 2. IndexDocumentBuilder.java : Added a new string filed containing
>>>> the corresponding file path which is needed to retrieve a document related
>>>> to an XML file.
>>>>
>>>>
>>>> *Metadata handling*
>>>> Here a POJO is created to properly manage the metadata for a file.
>>>> (XmlMetadata.java)
>>>> Currently it contains following fields.
>>>>
>>>>
>>>> 1. File path
>>>> 2. File Name (Not used)
>>>> 3. Checksum String
>>>>
>>>> When storing metadata, a HashMap is created with file path as the key
>>>> and XmlMetadata object. This map is then serialized and written to a file
>>>> named metadata.file and stored in the same directory where the index is
>>>> stored.
>>>>
>>>> *Update Index process*
>>>>
>>>> - If a file is detected as modified, the current index document
>>>> related to that file is deleted and newly created index document is added.
>>>> - If a new file is detected, a new index document will be created
>>>> and added to the existing index.
>>>> - If the file is deleted, delete the index document related to that
>>>> file.
>>>> - After every task, update the metadata object and after all
>>>> processes completed, write the new metadata map to the file.
>>>>
>>>> Please review the pull request and merge.
>>>>
>>>> https://github.com/apache/vxquery/pull/62
>>>>
>>>> Thank you
>>>> Menaka
>>>>
>>>>
>>>> --
>>>> *Menaka Madushanka Jayawardena*
>>>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>>>> University of Peradeniyaya.
>>>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>>>> TP:- 071 885 1183/ 071 350 5470
>>>>
>>>
>>>
>>
>
>
> --
> *Menaka Madushanka Jayawardena*
> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> University of Peradeniyaya.
> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> TP:- 071 885 1183/ 071 350 5470
>
--
*Menaka Madushanka Jayawardena*
Faculty of Engineering, <http://www.pdn.ac.lk/eng>
University of Peradeniyaya.
LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
TP:- 071 885 1183/ 071 350 5470
Re: Update Index Query completed.
Posted by Menaka Madushanka <me...@gmail.com>.
Hello Steven,
Almost done. :-)
On 24 June 2016 at 03:16, Steven Jacobs <sj...@ucr.edu> wrote:
> Auto-correct is always changing your name when I don't pay attention, I
> apologize Menaka.
> Steven
>
> On Thu, Jun 23, 2016 at 2:45 PM, Steven Jacobs <sj...@ucr.edu> wrote:
>
>> Melaka- One high level comment. I think it will be better to have
>> update-index take a single argument as we discussed (just the index
>> folder). The collection location can be saved as part of the metadata
>> information in the collection folder.
>> Steven
>>
>> On Wed, Jun 22, 2016 at 2:04 PM, Menaka Madushanka <menaka12350@gmail.com
>> > wrote:
>>
>>> Hello,
>>>
>>> This is the summary of the implementation. (Included in Pull Request
>>> message as well)
>>>
>>> *Update Index Query*
>>> The update-index query takes two arguments, collection directory and
>>> index directory.
>>> It shares some of the functionalities from build-index-on-collection
>>> query so, some changes were done to the following classes in order to use
>>> them in updating index process and to maximize code reuse
>>>
>>>
>>> 1. IndexConstructorUtil.java : Created a new function to get an
>>> instance of IndexDocumentBuilder which can be used in IndexUpdater.java
>>> class.
>>> 2. IndexDocumentBuilder.java : Added a new string filed containing
>>> the corresponding file path which is needed to retrieve a document related
>>> to an XML file.
>>>
>>>
>>> *Metadata handling*
>>> Here a POJO is created to properly manage the metadata for a file.
>>> (XmlMetadata.java)
>>> Currently it contains following fields.
>>>
>>>
>>> 1. File path
>>> 2. File Name (Not used)
>>> 3. Checksum String
>>>
>>> When storing metadata, a HashMap is created with file path as the key
>>> and XmlMetadata object. This map is then serialized and written to a file
>>> named metadata.file and stored in the same directory where the index is
>>> stored.
>>>
>>> *Update Index process*
>>>
>>> - If a file is detected as modified, the current index document
>>> related to that file is deleted and newly created index document is added.
>>> - If a new file is detected, a new index document will be created
>>> and added to the existing index.
>>> - If the file is deleted, delete the index document related to that
>>> file.
>>> - After every task, update the metadata object and after all
>>> processes completed, write the new metadata map to the file.
>>>
>>> Please review the pull request and merge.
>>>
>>> https://github.com/apache/vxquery/pull/62
>>>
>>> Thank you
>>> Menaka
>>>
>>>
>>> --
>>> *Menaka Madushanka Jayawardena*
>>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>>> University of Peradeniyaya.
>>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>>> TP:- 071 885 1183/ 071 350 5470
>>>
>>
>>
>
--
*Menaka Madushanka Jayawardena*
Faculty of Engineering, <http://www.pdn.ac.lk/eng>
University of Peradeniyaya.
LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
TP:- 071 885 1183/ 071 350 5470
Re: Update Index Query completed.
Posted by Steven Jacobs <sj...@ucr.edu>.
Auto-correct is always changing your name when I don't pay attention, I
apologize Menaka.
Steven
On Thu, Jun 23, 2016 at 2:45 PM, Steven Jacobs <sj...@ucr.edu> wrote:
> Melaka- One high level comment. I think it will be better to have
> update-index take a single argument as we discussed (just the index
> folder). The collection location can be saved as part of the metadata
> information in the collection folder.
> Steven
>
> On Wed, Jun 22, 2016 at 2:04 PM, Menaka Madushanka <me...@gmail.com>
> wrote:
>
>> Hello,
>>
>> This is the summary of the implementation. (Included in Pull Request
>> message as well)
>>
>> *Update Index Query*
>> The update-index query takes two arguments, collection directory and
>> index directory.
>> It shares some of the functionalities from build-index-on-collection
>> query so, some changes were done to the following classes in order to use
>> them in updating index process and to maximize code reuse
>>
>>
>> 1. IndexConstructorUtil.java : Created a new function to get an
>> instance of IndexDocumentBuilder which can be used in IndexUpdater.java
>> class.
>> 2. IndexDocumentBuilder.java : Added a new string filed containing
>> the corresponding file path which is needed to retrieve a document related
>> to an XML file.
>>
>>
>> *Metadata handling*
>> Here a POJO is created to properly manage the metadata for a file.
>> (XmlMetadata.java)
>> Currently it contains following fields.
>>
>>
>> 1. File path
>> 2. File Name (Not used)
>> 3. Checksum String
>>
>> When storing metadata, a HashMap is created with file path as the key and
>> XmlMetadata object. This map is then serialized and written to a file named
>> metadata.file and stored in the same directory where the index is stored.
>>
>> *Update Index process*
>>
>> - If a file is detected as modified, the current index document
>> related to that file is deleted and newly created index document is added.
>> - If a new file is detected, a new index document will be created and
>> added to the existing index.
>> - If the file is deleted, delete the index document related to that
>> file.
>> - After every task, update the metadata object and after all
>> processes completed, write the new metadata map to the file.
>>
>> Please review the pull request and merge.
>>
>> https://github.com/apache/vxquery/pull/62
>>
>> Thank you
>> Menaka
>>
>>
>> --
>> *Menaka Madushanka Jayawardena*
>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>> University of Peradeniyaya.
>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>> TP:- 071 885 1183/ 071 350 5470
>>
>
>
Re: Update Index Query completed.
Posted by Steven Jacobs <sj...@ucr.edu>.
Melaka- One high level comment. I think it will be better to have
update-index take a single argument as we discussed (just the index
folder). The collection location can be saved as part of the metadata
information in the collection folder.
Steven
On Wed, Jun 22, 2016 at 2:04 PM, Menaka Madushanka <me...@gmail.com>
wrote:
> Hello,
>
> This is the summary of the implementation. (Included in Pull Request
> message as well)
>
> *Update Index Query*
> The update-index query takes two arguments, collection directory and index
> directory.
> It shares some of the functionalities from build-index-on-collection query
> so, some changes were done to the following classes in order to use them in
> updating index process and to maximize code reuse
>
>
> 1. IndexConstructorUtil.java : Created a new function to get an
> instance of IndexDocumentBuilder which can be used in IndexUpdater.java
> class.
> 2. IndexDocumentBuilder.java : Added a new string filed containing the
> corresponding file path which is needed to retrieve a document related to
> an XML file.
>
>
> *Metadata handling*
> Here a POJO is created to properly manage the metadata for a file.
> (XmlMetadata.java)
> Currently it contains following fields.
>
>
> 1. File path
> 2. File Name (Not used)
> 3. Checksum String
>
> When storing metadata, a HashMap is created with file path as the key and
> XmlMetadata object. This map is then serialized and written to a file named
> metadata.file and stored in the same directory where the index is stored.
>
> *Update Index process*
>
> - If a file is detected as modified, the current index document
> related to that file is deleted and newly created index document is added.
> - If a new file is detected, a new index document will be created and
> added to the existing index.
> - If the file is deleted, delete the index document related to that
> file.
> - After every task, update the metadata object and after all processes
> completed, write the new metadata map to the file.
>
> Please review the pull request and merge.
>
> https://github.com/apache/vxquery/pull/62
>
> Thank you
> Menaka
>
>
> --
> *Menaka Madushanka Jayawardena*
> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> University of Peradeniyaya.
> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
> TP:- 071 885 1183/ 071 350 5470
>