You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drat.apache.org by Ahmed Ifhaam <ah...@gmail.com> on 2018/06/12 19:06:25 UTC
Re: Current Status and Plans (was Re: Joining on board)
Hi everyone,
I was updating my progress of GSOC time to time. Since i'm completing one
month i would like to summarize it.
My task was to evolve the current Proteus interface for DRAT. As you all
may know already, DRAT had four main functions
Crawl,Index,Map,Reduce. When ever the rest api of DRAT called these
functions, these were executing set of shell commands to perform the task.
Since my task was to evolve the Web interface, With the guidance of my
mentor and the community, i was tasked with creating the APIs
with direct java API calls rather than shell scripts. DRAT uses OODT to
perform these actions, while OODT currently moving to AvroRPC from XMLRpc
technology.
Though we were trying to implement the APIs with AvroRPC clients, Due to
some complications faced in implementing AvroRPC for OODT we had to stick
to the XMLRpc.
But as soon as OODT is released with the stable AvroRPC implementation
these will be updated to use AvroRPC instead of XMLRpc. Current changes
have been merged
with the master branch of DRAT.
So far i have successfully completed all the API calls and also have
introduced one new Rest API for DRAT to execute dynamic workflows with
given tasks.
On the other hand for the web interface implementation i have designed the
mockups and shared in mailing list as well. While waiting for your valuable
comments for those, I have started working on that too. For both the API
and web interface i have updated the DRAT wiki on Github.
For web interface i have planned to use angular and angular Material
technologies.I'm willing to start with the latest version of angular.
The help i got from you people was much helpful, I'm very grateful for
that. Special Thanks to Chris and Imesha for helping me constantly.
Your comments on my work and also suggestions for the rest would be very
helpful in we making this product a better one.
Thank you all.
On Tue, May 22, 2018 at 8:23 PM, Ahmed Ifhaam <ah...@gmail.com>
wrote:
> hi all
> cureently i have completed the task of moving indexing function from
> process wrapper to java calls.
> but other
> funcyions like crawling,mapping needs some more deeper understanding where
> i need more time so ill be completing one more api replacement from one
> week as of today.
>
> from you all i need some support to understand what is the workflow whats
> the sendingevent function does and some docs which describes it better.
>
> thank you
>
> On 22 May 2018 12:06 a.m., "Chris Mattmann" <ma...@apache.org> wrote:
>
> Dear Ahmed,
>
>
>
> Can you please update the community on your plans for what you are working
> on what is done and what you expect to complete in the next week or two?
>
>
>
> Thanks,
> Chris
>
>
>
>
>
>
>
> From: Ahmed Ifhaam <ah...@gmail.com>
> Reply-To: "dev@drat.apache.org" <de...@drat.apache.org>
> Date: Sunday, May 13, 2018 at 12:30 PM
> To: "dev@drat.apache.org" <de...@drat.apache.org>
> Subject: Joining on board
>
>
>
> hi all,
>
>
>
> As part of GSOC_2018 i'll be working on DRAT. I'm very pleased to work on
>
> this and expect your support and advice as well. I'ts great to joining you
>
> all. Thank you
>
>
>
>
> --
>
> *Ahmed Ifhaam *
>
> University of Kelaniya | MIT '19
>
> Tel :- 0662244937 | Mob :- 0776699609
>
> ahmedifhaam1@gmail.com <E-...@gmail.com>
>
>
>
>
>
--
*Ahmed Ifhaam *
University of Kelaniya | MIT '19
Tel :- 0662244937 | Mob :- 0776699609
ahmedifhaam1@gmail.com <E-...@gmail.com>
Current Status and Plans (was Re: Joining on board)
Posted by Ahmed Ifhaam <ah...@gmail.com>.
hi everyone,
As of today i have completed the integration of drat Proteus and drat viz
into single application. currently the pull request [1] consist the
complete implementation of the webapp from the initial commit.
all the functions of this webapplication have been implemented using rest
API calls to the backend.
i have used *single file component* based architecture (structure) for the
application and also so far almost 90% of the components are independed
from another.
I have used a simple state management technic so that the layouts can be
easily altered with future feedbacks. Each component has been developed
with vuetify.js material design library.
So far i have received some feedbacks from the community that tables are
taking too much width, i am waiting to finish the required features, as
soon as i finish i will focus on the feedbacks.
Still implementing dratstat functionality through java backend, which is
currently a seprate python script is the only thing left as major
requirement.
so far the support i got from this community and what i have learned is
something i couldn't even imagine. This has been a great opportunity, I'm
very thankful for each of you for helping me with this. I hope i can be
part of this forever but on my GSOC career this month would be the last,
hence i kindly request you all to take a look at what i have done and give
me your valuable feedbacks. i would be very grateful.
[1] https://github.com/apache/drat/pull/135
Thank you.
On Mon, Jul 9, 2018 at 11:06 AM, Ahmed Ifhaam <ah...@gmail.com>
wrote:
> hi everyone,
>
> First phase of Proteus (new) Web application with vue.js have been
> implemented successfully. Currently the application covers almost all the
> features of Proteus(old) application and few features of viz ui. Next phase
> of integrating viz or dratstat feature is currently on progress. This
> branch [1] has been opened specially for this task. This [2] Pull request
> has the progress so far.
> This webapplication totally works on rest apis and i have tried to make it
> loose coupled with the backend as much as posssible.
>
> I would be very glad to hear your feedbacks from you all.
>
>
> Thank you.
>
> [1] https://github.com/apache/drat/tree/gsoc18
> [2] https://github.com/apache/drat/pull/135
>
> On Fri, Jun 22, 2018 at 11:57 PM, Ahmed Ifhaam <ah...@gmail.com>
> wrote:
>
>> hi all,
>>
>> After the last email, I started working on Vue,js and have implemented
>> the first page as follows Please check and let me know the feedback
>> It would be awesome to get some comments.
>> Screenshot from 2018-06-22 22-50-36.png
>> <https://drive.google.com/file/d/1YWs3D5MBaRp4wquCSq1W3yBW_8wT4qLv/view?usp=drive_web>
>>
>> thank you
>>
>> On Wed, Jun 13, 2018 at 12:36 AM, Ahmed Ifhaam <ah...@gmail.com>
>> wrote:
>>
>>> Hi everyone,
>>>
>>> I was updating my progress of GSOC time to time. Since i'm completing
>>> one month i would like to summarize it.
>>> My task was to evolve the current Proteus interface for DRAT. As you all
>>> may know already, DRAT had four main functions
>>> Crawl,Index,Map,Reduce. When ever the rest api of DRAT called these
>>> functions, these were executing set of shell commands to perform the task.
>>>
>>> Since my task was to evolve the Web interface, With the guidance of my
>>> mentor and the community, i was tasked with creating the APIs
>>> with direct java API calls rather than shell scripts. DRAT uses OODT to
>>> perform these actions, while OODT currently moving to AvroRPC from XMLRpc
>>> technology.
>>> Though we were trying to implement the APIs with AvroRPC clients, Due to
>>> some complications faced in implementing AvroRPC for OODT we had to stick
>>> to the XMLRpc.
>>> But as soon as OODT is released with the stable AvroRPC implementation
>>> these will be updated to use AvroRPC instead of XMLRpc. Current changes
>>> have been merged
>>> with the master branch of DRAT.
>>>
>>> So far i have successfully completed all the API calls and also have
>>> introduced one new Rest API for DRAT to execute dynamic workflows with
>>> given tasks.
>>> On the other hand for the web interface implementation i have designed
>>> the mockups and shared in mailing list as well. While waiting for your
>>> valuable
>>> comments for those, I have started working on that too. For both the API
>>> and web interface i have updated the DRAT wiki on Github.
>>>
>>> For web interface i have planned to use angular and angular Material
>>> technologies.I'm willing to start with the latest version of angular.
>>> The help i got from you people was much helpful, I'm very grateful for
>>> that. Special Thanks to Chris and Imesha for helping me constantly.
>>>
>>> Your comments on my work and also suggestions for the rest would be very
>>> helpful in we making this product a better one.
>>>
>>> Thank you all.
>>>
>>> On Tue, May 22, 2018 at 8:23 PM, Ahmed Ifhaam <ah...@gmail.com>
>>> wrote:
>>>
>>>> hi all
>>>> cureently i have completed the task of moving indexing function from
>>>> process wrapper to java calls.
>>>> but other
>>>> funcyions like crawling,mapping needs some more deeper understanding
>>>> where i need more time so ill be completing one more api replacement from
>>>> one week as of today.
>>>>
>>>> from you all i need some support to understand what is the workflow
>>>> whats the sendingevent function does and some docs which describes it
>>>> better.
>>>>
>>>> thank you
>>>>
>>>> On 22 May 2018 12:06 a.m., "Chris Mattmann" <ma...@apache.org>
>>>> wrote:
>>>>
>>>> Dear Ahmed,
>>>>
>>>>
>>>>
>>>> Can you please update the community on your plans for what you are
>>>> working
>>>> on what is done and what you expect to complete in the next week or two?
>>>>
>>>>
>>>>
>>>> Thanks,
>>>> Chris
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> From: Ahmed Ifhaam <ah...@gmail.com>
>>>> Reply-To: "dev@drat.apache.org" <de...@drat.apache.org>
>>>> Date: Sunday, May 13, 2018 at 12:30 PM
>>>> To: "dev@drat.apache.org" <de...@drat.apache.org>
>>>> Subject: Joining on board
>>>>
>>>>
>>>>
>>>> hi all,
>>>>
>>>>
>>>>
>>>> As part of GSOC_2018 i'll be working on DRAT. I'm very pleased to work
>>>> on
>>>>
>>>> this and expect your support and advice as well. I'ts great to joining
>>>> you
>>>>
>>>> all. Thank you
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Ahmed Ifhaam *
>>>>
>>>> University of Kelaniya | MIT '19
>>>>
>>>> Tel :- 0662244937 | Mob :- 0776699609
>>>>
>>>> ahmedifhaam1@gmail.com <E-...@gmail.com>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Ahmed Ifhaam *
>>> University of Kelaniya | MIT '19
>>> Tel :- 0662244937 | Mob :- 0776699609
>>> ahmedifhaam1@gmail.com <E-...@gmail.com>
>>>
>>
>>
>>
>> --
>> *Ahmed Ifhaam *
>> University of Kelaniya | MIT '19
>> Tel :- 0662244937 | Mob :- 0776699609
>> ahmedifhaam1@gmail.com <E-...@gmail.com>
>>
>
>
>
> --
> *Ahmed Ifhaam *
> University of Kelaniya | MIT '19
> Tel :- 0662244937 | Mob :- 0776699609
> ahmedifhaam1@gmail.com <E-...@gmail.com>
>
--
*Ahmed Ifhaam *
University of Kelaniya | MIT '19
Tel :- 0662244937 | Mob :- 0776699609
ahmedifhaam1@gmail.com <E-...@gmail.com>
--
*Ahmed Ifhaam *
University of Kelaniya | MIT '19
Tel :- 0662244937 | Mob :- 0776699609
ahmedifhaam1@gmail.com <E-...@gmail.com>
Re: Current Status and Plans (was Re: Joining on board)
Posted by Ahmed Ifhaam <ah...@gmail.com>.
hi everyone,
First phase of Proteus (new) Web application with vue.js have been
implemented successfully. Currently the application covers almost all the
features of Proteus(old) application and few features of viz ui. Next phase
of integrating viz or dratstat feature is currently on progress. This
branch [1] has been opened specially for this task. This [2] Pull request
has the progress so far.
This webapplication totally works on rest apis and i have tried to make it
loose coupled with the backend as much as posssible.
I would be very glad to hear your feedbacks from you all.
Thank you.
[1] https://github.com/apache/drat/tree/gsoc18
[2] https://github.com/apache/drat/pull/135
On Fri, Jun 22, 2018 at 11:57 PM, Ahmed Ifhaam <ah...@gmail.com>
wrote:
> hi all,
>
> After the last email, I started working on Vue,js and have implemented the
> first page as follows Please check and let me know the feedback
> It would be awesome to get some comments.
> Screenshot from 2018-06-22 22-50-36.png
> <https://drive.google.com/file/d/1YWs3D5MBaRp4wquCSq1W3yBW_8wT4qLv/view?usp=drive_web>
>
> thank you
>
> On Wed, Jun 13, 2018 at 12:36 AM, Ahmed Ifhaam <ah...@gmail.com>
> wrote:
>
>> Hi everyone,
>>
>> I was updating my progress of GSOC time to time. Since i'm completing one
>> month i would like to summarize it.
>> My task was to evolve the current Proteus interface for DRAT. As you all
>> may know already, DRAT had four main functions
>> Crawl,Index,Map,Reduce. When ever the rest api of DRAT called these
>> functions, these were executing set of shell commands to perform the task.
>>
>> Since my task was to evolve the Web interface, With the guidance of my
>> mentor and the community, i was tasked with creating the APIs
>> with direct java API calls rather than shell scripts. DRAT uses OODT to
>> perform these actions, while OODT currently moving to AvroRPC from XMLRpc
>> technology.
>> Though we were trying to implement the APIs with AvroRPC clients, Due to
>> some complications faced in implementing AvroRPC for OODT we had to stick
>> to the XMLRpc.
>> But as soon as OODT is released with the stable AvroRPC implementation
>> these will be updated to use AvroRPC instead of XMLRpc. Current changes
>> have been merged
>> with the master branch of DRAT.
>>
>> So far i have successfully completed all the API calls and also have
>> introduced one new Rest API for DRAT to execute dynamic workflows with
>> given tasks.
>> On the other hand for the web interface implementation i have designed
>> the mockups and shared in mailing list as well. While waiting for your
>> valuable
>> comments for those, I have started working on that too. For both the API
>> and web interface i have updated the DRAT wiki on Github.
>>
>> For web interface i have planned to use angular and angular Material
>> technologies.I'm willing to start with the latest version of angular.
>> The help i got from you people was much helpful, I'm very grateful for
>> that. Special Thanks to Chris and Imesha for helping me constantly.
>>
>> Your comments on my work and also suggestions for the rest would be very
>> helpful in we making this product a better one.
>>
>> Thank you all.
>>
>> On Tue, May 22, 2018 at 8:23 PM, Ahmed Ifhaam <ah...@gmail.com>
>> wrote:
>>
>>> hi all
>>> cureently i have completed the task of moving indexing function from
>>> process wrapper to java calls.
>>> but other
>>> funcyions like crawling,mapping needs some more deeper understanding
>>> where i need more time so ill be completing one more api replacement from
>>> one week as of today.
>>>
>>> from you all i need some support to understand what is the workflow
>>> whats the sendingevent function does and some docs which describes it
>>> better.
>>>
>>> thank you
>>>
>>> On 22 May 2018 12:06 a.m., "Chris Mattmann" <ma...@apache.org> wrote:
>>>
>>> Dear Ahmed,
>>>
>>>
>>>
>>> Can you please update the community on your plans for what you are
>>> working
>>> on what is done and what you expect to complete in the next week or two?
>>>
>>>
>>>
>>> Thanks,
>>> Chris
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> From: Ahmed Ifhaam <ah...@gmail.com>
>>> Reply-To: "dev@drat.apache.org" <de...@drat.apache.org>
>>> Date: Sunday, May 13, 2018 at 12:30 PM
>>> To: "dev@drat.apache.org" <de...@drat.apache.org>
>>> Subject: Joining on board
>>>
>>>
>>>
>>> hi all,
>>>
>>>
>>>
>>> As part of GSOC_2018 i'll be working on DRAT. I'm very pleased to work on
>>>
>>> this and expect your support and advice as well. I'ts great to joining
>>> you
>>>
>>> all. Thank you
>>>
>>>
>>>
>>>
>>> --
>>>
>>> *Ahmed Ifhaam *
>>>
>>> University of Kelaniya | MIT '19
>>>
>>> Tel :- 0662244937 | Mob :- 0776699609
>>>
>>> ahmedifhaam1@gmail.com <E-...@gmail.com>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> *Ahmed Ifhaam *
>> University of Kelaniya | MIT '19
>> Tel :- 0662244937 | Mob :- 0776699609
>> ahmedifhaam1@gmail.com <E-...@gmail.com>
>>
>
>
>
> --
> *Ahmed Ifhaam *
> University of Kelaniya | MIT '19
> Tel :- 0662244937 | Mob :- 0776699609
> ahmedifhaam1@gmail.com <E-...@gmail.com>
>
--
*Ahmed Ifhaam *
University of Kelaniya | MIT '19
Tel :- 0662244937 | Mob :- 0776699609
ahmedifhaam1@gmail.com <E-...@gmail.com>
Re: Current Status and Plans (was Re: Joining on board)
Posted by Ahmed Ifhaam <ah...@gmail.com>.
hi all,
After the last email, I started working on Vue,js and have implemented the
first page as follows Please check and let me know the feedback
It would be awesome to get some comments.
Screenshot from 2018-06-22 22-50-36.png
<https://drive.google.com/file/d/1YWs3D5MBaRp4wquCSq1W3yBW_8wT4qLv/view?usp=drive_web>
thank you
On Wed, Jun 13, 2018 at 12:36 AM, Ahmed Ifhaam <ah...@gmail.com>
wrote:
> Hi everyone,
>
> I was updating my progress of GSOC time to time. Since i'm completing one
> month i would like to summarize it.
> My task was to evolve the current Proteus interface for DRAT. As you all
> may know already, DRAT had four main functions
> Crawl,Index,Map,Reduce. When ever the rest api of DRAT called these
> functions, these were executing set of shell commands to perform the task.
>
> Since my task was to evolve the Web interface, With the guidance of my
> mentor and the community, i was tasked with creating the APIs
> with direct java API calls rather than shell scripts. DRAT uses OODT to
> perform these actions, while OODT currently moving to AvroRPC from XMLRpc
> technology.
> Though we were trying to implement the APIs with AvroRPC clients, Due to
> some complications faced in implementing AvroRPC for OODT we had to stick
> to the XMLRpc.
> But as soon as OODT is released with the stable AvroRPC implementation
> these will be updated to use AvroRPC instead of XMLRpc. Current changes
> have been merged
> with the master branch of DRAT.
>
> So far i have successfully completed all the API calls and also have
> introduced one new Rest API for DRAT to execute dynamic workflows with
> given tasks.
> On the other hand for the web interface implementation i have designed
> the mockups and shared in mailing list as well. While waiting for your
> valuable
> comments for those, I have started working on that too. For both the API
> and web interface i have updated the DRAT wiki on Github.
>
> For web interface i have planned to use angular and angular Material
> technologies.I'm willing to start with the latest version of angular.
> The help i got from you people was much helpful, I'm very grateful for
> that. Special Thanks to Chris and Imesha for helping me constantly.
>
> Your comments on my work and also suggestions for the rest would be very
> helpful in we making this product a better one.
>
> Thank you all.
>
> On Tue, May 22, 2018 at 8:23 PM, Ahmed Ifhaam <ah...@gmail.com>
> wrote:
>
>> hi all
>> cureently i have completed the task of moving indexing function from
>> process wrapper to java calls.
>> but other
>> funcyions like crawling,mapping needs some more deeper understanding
>> where i need more time so ill be completing one more api replacement from
>> one week as of today.
>>
>> from you all i need some support to understand what is the workflow whats
>> the sendingevent function does and some docs which describes it better.
>>
>> thank you
>>
>> On 22 May 2018 12:06 a.m., "Chris Mattmann" <ma...@apache.org> wrote:
>>
>> Dear Ahmed,
>>
>>
>>
>> Can you please update the community on your plans for what you are working
>> on what is done and what you expect to complete in the next week or two?
>>
>>
>>
>> Thanks,
>> Chris
>>
>>
>>
>>
>>
>>
>>
>> From: Ahmed Ifhaam <ah...@gmail.com>
>> Reply-To: "dev@drat.apache.org" <de...@drat.apache.org>
>> Date: Sunday, May 13, 2018 at 12:30 PM
>> To: "dev@drat.apache.org" <de...@drat.apache.org>
>> Subject: Joining on board
>>
>>
>>
>> hi all,
>>
>>
>>
>> As part of GSOC_2018 i'll be working on DRAT. I'm very pleased to work on
>>
>> this and expect your support and advice as well. I'ts great to joining you
>>
>> all. Thank you
>>
>>
>>
>>
>> --
>>
>> *Ahmed Ifhaam *
>>
>> University of Kelaniya | MIT '19
>>
>> Tel :- 0662244937 | Mob :- 0776699609
>>
>> ahmedifhaam1@gmail.com <E-...@gmail.com>
>>
>>
>>
>>
>>
>
>
> --
> *Ahmed Ifhaam *
> University of Kelaniya | MIT '19
> Tel :- 0662244937 | Mob :- 0776699609
> ahmedifhaam1@gmail.com <E-...@gmail.com>
>
--
*Ahmed Ifhaam *
University of Kelaniya | MIT '19
Tel :- 0662244937 | Mob :- 0776699609
ahmedifhaam1@gmail.com <E-...@gmail.com>