You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by Casey Stella <ce...@gmail.com> on 2016/09/19 22:06:15 UTC
[DISCUSS] METRON-433: Documentation update
Hi all,
I noticed that our public facing docs were really getting stale. This is a
normal thing in a fast moving project, but I imagine it's really hard for
the community to get engaged with a project that has such details caught up
inside of code and unit tests. As such, I thought it is reasonable to
spend a little time ensuring the docs are better organized and clear, so I
spent the day focusing on that:
1. they were nonexistent in a lot of places like model as a service
which are new
2. they were stale (mostly around stellar functions)
3. they were incomplete (parsers take configs, we don't document
that..we now do)
4. they were not clear. I added some clarification and pictures of the
topologies
5. they were in the wrong place (e.g. enrichment config documentation
was in metron-common instead of metron-enrichment)
6. there wasn't a starting place from the top level Readme into the
scary subdepths of the individual projects
I also gave a small fully worked example for some of the commonly confusing
tasks, like doing a stellar enrichment or deploying a MaaS model. I wrote
them up, but they were caught up inside of PR comments, this just brought
them into the docs.
You can find the PR here
<https://github.com/apache/incubator-metron/pull/260>. It's already gotten
a +1, but I'd love to see some discussion around what ELSE we can do to
make it easier to get community contributions. After all, there would be
no Metron without its community.
Best,
Casey
Re: [DISCUSS] METRON-433: Documentation update
Posted by Carolyn Duby <cd...@hortonworks.com>.
Nice!
On 9/19/16, 6:06 PM, "Casey Stella" <ce...@gmail.com> wrote:
>Hi all,
>
>I noticed that our public facing docs were really getting stale. This is a
>normal thing in a fast moving project, but I imagine it's really hard for
>the community to get engaged with a project that has such details caught up
>inside of code and unit tests. As such, I thought it is reasonable to
>spend a little time ensuring the docs are better organized and clear, so I
>spent the day focusing on that:
>
> 1. they were nonexistent in a lot of places like model as a service
> which are new
> 2. they were stale (mostly around stellar functions)
> 3. they were incomplete (parsers take configs, we don't document
> that..we now do)
> 4. they were not clear. I added some clarification and pictures of the
> topologies
> 5. they were in the wrong place (e.g. enrichment config documentation
> was in metron-common instead of metron-enrichment)
> 6. there wasn't a starting place from the top level Readme into the
> scary subdepths of the individual projects
>
>I also gave a small fully worked example for some of the commonly confusing
>tasks, like doing a stellar enrichment or deploying a MaaS model. I wrote
>them up, but they were caught up inside of PR comments, this just brought
>them into the docs.
>
>You can find the PR here
><https://github.com/apache/incubator-metron/pull/260>. It's already gotten
>a +1, but I'd love to see some discussion around what ELSE we can do to
>make it easier to get community contributions. After all, there would be
>no Metron without its community.
>
>Best,
>
>Casey
Re: [DISCUSS] METRON-433: Documentation update
Posted by Anand Subramanian <as...@hortonworks.com>.
Thank you, Casey.
Please see comment here:
https://github.com/apache/incubator-metron/pull/260#issuecomment-248315513
Regards,
Anand
From: Casey Stella <ce...@gmail.com>>
Reply-To: "user@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <us...@metron.incubator.apache.org>>
Date: Tuesday, September 20, 2016 at 7:48 PM
To: "dev@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <de...@metron.incubator.apache.org>>
Cc: "user@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <us...@metron.incubator.apache.org>>
Subject: Re: [DISCUSS] METRON-433: Documentation update
Oh, it appears that attachments get stripped off sending to the mailing lists. Maybe you can make the comment on the PR (https://github.com/apache/incubator-metron/pull/260) and link to it here?
On Tue, Sep 20, 2016 at 10:10 AM, Anand Subramanian <as...@hortonworks.com>> wrote:
Would the Storm topology visualization diagram work for the parser, enrichment and indexing illustration?
Attached is a sample image for the parser topology. I can work on getting rid of the numbers beneath the names.
Cheers,
Anand
From: Casey Stella <ce...@gmail.com>>
Reply-To: "user@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <us...@metron.incubator.apache.org>>
Date: Tuesday, September 20, 2016 at 7:03 PM
To: "dev@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <de...@metron.incubator.apache.org>>
Cc: "user@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <us...@metron.incubator.apache.org>>
Subject: Re: [DISCUSS] METRON-433: Documentation update
I do think that's a good idea. Unfortunately, I made those images in powerpoint and I don't know how to get stencils with the storm symbols on them. Does anyone want to volunteer to transition the architecture docs over to use storm symbols?
Best,
Casey
On Tue, Sep 20, 2016 at 8:30 AM, Zeolla@GMail.com<ma...@GMail.com> <ze...@gmail.com>> wrote:
Do you think it would be worthwhile to revise the enrichment and parser
illustrations to use storm component symbols instead of generic symbols?
That may add additional clarity to people who aren't familiar with all of
the different components and how they work together. Still running through
the updated docs but so far it's great stuff.
Jon
On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson <ke...@gmail.com>>
wrote:
> I have a revised document for installation that I would like to
> contribute. (ubuntu).
> What I have not found is an easily accessible of contributing. This may be
> due to my time poor life where I simply do not have the time to be
> searching for such a link.
> Where can I find community contributions link?
>
> Kev
>
>
>
> On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella <ce...@gmail.com>> wrote:
>
>> Hi all,
>>
>> I noticed that our public facing docs were really getting stale. This is
>> a normal thing in a fast moving project, but I imagine it's really hard for
>> the community to get engaged with a project that has such details caught up
>> inside of code and unit tests. As such, I thought it is reasonable to
>> spend a little time ensuring the docs are better organized and clear, so I
>> spent the day focusing on that:
>>
>
>> 1. they were nonexistent in a lot of places like model as a service
>> which are new
>>
>>
>> 1. they were stale (mostly around stellar functions)
>>
>>
>> 1. they were incomplete (parsers take configs, we don't document
>> that..we now do)
>>
>>
>> 1. they were not clear. I added some clarification and pictures of
>> the topologies
>>
>>
>> 1. they were in the wrong place (e.g. enrichment config documentation
>> was in metron-common instead of metron-enrichment)
>>
>>
>> 1. there wasn't a starting place from the top level Readme into the
>> scary subdepths of the individual projects
>>
>> I also gave a small fully worked example for some of the commonly
>> confusing tasks, like doing a stellar enrichment or deploying a MaaS
>> model. I wrote them up, but they were caught up inside of PR comments,
>> this just brought them into the docs.
>>
>
>> You can find the PR here
>> <https://github.com/apache/incubator-metron/pull/260>. It's already
>> gotten a +1, but I'd love to see some discussion around what ELSE we can do
>> to make it easier to get community contributions. After all, there would
>> be no Metron without its community.
>>
>> Best,
>>
>> Casey
>>
>
>
>
> --
> --
> "Democracy is two wolves and a lamb voting on what to have for lunch.
> Liberty is a well-armed lamb contesting the vote."
>
--
Jon
Re: [DISCUSS] METRON-433: Documentation update
Posted by Anand Subramanian <as...@hortonworks.com>.
Thank you, Casey.
Please see comment here:
https://github.com/apache/incubator-metron/pull/260#issuecomment-248315513
Regards,
Anand
From: Casey Stella <ce...@gmail.com>>
Reply-To: "user@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <us...@metron.incubator.apache.org>>
Date: Tuesday, September 20, 2016 at 7:48 PM
To: "dev@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <de...@metron.incubator.apache.org>>
Cc: "user@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <us...@metron.incubator.apache.org>>
Subject: Re: [DISCUSS] METRON-433: Documentation update
Oh, it appears that attachments get stripped off sending to the mailing lists. Maybe you can make the comment on the PR (https://github.com/apache/incubator-metron/pull/260) and link to it here?
On Tue, Sep 20, 2016 at 10:10 AM, Anand Subramanian <as...@hortonworks.com>> wrote:
Would the Storm topology visualization diagram work for the parser, enrichment and indexing illustration?
Attached is a sample image for the parser topology. I can work on getting rid of the numbers beneath the names.
Cheers,
Anand
From: Casey Stella <ce...@gmail.com>>
Reply-To: "user@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <us...@metron.incubator.apache.org>>
Date: Tuesday, September 20, 2016 at 7:03 PM
To: "dev@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <de...@metron.incubator.apache.org>>
Cc: "user@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <us...@metron.incubator.apache.org>>
Subject: Re: [DISCUSS] METRON-433: Documentation update
I do think that's a good idea. Unfortunately, I made those images in powerpoint and I don't know how to get stencils with the storm symbols on them. Does anyone want to volunteer to transition the architecture docs over to use storm symbols?
Best,
Casey
On Tue, Sep 20, 2016 at 8:30 AM, Zeolla@GMail.com<ma...@GMail.com> <ze...@gmail.com>> wrote:
Do you think it would be worthwhile to revise the enrichment and parser
illustrations to use storm component symbols instead of generic symbols?
That may add additional clarity to people who aren't familiar with all of
the different components and how they work together. Still running through
the updated docs but so far it's great stuff.
Jon
On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson <ke...@gmail.com>>
wrote:
> I have a revised document for installation that I would like to
> contribute. (ubuntu).
> What I have not found is an easily accessible of contributing. This may be
> due to my time poor life where I simply do not have the time to be
> searching for such a link.
> Where can I find community contributions link?
>
> Kev
>
>
>
> On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella <ce...@gmail.com>> wrote:
>
>> Hi all,
>>
>> I noticed that our public facing docs were really getting stale. This is
>> a normal thing in a fast moving project, but I imagine it's really hard for
>> the community to get engaged with a project that has such details caught up
>> inside of code and unit tests. As such, I thought it is reasonable to
>> spend a little time ensuring the docs are better organized and clear, so I
>> spent the day focusing on that:
>>
>
>> 1. they were nonexistent in a lot of places like model as a service
>> which are new
>>
>>
>> 1. they were stale (mostly around stellar functions)
>>
>>
>> 1. they were incomplete (parsers take configs, we don't document
>> that..we now do)
>>
>>
>> 1. they were not clear. I added some clarification and pictures of
>> the topologies
>>
>>
>> 1. they were in the wrong place (e.g. enrichment config documentation
>> was in metron-common instead of metron-enrichment)
>>
>>
>> 1. there wasn't a starting place from the top level Readme into the
>> scary subdepths of the individual projects
>>
>> I also gave a small fully worked example for some of the commonly
>> confusing tasks, like doing a stellar enrichment or deploying a MaaS
>> model. I wrote them up, but they were caught up inside of PR comments,
>> this just brought them into the docs.
>>
>
>> You can find the PR here
>> <https://github.com/apache/incubator-metron/pull/260>. It's already
>> gotten a +1, but I'd love to see some discussion around what ELSE we can do
>> to make it easier to get community contributions. After all, there would
>> be no Metron without its community.
>>
>> Best,
>>
>> Casey
>>
>
>
>
> --
> --
> "Democracy is two wolves and a lamb voting on what to have for lunch.
> Liberty is a well-armed lamb contesting the vote."
>
--
Jon
Re: [DISCUSS] METRON-433: Documentation update
Posted by Casey Stella <ce...@gmail.com>.
Oh, it appears that attachments get stripped off sending to the mailing
lists. Maybe you can make the comment on the PR (
https://github.com/apache/incubator-metron/pull/260) and link to it here?
On Tue, Sep 20, 2016 at 10:10 AM, Anand Subramanian <
asubramanian@hortonworks.com> wrote:
> Would the Storm topology visualization diagram work for the parser,
> enrichment and indexing illustration?
>
> Attached is a sample image for the parser topology. I can work on getting
> rid of the numbers beneath the names.
>
> Cheers,
> Anand
>
> From: Casey Stella <ce...@gmail.com>
> Reply-To: "user@metron.incubator.apache.org" <
> user@metron.incubator.apache.org>
> Date: Tuesday, September 20, 2016 at 7:03 PM
> To: "dev@metron.incubator.apache.org" <de...@metron.incubator.apache.org>
> Cc: "user@metron.incubator.apache.org" <us...@metron.incubator.apache.org>
> Subject: Re: [DISCUSS] METRON-433: Documentation update
>
> I do think that's a good idea. Unfortunately, I made those images in
> powerpoint and I don't know how to get stencils with the storm symbols on
> them. Does anyone want to volunteer to transition the architecture docs
> over to use storm symbols?
>
> Best,
>
> Casey
>
> On Tue, Sep 20, 2016 at 8:30 AM, Zeolla@GMail.com <ze...@gmail.com>
> wrote:
>
>> Do you think it would be worthwhile to revise the enrichment and parser
>> illustrations to use storm component symbols instead of generic symbols?
>> That may add additional clarity to people who aren't familiar with all of
>> the different components and how they work together. Still running
>> through
>> the updated docs but so far it's great stuff.
>>
>> Jon
>>
>> On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson <ke...@gmail.com>
>> wrote:
>>
>> > I have a revised document for installation that I would like to
>> > contribute. (ubuntu).
>> > What I have not found is an easily accessible of contributing. This may
>> be
>> > due to my time poor life where I simply do not have the time to be
>> > searching for such a link.
>> > Where can I find community contributions link?
>> >
>> > Kev
>> >
>> >
>> >
>> > On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella <ce...@gmail.com>
>> wrote:
>> >
>> >> Hi all,
>> >>
>> >> I noticed that our public facing docs were really getting stale. This
>> is
>> >> a normal thing in a fast moving project, but I imagine it's really
>> hard for
>> >> the community to get engaged with a project that has such details
>> caught up
>> >> inside of code and unit tests. As such, I thought it is reasonable to
>> >> spend a little time ensuring the docs are better organized and clear,
>> so I
>> >> spent the day focusing on that:
>> >>
>> >
>> >> 1. they were nonexistent in a lot of places like model as a service
>> >> which are new
>> >>
>> >>
>> >> 1. they were stale (mostly around stellar functions)
>> >>
>> >>
>> >> 1. they were incomplete (parsers take configs, we don't document
>> >> that..we now do)
>> >>
>> >>
>> >> 1. they were not clear. I added some clarification and pictures of
>> >> the topologies
>> >>
>> >>
>> >> 1. they were in the wrong place (e.g. enrichment config
>> documentation
>> >> was in metron-common instead of metron-enrichment)
>> >>
>> >>
>> >> 1. there wasn't a starting place from the top level Readme into the
>> >> scary subdepths of the individual projects
>> >>
>> >> I also gave a small fully worked example for some of the commonly
>> >> confusing tasks, like doing a stellar enrichment or deploying a MaaS
>> >> model. I wrote them up, but they were caught up inside of PR comments,
>> >> this just brought them into the docs.
>> >>
>> >
>> >> You can find the PR here
>> >> <https://github.com/apache/incubator-metron/pull/260>. It's already
>> >> gotten a +1, but I'd love to see some discussion around what ELSE we
>> can do
>> >> to make it easier to get community contributions. After all, there
>> would
>> >> be no Metron without its community.
>> >>
>> >> Best,
>> >>
>> >> Casey
>> >>
>> >
>> >
>> >
>> > --
>> > --
>> > "Democracy is two wolves and a lamb voting on what to have for lunch.
>> > Liberty is a well-armed lamb contesting the vote."
>> >
>> --
>>
>> Jon
>>
>
>
Re: [DISCUSS] METRON-433: Documentation update
Posted by Casey Stella <ce...@gmail.com>.
Oh, it appears that attachments get stripped off sending to the mailing
lists. Maybe you can make the comment on the PR (
https://github.com/apache/incubator-metron/pull/260) and link to it here?
On Tue, Sep 20, 2016 at 10:10 AM, Anand Subramanian <
asubramanian@hortonworks.com> wrote:
> Would the Storm topology visualization diagram work for the parser,
> enrichment and indexing illustration?
>
> Attached is a sample image for the parser topology. I can work on getting
> rid of the numbers beneath the names.
>
> Cheers,
> Anand
>
> From: Casey Stella <ce...@gmail.com>
> Reply-To: "user@metron.incubator.apache.org" <
> user@metron.incubator.apache.org>
> Date: Tuesday, September 20, 2016 at 7:03 PM
> To: "dev@metron.incubator.apache.org" <de...@metron.incubator.apache.org>
> Cc: "user@metron.incubator.apache.org" <us...@metron.incubator.apache.org>
> Subject: Re: [DISCUSS] METRON-433: Documentation update
>
> I do think that's a good idea. Unfortunately, I made those images in
> powerpoint and I don't know how to get stencils with the storm symbols on
> them. Does anyone want to volunteer to transition the architecture docs
> over to use storm symbols?
>
> Best,
>
> Casey
>
> On Tue, Sep 20, 2016 at 8:30 AM, Zeolla@GMail.com <ze...@gmail.com>
> wrote:
>
>> Do you think it would be worthwhile to revise the enrichment and parser
>> illustrations to use storm component symbols instead of generic symbols?
>> That may add additional clarity to people who aren't familiar with all of
>> the different components and how they work together. Still running
>> through
>> the updated docs but so far it's great stuff.
>>
>> Jon
>>
>> On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson <ke...@gmail.com>
>> wrote:
>>
>> > I have a revised document for installation that I would like to
>> > contribute. (ubuntu).
>> > What I have not found is an easily accessible of contributing. This may
>> be
>> > due to my time poor life where I simply do not have the time to be
>> > searching for such a link.
>> > Where can I find community contributions link?
>> >
>> > Kev
>> >
>> >
>> >
>> > On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella <ce...@gmail.com>
>> wrote:
>> >
>> >> Hi all,
>> >>
>> >> I noticed that our public facing docs were really getting stale. This
>> is
>> >> a normal thing in a fast moving project, but I imagine it's really
>> hard for
>> >> the community to get engaged with a project that has such details
>> caught up
>> >> inside of code and unit tests. As such, I thought it is reasonable to
>> >> spend a little time ensuring the docs are better organized and clear,
>> so I
>> >> spent the day focusing on that:
>> >>
>> >
>> >> 1. they were nonexistent in a lot of places like model as a service
>> >> which are new
>> >>
>> >>
>> >> 1. they were stale (mostly around stellar functions)
>> >>
>> >>
>> >> 1. they were incomplete (parsers take configs, we don't document
>> >> that..we now do)
>> >>
>> >>
>> >> 1. they were not clear. I added some clarification and pictures of
>> >> the topologies
>> >>
>> >>
>> >> 1. they were in the wrong place (e.g. enrichment config
>> documentation
>> >> was in metron-common instead of metron-enrichment)
>> >>
>> >>
>> >> 1. there wasn't a starting place from the top level Readme into the
>> >> scary subdepths of the individual projects
>> >>
>> >> I also gave a small fully worked example for some of the commonly
>> >> confusing tasks, like doing a stellar enrichment or deploying a MaaS
>> >> model. I wrote them up, but they were caught up inside of PR comments,
>> >> this just brought them into the docs.
>> >>
>> >
>> >> You can find the PR here
>> >> <https://github.com/apache/incubator-metron/pull/260>. It's already
>> >> gotten a +1, but I'd love to see some discussion around what ELSE we
>> can do
>> >> to make it easier to get community contributions. After all, there
>> would
>> >> be no Metron without its community.
>> >>
>> >> Best,
>> >>
>> >> Casey
>> >>
>> >
>> >
>> >
>> > --
>> > --
>> > "Democracy is two wolves and a lamb voting on what to have for lunch.
>> > Liberty is a well-armed lamb contesting the vote."
>> >
>> --
>>
>> Jon
>>
>
>
Re: [DISCUSS] METRON-433: Documentation update
Posted by Anand Subramanian <as...@hortonworks.com>.
Would the Storm topology visualization diagram work for the parser, enrichment and indexing illustration?
Attached is a sample image for the parser topology. I can work on getting rid of the numbers beneath the names.
Cheers,
Anand
From: Casey Stella <ce...@gmail.com>>
Reply-To: "user@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <us...@metron.incubator.apache.org>>
Date: Tuesday, September 20, 2016 at 7:03 PM
To: "dev@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <de...@metron.incubator.apache.org>>
Cc: "user@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <us...@metron.incubator.apache.org>>
Subject: Re: [DISCUSS] METRON-433: Documentation update
I do think that's a good idea. Unfortunately, I made those images in powerpoint and I don't know how to get stencils with the storm symbols on them. Does anyone want to volunteer to transition the architecture docs over to use storm symbols?
Best,
Casey
On Tue, Sep 20, 2016 at 8:30 AM, Zeolla@GMail.com<ma...@GMail.com> <ze...@gmail.com>> wrote:
Do you think it would be worthwhile to revise the enrichment and parser
illustrations to use storm component symbols instead of generic symbols?
That may add additional clarity to people who aren't familiar with all of
the different components and how they work together. Still running through
the updated docs but so far it's great stuff.
Jon
On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson <ke...@gmail.com>>
wrote:
> I have a revised document for installation that I would like to
> contribute. (ubuntu).
> What I have not found is an easily accessible of contributing. This may be
> due to my time poor life where I simply do not have the time to be
> searching for such a link.
> Where can I find community contributions link?
>
> Kev
>
>
>
> On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella <ce...@gmail.com>> wrote:
>
>> Hi all,
>>
>> I noticed that our public facing docs were really getting stale. This is
>> a normal thing in a fast moving project, but I imagine it's really hard for
>> the community to get engaged with a project that has such details caught up
>> inside of code and unit tests. As such, I thought it is reasonable to
>> spend a little time ensuring the docs are better organized and clear, so I
>> spent the day focusing on that:
>>
>
>> 1. they were nonexistent in a lot of places like model as a service
>> which are new
>>
>>
>> 1. they were stale (mostly around stellar functions)
>>
>>
>> 1. they were incomplete (parsers take configs, we don't document
>> that..we now do)
>>
>>
>> 1. they were not clear. I added some clarification and pictures of
>> the topologies
>>
>>
>> 1. they were in the wrong place (e.g. enrichment config documentation
>> was in metron-common instead of metron-enrichment)
>>
>>
>> 1. there wasn't a starting place from the top level Readme into the
>> scary subdepths of the individual projects
>>
>> I also gave a small fully worked example for some of the commonly
>> confusing tasks, like doing a stellar enrichment or deploying a MaaS
>> model. I wrote them up, but they were caught up inside of PR comments,
>> this just brought them into the docs.
>>
>
>> You can find the PR here
>> <https://github.com/apache/incubator-metron/pull/260>. It's already
>> gotten a +1, but I'd love to see some discussion around what ELSE we can do
>> to make it easier to get community contributions. After all, there would
>> be no Metron without its community.
>>
>> Best,
>>
>> Casey
>>
>
>
>
> --
> --
> "Democracy is two wolves and a lamb voting on what to have for lunch.
> Liberty is a well-armed lamb contesting the vote."
>
--
Jon
Re: [DISCUSS] METRON-433: Documentation update
Posted by Anand Subramanian <as...@hortonworks.com>.
Would the Storm topology visualization diagram work for the parser, enrichment and indexing illustration?
Attached is a sample image for the parser topology. I can work on getting rid of the numbers beneath the names.
Cheers,
Anand
From: Casey Stella <ce...@gmail.com>>
Reply-To: "user@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <us...@metron.incubator.apache.org>>
Date: Tuesday, September 20, 2016 at 7:03 PM
To: "dev@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <de...@metron.incubator.apache.org>>
Cc: "user@metron.incubator.apache.org<ma...@metron.incubator.apache.org>" <us...@metron.incubator.apache.org>>
Subject: Re: [DISCUSS] METRON-433: Documentation update
I do think that's a good idea. Unfortunately, I made those images in powerpoint and I don't know how to get stencils with the storm symbols on them. Does anyone want to volunteer to transition the architecture docs over to use storm symbols?
Best,
Casey
On Tue, Sep 20, 2016 at 8:30 AM, Zeolla@GMail.com<ma...@GMail.com> <ze...@gmail.com>> wrote:
Do you think it would be worthwhile to revise the enrichment and parser
illustrations to use storm component symbols instead of generic symbols?
That may add additional clarity to people who aren't familiar with all of
the different components and how they work together. Still running through
the updated docs but so far it's great stuff.
Jon
On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson <ke...@gmail.com>>
wrote:
> I have a revised document for installation that I would like to
> contribute. (ubuntu).
> What I have not found is an easily accessible of contributing. This may be
> due to my time poor life where I simply do not have the time to be
> searching for such a link.
> Where can I find community contributions link?
>
> Kev
>
>
>
> On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella <ce...@gmail.com>> wrote:
>
>> Hi all,
>>
>> I noticed that our public facing docs were really getting stale. This is
>> a normal thing in a fast moving project, but I imagine it's really hard for
>> the community to get engaged with a project that has such details caught up
>> inside of code and unit tests. As such, I thought it is reasonable to
>> spend a little time ensuring the docs are better organized and clear, so I
>> spent the day focusing on that:
>>
>
>> 1. they were nonexistent in a lot of places like model as a service
>> which are new
>>
>>
>> 1. they were stale (mostly around stellar functions)
>>
>>
>> 1. they were incomplete (parsers take configs, we don't document
>> that..we now do)
>>
>>
>> 1. they were not clear. I added some clarification and pictures of
>> the topologies
>>
>>
>> 1. they were in the wrong place (e.g. enrichment config documentation
>> was in metron-common instead of metron-enrichment)
>>
>>
>> 1. there wasn't a starting place from the top level Readme into the
>> scary subdepths of the individual projects
>>
>> I also gave a small fully worked example for some of the commonly
>> confusing tasks, like doing a stellar enrichment or deploying a MaaS
>> model. I wrote them up, but they were caught up inside of PR comments,
>> this just brought them into the docs.
>>
>
>> You can find the PR here
>> <https://github.com/apache/incubator-metron/pull/260>. It's already
>> gotten a +1, but I'd love to see some discussion around what ELSE we can do
>> to make it easier to get community contributions. After all, there would
>> be no Metron without its community.
>>
>> Best,
>>
>> Casey
>>
>
>
>
> --
> --
> "Democracy is two wolves and a lamb voting on what to have for lunch.
> Liberty is a well-armed lamb contesting the vote."
>
--
Jon
Re: [DISCUSS] METRON-433: Documentation update
Posted by Casey Stella <ce...@gmail.com>.
I do think that's a good idea. Unfortunately, I made those images in
powerpoint and I don't know how to get stencils with the storm symbols on
them. Does anyone want to volunteer to transition the architecture docs
over to use storm symbols?
Best,
Casey
On Tue, Sep 20, 2016 at 8:30 AM, Zeolla@GMail.com <ze...@gmail.com> wrote:
> Do you think it would be worthwhile to revise the enrichment and parser
> illustrations to use storm component symbols instead of generic symbols?
> That may add additional clarity to people who aren't familiar with all of
> the different components and how they work together. Still running through
> the updated docs but so far it's great stuff.
>
> Jon
>
> On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson <ke...@gmail.com>
> wrote:
>
> > I have a revised document for installation that I would like to
> > contribute. (ubuntu).
> > What I have not found is an easily accessible of contributing. This may
> be
> > due to my time poor life where I simply do not have the time to be
> > searching for such a link.
> > Where can I find community contributions link?
> >
> > Kev
> >
> >
> >
> > On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella <ce...@gmail.com>
> wrote:
> >
> >> Hi all,
> >>
> >> I noticed that our public facing docs were really getting stale. This
> is
> >> a normal thing in a fast moving project, but I imagine it's really hard
> for
> >> the community to get engaged with a project that has such details
> caught up
> >> inside of code and unit tests. As such, I thought it is reasonable to
> >> spend a little time ensuring the docs are better organized and clear,
> so I
> >> spent the day focusing on that:
> >>
> >
> >> 1. they were nonexistent in a lot of places like model as a service
> >> which are new
> >>
> >>
> >> 1. they were stale (mostly around stellar functions)
> >>
> >>
> >> 1. they were incomplete (parsers take configs, we don't document
> >> that..we now do)
> >>
> >>
> >> 1. they were not clear. I added some clarification and pictures of
> >> the topologies
> >>
> >>
> >> 1. they were in the wrong place (e.g. enrichment config documentation
> >> was in metron-common instead of metron-enrichment)
> >>
> >>
> >> 1. there wasn't a starting place from the top level Readme into the
> >> scary subdepths of the individual projects
> >>
> >> I also gave a small fully worked example for some of the commonly
> >> confusing tasks, like doing a stellar enrichment or deploying a MaaS
> >> model. I wrote them up, but they were caught up inside of PR comments,
> >> this just brought them into the docs.
> >>
> >
> >> You can find the PR here
> >> <https://github.com/apache/incubator-metron/pull/260>. It's already
> >> gotten a +1, but I'd love to see some discussion around what ELSE we
> can do
> >> to make it easier to get community contributions. After all, there
> would
> >> be no Metron without its community.
> >>
> >> Best,
> >>
> >> Casey
> >>
> >
> >
> >
> > --
> > --
> > "Democracy is two wolves and a lamb voting on what to have for lunch.
> > Liberty is a well-armed lamb contesting the vote."
> >
> --
>
> Jon
>
Re: [DISCUSS] METRON-433: Documentation update
Posted by Casey Stella <ce...@gmail.com>.
I do think that's a good idea. Unfortunately, I made those images in
powerpoint and I don't know how to get stencils with the storm symbols on
them. Does anyone want to volunteer to transition the architecture docs
over to use storm symbols?
Best,
Casey
On Tue, Sep 20, 2016 at 8:30 AM, Zeolla@GMail.com <ze...@gmail.com> wrote:
> Do you think it would be worthwhile to revise the enrichment and parser
> illustrations to use storm component symbols instead of generic symbols?
> That may add additional clarity to people who aren't familiar with all of
> the different components and how they work together. Still running through
> the updated docs but so far it's great stuff.
>
> Jon
>
> On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson <ke...@gmail.com>
> wrote:
>
> > I have a revised document for installation that I would like to
> > contribute. (ubuntu).
> > What I have not found is an easily accessible of contributing. This may
> be
> > due to my time poor life where I simply do not have the time to be
> > searching for such a link.
> > Where can I find community contributions link?
> >
> > Kev
> >
> >
> >
> > On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella <ce...@gmail.com>
> wrote:
> >
> >> Hi all,
> >>
> >> I noticed that our public facing docs were really getting stale. This
> is
> >> a normal thing in a fast moving project, but I imagine it's really hard
> for
> >> the community to get engaged with a project that has such details
> caught up
> >> inside of code and unit tests. As such, I thought it is reasonable to
> >> spend a little time ensuring the docs are better organized and clear,
> so I
> >> spent the day focusing on that:
> >>
> >
> >> 1. they were nonexistent in a lot of places like model as a service
> >> which are new
> >>
> >>
> >> 1. they were stale (mostly around stellar functions)
> >>
> >>
> >> 1. they were incomplete (parsers take configs, we don't document
> >> that..we now do)
> >>
> >>
> >> 1. they were not clear. I added some clarification and pictures of
> >> the topologies
> >>
> >>
> >> 1. they were in the wrong place (e.g. enrichment config documentation
> >> was in metron-common instead of metron-enrichment)
> >>
> >>
> >> 1. there wasn't a starting place from the top level Readme into the
> >> scary subdepths of the individual projects
> >>
> >> I also gave a small fully worked example for some of the commonly
> >> confusing tasks, like doing a stellar enrichment or deploying a MaaS
> >> model. I wrote them up, but they were caught up inside of PR comments,
> >> this just brought them into the docs.
> >>
> >
> >> You can find the PR here
> >> <https://github.com/apache/incubator-metron/pull/260>. It's already
> >> gotten a +1, but I'd love to see some discussion around what ELSE we
> can do
> >> to make it easier to get community contributions. After all, there
> would
> >> be no Metron without its community.
> >>
> >> Best,
> >>
> >> Casey
> >>
> >
> >
> >
> > --
> > --
> > "Democracy is two wolves and a lamb voting on what to have for lunch.
> > Liberty is a well-armed lamb contesting the vote."
> >
> --
>
> Jon
>
Re: [DISCUSS] METRON-433: Documentation update
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
Do you think it would be worthwhile to revise the enrichment and parser
illustrations to use storm component symbols instead of generic symbols?
That may add additional clarity to people who aren't familiar with all of
the different components and how they work together. Still running through
the updated docs but so far it's great stuff.
Jon
On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson <ke...@gmail.com>
wrote:
> I have a revised document for installation that I would like to
> contribute. (ubuntu).
> What I have not found is an easily accessible of contributing. This may be
> due to my time poor life where I simply do not have the time to be
> searching for such a link.
> Where can I find community contributions link?
>
> Kev
>
>
>
> On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella <ce...@gmail.com> wrote:
>
>> Hi all,
>>
>> I noticed that our public facing docs were really getting stale. This is
>> a normal thing in a fast moving project, but I imagine it's really hard for
>> the community to get engaged with a project that has such details caught up
>> inside of code and unit tests. As such, I thought it is reasonable to
>> spend a little time ensuring the docs are better organized and clear, so I
>> spent the day focusing on that:
>>
>
>> 1. they were nonexistent in a lot of places like model as a service
>> which are new
>>
>>
>> 1. they were stale (mostly around stellar functions)
>>
>>
>> 1. they were incomplete (parsers take configs, we don't document
>> that..we now do)
>>
>>
>> 1. they were not clear. I added some clarification and pictures of
>> the topologies
>>
>>
>> 1. they were in the wrong place (e.g. enrichment config documentation
>> was in metron-common instead of metron-enrichment)
>>
>>
>> 1. there wasn't a starting place from the top level Readme into the
>> scary subdepths of the individual projects
>>
>> I also gave a small fully worked example for some of the commonly
>> confusing tasks, like doing a stellar enrichment or deploying a MaaS
>> model. I wrote them up, but they were caught up inside of PR comments,
>> this just brought them into the docs.
>>
>
>> You can find the PR here
>> <https://github.com/apache/incubator-metron/pull/260>. It's already
>> gotten a +1, but I'd love to see some discussion around what ELSE we can do
>> to make it easier to get community contributions. After all, there would
>> be no Metron without its community.
>>
>> Best,
>>
>> Casey
>>
>
>
>
> --
> --
> "Democracy is two wolves and a lamb voting on what to have for lunch.
> Liberty is a well-armed lamb contesting the vote."
>
--
Jon
Re: [DISCUSS] METRON-433: Documentation update
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
Do you think it would be worthwhile to revise the enrichment and parser
illustrations to use storm component symbols instead of generic symbols?
That may add additional clarity to people who aren't familiar with all of
the different components and how they work together. Still running through
the updated docs but so far it's great stuff.
Jon
On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson <ke...@gmail.com>
wrote:
> I have a revised document for installation that I would like to
> contribute. (ubuntu).
> What I have not found is an easily accessible of contributing. This may be
> due to my time poor life where I simply do not have the time to be
> searching for such a link.
> Where can I find community contributions link?
>
> Kev
>
>
>
> On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella <ce...@gmail.com> wrote:
>
>> Hi all,
>>
>> I noticed that our public facing docs were really getting stale. This is
>> a normal thing in a fast moving project, but I imagine it's really hard for
>> the community to get engaged with a project that has such details caught up
>> inside of code and unit tests. As such, I thought it is reasonable to
>> spend a little time ensuring the docs are better organized and clear, so I
>> spent the day focusing on that:
>>
>
>> 1. they were nonexistent in a lot of places like model as a service
>> which are new
>>
>>
>> 1. they were stale (mostly around stellar functions)
>>
>>
>> 1. they were incomplete (parsers take configs, we don't document
>> that..we now do)
>>
>>
>> 1. they were not clear. I added some clarification and pictures of
>> the topologies
>>
>>
>> 1. they were in the wrong place (e.g. enrichment config documentation
>> was in metron-common instead of metron-enrichment)
>>
>>
>> 1. there wasn't a starting place from the top level Readme into the
>> scary subdepths of the individual projects
>>
>> I also gave a small fully worked example for some of the commonly
>> confusing tasks, like doing a stellar enrichment or deploying a MaaS
>> model. I wrote them up, but they were caught up inside of PR comments,
>> this just brought them into the docs.
>>
>
>> You can find the PR here
>> <https://github.com/apache/incubator-metron/pull/260>. It's already
>> gotten a +1, but I'd love to see some discussion around what ELSE we can do
>> to make it easier to get community contributions. After all, there would
>> be no Metron without its community.
>>
>> Best,
>>
>> Casey
>>
>
>
>
> --
> --
> "Democracy is two wolves and a lamb voting on what to have for lunch.
> Liberty is a well-armed lamb contesting the vote."
>
--
Jon
Re: [DISCUSS] METRON-433: Documentation update
Posted by Kevin Waterson <ke...@gmail.com>.
I have a revised document for installation that I would like to contribute.
(ubuntu).
What I have not found is an easily accessible of contributing. This may be
due to my time poor life where I simply do not have the time to be
searching for such a link.
Where can I find community contributions link?
Kev
On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella <ce...@gmail.com> wrote:
> Hi all,
>
> I noticed that our public facing docs were really getting stale. This is
> a normal thing in a fast moving project, but I imagine it's really hard for
> the community to get engaged with a project that has such details caught up
> inside of code and unit tests. As such, I thought it is reasonable to
> spend a little time ensuring the docs are better organized and clear, so I
> spent the day focusing on that:
>
> 1. they were nonexistent in a lot of places like model as a service
> which are new
> 2. they were stale (mostly around stellar functions)
> 3. they were incomplete (parsers take configs, we don't document
> that..we now do)
> 4. they were not clear. I added some clarification and pictures of the
> topologies
> 5. they were in the wrong place (e.g. enrichment config documentation
> was in metron-common instead of metron-enrichment)
> 6. there wasn't a starting place from the top level Readme into the
> scary subdepths of the individual projects
>
> I also gave a small fully worked example for some of the commonly
> confusing tasks, like doing a stellar enrichment or deploying a MaaS
> model. I wrote them up, but they were caught up inside of PR comments,
> this just brought them into the docs.
>
> You can find the PR here
> <https://github.com/apache/incubator-metron/pull/260>. It's already
> gotten a +1, but I'd love to see some discussion around what ELSE we can do
> to make it easier to get community contributions. After all, there would
> be no Metron without its community.
>
> Best,
>
> Casey
>
--
--
"Democracy is two wolves and a lamb voting on what to have for lunch.
Liberty is a well-armed lamb contesting the vote."
Re: [DISCUSS] METRON-433: Documentation update
Posted by Casey Stella <ce...@gmail.com>.
One quick update, I also added a section in metron-parsers,
metron-enrichment and metron-indexing detailing the performance knobs to
tweak. I am not giving direct suggestions as to the numbers, but just the
available knobs available to tweak in each context, how to tweak them and
some hint as to what they might do. I notice a lot of questions that I get
from people actually deploying Metron is around that, so I thought it
worthwhile to bring out in the docs.
On Mon, Sep 19, 2016 at 6:06 PM, Casey Stella <ce...@gmail.com> wrote:
> Hi all,
>
> I noticed that our public facing docs were really getting stale. This is
> a normal thing in a fast moving project, but I imagine it's really hard for
> the community to get engaged with a project that has such details caught up
> inside of code and unit tests. As such, I thought it is reasonable to
> spend a little time ensuring the docs are better organized and clear, so I
> spent the day focusing on that:
>
> 1. they were nonexistent in a lot of places like model as a service
> which are new
> 2. they were stale (mostly around stellar functions)
> 3. they were incomplete (parsers take configs, we don't document
> that..we now do)
> 4. they were not clear. I added some clarification and pictures of the
> topologies
> 5. they were in the wrong place (e.g. enrichment config documentation
> was in metron-common instead of metron-enrichment)
> 6. there wasn't a starting place from the top level Readme into the
> scary subdepths of the individual projects
>
> I also gave a small fully worked example for some of the commonly
> confusing tasks, like doing a stellar enrichment or deploying a MaaS
> model. I wrote them up, but they were caught up inside of PR comments,
> this just brought them into the docs.
>
> You can find the PR here
> <https://github.com/apache/incubator-metron/pull/260>. It's already
> gotten a +1, but I'd love to see some discussion around what ELSE we can do
> to make it easier to get community contributions. After all, there would
> be no Metron without its community.
>
> Best,
>
> Casey
>
Re: [DISCUSS] METRON-433: Documentation update
Posted by Casey Stella <ce...@gmail.com>.
One quick update, I also added a section in metron-parsers,
metron-enrichment and metron-indexing detailing the performance knobs to
tweak. I am not giving direct suggestions as to the numbers, but just the
available knobs available to tweak in each context, how to tweak them and
some hint as to what they might do. I notice a lot of questions that I get
from people actually deploying Metron is around that, so I thought it
worthwhile to bring out in the docs.
On Mon, Sep 19, 2016 at 6:06 PM, Casey Stella <ce...@gmail.com> wrote:
> Hi all,
>
> I noticed that our public facing docs were really getting stale. This is
> a normal thing in a fast moving project, but I imagine it's really hard for
> the community to get engaged with a project that has such details caught up
> inside of code and unit tests. As such, I thought it is reasonable to
> spend a little time ensuring the docs are better organized and clear, so I
> spent the day focusing on that:
>
> 1. they were nonexistent in a lot of places like model as a service
> which are new
> 2. they were stale (mostly around stellar functions)
> 3. they were incomplete (parsers take configs, we don't document
> that..we now do)
> 4. they were not clear. I added some clarification and pictures of the
> topologies
> 5. they were in the wrong place (e.g. enrichment config documentation
> was in metron-common instead of metron-enrichment)
> 6. there wasn't a starting place from the top level Readme into the
> scary subdepths of the individual projects
>
> I also gave a small fully worked example for some of the commonly
> confusing tasks, like doing a stellar enrichment or deploying a MaaS
> model. I wrote them up, but they were caught up inside of PR comments,
> this just brought them into the docs.
>
> You can find the PR here
> <https://github.com/apache/incubator-metron/pull/260>. It's already
> gotten a +1, but I'd love to see some discussion around what ELSE we can do
> to make it easier to get community contributions. After all, there would
> be no Metron without its community.
>
> Best,
>
> Casey
>