You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by Dima Kovalyov <Di...@sstech.us> on 2016/12/01 14:33:22 UTC
Confluence write access to a space
Hello,
Who can provide me with an access to add new page in this Installation space below?
https://cwiki.apache.org/confluence/display/METRON/Installation
I want to add up-to-date article on how-to install Metron using Ambari mpack.
Please advise.
My username is "dima" registered with electronic mail dimdroll hosted by gmail.
Right now I have it posted here:
https://goo.gl/HWGwpj
But instead of maintaining it there it will be convenient for me to have it in confluence instead of gdocs as I am using confluence in our internal project as well, so I can basically copy formatted doc between each other.
Thank you.
- Dima
On 11/30/2016 04:57 PM, Zeolla@GMail.com<ma...@GMail.com> wrote:
Probably the best long term place for this would be here<https://cwiki.apache.org/confluence/display/METRON/Installation>, potentially with a new mpack article (to differentiate from the ambari install which is already there, but dated). I can help with migrating it to the wiki - I will run through your documentation later today and request edit permissions if I find anything worth updating.
Would you be OK if I put your doc in the wiki at some point?
Jon
On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <Di...@sstech.us>> wrote:
As multiple users asked I have published document for installing Metron on Centos 6 using Ambari 2.4 with HDP 2.5:
https://goo.gl/HWGwpj
It may look ugly as this is direct export from internal confluence with cosmetic changes and our internal details removed. Let me know if there is a better place to publish it.
Feel free to request edit permissions, I would like the document to be up-to-date, simplified and corrected if there are any errors. The goal for this document is to provide installation guidance in a manner that Linux administrator with basic knowledge could perform installation without opening any other links or searching for commands, step by step.
Thank you.
p.s. feel free to let me know if you will encounter any errors as I've seen them bunch.
- Dima
On 11/30/2016 04:45 PM, Zeolla@GMail.com<ma...@GMail.com> wrote:
I also have written up a method to retrieve and setup the required packages in my script<https://github.com/JonZeolla/Development/blob/master/Bash/setupMetron.sh> which could probably be cleaned up and incorporated.
Jon
On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <ni...@nickallen.org>> wrote:
In regards to your comment about a "full package version dump" there is at least a start at `metron-deployment/scripts/platform-info.sh`. If we can enhance that, let's do it.
On Wed, Nov 30, 2016 at 4:04 AM, shoggi <sh...@gmail.com>> wrote:
Hi
Just to wrap this up. I've tried to deploy it on HDP 2.5 but I got the exact same problem. What I did was removing all components for 2.4.3.0 and then created a new cluster based on 2.5. I left it at that as the most likely issue could be with my base install. May that be wrong version of something or even a missing dev package.
As my work also coincided with the announcement by Casey I went ahead and installed 0.3.0 as a dev installation instead. That worked great as the dev-full-platform guide is very complete.
As a general feedback, in order to get more people interested and not having noobs like me come up with redundant questions - At this stage also sorry for not providing a meaningful subject, I simply was too excited :)
There is much to win with a small extension of the installation guide. I don't think it needs much more prose around but a full package version dump with every new release would help greatly to get things going. For the things which need to be compiled in, the versioning information should be enough. I am saying this, as everyone starts with a different base system. Some have the chance to start from scratch, some have a system running and build something on top and then there are the experienced Metron users, who have many base requirements.. they tend to forget. And I believe at the end
it always fails just because of some very tiny small difference.
If there is a space somewhere available, this could even be a spot, where people can say, "hey, I got this running on XYZ and here is the package version dump". A simple table with 'Metron version', 'OS release', 'Package list file', 'Version Details of every other needed to-be-compiled package', is already a big help. This really could supplement the existing guide found here: https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+an+Ambari-Managed+Cluster.
But then, I could be alone and others did not experience the same and simply got it to run, I don't know. Anyway, I would be happy to help if help is needed.
Cheers
Shoggi
On Sun, Nov 27, 2016 at 3:00 PM, shoggi <sh...@gmail.com>> wrote:
Hi Simon
Thank you for your very quick response. I installed it on HDP 2.4.3.0.
I had a 0.2.0 instance running and was very conscious about versioning the various components. Somehow I had the understanding of not touching 2.5 as of yet. I give that a try.
Thanks heaps !
Shoggi
On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <si...@simonellistonball.com>> wrote:
Hi Shoggi,
Can I ask which version of HDP you used as a base. The article is a little out of date in that is calls for 2.4, which the 0.3.0 release is based on 2.5. There was a major version difference in storm between the two, which seems likely to have caused the problem you're seeing.
Simon
Sent from my iPad
On 27 Nov 2016, at 13:48, shoggi <sh...@gmail.com>> wrote:
Hi all
I followed the installation guide to deploy Metron via Ambari https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html.
I ended up with all but the Metron Components (Enrichment, Parsing, Parsers) to start up. The closest to pinpoint the issue is, when I try to start any new parser topology such as with bro:
/usr/metron/0.3.0/bin/start_parser_topology.sh -k x.localdomain:6667 -z x.localdomain:2181 -s bro
The error I get is:
Exception in thread "main" java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: org/apache/storm/Config
Or in more details:
Running: /usr/jdk64/jdk1.8.0_77/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-227/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.4.3.0-227/storm/bin -Dstorm.jar=/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar org.apache.metron.parsers.topology.ParserTopologyCLI -k x.localdomain:6667 -z x.localdomain:2181 -s bro
Exception in thread "main" java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: org/apache/storm/Config
at org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<init>(ParserTopologyCLI.java:199)
at org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<clinit>(ParserTopologyCLI.java:47)
at org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:266)
Caused by: java.lang.NoClassDefFoundError: org/apache/storm/Config
... 3 more
Caused by: java.lang.ClassNotFoundException: org.apache.storm.Config
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 3 more
Must be something wrong with my config, possibly I have overlooked something but can't put my finger on it. Any help is appreciated.
Many thanks
Shoggi
--
Nick Allen <ni...@nickallen.org>>
--
Jon
Sent from my mobile device
--
Jon
Sent from my mobile device
Re: Confluence write access to a space
Posted by Matt Foley <mf...@hortonworks.com>.
No, node.data and node.master are both correctly set to true (with Ambari’s agreement/participation) in the elasticsearch.yml file in CONF_DIR, and this is being correctly picked up by ES when launched interactively. I really think this is in the service management stuff in /etc/init.d/elasticsearch and /usr/share/elasticsearch/bin/elasticsearch . Remains to be proven, of course…
The reason I think ES isn’t even being successfully launched by systemd, is there is zero logging anywhere, except in ambari where it shows nothing but a successful service launch. No files created in /var/log/elasticsearch, which all scripts agree is the value of LOG_DIR, despite permissions set to “drwxr-xr-x. elasticsearch elasticsearch”
Thanks,
--Matt
On 12/14/16, 2:07 PM, "David Lyle" <dl...@gmail.com> wrote:
Aha! There's your problem. :)
Kidding aside, that is weird. I would expect the ES instance to come up and
go status red right away, not up and die.
I did have a horrible, horrible hack that made all that work, it involved
modifying the stored es templates to both have node.master and node.data
set to true in
/var/lib/ambari-server/resources/common-services/ELASTICSEARCH (from
memory, path may be a bit off). It occurred to me that an easy incremental
step toward to full METRON-608 would be to simply expose the templates to
the config pages...
-D...
On Wed, Dec 14, 2016 at 4:59 PM, Matt Foley <mf...@hortonworks.com> wrote:
> David,
> No, it’s in my METRON-608 single-node deployment :-)
>
> On 12/14/16, 12:46 PM, "David Lyle" <dl...@gmail.com> wrote:
>
> Hi Matt and Jon,
>
> FWIW, Metron with the MPack has been tested extensively on CentOS 7.
> Works
> like a champ. The issue right now is that sensor install is a CentOS
> 6/Ansible proposition.
>
> I haven't seen the issue with ES that you're experiencing, Matt. Is
> that in
> a 1 Master 3 Data node ES config or something else?
>
> -D...
>
>
> On Wed, Dec 14, 2016 at 3:29 PM, Matt Foley <mf...@hortonworks.com>
> wrote:
>
> > I hope we will try to support Centos7. Many of my company’s
> customers are
> > requiring new installs to use RHEL 7 or Centos 7 rather than 6. In
> > addition, we have the benefit that Centos 6 RPMs basically always
> run fine
> > in Centos7 (barring heavy-duty low-level system manipulations that
> Metron,
> > Kibana, and ES just don’t use), ES doesn’t distinguish between them,
> and
> > Kibana install support for Centos7 is already in place in the
> Mpack. And
> > of course the Hadoop Stack runs fine in 7.
> >
> > For what it’s worth, I’ve been working with the Mpack installation on
> > Centos7 for the last few weeks, and it works fine except for a
> persistent
> > issue where the Elasticsearch service works fine if launched
> interactively
> > but terminates immediately if launched with same arguments as a
> service.
> > Hope to find the cause in the next couple days. (If anyone knows
> why,
> > would love to hear. The daemonized launch scripts come from ES, not
> from
> > Metron, so “should just work”.)
> >
> > Thanks,
> > --Matt
> >
> > On 12/13/16, 6:46 PM, "Zeolla@GMail.com" <ze...@gmail.com> wrote:
> >
> > For now, I'm not sure what the solution is, but I would think
> choosing
> > one
> > specific list of required/supported software (including OS) and
> > documenting
> > that thoroughly is the right start. Something like what
> currently
> > exists
> > for the vagrant side of things.
> >
> > That said, long term I would love to see broader support and
> > automatically
> > generating documentation derived from the code. Isn't CentOS 6
> still
> > preferred over 7? That's what I've been working on solely for
> that
> > reason.
> >
> > Jon
> >
> > On Tue, Dec 13, 2016, 21:07 Matt Foley <ma...@apache.org> wrote:
> >
> > > In my work on METRON-608, I’ve found a lot of small but
> significant
> > bugs
> > > in the existing Mpack (version 0.3.0). These bugs integrate
> with a
> > lot of
> > > the oddball tweaks specified in the draft Install docs I saw.
> I
> > would like
> > > to submit a PR for these bugs, but they must be accompanied by
> > changes in
> > > the Install doc.
> > >
> > > How do we want to manage this? Should we version the Install
> doc?
> > Or
> > > have multiple sections for the different versions? Or have
> > footnotes or
> > > sidebar comments about which version certain paragraphs do and
> do
> > not apply
> > > to?
> > >
> > > I’m inclined to use sidebars within the document, because the
> > dependencies
> > > aren’t just on which versions of HDP, Ambari, and Metron you
> use.
> > They
> > > also depend on Python 2.6 vs 2.7, and Centos 6 vs 7.
> > >
> > > Thanks,
> > > --Matt
> > >
> > >
> > > On 12/13/16, 4:13 PM, "Zeolla@GMail.com" <ze...@gmail.com>
> wrote:
> > >
> > > Sorry about the delay. To be honest I delayed once I saw
> the
> > > management UI
> > > PR because I was going to wait for that to be merged into
> master
> > > before I
> > > did my mpack install. My initial thought is that there
> may need
> > to be
> > > some
> > > sort of a merger between what you provided and the
> Hortonworks
> > blog
> > > post,
> > > as I would prefer a single, comprehensive post over
> > fragmentation.
> > >
> > > Jon
> > >
> > > On Tue, Dec 13, 2016, 18:29 Dima Kovalyov <
> > Dima.Kovalyov@sstech.us>
> > > wrote:
> > >
> > > > Thank you Jon,
> > > >
> > > > Just wondering if you have finished polishing the
> document? I
> > will
> > > have
> > > > some time this week to polish and publish it if there is
> > anything
> > > left
> > > > to format.
> > > >
> > > > - Dima
> > > >
> > > > On 12/01/2016 04:41 PM, Zeolla@GMail.com wrote:
> > > > > While I can't delegate access to the wiki, I am
> working on
> > > migrating the
> > > > > documentation over. I have been modifying the
> document for
> > > clarity and
> > > > > updating up some of the formatting, so please give me
> some
> > time to
> > > do so.
> > > > >
> > > > > Jon
> > > > >
> > > > > On Thu, Dec 1, 2016 at 9:33 AM Dima Kovalyov <
> > > Dima.Kovalyov@sstech.us>
> > > > > wrote:
> > > > >
> > > > >> Hello,
> > > > >>
> > > > >> Who can provide me with an access to add new page in
> this
> > > Installation
> > > > >> space below?
> > > > >> https://cwiki.apache.org/confluence/display/METRON/
> > Installation
> > > > >> I want to add up-to-date article on how-to install
> Metron
> > using
> > > Ambari
> > > > >> mpack.
> > > > >> Please advise.
> > > > >>
> > > > >> My username is "dima" registered with electronic mail
> > dimdroll
> > > hosted by
> > > > >> gmail.
> > > > >>
> > > > >> Right now I have it posted here:
> > > > >> https://goo.gl/HWGwpj
> > > > >> But instead of maintaining it there it will be
> convenient
> > for me
> > > to have
> > > > >> it in confluence instead of gdocs as I am using
> confluence
> > in our
> > > > internal
> > > > >> project as well, so I can basically copy formatted doc
> > between
> > > each
> > > > other.
> > > > >>
> > > > >> Thank you.
> > > > >>
> > > > >> - Dima
> > > > >>
> > > > >> On 11/30/2016 04:57 PM, Zeolla@GMail.com<mailto:
> Zeolla
> > @GMail.com>
> > > > wrote:
> > > > >> Probably the best long term place for this would be
> here<
> > > > >> https://cwiki.apache.org/confluence/display/METRON/
> > Installation>,
> > > > >> potentially with a new mpack article (to
> differentiate from
> > the
> > > ambari
> > > > >> install which is already there, but dated). I can
> help with
> > > migrating
> > > > it
> > > > >> to the wiki - I will run through your documentation
> later
> > today
> > > and
> > > > request
> > > > >> edit permissions if I find anything worth updating.
> > > > >>
> > > > >> Would you be OK if I put your doc in the wiki at some
> point?
> > > > >>
> > > > >> Jon
> > > > >>
> > > > >> On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <
> > > Dima.Kovalyov@sstech.us
> > > > >> <ma...@sstech.us>> wrote:
> > > > >> As multiple users asked I have published document for
> > installing
> > > Metron
> > > > on
> > > > >> Centos 6 using Ambari 2.4 with HDP 2.5:
> > > > >> https://goo.gl/HWGwpj
> > > > >> It may look ugly as this is direct export from
> internal
> > > confluence with
> > > > >> cosmetic changes and our internal details removed.
> Let me
> > know if
> > > there
> > > > is
> > > > >> a better place to publish it.
> > > > >>
> > > > >> Feel free to request edit permissions, I would like
> the
> > document
> > > to be
> > > > >> up-to-date, simplified and corrected if there are any
> > errors. The
> > > goal
> > > > for
> > > > >> this document is to provide installation guidance in a
> > manner
> > > that Linux
> > > > >> administrator with basic knowledge could perform
> > installation
> > > without
> > > > >> opening any other links or searching for commands,
> step by
> > step.
> > > > >> Thank you.
> > > > >>
> > > > >> p.s. feel free to let me know if you will encounter
> any
> > errors as
> > > I've
> > > > >> seen them bunch.
> > > > >>
> > > > >> - Dima
> > > > >>
> > > > >>
> > > > >>
> > > > >> On 11/30/2016 04:45 PM, Zeolla@GMail.com<mailto:
> Zeolla
> > @GMail.com>
> > > > wrote:
> > > > >> I also have written up a method to retrieve and setup
> the
> > required
> > > > >> packages in my script<
> > > > >>
> > > >
> > > https://github.com/JonZeolla/Development/blob/master/Bash/
> > setupMetron.sh>
> > > > >> which could probably be cleaned up and incorporated.
> > > > >>
> > > > >> Jon
> > > > >>
> > > > >> On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <
> > nick@nickallen.org
> > > <mailto:
> > > > >> nick@nickallen.org>> wrote:
> > > > >> In regards to your comment about a "full package
> version
> > dump"
> > > there is
> > > > at
> > > > >> least a start at `metron-deployment/scripts/
> > platform-info.sh`.
> > > If we
> > > > can
> > > > >> enhance that, let's do it.
> > > > >>
> > > > >> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <
> shoggi@gmail.com
> > <mailto:
> > > > >> shoggi@gmail.com>> wrote:
> > > > >> Hi
> > > > >>
> > > > >> Just to wrap this up. I've tried to deploy it on HDP
> 2.5
> > but I
> > > got the
> > > > >> exact same problem. What I did was removing all
> components
> > for
> > > 2.4.3.0
> > > > and
> > > > >> then created a new cluster based on 2.5. I left it at
> that
> > as the
> > > most
> > > > >> likely issue could be with my base install. May that
> be
> > wrong
> > > version of
> > > > >> something or even a missing dev package.
> > > > >>
> > > > >> As my work also coincided with the announcement by
> Casey I
> > went
> > > ahead
> > > > and
> > > > >> installed 0.3.0 as a dev installation instead. That
> worked
> > great
> > > as the
> > > > >> dev-full-platform guide is very complete.
> > > > >>
> > > > >> As a general feedback, in order to get more people
> > interested and
> > > not
> > > > >> having noobs like me come up with redundant questions
> - At
> > this
> > > stage
> > > > also
> > > > >> sorry for not providing a meaningful subject, I
> simply was
> > too
> > > excited
> > > > :)
> > > > >>
> > > > >> There is much to win with a small extension of the
> > installation
> > > guide. I
> > > > >> don't think it needs much more prose around but a full
> > package
> > > version
> > > > dump
> > > > >> with every new release would help greatly to get
> things
> > going.
> > > For the
> > > > >> things which need to be compiled in, the versioning
> > information
> > > should
> > > > be
> > > > >> enough. I am saying this, as everyone starts with a
> > different base
> > > > system.
> > > > >> Some have the chance to start from scratch, some have
> a
> > system
> > > running
> > > > and
> > > > >> build something on top and then there are the
> experienced
> > Metron
> > > users,
> > > > who
> > > > >> have many base requirements.. they tend to forget.
> And I
> > believe
> > > at the
> > > > end
> > > > >> it always fails just because of some very tiny small
> > difference.
> > > > >>
> > > > >> If there is a space somewhere available, this could
> even be
> > a
> > > spot,
> > > > where
> > > > >> people can say, "hey, I got this running on XYZ and
> here is
> > the
> > > package
> > > > >> version dump". A simple table with 'Metron version',
> 'OS
> > release',
> > > > 'Package
> > > > >> list file', 'Version Details of every other needed
> > to-be-compiled
> > > > package',
> > > > >> is already a big help. This really could supplement
> the
> > existing
> > > guide
> > > > >> found here:
> > > > >>
> > > >
> > > https://cwiki.apache.org/confluence/display/METRON/
> > Metron+Installation+on+an+Ambari-Managed+Cluster
> > > > >> .
> > > > >>
> > > > >> But then, I could be alone and others did not
> experience
> > the same
> > > and
> > > > >> simply got it to run, I don't know. Anyway, I would be
> > happy to
> > > help if
> > > > >> help is needed.
> > > > >>
> > > > >> Cheers
> > > > >> Shoggi
> > > > >>
> > > > >> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <
> shoggi@gmail.com
> > <mailto:
> > > > >> shoggi@gmail.com>> wrote:
> > > > >> Hi Simon
> > > > >>
> > > > >> Thank you for your very quick response. I installed
> it on
> > HDP
> > > 2.4.3.0.
> > > > >>
> > > > >> I had a 0.2.0 instance running and was very conscious
> about
> > > versioning
> > > > the
> > > > >> various components. Somehow I had the understanding
> of not
> > > touching 2.5
> > > > as
> > > > >> of yet. I give that a try.
> > > > >>
> > > > >> Thanks heaps !
> > > > >>
> > > > >> Shoggi
> > > > >>
> > > > >> On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <
> > > > >> simon@simonellistonball.com<mailto:simon@
> > simonellistonball.com>>
> > > wrote:
> > > > >> Hi Shoggi,
> > > > >>
> > > > >> Can I ask which version of HDP you used as a base. The
> > article is
> > > a
> > > > little
> > > > >> out of date in that is calls for 2.4, which the 0.3.0
> > release is
> > > based
> > > > on
> > > > >> 2.5. There was a major version difference in storm
> between
> > the
> > > two,
> > > > which
> > > > >> seems likely to have caused the problem you're seeing.
> > > > >>
> > > > >> Simon
> > > > >>
> > > > >> Sent from my iPad
> > > > >>
> > > > >> On 27 Nov 2016, at 13:48, shoggi <shoggi@gmail.com
> <mailto:
> > > > shoggi@gmail.com>>
> > > > >> wrote:
> > > > >>
> > > > >> Hi all
> > > > >>
> > > > >> I followed the installation guide to deploy Metron via
> > Ambari
> > > > >>
> > > >
> > > https://community.hortonworks.com/articles/60805/deploying-
> > a-fresh-metron-cluster-using-ambari-serv.html
> > > > >> .
> > > > >>
> > > > >> I ended up with all but the Metron Components
> (Enrichment,
> > > Parsing,
> > > > >> Parsers) to start up. The closest to pinpoint the
> issue is,
> > when
> > > I try
> > > > to
> > > > >> start any new parser topology such as with bro:
> > > > >>
> > > > >> /usr/metron/0.3.0/bin/start_parser_topology.sh -k
> > > x.localdomain:6667 -z
> > > > >> x.localdomain:2181 -s bro
> > > > >>
> > > > >> The error I get is:
> > > > >>
> > > > >> Exception in thread "main" java.lang.
> BootstrapMethodError:
> > > > >> java.lang.NoClassDefFoundError:
> org/apache/storm/Config
> > > > >>
> > > > >> Or in more details:
> > > > >>
> > > > >> Running: /usr/jdk64/jdk1.8.0_77/bin/java -client
> > -Ddaemon.name=
> > > > >> -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-
> 227/storm
> > > > >> -Dstorm.log.dir=/var/log/storm
> > > > >>
> > > >
> > > -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/
> > lib:/usr/hdp/current/storm-client/lib
> > > > >> -Dstorm.conf.file= -cp
> > > > >>
> > > >
> > > /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.
> > 3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > clojure-1.6.0.jar:/usr/hdp/
> > > > >> 2.4.3.<http://2.4.3
> > > > >>
> > > >
> > > .>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/
> > lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-
> > auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/
> > > > >> 2.4.3.<http://2.4.3
> > > > >>
> > > >
> > > .>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/jackson-core-2.3.1.jar:/usr/hdp/
> > > > >> 2.4.3.<http://2.4.3
> > > > >>
> > > >
> > > .>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/
> > usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/
> > hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.
> > v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.
> > 94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/
> > > > >> 2.4.3.<http://2.4.3
> > > > >>
> > > >
> > > .>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > minlog-1.2.jar:/usr/hdp/
> > > > >> 2.4.3.<http://2.4.3
> > > >
> > > .>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-
> > 227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-
> > 227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-
> > 227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/
> > lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-
> > uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/
> > hdp/2.4.3.0-227/storm/bin
> > > > >> -Dstorm.jar=/usr/metron/0.3.0/
> > lib/metron-parsers-0.3.0-uber.jar
> > > > >> org.apache.metron.parsers.topology.ParserTopologyCLI
> -k
> > > > x.localdomain:6667
> > > > >> -z x.localdomain:2181 -s bro
> > > > >> Exception in thread "main" java.lang.
> BootstrapMethodError:
> > > > >> java.lang.NoClassDefFoundError:
> org/apache/storm/Config
> > > > >> at
> > > > >>
> > > >
> > > org.apache.metron.parsers.topology.ParserTopologyCLI$
> > ParserOptions.<init>(ParserTopologyCLI.java:199)
> > > > >> at
> > > > >>
> > > >
> > > org.apache.metron.parsers.topology.ParserTopologyCLI$
> > ParserOptions.<clinit>(ParserTopologyCLI.java:47)
> > > > >> at
> > > > >>
> > > >
> > > org.apache.metron.parsers.topology.ParserTopologyCLI.
> > main(ParserTopologyCLI.java:266)
> > > > >> Caused by: java.lang.NoClassDefFoundError:
> > org/apache/storm/Config
> > > > >> ... 3 more
> > > > >> Caused by: java.lang.ClassNotFoundException:
> > > org.apache.storm.Config
> > > > >> at java.net.URLClassLoader.
> findClass(URLClassLoader.java:
> > 381)
> > > > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:
> 424)
> > > > >> at sun.misc.Launcher$AppClassLoader.loadClass(
> > Launcher.java:331)
> > > > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:
> 357)
> > > > >> ... 3 more
> > > > >>
> > > > >> Must be something wrong with my config, possibly I
> have
> > overlooked
> > > > >> something but can't put my finger on it. Any help is
> > appreciated.
> > > > >>
> > > > >> Many thanks
> > > > >> Shoggi
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Nick Allen <nick@nickallen.org<mailto:nic
> k@nickallen.org>>
> > > > >> --
> > > > >>
> > > > >> Jon
> > > > >>
> > > > >> Sent from my mobile device
> > > > >>
> > > > >> --
> > > > >>
> > > > >> Jon
> > > > >>
> > > > >> Sent from my mobile device
> > > > >>
> > > > >> --
> > > > > Jon
> > > > >
> > > > > Sent from my mobile device
> > > > >
> > > >
> > > > --
> > >
> > > Jon
> > >
> > > Sent from my mobile device
> > >
> > >
> > >
> > > --
> >
> > Jon
> >
> > Sent from my mobile device
> >
> >
> >
>
>
>
Re: Confluence write access to a space
Posted by David Lyle <dl...@gmail.com>.
Aha! There's your problem. :)
Kidding aside, that is weird. I would expect the ES instance to come up and
go status red right away, not up and die.
I did have a horrible, horrible hack that made all that work, it involved
modifying the stored es templates to both have node.master and node.data
set to true in
/var/lib/ambari-server/resources/common-services/ELASTICSEARCH (from
memory, path may be a bit off). It occurred to me that an easy incremental
step toward to full METRON-608 would be to simply expose the templates to
the config pages...
-D...
On Wed, Dec 14, 2016 at 4:59 PM, Matt Foley <mf...@hortonworks.com> wrote:
> David,
> No, it’s in my METRON-608 single-node deployment :-)
>
> On 12/14/16, 12:46 PM, "David Lyle" <dl...@gmail.com> wrote:
>
> Hi Matt and Jon,
>
> FWIW, Metron with the MPack has been tested extensively on CentOS 7.
> Works
> like a champ. The issue right now is that sensor install is a CentOS
> 6/Ansible proposition.
>
> I haven't seen the issue with ES that you're experiencing, Matt. Is
> that in
> a 1 Master 3 Data node ES config or something else?
>
> -D...
>
>
> On Wed, Dec 14, 2016 at 3:29 PM, Matt Foley <mf...@hortonworks.com>
> wrote:
>
> > I hope we will try to support Centos7. Many of my company’s
> customers are
> > requiring new installs to use RHEL 7 or Centos 7 rather than 6. In
> > addition, we have the benefit that Centos 6 RPMs basically always
> run fine
> > in Centos7 (barring heavy-duty low-level system manipulations that
> Metron,
> > Kibana, and ES just don’t use), ES doesn’t distinguish between them,
> and
> > Kibana install support for Centos7 is already in place in the
> Mpack. And
> > of course the Hadoop Stack runs fine in 7.
> >
> > For what it’s worth, I’ve been working with the Mpack installation on
> > Centos7 for the last few weeks, and it works fine except for a
> persistent
> > issue where the Elasticsearch service works fine if launched
> interactively
> > but terminates immediately if launched with same arguments as a
> service.
> > Hope to find the cause in the next couple days. (If anyone knows
> why,
> > would love to hear. The daemonized launch scripts come from ES, not
> from
> > Metron, so “should just work”.)
> >
> > Thanks,
> > --Matt
> >
> > On 12/13/16, 6:46 PM, "Zeolla@GMail.com" <ze...@gmail.com> wrote:
> >
> > For now, I'm not sure what the solution is, but I would think
> choosing
> > one
> > specific list of required/supported software (including OS) and
> > documenting
> > that thoroughly is the right start. Something like what
> currently
> > exists
> > for the vagrant side of things.
> >
> > That said, long term I would love to see broader support and
> > automatically
> > generating documentation derived from the code. Isn't CentOS 6
> still
> > preferred over 7? That's what I've been working on solely for
> that
> > reason.
> >
> > Jon
> >
> > On Tue, Dec 13, 2016, 21:07 Matt Foley <ma...@apache.org> wrote:
> >
> > > In my work on METRON-608, I’ve found a lot of small but
> significant
> > bugs
> > > in the existing Mpack (version 0.3.0). These bugs integrate
> with a
> > lot of
> > > the oddball tweaks specified in the draft Install docs I saw.
> I
> > would like
> > > to submit a PR for these bugs, but they must be accompanied by
> > changes in
> > > the Install doc.
> > >
> > > How do we want to manage this? Should we version the Install
> doc?
> > Or
> > > have multiple sections for the different versions? Or have
> > footnotes or
> > > sidebar comments about which version certain paragraphs do and
> do
> > not apply
> > > to?
> > >
> > > I’m inclined to use sidebars within the document, because the
> > dependencies
> > > aren’t just on which versions of HDP, Ambari, and Metron you
> use.
> > They
> > > also depend on Python 2.6 vs 2.7, and Centos 6 vs 7.
> > >
> > > Thanks,
> > > --Matt
> > >
> > >
> > > On 12/13/16, 4:13 PM, "Zeolla@GMail.com" <ze...@gmail.com>
> wrote:
> > >
> > > Sorry about the delay. To be honest I delayed once I saw
> the
> > > management UI
> > > PR because I was going to wait for that to be merged into
> master
> > > before I
> > > did my mpack install. My initial thought is that there
> may need
> > to be
> > > some
> > > sort of a merger between what you provided and the
> Hortonworks
> > blog
> > > post,
> > > as I would prefer a single, comprehensive post over
> > fragmentation.
> > >
> > > Jon
> > >
> > > On Tue, Dec 13, 2016, 18:29 Dima Kovalyov <
> > Dima.Kovalyov@sstech.us>
> > > wrote:
> > >
> > > > Thank you Jon,
> > > >
> > > > Just wondering if you have finished polishing the
> document? I
> > will
> > > have
> > > > some time this week to polish and publish it if there is
> > anything
> > > left
> > > > to format.
> > > >
> > > > - Dima
> > > >
> > > > On 12/01/2016 04:41 PM, Zeolla@GMail.com wrote:
> > > > > While I can't delegate access to the wiki, I am
> working on
> > > migrating the
> > > > > documentation over. I have been modifying the
> document for
> > > clarity and
> > > > > updating up some of the formatting, so please give me
> some
> > time to
> > > do so.
> > > > >
> > > > > Jon
> > > > >
> > > > > On Thu, Dec 1, 2016 at 9:33 AM Dima Kovalyov <
> > > Dima.Kovalyov@sstech.us>
> > > > > wrote:
> > > > >
> > > > >> Hello,
> > > > >>
> > > > >> Who can provide me with an access to add new page in
> this
> > > Installation
> > > > >> space below?
> > > > >> https://cwiki.apache.org/confluence/display/METRON/
> > Installation
> > > > >> I want to add up-to-date article on how-to install
> Metron
> > using
> > > Ambari
> > > > >> mpack.
> > > > >> Please advise.
> > > > >>
> > > > >> My username is "dima" registered with electronic mail
> > dimdroll
> > > hosted by
> > > > >> gmail.
> > > > >>
> > > > >> Right now I have it posted here:
> > > > >> https://goo.gl/HWGwpj
> > > > >> But instead of maintaining it there it will be
> convenient
> > for me
> > > to have
> > > > >> it in confluence instead of gdocs as I am using
> confluence
> > in our
> > > > internal
> > > > >> project as well, so I can basically copy formatted doc
> > between
> > > each
> > > > other.
> > > > >>
> > > > >> Thank you.
> > > > >>
> > > > >> - Dima
> > > > >>
> > > > >> On 11/30/2016 04:57 PM, Zeolla@GMail.com<mailto:
> Zeolla
> > @GMail.com>
> > > > wrote:
> > > > >> Probably the best long term place for this would be
> here<
> > > > >> https://cwiki.apache.org/confluence/display/METRON/
> > Installation>,
> > > > >> potentially with a new mpack article (to
> differentiate from
> > the
> > > ambari
> > > > >> install which is already there, but dated). I can
> help with
> > > migrating
> > > > it
> > > > >> to the wiki - I will run through your documentation
> later
> > today
> > > and
> > > > request
> > > > >> edit permissions if I find anything worth updating.
> > > > >>
> > > > >> Would you be OK if I put your doc in the wiki at some
> point?
> > > > >>
> > > > >> Jon
> > > > >>
> > > > >> On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <
> > > Dima.Kovalyov@sstech.us
> > > > >> <ma...@sstech.us>> wrote:
> > > > >> As multiple users asked I have published document for
> > installing
> > > Metron
> > > > on
> > > > >> Centos 6 using Ambari 2.4 with HDP 2.5:
> > > > >> https://goo.gl/HWGwpj
> > > > >> It may look ugly as this is direct export from
> internal
> > > confluence with
> > > > >> cosmetic changes and our internal details removed.
> Let me
> > know if
> > > there
> > > > is
> > > > >> a better place to publish it.
> > > > >>
> > > > >> Feel free to request edit permissions, I would like
> the
> > document
> > > to be
> > > > >> up-to-date, simplified and corrected if there are any
> > errors. The
> > > goal
> > > > for
> > > > >> this document is to provide installation guidance in a
> > manner
> > > that Linux
> > > > >> administrator with basic knowledge could perform
> > installation
> > > without
> > > > >> opening any other links or searching for commands,
> step by
> > step.
> > > > >> Thank you.
> > > > >>
> > > > >> p.s. feel free to let me know if you will encounter
> any
> > errors as
> > > I've
> > > > >> seen them bunch.
> > > > >>
> > > > >> - Dima
> > > > >>
> > > > >>
> > > > >>
> > > > >> On 11/30/2016 04:45 PM, Zeolla@GMail.com<mailto:
> Zeolla
> > @GMail.com>
> > > > wrote:
> > > > >> I also have written up a method to retrieve and setup
> the
> > required
> > > > >> packages in my script<
> > > > >>
> > > >
> > > https://github.com/JonZeolla/Development/blob/master/Bash/
> > setupMetron.sh>
> > > > >> which could probably be cleaned up and incorporated.
> > > > >>
> > > > >> Jon
> > > > >>
> > > > >> On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <
> > nick@nickallen.org
> > > <mailto:
> > > > >> nick@nickallen.org>> wrote:
> > > > >> In regards to your comment about a "full package
> version
> > dump"
> > > there is
> > > > at
> > > > >> least a start at `metron-deployment/scripts/
> > platform-info.sh`.
> > > If we
> > > > can
> > > > >> enhance that, let's do it.
> > > > >>
> > > > >> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <
> shoggi@gmail.com
> > <mailto:
> > > > >> shoggi@gmail.com>> wrote:
> > > > >> Hi
> > > > >>
> > > > >> Just to wrap this up. I've tried to deploy it on HDP
> 2.5
> > but I
> > > got the
> > > > >> exact same problem. What I did was removing all
> components
> > for
> > > 2.4.3.0
> > > > and
> > > > >> then created a new cluster based on 2.5. I left it at
> that
> > as the
> > > most
> > > > >> likely issue could be with my base install. May that
> be
> > wrong
> > > version of
> > > > >> something or even a missing dev package.
> > > > >>
> > > > >> As my work also coincided with the announcement by
> Casey I
> > went
> > > ahead
> > > > and
> > > > >> installed 0.3.0 as a dev installation instead. That
> worked
> > great
> > > as the
> > > > >> dev-full-platform guide is very complete.
> > > > >>
> > > > >> As a general feedback, in order to get more people
> > interested and
> > > not
> > > > >> having noobs like me come up with redundant questions
> - At
> > this
> > > stage
> > > > also
> > > > >> sorry for not providing a meaningful subject, I
> simply was
> > too
> > > excited
> > > > :)
> > > > >>
> > > > >> There is much to win with a small extension of the
> > installation
> > > guide. I
> > > > >> don't think it needs much more prose around but a full
> > package
> > > version
> > > > dump
> > > > >> with every new release would help greatly to get
> things
> > going.
> > > For the
> > > > >> things which need to be compiled in, the versioning
> > information
> > > should
> > > > be
> > > > >> enough. I am saying this, as everyone starts with a
> > different base
> > > > system.
> > > > >> Some have the chance to start from scratch, some have
> a
> > system
> > > running
> > > > and
> > > > >> build something on top and then there are the
> experienced
> > Metron
> > > users,
> > > > who
> > > > >> have many base requirements.. they tend to forget.
> And I
> > believe
> > > at the
> > > > end
> > > > >> it always fails just because of some very tiny small
> > difference.
> > > > >>
> > > > >> If there is a space somewhere available, this could
> even be
> > a
> > > spot,
> > > > where
> > > > >> people can say, "hey, I got this running on XYZ and
> here is
> > the
> > > package
> > > > >> version dump". A simple table with 'Metron version',
> 'OS
> > release',
> > > > 'Package
> > > > >> list file', 'Version Details of every other needed
> > to-be-compiled
> > > > package',
> > > > >> is already a big help. This really could supplement
> the
> > existing
> > > guide
> > > > >> found here:
> > > > >>
> > > >
> > > https://cwiki.apache.org/confluence/display/METRON/
> > Metron+Installation+on+an+Ambari-Managed+Cluster
> > > > >> .
> > > > >>
> > > > >> But then, I could be alone and others did not
> experience
> > the same
> > > and
> > > > >> simply got it to run, I don't know. Anyway, I would be
> > happy to
> > > help if
> > > > >> help is needed.
> > > > >>
> > > > >> Cheers
> > > > >> Shoggi
> > > > >>
> > > > >> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <
> shoggi@gmail.com
> > <mailto:
> > > > >> shoggi@gmail.com>> wrote:
> > > > >> Hi Simon
> > > > >>
> > > > >> Thank you for your very quick response. I installed
> it on
> > HDP
> > > 2.4.3.0.
> > > > >>
> > > > >> I had a 0.2.0 instance running and was very conscious
> about
> > > versioning
> > > > the
> > > > >> various components. Somehow I had the understanding
> of not
> > > touching 2.5
> > > > as
> > > > >> of yet. I give that a try.
> > > > >>
> > > > >> Thanks heaps !
> > > > >>
> > > > >> Shoggi
> > > > >>
> > > > >> On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <
> > > > >> simon@simonellistonball.com<mailto:simon@
> > simonellistonball.com>>
> > > wrote:
> > > > >> Hi Shoggi,
> > > > >>
> > > > >> Can I ask which version of HDP you used as a base. The
> > article is
> > > a
> > > > little
> > > > >> out of date in that is calls for 2.4, which the 0.3.0
> > release is
> > > based
> > > > on
> > > > >> 2.5. There was a major version difference in storm
> between
> > the
> > > two,
> > > > which
> > > > >> seems likely to have caused the problem you're seeing.
> > > > >>
> > > > >> Simon
> > > > >>
> > > > >> Sent from my iPad
> > > > >>
> > > > >> On 27 Nov 2016, at 13:48, shoggi <shoggi@gmail.com
> <mailto:
> > > > shoggi@gmail.com>>
> > > > >> wrote:
> > > > >>
> > > > >> Hi all
> > > > >>
> > > > >> I followed the installation guide to deploy Metron via
> > Ambari
> > > > >>
> > > >
> > > https://community.hortonworks.com/articles/60805/deploying-
> > a-fresh-metron-cluster-using-ambari-serv.html
> > > > >> .
> > > > >>
> > > > >> I ended up with all but the Metron Components
> (Enrichment,
> > > Parsing,
> > > > >> Parsers) to start up. The closest to pinpoint the
> issue is,
> > when
> > > I try
> > > > to
> > > > >> start any new parser topology such as with bro:
> > > > >>
> > > > >> /usr/metron/0.3.0/bin/start_parser_topology.sh -k
> > > x.localdomain:6667 -z
> > > > >> x.localdomain:2181 -s bro
> > > > >>
> > > > >> The error I get is:
> > > > >>
> > > > >> Exception in thread "main" java.lang.
> BootstrapMethodError:
> > > > >> java.lang.NoClassDefFoundError:
> org/apache/storm/Config
> > > > >>
> > > > >> Or in more details:
> > > > >>
> > > > >> Running: /usr/jdk64/jdk1.8.0_77/bin/java -client
> > -Ddaemon.name=
> > > > >> -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-
> 227/storm
> > > > >> -Dstorm.log.dir=/var/log/storm
> > > > >>
> > > >
> > > -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/
> > lib:/usr/hdp/current/storm-client/lib
> > > > >> -Dstorm.conf.file= -cp
> > > > >>
> > > >
> > > /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.
> > 3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > clojure-1.6.0.jar:/usr/hdp/
> > > > >> 2.4.3.<http://2.4.3
> > > > >>
> > > >
> > > .>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/
> > lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-
> > auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/
> > > > >> 2.4.3.<http://2.4.3
> > > > >>
> > > >
> > > .>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/jackson-core-2.3.1.jar:/usr/hdp/
> > > > >> 2.4.3.<http://2.4.3
> > > > >>
> > > >
> > > .>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/
> > usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/
> > hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.
> > v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.
> > 94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/
> > > > >> 2.4.3.<http://2.4.3
> > > > >>
> > > >
> > > .>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > minlog-1.2.jar:/usr/hdp/
> > > > >> 2.4.3.<http://2.4.3
> > > >
> > > .>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-
> > 227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-
> > 227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-
> > 227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/
> > lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/
> > storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> > zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-
> > uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/
> > hdp/2.4.3.0-227/storm/bin
> > > > >> -Dstorm.jar=/usr/metron/0.3.0/
> > lib/metron-parsers-0.3.0-uber.jar
> > > > >> org.apache.metron.parsers.topology.ParserTopologyCLI
> -k
> > > > x.localdomain:6667
> > > > >> -z x.localdomain:2181 -s bro
> > > > >> Exception in thread "main" java.lang.
> BootstrapMethodError:
> > > > >> java.lang.NoClassDefFoundError:
> org/apache/storm/Config
> > > > >> at
> > > > >>
> > > >
> > > org.apache.metron.parsers.topology.ParserTopologyCLI$
> > ParserOptions.<init>(ParserTopologyCLI.java:199)
> > > > >> at
> > > > >>
> > > >
> > > org.apache.metron.parsers.topology.ParserTopologyCLI$
> > ParserOptions.<clinit>(ParserTopologyCLI.java:47)
> > > > >> at
> > > > >>
> > > >
> > > org.apache.metron.parsers.topology.ParserTopologyCLI.
> > main(ParserTopologyCLI.java:266)
> > > > >> Caused by: java.lang.NoClassDefFoundError:
> > org/apache/storm/Config
> > > > >> ... 3 more
> > > > >> Caused by: java.lang.ClassNotFoundException:
> > > org.apache.storm.Config
> > > > >> at java.net.URLClassLoader.
> findClass(URLClassLoader.java:
> > 381)
> > > > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:
> 424)
> > > > >> at sun.misc.Launcher$AppClassLoader.loadClass(
> > Launcher.java:331)
> > > > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:
> 357)
> > > > >> ... 3 more
> > > > >>
> > > > >> Must be something wrong with my config, possibly I
> have
> > overlooked
> > > > >> something but can't put my finger on it. Any help is
> > appreciated.
> > > > >>
> > > > >> Many thanks
> > > > >> Shoggi
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Nick Allen <nick@nickallen.org<mailto:nic
> k@nickallen.org>>
> > > > >> --
> > > > >>
> > > > >> Jon
> > > > >>
> > > > >> Sent from my mobile device
> > > > >>
> > > > >> --
> > > > >>
> > > > >> Jon
> > > > >>
> > > > >> Sent from my mobile device
> > > > >>
> > > > >> --
> > > > > Jon
> > > > >
> > > > > Sent from my mobile device
> > > > >
> > > >
> > > > --
> > >
> > > Jon
> > >
> > > Sent from my mobile device
> > >
> > >
> > >
> > > --
> >
> > Jon
> >
> > Sent from my mobile device
> >
> >
> >
>
>
>
Re: Confluence write access to a space
Posted by Matt Foley <mf...@hortonworks.com>.
David,
No, it’s in my METRON-608 single-node deployment :-)
On 12/14/16, 12:46 PM, "David Lyle" <dl...@gmail.com> wrote:
Hi Matt and Jon,
FWIW, Metron with the MPack has been tested extensively on CentOS 7. Works
like a champ. The issue right now is that sensor install is a CentOS
6/Ansible proposition.
I haven't seen the issue with ES that you're experiencing, Matt. Is that in
a 1 Master 3 Data node ES config or something else?
-D...
On Wed, Dec 14, 2016 at 3:29 PM, Matt Foley <mf...@hortonworks.com> wrote:
> I hope we will try to support Centos7. Many of my company’s customers are
> requiring new installs to use RHEL 7 or Centos 7 rather than 6. In
> addition, we have the benefit that Centos 6 RPMs basically always run fine
> in Centos7 (barring heavy-duty low-level system manipulations that Metron,
> Kibana, and ES just don’t use), ES doesn’t distinguish between them, and
> Kibana install support for Centos7 is already in place in the Mpack. And
> of course the Hadoop Stack runs fine in 7.
>
> For what it’s worth, I’ve been working with the Mpack installation on
> Centos7 for the last few weeks, and it works fine except for a persistent
> issue where the Elasticsearch service works fine if launched interactively
> but terminates immediately if launched with same arguments as a service.
> Hope to find the cause in the next couple days. (If anyone knows why,
> would love to hear. The daemonized launch scripts come from ES, not from
> Metron, so “should just work”.)
>
> Thanks,
> --Matt
>
> On 12/13/16, 6:46 PM, "Zeolla@GMail.com" <ze...@gmail.com> wrote:
>
> For now, I'm not sure what the solution is, but I would think choosing
> one
> specific list of required/supported software (including OS) and
> documenting
> that thoroughly is the right start. Something like what currently
> exists
> for the vagrant side of things.
>
> That said, long term I would love to see broader support and
> automatically
> generating documentation derived from the code. Isn't CentOS 6 still
> preferred over 7? That's what I've been working on solely for that
> reason.
>
> Jon
>
> On Tue, Dec 13, 2016, 21:07 Matt Foley <ma...@apache.org> wrote:
>
> > In my work on METRON-608, I’ve found a lot of small but significant
> bugs
> > in the existing Mpack (version 0.3.0). These bugs integrate with a
> lot of
> > the oddball tweaks specified in the draft Install docs I saw. I
> would like
> > to submit a PR for these bugs, but they must be accompanied by
> changes in
> > the Install doc.
> >
> > How do we want to manage this? Should we version the Install doc?
> Or
> > have multiple sections for the different versions? Or have
> footnotes or
> > sidebar comments about which version certain paragraphs do and do
> not apply
> > to?
> >
> > I’m inclined to use sidebars within the document, because the
> dependencies
> > aren’t just on which versions of HDP, Ambari, and Metron you use.
> They
> > also depend on Python 2.6 vs 2.7, and Centos 6 vs 7.
> >
> > Thanks,
> > --Matt
> >
> >
> > On 12/13/16, 4:13 PM, "Zeolla@GMail.com" <ze...@gmail.com> wrote:
> >
> > Sorry about the delay. To be honest I delayed once I saw the
> > management UI
> > PR because I was going to wait for that to be merged into master
> > before I
> > did my mpack install. My initial thought is that there may need
> to be
> > some
> > sort of a merger between what you provided and the Hortonworks
> blog
> > post,
> > as I would prefer a single, comprehensive post over
> fragmentation.
> >
> > Jon
> >
> > On Tue, Dec 13, 2016, 18:29 Dima Kovalyov <
> Dima.Kovalyov@sstech.us>
> > wrote:
> >
> > > Thank you Jon,
> > >
> > > Just wondering if you have finished polishing the document? I
> will
> > have
> > > some time this week to polish and publish it if there is
> anything
> > left
> > > to format.
> > >
> > > - Dima
> > >
> > > On 12/01/2016 04:41 PM, Zeolla@GMail.com wrote:
> > > > While I can't delegate access to the wiki, I am working on
> > migrating the
> > > > documentation over. I have been modifying the document for
> > clarity and
> > > > updating up some of the formatting, so please give me some
> time to
> > do so.
> > > >
> > > > Jon
> > > >
> > > > On Thu, Dec 1, 2016 at 9:33 AM Dima Kovalyov <
> > Dima.Kovalyov@sstech.us>
> > > > wrote:
> > > >
> > > >> Hello,
> > > >>
> > > >> Who can provide me with an access to add new page in this
> > Installation
> > > >> space below?
> > > >> https://cwiki.apache.org/confluence/display/METRON/
> Installation
> > > >> I want to add up-to-date article on how-to install Metron
> using
> > Ambari
> > > >> mpack.
> > > >> Please advise.
> > > >>
> > > >> My username is "dima" registered with electronic mail
> dimdroll
> > hosted by
> > > >> gmail.
> > > >>
> > > >> Right now I have it posted here:
> > > >> https://goo.gl/HWGwpj
> > > >> But instead of maintaining it there it will be convenient
> for me
> > to have
> > > >> it in confluence instead of gdocs as I am using confluence
> in our
> > > internal
> > > >> project as well, so I can basically copy formatted doc
> between
> > each
> > > other.
> > > >>
> > > >> Thank you.
> > > >>
> > > >> - Dima
> > > >>
> > > >> On 11/30/2016 04:57 PM, Zeolla@GMail.com<mailto:Zeolla
> @GMail.com>
> > > wrote:
> > > >> Probably the best long term place for this would be here<
> > > >> https://cwiki.apache.org/confluence/display/METRON/
> Installation>,
> > > >> potentially with a new mpack article (to differentiate from
> the
> > ambari
> > > >> install which is already there, but dated). I can help with
> > migrating
> > > it
> > > >> to the wiki - I will run through your documentation later
> today
> > and
> > > request
> > > >> edit permissions if I find anything worth updating.
> > > >>
> > > >> Would you be OK if I put your doc in the wiki at some point?
> > > >>
> > > >> Jon
> > > >>
> > > >> On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <
> > Dima.Kovalyov@sstech.us
> > > >> <ma...@sstech.us>> wrote:
> > > >> As multiple users asked I have published document for
> installing
> > Metron
> > > on
> > > >> Centos 6 using Ambari 2.4 with HDP 2.5:
> > > >> https://goo.gl/HWGwpj
> > > >> It may look ugly as this is direct export from internal
> > confluence with
> > > >> cosmetic changes and our internal details removed. Let me
> know if
> > there
> > > is
> > > >> a better place to publish it.
> > > >>
> > > >> Feel free to request edit permissions, I would like the
> document
> > to be
> > > >> up-to-date, simplified and corrected if there are any
> errors. The
> > goal
> > > for
> > > >> this document is to provide installation guidance in a
> manner
> > that Linux
> > > >> administrator with basic knowledge could perform
> installation
> > without
> > > >> opening any other links or searching for commands, step by
> step.
> > > >> Thank you.
> > > >>
> > > >> p.s. feel free to let me know if you will encounter any
> errors as
> > I've
> > > >> seen them bunch.
> > > >>
> > > >> - Dima
> > > >>
> > > >>
> > > >>
> > > >> On 11/30/2016 04:45 PM, Zeolla@GMail.com<mailto:Zeolla
> @GMail.com>
> > > wrote:
> > > >> I also have written up a method to retrieve and setup the
> required
> > > >> packages in my script<
> > > >>
> > >
> > https://github.com/JonZeolla/Development/blob/master/Bash/
> setupMetron.sh>
> > > >> which could probably be cleaned up and incorporated.
> > > >>
> > > >> Jon
> > > >>
> > > >> On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <
> nick@nickallen.org
> > <mailto:
> > > >> nick@nickallen.org>> wrote:
> > > >> In regards to your comment about a "full package version
> dump"
> > there is
> > > at
> > > >> least a start at `metron-deployment/scripts/
> platform-info.sh`.
> > If we
> > > can
> > > >> enhance that, let's do it.
> > > >>
> > > >> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <shoggi@gmail.com
> <mailto:
> > > >> shoggi@gmail.com>> wrote:
> > > >> Hi
> > > >>
> > > >> Just to wrap this up. I've tried to deploy it on HDP 2.5
> but I
> > got the
> > > >> exact same problem. What I did was removing all components
> for
> > 2.4.3.0
> > > and
> > > >> then created a new cluster based on 2.5. I left it at that
> as the
> > most
> > > >> likely issue could be with my base install. May that be
> wrong
> > version of
> > > >> something or even a missing dev package.
> > > >>
> > > >> As my work also coincided with the announcement by Casey I
> went
> > ahead
> > > and
> > > >> installed 0.3.0 as a dev installation instead. That worked
> great
> > as the
> > > >> dev-full-platform guide is very complete.
> > > >>
> > > >> As a general feedback, in order to get more people
> interested and
> > not
> > > >> having noobs like me come up with redundant questions - At
> this
> > stage
> > > also
> > > >> sorry for not providing a meaningful subject, I simply was
> too
> > excited
> > > :)
> > > >>
> > > >> There is much to win with a small extension of the
> installation
> > guide. I
> > > >> don't think it needs much more prose around but a full
> package
> > version
> > > dump
> > > >> with every new release would help greatly to get things
> going.
> > For the
> > > >> things which need to be compiled in, the versioning
> information
> > should
> > > be
> > > >> enough. I am saying this, as everyone starts with a
> different base
> > > system.
> > > >> Some have the chance to start from scratch, some have a
> system
> > running
> > > and
> > > >> build something on top and then there are the experienced
> Metron
> > users,
> > > who
> > > >> have many base requirements.. they tend to forget. And I
> believe
> > at the
> > > end
> > > >> it always fails just because of some very tiny small
> difference.
> > > >>
> > > >> If there is a space somewhere available, this could even be
> a
> > spot,
> > > where
> > > >> people can say, "hey, I got this running on XYZ and here is
> the
> > package
> > > >> version dump". A simple table with 'Metron version', 'OS
> release',
> > > 'Package
> > > >> list file', 'Version Details of every other needed
> to-be-compiled
> > > package',
> > > >> is already a big help. This really could supplement the
> existing
> > guide
> > > >> found here:
> > > >>
> > >
> > https://cwiki.apache.org/confluence/display/METRON/
> Metron+Installation+on+an+Ambari-Managed+Cluster
> > > >> .
> > > >>
> > > >> But then, I could be alone and others did not experience
> the same
> > and
> > > >> simply got it to run, I don't know. Anyway, I would be
> happy to
> > help if
> > > >> help is needed.
> > > >>
> > > >> Cheers
> > > >> Shoggi
> > > >>
> > > >> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <shoggi@gmail.com
> <mailto:
> > > >> shoggi@gmail.com>> wrote:
> > > >> Hi Simon
> > > >>
> > > >> Thank you for your very quick response. I installed it on
> HDP
> > 2.4.3.0.
> > > >>
> > > >> I had a 0.2.0 instance running and was very conscious about
> > versioning
> > > the
> > > >> various components. Somehow I had the understanding of not
> > touching 2.5
> > > as
> > > >> of yet. I give that a try.
> > > >>
> > > >> Thanks heaps !
> > > >>
> > > >> Shoggi
> > > >>
> > > >> On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <
> > > >> simon@simonellistonball.com<mailto:simon@
> simonellistonball.com>>
> > wrote:
> > > >> Hi Shoggi,
> > > >>
> > > >> Can I ask which version of HDP you used as a base. The
> article is
> > a
> > > little
> > > >> out of date in that is calls for 2.4, which the 0.3.0
> release is
> > based
> > > on
> > > >> 2.5. There was a major version difference in storm between
> the
> > two,
> > > which
> > > >> seems likely to have caused the problem you're seeing.
> > > >>
> > > >> Simon
> > > >>
> > > >> Sent from my iPad
> > > >>
> > > >> On 27 Nov 2016, at 13:48, shoggi <shoggi@gmail.com<mailto:
> > > shoggi@gmail.com>>
> > > >> wrote:
> > > >>
> > > >> Hi all
> > > >>
> > > >> I followed the installation guide to deploy Metron via
> Ambari
> > > >>
> > >
> > https://community.hortonworks.com/articles/60805/deploying-
> a-fresh-metron-cluster-using-ambari-serv.html
> > > >> .
> > > >>
> > > >> I ended up with all but the Metron Components (Enrichment,
> > Parsing,
> > > >> Parsers) to start up. The closest to pinpoint the issue is,
> when
> > I try
> > > to
> > > >> start any new parser topology such as with bro:
> > > >>
> > > >> /usr/metron/0.3.0/bin/start_parser_topology.sh -k
> > x.localdomain:6667 -z
> > > >> x.localdomain:2181 -s bro
> > > >>
> > > >> The error I get is:
> > > >>
> > > >> Exception in thread "main" java.lang.BootstrapMethodError:
> > > >> java.lang.NoClassDefFoundError: org/apache/storm/Config
> > > >>
> > > >> Or in more details:
> > > >>
> > > >> Running: /usr/jdk64/jdk1.8.0_77/bin/java -client
> -Ddaemon.name=
> > > >> -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-227/storm
> > > >> -Dstorm.log.dir=/var/log/storm
> > > >>
> > >
> > -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/
> lib:/usr/hdp/current/storm-client/lib
> > > >> -Dstorm.conf.file= -cp
> > > >>
> > >
> > /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.
> 3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> clojure-1.6.0.jar:/usr/hdp/
> > > >> 2.4.3.<http://2.4.3
> > > >>
> > >
> > .>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/
> lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-
> auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/
> > > >> 2.4.3.<http://2.4.3
> > > >>
> > >
> > .>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/jackson-core-2.3.1.jar:/usr/hdp/
> > > >> 2.4.3.<http://2.4.3
> > > >>
> > >
> > .>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/
> usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/
> hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.
> v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.
> 94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/
> > > >> 2.4.3.<http://2.4.3
> > > >>
> > >
> > .>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> minlog-1.2.jar:/usr/hdp/
> > > >> 2.4.3.<http://2.4.3
> > >
> > .>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-
> 227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-
> 227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-
> 227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/
> lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-
> uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/
> hdp/2.4.3.0-227/storm/bin
> > > >> -Dstorm.jar=/usr/metron/0.3.0/
> lib/metron-parsers-0.3.0-uber.jar
> > > >> org.apache.metron.parsers.topology.ParserTopologyCLI -k
> > > x.localdomain:6667
> > > >> -z x.localdomain:2181 -s bro
> > > >> Exception in thread "main" java.lang.BootstrapMethodError:
> > > >> java.lang.NoClassDefFoundError: org/apache/storm/Config
> > > >> at
> > > >>
> > >
> > org.apache.metron.parsers.topology.ParserTopologyCLI$
> ParserOptions.<init>(ParserTopologyCLI.java:199)
> > > >> at
> > > >>
> > >
> > org.apache.metron.parsers.topology.ParserTopologyCLI$
> ParserOptions.<clinit>(ParserTopologyCLI.java:47)
> > > >> at
> > > >>
> > >
> > org.apache.metron.parsers.topology.ParserTopologyCLI.
> main(ParserTopologyCLI.java:266)
> > > >> Caused by: java.lang.NoClassDefFoundError:
> org/apache/storm/Config
> > > >> ... 3 more
> > > >> Caused by: java.lang.ClassNotFoundException:
> > org.apache.storm.Config
> > > >> at java.net.URLClassLoader.findClass(URLClassLoader.java:
> 381)
> > > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> > > >> at sun.misc.Launcher$AppClassLoader.loadClass(
> Launcher.java:331)
> > > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> > > >> ... 3 more
> > > >>
> > > >> Must be something wrong with my config, possibly I have
> overlooked
> > > >> something but can't put my finger on it. Any help is
> appreciated.
> > > >>
> > > >> Many thanks
> > > >> Shoggi
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Nick Allen <ni...@nickallen.org>>
> > > >> --
> > > >>
> > > >> Jon
> > > >>
> > > >> Sent from my mobile device
> > > >>
> > > >> --
> > > >>
> > > >> Jon
> > > >>
> > > >> Sent from my mobile device
> > > >>
> > > >> --
> > > > Jon
> > > >
> > > > Sent from my mobile device
> > > >
> > >
> > > --
> >
> > Jon
> >
> > Sent from my mobile device
> >
> >
> >
> > --
>
> Jon
>
> Sent from my mobile device
>
>
>
Re: Confluence write access to a space
Posted by David Lyle <dl...@gmail.com>.
Hi Matt and Jon,
FWIW, Metron with the MPack has been tested extensively on CentOS 7. Works
like a champ. The issue right now is that sensor install is a CentOS
6/Ansible proposition.
I haven't seen the issue with ES that you're experiencing, Matt. Is that in
a 1 Master 3 Data node ES config or something else?
-D...
On Wed, Dec 14, 2016 at 3:29 PM, Matt Foley <mf...@hortonworks.com> wrote:
> I hope we will try to support Centos7. Many of my company’s customers are
> requiring new installs to use RHEL 7 or Centos 7 rather than 6. In
> addition, we have the benefit that Centos 6 RPMs basically always run fine
> in Centos7 (barring heavy-duty low-level system manipulations that Metron,
> Kibana, and ES just don’t use), ES doesn’t distinguish between them, and
> Kibana install support for Centos7 is already in place in the Mpack. And
> of course the Hadoop Stack runs fine in 7.
>
> For what it’s worth, I’ve been working with the Mpack installation on
> Centos7 for the last few weeks, and it works fine except for a persistent
> issue where the Elasticsearch service works fine if launched interactively
> but terminates immediately if launched with same arguments as a service.
> Hope to find the cause in the next couple days. (If anyone knows why,
> would love to hear. The daemonized launch scripts come from ES, not from
> Metron, so “should just work”.)
>
> Thanks,
> --Matt
>
> On 12/13/16, 6:46 PM, "Zeolla@GMail.com" <ze...@gmail.com> wrote:
>
> For now, I'm not sure what the solution is, but I would think choosing
> one
> specific list of required/supported software (including OS) and
> documenting
> that thoroughly is the right start. Something like what currently
> exists
> for the vagrant side of things.
>
> That said, long term I would love to see broader support and
> automatically
> generating documentation derived from the code. Isn't CentOS 6 still
> preferred over 7? That's what I've been working on solely for that
> reason.
>
> Jon
>
> On Tue, Dec 13, 2016, 21:07 Matt Foley <ma...@apache.org> wrote:
>
> > In my work on METRON-608, I’ve found a lot of small but significant
> bugs
> > in the existing Mpack (version 0.3.0). These bugs integrate with a
> lot of
> > the oddball tweaks specified in the draft Install docs I saw. I
> would like
> > to submit a PR for these bugs, but they must be accompanied by
> changes in
> > the Install doc.
> >
> > How do we want to manage this? Should we version the Install doc?
> Or
> > have multiple sections for the different versions? Or have
> footnotes or
> > sidebar comments about which version certain paragraphs do and do
> not apply
> > to?
> >
> > I’m inclined to use sidebars within the document, because the
> dependencies
> > aren’t just on which versions of HDP, Ambari, and Metron you use.
> They
> > also depend on Python 2.6 vs 2.7, and Centos 6 vs 7.
> >
> > Thanks,
> > --Matt
> >
> >
> > On 12/13/16, 4:13 PM, "Zeolla@GMail.com" <ze...@gmail.com> wrote:
> >
> > Sorry about the delay. To be honest I delayed once I saw the
> > management UI
> > PR because I was going to wait for that to be merged into master
> > before I
> > did my mpack install. My initial thought is that there may need
> to be
> > some
> > sort of a merger between what you provided and the Hortonworks
> blog
> > post,
> > as I would prefer a single, comprehensive post over
> fragmentation.
> >
> > Jon
> >
> > On Tue, Dec 13, 2016, 18:29 Dima Kovalyov <
> Dima.Kovalyov@sstech.us>
> > wrote:
> >
> > > Thank you Jon,
> > >
> > > Just wondering if you have finished polishing the document? I
> will
> > have
> > > some time this week to polish and publish it if there is
> anything
> > left
> > > to format.
> > >
> > > - Dima
> > >
> > > On 12/01/2016 04:41 PM, Zeolla@GMail.com wrote:
> > > > While I can't delegate access to the wiki, I am working on
> > migrating the
> > > > documentation over. I have been modifying the document for
> > clarity and
> > > > updating up some of the formatting, so please give me some
> time to
> > do so.
> > > >
> > > > Jon
> > > >
> > > > On Thu, Dec 1, 2016 at 9:33 AM Dima Kovalyov <
> > Dima.Kovalyov@sstech.us>
> > > > wrote:
> > > >
> > > >> Hello,
> > > >>
> > > >> Who can provide me with an access to add new page in this
> > Installation
> > > >> space below?
> > > >> https://cwiki.apache.org/confluence/display/METRON/
> Installation
> > > >> I want to add up-to-date article on how-to install Metron
> using
> > Ambari
> > > >> mpack.
> > > >> Please advise.
> > > >>
> > > >> My username is "dima" registered with electronic mail
> dimdroll
> > hosted by
> > > >> gmail.
> > > >>
> > > >> Right now I have it posted here:
> > > >> https://goo.gl/HWGwpj
> > > >> But instead of maintaining it there it will be convenient
> for me
> > to have
> > > >> it in confluence instead of gdocs as I am using confluence
> in our
> > > internal
> > > >> project as well, so I can basically copy formatted doc
> between
> > each
> > > other.
> > > >>
> > > >> Thank you.
> > > >>
> > > >> - Dima
> > > >>
> > > >> On 11/30/2016 04:57 PM, Zeolla@GMail.com<mailto:Zeolla
> @GMail.com>
> > > wrote:
> > > >> Probably the best long term place for this would be here<
> > > >> https://cwiki.apache.org/confluence/display/METRON/
> Installation>,
> > > >> potentially with a new mpack article (to differentiate from
> the
> > ambari
> > > >> install which is already there, but dated). I can help with
> > migrating
> > > it
> > > >> to the wiki - I will run through your documentation later
> today
> > and
> > > request
> > > >> edit permissions if I find anything worth updating.
> > > >>
> > > >> Would you be OK if I put your doc in the wiki at some point?
> > > >>
> > > >> Jon
> > > >>
> > > >> On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <
> > Dima.Kovalyov@sstech.us
> > > >> <ma...@sstech.us>> wrote:
> > > >> As multiple users asked I have published document for
> installing
> > Metron
> > > on
> > > >> Centos 6 using Ambari 2.4 with HDP 2.5:
> > > >> https://goo.gl/HWGwpj
> > > >> It may look ugly as this is direct export from internal
> > confluence with
> > > >> cosmetic changes and our internal details removed. Let me
> know if
> > there
> > > is
> > > >> a better place to publish it.
> > > >>
> > > >> Feel free to request edit permissions, I would like the
> document
> > to be
> > > >> up-to-date, simplified and corrected if there are any
> errors. The
> > goal
> > > for
> > > >> this document is to provide installation guidance in a
> manner
> > that Linux
> > > >> administrator with basic knowledge could perform
> installation
> > without
> > > >> opening any other links or searching for commands, step by
> step.
> > > >> Thank you.
> > > >>
> > > >> p.s. feel free to let me know if you will encounter any
> errors as
> > I've
> > > >> seen them bunch.
> > > >>
> > > >> - Dima
> > > >>
> > > >>
> > > >>
> > > >> On 11/30/2016 04:45 PM, Zeolla@GMail.com<mailto:Zeolla
> @GMail.com>
> > > wrote:
> > > >> I also have written up a method to retrieve and setup the
> required
> > > >> packages in my script<
> > > >>
> > >
> > https://github.com/JonZeolla/Development/blob/master/Bash/
> setupMetron.sh>
> > > >> which could probably be cleaned up and incorporated.
> > > >>
> > > >> Jon
> > > >>
> > > >> On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <
> nick@nickallen.org
> > <mailto:
> > > >> nick@nickallen.org>> wrote:
> > > >> In regards to your comment about a "full package version
> dump"
> > there is
> > > at
> > > >> least a start at `metron-deployment/scripts/
> platform-info.sh`.
> > If we
> > > can
> > > >> enhance that, let's do it.
> > > >>
> > > >> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <shoggi@gmail.com
> <mailto:
> > > >> shoggi@gmail.com>> wrote:
> > > >> Hi
> > > >>
> > > >> Just to wrap this up. I've tried to deploy it on HDP 2.5
> but I
> > got the
> > > >> exact same problem. What I did was removing all components
> for
> > 2.4.3.0
> > > and
> > > >> then created a new cluster based on 2.5. I left it at that
> as the
> > most
> > > >> likely issue could be with my base install. May that be
> wrong
> > version of
> > > >> something or even a missing dev package.
> > > >>
> > > >> As my work also coincided with the announcement by Casey I
> went
> > ahead
> > > and
> > > >> installed 0.3.0 as a dev installation instead. That worked
> great
> > as the
> > > >> dev-full-platform guide is very complete.
> > > >>
> > > >> As a general feedback, in order to get more people
> interested and
> > not
> > > >> having noobs like me come up with redundant questions - At
> this
> > stage
> > > also
> > > >> sorry for not providing a meaningful subject, I simply was
> too
> > excited
> > > :)
> > > >>
> > > >> There is much to win with a small extension of the
> installation
> > guide. I
> > > >> don't think it needs much more prose around but a full
> package
> > version
> > > dump
> > > >> with every new release would help greatly to get things
> going.
> > For the
> > > >> things which need to be compiled in, the versioning
> information
> > should
> > > be
> > > >> enough. I am saying this, as everyone starts with a
> different base
> > > system.
> > > >> Some have the chance to start from scratch, some have a
> system
> > running
> > > and
> > > >> build something on top and then there are the experienced
> Metron
> > users,
> > > who
> > > >> have many base requirements.. they tend to forget. And I
> believe
> > at the
> > > end
> > > >> it always fails just because of some very tiny small
> difference.
> > > >>
> > > >> If there is a space somewhere available, this could even be
> a
> > spot,
> > > where
> > > >> people can say, "hey, I got this running on XYZ and here is
> the
> > package
> > > >> version dump". A simple table with 'Metron version', 'OS
> release',
> > > 'Package
> > > >> list file', 'Version Details of every other needed
> to-be-compiled
> > > package',
> > > >> is already a big help. This really could supplement the
> existing
> > guide
> > > >> found here:
> > > >>
> > >
> > https://cwiki.apache.org/confluence/display/METRON/
> Metron+Installation+on+an+Ambari-Managed+Cluster
> > > >> .
> > > >>
> > > >> But then, I could be alone and others did not experience
> the same
> > and
> > > >> simply got it to run, I don't know. Anyway, I would be
> happy to
> > help if
> > > >> help is needed.
> > > >>
> > > >> Cheers
> > > >> Shoggi
> > > >>
> > > >> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <shoggi@gmail.com
> <mailto:
> > > >> shoggi@gmail.com>> wrote:
> > > >> Hi Simon
> > > >>
> > > >> Thank you for your very quick response. I installed it on
> HDP
> > 2.4.3.0.
> > > >>
> > > >> I had a 0.2.0 instance running and was very conscious about
> > versioning
> > > the
> > > >> various components. Somehow I had the understanding of not
> > touching 2.5
> > > as
> > > >> of yet. I give that a try.
> > > >>
> > > >> Thanks heaps !
> > > >>
> > > >> Shoggi
> > > >>
> > > >> On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <
> > > >> simon@simonellistonball.com<mailto:simon@
> simonellistonball.com>>
> > wrote:
> > > >> Hi Shoggi,
> > > >>
> > > >> Can I ask which version of HDP you used as a base. The
> article is
> > a
> > > little
> > > >> out of date in that is calls for 2.4, which the 0.3.0
> release is
> > based
> > > on
> > > >> 2.5. There was a major version difference in storm between
> the
> > two,
> > > which
> > > >> seems likely to have caused the problem you're seeing.
> > > >>
> > > >> Simon
> > > >>
> > > >> Sent from my iPad
> > > >>
> > > >> On 27 Nov 2016, at 13:48, shoggi <shoggi@gmail.com<mailto:
> > > shoggi@gmail.com>>
> > > >> wrote:
> > > >>
> > > >> Hi all
> > > >>
> > > >> I followed the installation guide to deploy Metron via
> Ambari
> > > >>
> > >
> > https://community.hortonworks.com/articles/60805/deploying-
> a-fresh-metron-cluster-using-ambari-serv.html
> > > >> .
> > > >>
> > > >> I ended up with all but the Metron Components (Enrichment,
> > Parsing,
> > > >> Parsers) to start up. The closest to pinpoint the issue is,
> when
> > I try
> > > to
> > > >> start any new parser topology such as with bro:
> > > >>
> > > >> /usr/metron/0.3.0/bin/start_parser_topology.sh -k
> > x.localdomain:6667 -z
> > > >> x.localdomain:2181 -s bro
> > > >>
> > > >> The error I get is:
> > > >>
> > > >> Exception in thread "main" java.lang.BootstrapMethodError:
> > > >> java.lang.NoClassDefFoundError: org/apache/storm/Config
> > > >>
> > > >> Or in more details:
> > > >>
> > > >> Running: /usr/jdk64/jdk1.8.0_77/bin/java -client
> -Ddaemon.name=
> > > >> -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-227/storm
> > > >> -Dstorm.log.dir=/var/log/storm
> > > >>
> > >
> > -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/
> lib:/usr/hdp/current/storm-client/lib
> > > >> -Dstorm.conf.file= -cp
> > > >>
> > >
> > /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.
> 3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> clojure-1.6.0.jar:/usr/hdp/
> > > >> 2.4.3.<http://2.4.3
> > > >>
> > >
> > .>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/
> lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-
> auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/
> > > >> 2.4.3.<http://2.4.3
> > > >>
> > >
> > .>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/jackson-core-2.3.1.jar:/usr/hdp/
> > > >> 2.4.3.<http://2.4.3
> > > >>
> > >
> > .>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/
> usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/
> hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.
> v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.
> 94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/
> > > >> 2.4.3.<http://2.4.3
> > > >>
> > >
> > .>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> minlog-1.2.jar:/usr/hdp/
> > > >> 2.4.3.<http://2.4.3
> > >
> > .>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-
> 227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-
> 227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-
> 227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/
> lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/
> storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/
> zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-
> uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/
> hdp/2.4.3.0-227/storm/bin
> > > >> -Dstorm.jar=/usr/metron/0.3.0/
> lib/metron-parsers-0.3.0-uber.jar
> > > >> org.apache.metron.parsers.topology.ParserTopologyCLI -k
> > > x.localdomain:6667
> > > >> -z x.localdomain:2181 -s bro
> > > >> Exception in thread "main" java.lang.BootstrapMethodError:
> > > >> java.lang.NoClassDefFoundError: org/apache/storm/Config
> > > >> at
> > > >>
> > >
> > org.apache.metron.parsers.topology.ParserTopologyCLI$
> ParserOptions.<init>(ParserTopologyCLI.java:199)
> > > >> at
> > > >>
> > >
> > org.apache.metron.parsers.topology.ParserTopologyCLI$
> ParserOptions.<clinit>(ParserTopologyCLI.java:47)
> > > >> at
> > > >>
> > >
> > org.apache.metron.parsers.topology.ParserTopologyCLI.
> main(ParserTopologyCLI.java:266)
> > > >> Caused by: java.lang.NoClassDefFoundError:
> org/apache/storm/Config
> > > >> ... 3 more
> > > >> Caused by: java.lang.ClassNotFoundException:
> > org.apache.storm.Config
> > > >> at java.net.URLClassLoader.findClass(URLClassLoader.java:
> 381)
> > > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> > > >> at sun.misc.Launcher$AppClassLoader.loadClass(
> Launcher.java:331)
> > > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> > > >> ... 3 more
> > > >>
> > > >> Must be something wrong with my config, possibly I have
> overlooked
> > > >> something but can't put my finger on it. Any help is
> appreciated.
> > > >>
> > > >> Many thanks
> > > >> Shoggi
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Nick Allen <ni...@nickallen.org>>
> > > >> --
> > > >>
> > > >> Jon
> > > >>
> > > >> Sent from my mobile device
> > > >>
> > > >> --
> > > >>
> > > >> Jon
> > > >>
> > > >> Sent from my mobile device
> > > >>
> > > >> --
> > > > Jon
> > > >
> > > > Sent from my mobile device
> > > >
> > >
> > > --
> >
> > Jon
> >
> > Sent from my mobile device
> >
> >
> >
> > --
>
> Jon
>
> Sent from my mobile device
>
>
>
Re: Confluence write access to a space
Posted by Matt Foley <mf...@hortonworks.com>.
I hope we will try to support Centos7. Many of my company’s customers are requiring new installs to use RHEL 7 or Centos 7 rather than 6. In addition, we have the benefit that Centos 6 RPMs basically always run fine in Centos7 (barring heavy-duty low-level system manipulations that Metron, Kibana, and ES just don’t use), ES doesn’t distinguish between them, and Kibana install support for Centos7 is already in place in the Mpack. And of course the Hadoop Stack runs fine in 7.
For what it’s worth, I’ve been working with the Mpack installation on Centos7 for the last few weeks, and it works fine except for a persistent issue where the Elasticsearch service works fine if launched interactively but terminates immediately if launched with same arguments as a service. Hope to find the cause in the next couple days. (If anyone knows why, would love to hear. The daemonized launch scripts come from ES, not from Metron, so “should just work”.)
Thanks,
--Matt
On 12/13/16, 6:46 PM, "Zeolla@GMail.com" <ze...@gmail.com> wrote:
For now, I'm not sure what the solution is, but I would think choosing one
specific list of required/supported software (including OS) and documenting
that thoroughly is the right start. Something like what currently exists
for the vagrant side of things.
That said, long term I would love to see broader support and automatically
generating documentation derived from the code. Isn't CentOS 6 still
preferred over 7? That's what I've been working on solely for that
reason.
Jon
On Tue, Dec 13, 2016, 21:07 Matt Foley <ma...@apache.org> wrote:
> In my work on METRON-608, I’ve found a lot of small but significant bugs
> in the existing Mpack (version 0.3.0). These bugs integrate with a lot of
> the oddball tweaks specified in the draft Install docs I saw. I would like
> to submit a PR for these bugs, but they must be accompanied by changes in
> the Install doc.
>
> How do we want to manage this? Should we version the Install doc? Or
> have multiple sections for the different versions? Or have footnotes or
> sidebar comments about which version certain paragraphs do and do not apply
> to?
>
> I’m inclined to use sidebars within the document, because the dependencies
> aren’t just on which versions of HDP, Ambari, and Metron you use. They
> also depend on Python 2.6 vs 2.7, and Centos 6 vs 7.
>
> Thanks,
> --Matt
>
>
> On 12/13/16, 4:13 PM, "Zeolla@GMail.com" <ze...@gmail.com> wrote:
>
> Sorry about the delay. To be honest I delayed once I saw the
> management UI
> PR because I was going to wait for that to be merged into master
> before I
> did my mpack install. My initial thought is that there may need to be
> some
> sort of a merger between what you provided and the Hortonworks blog
> post,
> as I would prefer a single, comprehensive post over fragmentation.
>
> Jon
>
> On Tue, Dec 13, 2016, 18:29 Dima Kovalyov <Di...@sstech.us>
> wrote:
>
> > Thank you Jon,
> >
> > Just wondering if you have finished polishing the document? I will
> have
> > some time this week to polish and publish it if there is anything
> left
> > to format.
> >
> > - Dima
> >
> > On 12/01/2016 04:41 PM, Zeolla@GMail.com wrote:
> > > While I can't delegate access to the wiki, I am working on
> migrating the
> > > documentation over. I have been modifying the document for
> clarity and
> > > updating up some of the formatting, so please give me some time to
> do so.
> > >
> > > Jon
> > >
> > > On Thu, Dec 1, 2016 at 9:33 AM Dima Kovalyov <
> Dima.Kovalyov@sstech.us>
> > > wrote:
> > >
> > >> Hello,
> > >>
> > >> Who can provide me with an access to add new page in this
> Installation
> > >> space below?
> > >> https://cwiki.apache.org/confluence/display/METRON/Installation
> > >> I want to add up-to-date article on how-to install Metron using
> Ambari
> > >> mpack.
> > >> Please advise.
> > >>
> > >> My username is "dima" registered with electronic mail dimdroll
> hosted by
> > >> gmail.
> > >>
> > >> Right now I have it posted here:
> > >> https://goo.gl/HWGwpj
> > >> But instead of maintaining it there it will be convenient for me
> to have
> > >> it in confluence instead of gdocs as I am using confluence in our
> > internal
> > >> project as well, so I can basically copy formatted doc between
> each
> > other.
> > >>
> > >> Thank you.
> > >>
> > >> - Dima
> > >>
> > >> On 11/30/2016 04:57 PM, Zeolla@GMail.com<ma...@GMail.com>
> > wrote:
> > >> Probably the best long term place for this would be here<
> > >> https://cwiki.apache.org/confluence/display/METRON/Installation>,
> > >> potentially with a new mpack article (to differentiate from the
> ambari
> > >> install which is already there, but dated). I can help with
> migrating
> > it
> > >> to the wiki - I will run through your documentation later today
> and
> > request
> > >> edit permissions if I find anything worth updating.
> > >>
> > >> Would you be OK if I put your doc in the wiki at some point?
> > >>
> > >> Jon
> > >>
> > >> On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <
> Dima.Kovalyov@sstech.us
> > >> <ma...@sstech.us>> wrote:
> > >> As multiple users asked I have published document for installing
> Metron
> > on
> > >> Centos 6 using Ambari 2.4 with HDP 2.5:
> > >> https://goo.gl/HWGwpj
> > >> It may look ugly as this is direct export from internal
> confluence with
> > >> cosmetic changes and our internal details removed. Let me know if
> there
> > is
> > >> a better place to publish it.
> > >>
> > >> Feel free to request edit permissions, I would like the document
> to be
> > >> up-to-date, simplified and corrected if there are any errors. The
> goal
> > for
> > >> this document is to provide installation guidance in a manner
> that Linux
> > >> administrator with basic knowledge could perform installation
> without
> > >> opening any other links or searching for commands, step by step.
> > >> Thank you.
> > >>
> > >> p.s. feel free to let me know if you will encounter any errors as
> I've
> > >> seen them bunch.
> > >>
> > >> - Dima
> > >>
> > >>
> > >>
> > >> On 11/30/2016 04:45 PM, Zeolla@GMail.com<ma...@GMail.com>
> > wrote:
> > >> I also have written up a method to retrieve and setup the required
> > >> packages in my script<
> > >>
> >
> https://github.com/JonZeolla/Development/blob/master/Bash/setupMetron.sh>
> > >> which could probably be cleaned up and incorporated.
> > >>
> > >> Jon
> > >>
> > >> On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <nick@nickallen.org
> <mailto:
> > >> nick@nickallen.org>> wrote:
> > >> In regards to your comment about a "full package version dump"
> there is
> > at
> > >> least a start at `metron-deployment/scripts/platform-info.sh`.
> If we
> > can
> > >> enhance that, let's do it.
> > >>
> > >> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <shoggi@gmail.com<mailto:
> > >> shoggi@gmail.com>> wrote:
> > >> Hi
> > >>
> > >> Just to wrap this up. I've tried to deploy it on HDP 2.5 but I
> got the
> > >> exact same problem. What I did was removing all components for
> 2.4.3.0
> > and
> > >> then created a new cluster based on 2.5. I left it at that as the
> most
> > >> likely issue could be with my base install. May that be wrong
> version of
> > >> something or even a missing dev package.
> > >>
> > >> As my work also coincided with the announcement by Casey I went
> ahead
> > and
> > >> installed 0.3.0 as a dev installation instead. That worked great
> as the
> > >> dev-full-platform guide is very complete.
> > >>
> > >> As a general feedback, in order to get more people interested and
> not
> > >> having noobs like me come up with redundant questions - At this
> stage
> > also
> > >> sorry for not providing a meaningful subject, I simply was too
> excited
> > :)
> > >>
> > >> There is much to win with a small extension of the installation
> guide. I
> > >> don't think it needs much more prose around but a full package
> version
> > dump
> > >> with every new release would help greatly to get things going.
> For the
> > >> things which need to be compiled in, the versioning information
> should
> > be
> > >> enough. I am saying this, as everyone starts with a different base
> > system.
> > >> Some have the chance to start from scratch, some have a system
> running
> > and
> > >> build something on top and then there are the experienced Metron
> users,
> > who
> > >> have many base requirements.. they tend to forget. And I believe
> at the
> > end
> > >> it always fails just because of some very tiny small difference.
> > >>
> > >> If there is a space somewhere available, this could even be a
> spot,
> > where
> > >> people can say, "hey, I got this running on XYZ and here is the
> package
> > >> version dump". A simple table with 'Metron version', 'OS release',
> > 'Package
> > >> list file', 'Version Details of every other needed to-be-compiled
> > package',
> > >> is already a big help. This really could supplement the existing
> guide
> > >> found here:
> > >>
> >
> https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+an+Ambari-Managed+Cluster
> > >> .
> > >>
> > >> But then, I could be alone and others did not experience the same
> and
> > >> simply got it to run, I don't know. Anyway, I would be happy to
> help if
> > >> help is needed.
> > >>
> > >> Cheers
> > >> Shoggi
> > >>
> > >> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <shoggi@gmail.com<mailto:
> > >> shoggi@gmail.com>> wrote:
> > >> Hi Simon
> > >>
> > >> Thank you for your very quick response. I installed it on HDP
> 2.4.3.0.
> > >>
> > >> I had a 0.2.0 instance running and was very conscious about
> versioning
> > the
> > >> various components. Somehow I had the understanding of not
> touching 2.5
> > as
> > >> of yet. I give that a try.
> > >>
> > >> Thanks heaps !
> > >>
> > >> Shoggi
> > >>
> > >> On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <
> > >> simon@simonellistonball.com<ma...@simonellistonball.com>>
> wrote:
> > >> Hi Shoggi,
> > >>
> > >> Can I ask which version of HDP you used as a base. The article is
> a
> > little
> > >> out of date in that is calls for 2.4, which the 0.3.0 release is
> based
> > on
> > >> 2.5. There was a major version difference in storm between the
> two,
> > which
> > >> seems likely to have caused the problem you're seeing.
> > >>
> > >> Simon
> > >>
> > >> Sent from my iPad
> > >>
> > >> On 27 Nov 2016, at 13:48, shoggi <shoggi@gmail.com<mailto:
> > shoggi@gmail.com>>
> > >> wrote:
> > >>
> > >> Hi all
> > >>
> > >> I followed the installation guide to deploy Metron via Ambari
> > >>
> >
> https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html
> > >> .
> > >>
> > >> I ended up with all but the Metron Components (Enrichment,
> Parsing,
> > >> Parsers) to start up. The closest to pinpoint the issue is, when
> I try
> > to
> > >> start any new parser topology such as with bro:
> > >>
> > >> /usr/metron/0.3.0/bin/start_parser_topology.sh -k
> x.localdomain:6667 -z
> > >> x.localdomain:2181 -s bro
> > >>
> > >> The error I get is:
> > >>
> > >> Exception in thread "main" java.lang.BootstrapMethodError:
> > >> java.lang.NoClassDefFoundError: org/apache/storm/Config
> > >>
> > >> Or in more details:
> > >>
> > >> Running: /usr/jdk64/jdk1.8.0_77/bin/java -client -Ddaemon.name=
> > >> -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-227/storm
> > >> -Dstorm.log.dir=/var/log/storm
> > >>
> >
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> > >> -Dstorm.conf.file= -cp
> > >>
> >
> /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/clojure-1.6.0.jar:/usr/hdp/
> > >> 2.4.3.<http://2.4.3
> > >>
> >
> .>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/
> > >> 2.4.3.<http://2.4.3
> > >>
> >
> .>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/
> > >> 2.4.3.<http://2.4.3
> > >>
> >
> .>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/
> > >> 2.4.3.<http://2.4.3
> > >>
> >
> .>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/minlog-1.2.jar:/usr/hdp/
> > >> 2.4.3.<http://2.4.3
> >
> .>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.4.3.0-227/storm/bin
> > >> -Dstorm.jar=/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar
> > >> org.apache.metron.parsers.topology.ParserTopologyCLI -k
> > x.localdomain:6667
> > >> -z x.localdomain:2181 -s bro
> > >> Exception in thread "main" java.lang.BootstrapMethodError:
> > >> java.lang.NoClassDefFoundError: org/apache/storm/Config
> > >> at
> > >>
> >
> org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<init>(ParserTopologyCLI.java:199)
> > >> at
> > >>
> >
> org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<clinit>(ParserTopologyCLI.java:47)
> > >> at
> > >>
> >
> org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:266)
> > >> Caused by: java.lang.NoClassDefFoundError: org/apache/storm/Config
> > >> ... 3 more
> > >> Caused by: java.lang.ClassNotFoundException:
> org.apache.storm.Config
> > >> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> > >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> > >> ... 3 more
> > >>
> > >> Must be something wrong with my config, possibly I have overlooked
> > >> something but can't put my finger on it. Any help is appreciated.
> > >>
> > >> Many thanks
> > >> Shoggi
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> --
> > >> Nick Allen <ni...@nickallen.org>>
> > >> --
> > >>
> > >> Jon
> > >>
> > >> Sent from my mobile device
> > >>
> > >> --
> > >>
> > >> Jon
> > >>
> > >> Sent from my mobile device
> > >>
> > >> --
> > > Jon
> > >
> > > Sent from my mobile device
> > >
> >
> > --
>
> Jon
>
> Sent from my mobile device
>
>
>
> --
Jon
Sent from my mobile device
Re: Confluence write access to a space
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
For now, I'm not sure what the solution is, but I would think choosing one
specific list of required/supported software (including OS) and documenting
that thoroughly is the right start. Something like what currently exists
for the vagrant side of things.
That said, long term I would love to see broader support and automatically
generating documentation derived from the code. Isn't CentOS 6 still
preferred over 7? That's what I've been working on solely for that
reason.
Jon
On Tue, Dec 13, 2016, 21:07 Matt Foley <ma...@apache.org> wrote:
> In my work on METRON-608, I’ve found a lot of small but significant bugs
> in the existing Mpack (version 0.3.0). These bugs integrate with a lot of
> the oddball tweaks specified in the draft Install docs I saw. I would like
> to submit a PR for these bugs, but they must be accompanied by changes in
> the Install doc.
>
> How do we want to manage this? Should we version the Install doc? Or
> have multiple sections for the different versions? Or have footnotes or
> sidebar comments about which version certain paragraphs do and do not apply
> to?
>
> I’m inclined to use sidebars within the document, because the dependencies
> aren’t just on which versions of HDP, Ambari, and Metron you use. They
> also depend on Python 2.6 vs 2.7, and Centos 6 vs 7.
>
> Thanks,
> --Matt
>
>
> On 12/13/16, 4:13 PM, "Zeolla@GMail.com" <ze...@gmail.com> wrote:
>
> Sorry about the delay. To be honest I delayed once I saw the
> management UI
> PR because I was going to wait for that to be merged into master
> before I
> did my mpack install. My initial thought is that there may need to be
> some
> sort of a merger between what you provided and the Hortonworks blog
> post,
> as I would prefer a single, comprehensive post over fragmentation.
>
> Jon
>
> On Tue, Dec 13, 2016, 18:29 Dima Kovalyov <Di...@sstech.us>
> wrote:
>
> > Thank you Jon,
> >
> > Just wondering if you have finished polishing the document? I will
> have
> > some time this week to polish and publish it if there is anything
> left
> > to format.
> >
> > - Dima
> >
> > On 12/01/2016 04:41 PM, Zeolla@GMail.com wrote:
> > > While I can't delegate access to the wiki, I am working on
> migrating the
> > > documentation over. I have been modifying the document for
> clarity and
> > > updating up some of the formatting, so please give me some time to
> do so.
> > >
> > > Jon
> > >
> > > On Thu, Dec 1, 2016 at 9:33 AM Dima Kovalyov <
> Dima.Kovalyov@sstech.us>
> > > wrote:
> > >
> > >> Hello,
> > >>
> > >> Who can provide me with an access to add new page in this
> Installation
> > >> space below?
> > >> https://cwiki.apache.org/confluence/display/METRON/Installation
> > >> I want to add up-to-date article on how-to install Metron using
> Ambari
> > >> mpack.
> > >> Please advise.
> > >>
> > >> My username is "dima" registered with electronic mail dimdroll
> hosted by
> > >> gmail.
> > >>
> > >> Right now I have it posted here:
> > >> https://goo.gl/HWGwpj
> > >> But instead of maintaining it there it will be convenient for me
> to have
> > >> it in confluence instead of gdocs as I am using confluence in our
> > internal
> > >> project as well, so I can basically copy formatted doc between
> each
> > other.
> > >>
> > >> Thank you.
> > >>
> > >> - Dima
> > >>
> > >> On 11/30/2016 04:57 PM, Zeolla@GMail.com<ma...@GMail.com>
> > wrote:
> > >> Probably the best long term place for this would be here<
> > >> https://cwiki.apache.org/confluence/display/METRON/Installation>,
> > >> potentially with a new mpack article (to differentiate from the
> ambari
> > >> install which is already there, but dated). I can help with
> migrating
> > it
> > >> to the wiki - I will run through your documentation later today
> and
> > request
> > >> edit permissions if I find anything worth updating.
> > >>
> > >> Would you be OK if I put your doc in the wiki at some point?
> > >>
> > >> Jon
> > >>
> > >> On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <
> Dima.Kovalyov@sstech.us
> > >> <ma...@sstech.us>> wrote:
> > >> As multiple users asked I have published document for installing
> Metron
> > on
> > >> Centos 6 using Ambari 2.4 with HDP 2.5:
> > >> https://goo.gl/HWGwpj
> > >> It may look ugly as this is direct export from internal
> confluence with
> > >> cosmetic changes and our internal details removed. Let me know if
> there
> > is
> > >> a better place to publish it.
> > >>
> > >> Feel free to request edit permissions, I would like the document
> to be
> > >> up-to-date, simplified and corrected if there are any errors. The
> goal
> > for
> > >> this document is to provide installation guidance in a manner
> that Linux
> > >> administrator with basic knowledge could perform installation
> without
> > >> opening any other links or searching for commands, step by step.
> > >> Thank you.
> > >>
> > >> p.s. feel free to let me know if you will encounter any errors as
> I've
> > >> seen them bunch.
> > >>
> > >> - Dima
> > >>
> > >>
> > >>
> > >> On 11/30/2016 04:45 PM, Zeolla@GMail.com<ma...@GMail.com>
> > wrote:
> > >> I also have written up a method to retrieve and setup the required
> > >> packages in my script<
> > >>
> >
> https://github.com/JonZeolla/Development/blob/master/Bash/setupMetron.sh>
> > >> which could probably be cleaned up and incorporated.
> > >>
> > >> Jon
> > >>
> > >> On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <nick@nickallen.org
> <mailto:
> > >> nick@nickallen.org>> wrote:
> > >> In regards to your comment about a "full package version dump"
> there is
> > at
> > >> least a start at `metron-deployment/scripts/platform-info.sh`.
> If we
> > can
> > >> enhance that, let's do it.
> > >>
> > >> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <shoggi@gmail.com<mailto:
> > >> shoggi@gmail.com>> wrote:
> > >> Hi
> > >>
> > >> Just to wrap this up. I've tried to deploy it on HDP 2.5 but I
> got the
> > >> exact same problem. What I did was removing all components for
> 2.4.3.0
> > and
> > >> then created a new cluster based on 2.5. I left it at that as the
> most
> > >> likely issue could be with my base install. May that be wrong
> version of
> > >> something or even a missing dev package.
> > >>
> > >> As my work also coincided with the announcement by Casey I went
> ahead
> > and
> > >> installed 0.3.0 as a dev installation instead. That worked great
> as the
> > >> dev-full-platform guide is very complete.
> > >>
> > >> As a general feedback, in order to get more people interested and
> not
> > >> having noobs like me come up with redundant questions - At this
> stage
> > also
> > >> sorry for not providing a meaningful subject, I simply was too
> excited
> > :)
> > >>
> > >> There is much to win with a small extension of the installation
> guide. I
> > >> don't think it needs much more prose around but a full package
> version
> > dump
> > >> with every new release would help greatly to get things going.
> For the
> > >> things which need to be compiled in, the versioning information
> should
> > be
> > >> enough. I am saying this, as everyone starts with a different base
> > system.
> > >> Some have the chance to start from scratch, some have a system
> running
> > and
> > >> build something on top and then there are the experienced Metron
> users,
> > who
> > >> have many base requirements.. they tend to forget. And I believe
> at the
> > end
> > >> it always fails just because of some very tiny small difference.
> > >>
> > >> If there is a space somewhere available, this could even be a
> spot,
> > where
> > >> people can say, "hey, I got this running on XYZ and here is the
> package
> > >> version dump". A simple table with 'Metron version', 'OS release',
> > 'Package
> > >> list file', 'Version Details of every other needed to-be-compiled
> > package',
> > >> is already a big help. This really could supplement the existing
> guide
> > >> found here:
> > >>
> >
> https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+an+Ambari-Managed+Cluster
> > >> .
> > >>
> > >> But then, I could be alone and others did not experience the same
> and
> > >> simply got it to run, I don't know. Anyway, I would be happy to
> help if
> > >> help is needed.
> > >>
> > >> Cheers
> > >> Shoggi
> > >>
> > >> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <shoggi@gmail.com<mailto:
> > >> shoggi@gmail.com>> wrote:
> > >> Hi Simon
> > >>
> > >> Thank you for your very quick response. I installed it on HDP
> 2.4.3.0.
> > >>
> > >> I had a 0.2.0 instance running and was very conscious about
> versioning
> > the
> > >> various components. Somehow I had the understanding of not
> touching 2.5
> > as
> > >> of yet. I give that a try.
> > >>
> > >> Thanks heaps !
> > >>
> > >> Shoggi
> > >>
> > >> On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <
> > >> simon@simonellistonball.com<ma...@simonellistonball.com>>
> wrote:
> > >> Hi Shoggi,
> > >>
> > >> Can I ask which version of HDP you used as a base. The article is
> a
> > little
> > >> out of date in that is calls for 2.4, which the 0.3.0 release is
> based
> > on
> > >> 2.5. There was a major version difference in storm between the
> two,
> > which
> > >> seems likely to have caused the problem you're seeing.
> > >>
> > >> Simon
> > >>
> > >> Sent from my iPad
> > >>
> > >> On 27 Nov 2016, at 13:48, shoggi <shoggi@gmail.com<mailto:
> > shoggi@gmail.com>>
> > >> wrote:
> > >>
> > >> Hi all
> > >>
> > >> I followed the installation guide to deploy Metron via Ambari
> > >>
> >
> https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html
> > >> .
> > >>
> > >> I ended up with all but the Metron Components (Enrichment,
> Parsing,
> > >> Parsers) to start up. The closest to pinpoint the issue is, when
> I try
> > to
> > >> start any new parser topology such as with bro:
> > >>
> > >> /usr/metron/0.3.0/bin/start_parser_topology.sh -k
> x.localdomain:6667 -z
> > >> x.localdomain:2181 -s bro
> > >>
> > >> The error I get is:
> > >>
> > >> Exception in thread "main" java.lang.BootstrapMethodError:
> > >> java.lang.NoClassDefFoundError: org/apache/storm/Config
> > >>
> > >> Or in more details:
> > >>
> > >> Running: /usr/jdk64/jdk1.8.0_77/bin/java -client -Ddaemon.name=
> > >> -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-227/storm
> > >> -Dstorm.log.dir=/var/log/storm
> > >>
> >
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> > >> -Dstorm.conf.file= -cp
> > >>
> >
> /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/clojure-1.6.0.jar:/usr/hdp/
> > >> 2.4.3.<http://2.4.3
> > >>
> >
> .>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/
> > >> 2.4.3.<http://2.4.3
> > >>
> >
> .>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/
> > >> 2.4.3.<http://2.4.3
> > >>
> >
> .>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/
> > >> 2.4.3.<http://2.4.3
> > >>
> >
> .>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/minlog-1.2.jar:/usr/hdp/
> > >> 2.4.3.<http://2.4.3
> >
> .>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.4.3.0-227/storm/bin
> > >> -Dstorm.jar=/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar
> > >> org.apache.metron.parsers.topology.ParserTopologyCLI -k
> > x.localdomain:6667
> > >> -z x.localdomain:2181 -s bro
> > >> Exception in thread "main" java.lang.BootstrapMethodError:
> > >> java.lang.NoClassDefFoundError: org/apache/storm/Config
> > >> at
> > >>
> >
> org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<init>(ParserTopologyCLI.java:199)
> > >> at
> > >>
> >
> org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<clinit>(ParserTopologyCLI.java:47)
> > >> at
> > >>
> >
> org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:266)
> > >> Caused by: java.lang.NoClassDefFoundError: org/apache/storm/Config
> > >> ... 3 more
> > >> Caused by: java.lang.ClassNotFoundException:
> org.apache.storm.Config
> > >> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> > >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> > >> ... 3 more
> > >>
> > >> Must be something wrong with my config, possibly I have overlooked
> > >> something but can't put my finger on it. Any help is appreciated.
> > >>
> > >> Many thanks
> > >> Shoggi
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> --
> > >> Nick Allen <ni...@nickallen.org>>
> > >> --
> > >>
> > >> Jon
> > >>
> > >> Sent from my mobile device
> > >>
> > >> --
> > >>
> > >> Jon
> > >>
> > >> Sent from my mobile device
> > >>
> > >> --
> > > Jon
> > >
> > > Sent from my mobile device
> > >
> >
> > --
>
> Jon
>
> Sent from my mobile device
>
>
>
> --
Jon
Sent from my mobile device
Re: Confluence write access to a space
Posted by Matt Foley <ma...@apache.org>.
In my work on METRON-608, I’ve found a lot of small but significant bugs in the existing Mpack (version 0.3.0). These bugs integrate with a lot of the oddball tweaks specified in the draft Install docs I saw. I would like to submit a PR for these bugs, but they must be accompanied by changes in the Install doc.
How do we want to manage this? Should we version the Install doc? Or have multiple sections for the different versions? Or have footnotes or sidebar comments about which version certain paragraphs do and do not apply to?
I’m inclined to use sidebars within the document, because the dependencies aren’t just on which versions of HDP, Ambari, and Metron you use. They also depend on Python 2.6 vs 2.7, and Centos 6 vs 7.
Thanks,
--Matt
On 12/13/16, 4:13 PM, "Zeolla@GMail.com" <ze...@gmail.com> wrote:
Sorry about the delay. To be honest I delayed once I saw the management UI
PR because I was going to wait for that to be merged into master before I
did my mpack install. My initial thought is that there may need to be some
sort of a merger between what you provided and the Hortonworks blog post,
as I would prefer a single, comprehensive post over fragmentation.
Jon
On Tue, Dec 13, 2016, 18:29 Dima Kovalyov <Di...@sstech.us> wrote:
> Thank you Jon,
>
> Just wondering if you have finished polishing the document? I will have
> some time this week to polish and publish it if there is anything left
> to format.
>
> - Dima
>
> On 12/01/2016 04:41 PM, Zeolla@GMail.com wrote:
> > While I can't delegate access to the wiki, I am working on migrating the
> > documentation over. I have been modifying the document for clarity and
> > updating up some of the formatting, so please give me some time to do so.
> >
> > Jon
> >
> > On Thu, Dec 1, 2016 at 9:33 AM Dima Kovalyov <Di...@sstech.us>
> > wrote:
> >
> >> Hello,
> >>
> >> Who can provide me with an access to add new page in this Installation
> >> space below?
> >> https://cwiki.apache.org/confluence/display/METRON/Installation
> >> I want to add up-to-date article on how-to install Metron using Ambari
> >> mpack.
> >> Please advise.
> >>
> >> My username is "dima" registered with electronic mail dimdroll hosted by
> >> gmail.
> >>
> >> Right now I have it posted here:
> >> https://goo.gl/HWGwpj
> >> But instead of maintaining it there it will be convenient for me to have
> >> it in confluence instead of gdocs as I am using confluence in our
> internal
> >> project as well, so I can basically copy formatted doc between each
> other.
> >>
> >> Thank you.
> >>
> >> - Dima
> >>
> >> On 11/30/2016 04:57 PM, Zeolla@GMail.com<ma...@GMail.com>
> wrote:
> >> Probably the best long term place for this would be here<
> >> https://cwiki.apache.org/confluence/display/METRON/Installation>,
> >> potentially with a new mpack article (to differentiate from the ambari
> >> install which is already there, but dated). I can help with migrating
> it
> >> to the wiki - I will run through your documentation later today and
> request
> >> edit permissions if I find anything worth updating.
> >>
> >> Would you be OK if I put your doc in the wiki at some point?
> >>
> >> Jon
> >>
> >> On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <Dima.Kovalyov@sstech.us
> >> <ma...@sstech.us>> wrote:
> >> As multiple users asked I have published document for installing Metron
> on
> >> Centos 6 using Ambari 2.4 with HDP 2.5:
> >> https://goo.gl/HWGwpj
> >> It may look ugly as this is direct export from internal confluence with
> >> cosmetic changes and our internal details removed. Let me know if there
> is
> >> a better place to publish it.
> >>
> >> Feel free to request edit permissions, I would like the document to be
> >> up-to-date, simplified and corrected if there are any errors. The goal
> for
> >> this document is to provide installation guidance in a manner that Linux
> >> administrator with basic knowledge could perform installation without
> >> opening any other links or searching for commands, step by step.
> >> Thank you.
> >>
> >> p.s. feel free to let me know if you will encounter any errors as I've
> >> seen them bunch.
> >>
> >> - Dima
> >>
> >>
> >>
> >> On 11/30/2016 04:45 PM, Zeolla@GMail.com<ma...@GMail.com>
> wrote:
> >> I also have written up a method to retrieve and setup the required
> >> packages in my script<
> >>
> https://github.com/JonZeolla/Development/blob/master/Bash/setupMetron.sh>
> >> which could probably be cleaned up and incorporated.
> >>
> >> Jon
> >>
> >> On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <nick@nickallen.org<mailto:
> >> nick@nickallen.org>> wrote:
> >> In regards to your comment about a "full package version dump" there is
> at
> >> least a start at `metron-deployment/scripts/platform-info.sh`. If we
> can
> >> enhance that, let's do it.
> >>
> >> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <shoggi@gmail.com<mailto:
> >> shoggi@gmail.com>> wrote:
> >> Hi
> >>
> >> Just to wrap this up. I've tried to deploy it on HDP 2.5 but I got the
> >> exact same problem. What I did was removing all components for 2.4.3.0
> and
> >> then created a new cluster based on 2.5. I left it at that as the most
> >> likely issue could be with my base install. May that be wrong version of
> >> something or even a missing dev package.
> >>
> >> As my work also coincided with the announcement by Casey I went ahead
> and
> >> installed 0.3.0 as a dev installation instead. That worked great as the
> >> dev-full-platform guide is very complete.
> >>
> >> As a general feedback, in order to get more people interested and not
> >> having noobs like me come up with redundant questions - At this stage
> also
> >> sorry for not providing a meaningful subject, I simply was too excited
> :)
> >>
> >> There is much to win with a small extension of the installation guide. I
> >> don't think it needs much more prose around but a full package version
> dump
> >> with every new release would help greatly to get things going. For the
> >> things which need to be compiled in, the versioning information should
> be
> >> enough. I am saying this, as everyone starts with a different base
> system.
> >> Some have the chance to start from scratch, some have a system running
> and
> >> build something on top and then there are the experienced Metron users,
> who
> >> have many base requirements.. they tend to forget. And I believe at the
> end
> >> it always fails just because of some very tiny small difference.
> >>
> >> If there is a space somewhere available, this could even be a spot,
> where
> >> people can say, "hey, I got this running on XYZ and here is the package
> >> version dump". A simple table with 'Metron version', 'OS release',
> 'Package
> >> list file', 'Version Details of every other needed to-be-compiled
> package',
> >> is already a big help. This really could supplement the existing guide
> >> found here:
> >>
> https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+an+Ambari-Managed+Cluster
> >> .
> >>
> >> But then, I could be alone and others did not experience the same and
> >> simply got it to run, I don't know. Anyway, I would be happy to help if
> >> help is needed.
> >>
> >> Cheers
> >> Shoggi
> >>
> >> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <shoggi@gmail.com<mailto:
> >> shoggi@gmail.com>> wrote:
> >> Hi Simon
> >>
> >> Thank you for your very quick response. I installed it on HDP 2.4.3.0.
> >>
> >> I had a 0.2.0 instance running and was very conscious about versioning
> the
> >> various components. Somehow I had the understanding of not touching 2.5
> as
> >> of yet. I give that a try.
> >>
> >> Thanks heaps !
> >>
> >> Shoggi
> >>
> >> On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <
> >> simon@simonellistonball.com<ma...@simonellistonball.com>> wrote:
> >> Hi Shoggi,
> >>
> >> Can I ask which version of HDP you used as a base. The article is a
> little
> >> out of date in that is calls for 2.4, which the 0.3.0 release is based
> on
> >> 2.5. There was a major version difference in storm between the two,
> which
> >> seems likely to have caused the problem you're seeing.
> >>
> >> Simon
> >>
> >> Sent from my iPad
> >>
> >> On 27 Nov 2016, at 13:48, shoggi <shoggi@gmail.com<mailto:
> shoggi@gmail.com>>
> >> wrote:
> >>
> >> Hi all
> >>
> >> I followed the installation guide to deploy Metron via Ambari
> >>
> https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html
> >> .
> >>
> >> I ended up with all but the Metron Components (Enrichment, Parsing,
> >> Parsers) to start up. The closest to pinpoint the issue is, when I try
> to
> >> start any new parser topology such as with bro:
> >>
> >> /usr/metron/0.3.0/bin/start_parser_topology.sh -k x.localdomain:6667 -z
> >> x.localdomain:2181 -s bro
> >>
> >> The error I get is:
> >>
> >> Exception in thread "main" java.lang.BootstrapMethodError:
> >> java.lang.NoClassDefFoundError: org/apache/storm/Config
> >>
> >> Or in more details:
> >>
> >> Running: /usr/jdk64/jdk1.8.0_77/bin/java -client -Ddaemon.name=
> >> -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-227/storm
> >> -Dstorm.log.dir=/var/log/storm
> >>
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> >> -Dstorm.conf.file= -cp
> >>
> /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/clojure-1.6.0.jar:/usr/hdp/
> >> 2.4.3.<http://2.4.3
> >>
> .>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/
> >> 2.4.3.<http://2.4.3
> >>
> .>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/
> >> 2.4.3.<http://2.4.3
> >>
> .>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/
> >> 2.4.3.<http://2.4.3
> >>
> .>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/minlog-1.2.jar:/usr/hdp/
> >> 2.4.3.<http://2.4.3
> .>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.4.3.0-227/storm/bin
> >> -Dstorm.jar=/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar
> >> org.apache.metron.parsers.topology.ParserTopologyCLI -k
> x.localdomain:6667
> >> -z x.localdomain:2181 -s bro
> >> Exception in thread "main" java.lang.BootstrapMethodError:
> >> java.lang.NoClassDefFoundError: org/apache/storm/Config
> >> at
> >>
> org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<init>(ParserTopologyCLI.java:199)
> >> at
> >>
> org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<clinit>(ParserTopologyCLI.java:47)
> >> at
> >>
> org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:266)
> >> Caused by: java.lang.NoClassDefFoundError: org/apache/storm/Config
> >> ... 3 more
> >> Caused by: java.lang.ClassNotFoundException: org.apache.storm.Config
> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> >> ... 3 more
> >>
> >> Must be something wrong with my config, possibly I have overlooked
> >> something but can't put my finger on it. Any help is appreciated.
> >>
> >> Many thanks
> >> Shoggi
> >>
> >>
> >>
> >>
> >>
> >> --
> >> Nick Allen <ni...@nickallen.org>>
> >> --
> >>
> >> Jon
> >>
> >> Sent from my mobile device
> >>
> >> --
> >>
> >> Jon
> >>
> >> Sent from my mobile device
> >>
> >> --
> > Jon
> >
> > Sent from my mobile device
> >
>
> --
Jon
Sent from my mobile device
Re: Confluence write access to a space
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
Sorry about the delay. To be honest I delayed once I saw the management UI
PR because I was going to wait for that to be merged into master before I
did my mpack install. My initial thought is that there may need to be some
sort of a merger between what you provided and the Hortonworks blog post,
as I would prefer a single, comprehensive post over fragmentation.
Jon
On Tue, Dec 13, 2016, 18:29 Dima Kovalyov <Di...@sstech.us> wrote:
> Thank you Jon,
>
> Just wondering if you have finished polishing the document? I will have
> some time this week to polish and publish it if there is anything left
> to format.
>
> - Dima
>
> On 12/01/2016 04:41 PM, Zeolla@GMail.com wrote:
> > While I can't delegate access to the wiki, I am working on migrating the
> > documentation over. I have been modifying the document for clarity and
> > updating up some of the formatting, so please give me some time to do so.
> >
> > Jon
> >
> > On Thu, Dec 1, 2016 at 9:33 AM Dima Kovalyov <Di...@sstech.us>
> > wrote:
> >
> >> Hello,
> >>
> >> Who can provide me with an access to add new page in this Installation
> >> space below?
> >> https://cwiki.apache.org/confluence/display/METRON/Installation
> >> I want to add up-to-date article on how-to install Metron using Ambari
> >> mpack.
> >> Please advise.
> >>
> >> My username is "dima" registered with electronic mail dimdroll hosted by
> >> gmail.
> >>
> >> Right now I have it posted here:
> >> https://goo.gl/HWGwpj
> >> But instead of maintaining it there it will be convenient for me to have
> >> it in confluence instead of gdocs as I am using confluence in our
> internal
> >> project as well, so I can basically copy formatted doc between each
> other.
> >>
> >> Thank you.
> >>
> >> - Dima
> >>
> >> On 11/30/2016 04:57 PM, Zeolla@GMail.com<ma...@GMail.com>
> wrote:
> >> Probably the best long term place for this would be here<
> >> https://cwiki.apache.org/confluence/display/METRON/Installation>,
> >> potentially with a new mpack article (to differentiate from the ambari
> >> install which is already there, but dated). I can help with migrating
> it
> >> to the wiki - I will run through your documentation later today and
> request
> >> edit permissions if I find anything worth updating.
> >>
> >> Would you be OK if I put your doc in the wiki at some point?
> >>
> >> Jon
> >>
> >> On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <Dima.Kovalyov@sstech.us
> >> <ma...@sstech.us>> wrote:
> >> As multiple users asked I have published document for installing Metron
> on
> >> Centos 6 using Ambari 2.4 with HDP 2.5:
> >> https://goo.gl/HWGwpj
> >> It may look ugly as this is direct export from internal confluence with
> >> cosmetic changes and our internal details removed. Let me know if there
> is
> >> a better place to publish it.
> >>
> >> Feel free to request edit permissions, I would like the document to be
> >> up-to-date, simplified and corrected if there are any errors. The goal
> for
> >> this document is to provide installation guidance in a manner that Linux
> >> administrator with basic knowledge could perform installation without
> >> opening any other links or searching for commands, step by step.
> >> Thank you.
> >>
> >> p.s. feel free to let me know if you will encounter any errors as I've
> >> seen them bunch.
> >>
> >> - Dima
> >>
> >>
> >>
> >> On 11/30/2016 04:45 PM, Zeolla@GMail.com<ma...@GMail.com>
> wrote:
> >> I also have written up a method to retrieve and setup the required
> >> packages in my script<
> >>
> https://github.com/JonZeolla/Development/blob/master/Bash/setupMetron.sh>
> >> which could probably be cleaned up and incorporated.
> >>
> >> Jon
> >>
> >> On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <nick@nickallen.org<mailto:
> >> nick@nickallen.org>> wrote:
> >> In regards to your comment about a "full package version dump" there is
> at
> >> least a start at `metron-deployment/scripts/platform-info.sh`. If we
> can
> >> enhance that, let's do it.
> >>
> >> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <shoggi@gmail.com<mailto:
> >> shoggi@gmail.com>> wrote:
> >> Hi
> >>
> >> Just to wrap this up. I've tried to deploy it on HDP 2.5 but I got the
> >> exact same problem. What I did was removing all components for 2.4.3.0
> and
> >> then created a new cluster based on 2.5. I left it at that as the most
> >> likely issue could be with my base install. May that be wrong version of
> >> something or even a missing dev package.
> >>
> >> As my work also coincided with the announcement by Casey I went ahead
> and
> >> installed 0.3.0 as a dev installation instead. That worked great as the
> >> dev-full-platform guide is very complete.
> >>
> >> As a general feedback, in order to get more people interested and not
> >> having noobs like me come up with redundant questions - At this stage
> also
> >> sorry for not providing a meaningful subject, I simply was too excited
> :)
> >>
> >> There is much to win with a small extension of the installation guide. I
> >> don't think it needs much more prose around but a full package version
> dump
> >> with every new release would help greatly to get things going. For the
> >> things which need to be compiled in, the versioning information should
> be
> >> enough. I am saying this, as everyone starts with a different base
> system.
> >> Some have the chance to start from scratch, some have a system running
> and
> >> build something on top and then there are the experienced Metron users,
> who
> >> have many base requirements.. they tend to forget. And I believe at the
> end
> >> it always fails just because of some very tiny small difference.
> >>
> >> If there is a space somewhere available, this could even be a spot,
> where
> >> people can say, "hey, I got this running on XYZ and here is the package
> >> version dump". A simple table with 'Metron version', 'OS release',
> 'Package
> >> list file', 'Version Details of every other needed to-be-compiled
> package',
> >> is already a big help. This really could supplement the existing guide
> >> found here:
> >>
> https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+an+Ambari-Managed+Cluster
> >> .
> >>
> >> But then, I could be alone and others did not experience the same and
> >> simply got it to run, I don't know. Anyway, I would be happy to help if
> >> help is needed.
> >>
> >> Cheers
> >> Shoggi
> >>
> >> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <shoggi@gmail.com<mailto:
> >> shoggi@gmail.com>> wrote:
> >> Hi Simon
> >>
> >> Thank you for your very quick response. I installed it on HDP 2.4.3.0.
> >>
> >> I had a 0.2.0 instance running and was very conscious about versioning
> the
> >> various components. Somehow I had the understanding of not touching 2.5
> as
> >> of yet. I give that a try.
> >>
> >> Thanks heaps !
> >>
> >> Shoggi
> >>
> >> On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <
> >> simon@simonellistonball.com<ma...@simonellistonball.com>> wrote:
> >> Hi Shoggi,
> >>
> >> Can I ask which version of HDP you used as a base. The article is a
> little
> >> out of date in that is calls for 2.4, which the 0.3.0 release is based
> on
> >> 2.5. There was a major version difference in storm between the two,
> which
> >> seems likely to have caused the problem you're seeing.
> >>
> >> Simon
> >>
> >> Sent from my iPad
> >>
> >> On 27 Nov 2016, at 13:48, shoggi <shoggi@gmail.com<mailto:
> shoggi@gmail.com>>
> >> wrote:
> >>
> >> Hi all
> >>
> >> I followed the installation guide to deploy Metron via Ambari
> >>
> https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html
> >> .
> >>
> >> I ended up with all but the Metron Components (Enrichment, Parsing,
> >> Parsers) to start up. The closest to pinpoint the issue is, when I try
> to
> >> start any new parser topology such as with bro:
> >>
> >> /usr/metron/0.3.0/bin/start_parser_topology.sh -k x.localdomain:6667 -z
> >> x.localdomain:2181 -s bro
> >>
> >> The error I get is:
> >>
> >> Exception in thread "main" java.lang.BootstrapMethodError:
> >> java.lang.NoClassDefFoundError: org/apache/storm/Config
> >>
> >> Or in more details:
> >>
> >> Running: /usr/jdk64/jdk1.8.0_77/bin/java -client -Ddaemon.name=
> >> -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-227/storm
> >> -Dstorm.log.dir=/var/log/storm
> >>
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> >> -Dstorm.conf.file= -cp
> >>
> /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/clojure-1.6.0.jar:/usr/hdp/
> >> 2.4.3.<http://2.4.3
> >>
> .>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/
> >> 2.4.3.<http://2.4.3
> >>
> .>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/
> >> 2.4.3.<http://2.4.3
> >>
> .>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/
> >> 2.4.3.<http://2.4.3
> >>
> .>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/minlog-1.2.jar:/usr/hdp/
> >> 2.4.3.<http://2.4.3
> .>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.4.3.0-227/storm/bin
> >> -Dstorm.jar=/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar
> >> org.apache.metron.parsers.topology.ParserTopologyCLI -k
> x.localdomain:6667
> >> -z x.localdomain:2181 -s bro
> >> Exception in thread "main" java.lang.BootstrapMethodError:
> >> java.lang.NoClassDefFoundError: org/apache/storm/Config
> >> at
> >>
> org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<init>(ParserTopologyCLI.java:199)
> >> at
> >>
> org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<clinit>(ParserTopologyCLI.java:47)
> >> at
> >>
> org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:266)
> >> Caused by: java.lang.NoClassDefFoundError: org/apache/storm/Config
> >> ... 3 more
> >> Caused by: java.lang.ClassNotFoundException: org.apache.storm.Config
> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> >> ... 3 more
> >>
> >> Must be something wrong with my config, possibly I have overlooked
> >> something but can't put my finger on it. Any help is appreciated.
> >>
> >> Many thanks
> >> Shoggi
> >>
> >>
> >>
> >>
> >>
> >> --
> >> Nick Allen <ni...@nickallen.org>>
> >> --
> >>
> >> Jon
> >>
> >> Sent from my mobile device
> >>
> >> --
> >>
> >> Jon
> >>
> >> Sent from my mobile device
> >>
> >> --
> > Jon
> >
> > Sent from my mobile device
> >
>
> --
Jon
Sent from my mobile device
Re: Confluence write access to a space
Posted by Dima Kovalyov <Di...@sstech.us>.
Thank you Jon,
Just wondering if you have finished polishing the document? I will have
some time this week to polish and publish it if there is anything left
to format.
- Dima
On 12/01/2016 04:41 PM, Zeolla@GMail.com wrote:
> While I can't delegate access to the wiki, I am working on migrating the
> documentation over. I have been modifying the document for clarity and
> updating up some of the formatting, so please give me some time to do so.
>
> Jon
>
> On Thu, Dec 1, 2016 at 9:33 AM Dima Kovalyov <Di...@sstech.us>
> wrote:
>
>> Hello,
>>
>> Who can provide me with an access to add new page in this Installation
>> space below?
>> https://cwiki.apache.org/confluence/display/METRON/Installation
>> I want to add up-to-date article on how-to install Metron using Ambari
>> mpack.
>> Please advise.
>>
>> My username is "dima" registered with electronic mail dimdroll hosted by
>> gmail.
>>
>> Right now I have it posted here:
>> https://goo.gl/HWGwpj
>> But instead of maintaining it there it will be convenient for me to have
>> it in confluence instead of gdocs as I am using confluence in our internal
>> project as well, so I can basically copy formatted doc between each other.
>>
>> Thank you.
>>
>> - Dima
>>
>> On 11/30/2016 04:57 PM, Zeolla@GMail.com<ma...@GMail.com> wrote:
>> Probably the best long term place for this would be here<
>> https://cwiki.apache.org/confluence/display/METRON/Installation>,
>> potentially with a new mpack article (to differentiate from the ambari
>> install which is already there, but dated). I can help with migrating it
>> to the wiki - I will run through your documentation later today and request
>> edit permissions if I find anything worth updating.
>>
>> Would you be OK if I put your doc in the wiki at some point?
>>
>> Jon
>>
>> On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <Dima.Kovalyov@sstech.us
>> <ma...@sstech.us>> wrote:
>> As multiple users asked I have published document for installing Metron on
>> Centos 6 using Ambari 2.4 with HDP 2.5:
>> https://goo.gl/HWGwpj
>> It may look ugly as this is direct export from internal confluence with
>> cosmetic changes and our internal details removed. Let me know if there is
>> a better place to publish it.
>>
>> Feel free to request edit permissions, I would like the document to be
>> up-to-date, simplified and corrected if there are any errors. The goal for
>> this document is to provide installation guidance in a manner that Linux
>> administrator with basic knowledge could perform installation without
>> opening any other links or searching for commands, step by step.
>> Thank you.
>>
>> p.s. feel free to let me know if you will encounter any errors as I've
>> seen them bunch.
>>
>> - Dima
>>
>>
>>
>> On 11/30/2016 04:45 PM, Zeolla@GMail.com<ma...@GMail.com> wrote:
>> I also have written up a method to retrieve and setup the required
>> packages in my script<
>> https://github.com/JonZeolla/Development/blob/master/Bash/setupMetron.sh>
>> which could probably be cleaned up and incorporated.
>>
>> Jon
>>
>> On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <nick@nickallen.org<mailto:
>> nick@nickallen.org>> wrote:
>> In regards to your comment about a "full package version dump" there is at
>> least a start at `metron-deployment/scripts/platform-info.sh`. If we can
>> enhance that, let's do it.
>>
>> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <shoggi@gmail.com<mailto:
>> shoggi@gmail.com>> wrote:
>> Hi
>>
>> Just to wrap this up. I've tried to deploy it on HDP 2.5 but I got the
>> exact same problem. What I did was removing all components for 2.4.3.0 and
>> then created a new cluster based on 2.5. I left it at that as the most
>> likely issue could be with my base install. May that be wrong version of
>> something or even a missing dev package.
>>
>> As my work also coincided with the announcement by Casey I went ahead and
>> installed 0.3.0 as a dev installation instead. That worked great as the
>> dev-full-platform guide is very complete.
>>
>> As a general feedback, in order to get more people interested and not
>> having noobs like me come up with redundant questions - At this stage also
>> sorry for not providing a meaningful subject, I simply was too excited :)
>>
>> There is much to win with a small extension of the installation guide. I
>> don't think it needs much more prose around but a full package version dump
>> with every new release would help greatly to get things going. For the
>> things which need to be compiled in, the versioning information should be
>> enough. I am saying this, as everyone starts with a different base system.
>> Some have the chance to start from scratch, some have a system running and
>> build something on top and then there are the experienced Metron users, who
>> have many base requirements.. they tend to forget. And I believe at the end
>> it always fails just because of some very tiny small difference.
>>
>> If there is a space somewhere available, this could even be a spot, where
>> people can say, "hey, I got this running on XYZ and here is the package
>> version dump". A simple table with 'Metron version', 'OS release', 'Package
>> list file', 'Version Details of every other needed to-be-compiled package',
>> is already a big help. This really could supplement the existing guide
>> found here:
>> https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+an+Ambari-Managed+Cluster
>> .
>>
>> But then, I could be alone and others did not experience the same and
>> simply got it to run, I don't know. Anyway, I would be happy to help if
>> help is needed.
>>
>> Cheers
>> Shoggi
>>
>> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <shoggi@gmail.com<mailto:
>> shoggi@gmail.com>> wrote:
>> Hi Simon
>>
>> Thank you for your very quick response. I installed it on HDP 2.4.3.0.
>>
>> I had a 0.2.0 instance running and was very conscious about versioning the
>> various components. Somehow I had the understanding of not touching 2.5 as
>> of yet. I give that a try.
>>
>> Thanks heaps !
>>
>> Shoggi
>>
>> On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <
>> simon@simonellistonball.com<ma...@simonellistonball.com>> wrote:
>> Hi Shoggi,
>>
>> Can I ask which version of HDP you used as a base. The article is a little
>> out of date in that is calls for 2.4, which the 0.3.0 release is based on
>> 2.5. There was a major version difference in storm between the two, which
>> seems likely to have caused the problem you're seeing.
>>
>> Simon
>>
>> Sent from my iPad
>>
>> On 27 Nov 2016, at 13:48, shoggi <sh...@gmail.com>>
>> wrote:
>>
>> Hi all
>>
>> I followed the installation guide to deploy Metron via Ambari
>> https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html
>> .
>>
>> I ended up with all but the Metron Components (Enrichment, Parsing,
>> Parsers) to start up. The closest to pinpoint the issue is, when I try to
>> start any new parser topology such as with bro:
>>
>> /usr/metron/0.3.0/bin/start_parser_topology.sh -k x.localdomain:6667 -z
>> x.localdomain:2181 -s bro
>>
>> The error I get is:
>>
>> Exception in thread "main" java.lang.BootstrapMethodError:
>> java.lang.NoClassDefFoundError: org/apache/storm/Config
>>
>> Or in more details:
>>
>> Running: /usr/jdk64/jdk1.8.0_77/bin/java -client -Ddaemon.name=
>> -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-227/storm
>> -Dstorm.log.dir=/var/log/storm
>> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
>> -Dstorm.conf.file= -cp
>> /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/clojure-1.6.0.jar:/usr/hdp/
>> 2.4.3.<http://2.4.3
>> .>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/
>> 2.4.3.<http://2.4.3
>> .>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/
>> 2.4.3.<http://2.4.3
>> .>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/
>> 2.4.3.<http://2.4.3
>> .>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/minlog-1.2.jar:/usr/hdp/
>> 2.4.3.<http://2.4.3.>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.4.3.0-227/storm/bin
>> -Dstorm.jar=/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar
>> org.apache.metron.parsers.topology.ParserTopologyCLI -k x.localdomain:6667
>> -z x.localdomain:2181 -s bro
>> Exception in thread "main" java.lang.BootstrapMethodError:
>> java.lang.NoClassDefFoundError: org/apache/storm/Config
>> at
>> org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<init>(ParserTopologyCLI.java:199)
>> at
>> org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<clinit>(ParserTopologyCLI.java:47)
>> at
>> org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:266)
>> Caused by: java.lang.NoClassDefFoundError: org/apache/storm/Config
>> ... 3 more
>> Caused by: java.lang.ClassNotFoundException: org.apache.storm.Config
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>> ... 3 more
>>
>> Must be something wrong with my config, possibly I have overlooked
>> something but can't put my finger on it. Any help is appreciated.
>>
>> Many thanks
>> Shoggi
>>
>>
>>
>>
>>
>> --
>> Nick Allen <ni...@nickallen.org>>
>> --
>>
>> Jon
>>
>> Sent from my mobile device
>>
>> --
>>
>> Jon
>>
>> Sent from my mobile device
>>
>> --
> Jon
>
> Sent from my mobile device
>
Re: Confluence write access to a space
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
While I can't delegate access to the wiki, I am working on migrating the
documentation over. I have been modifying the document for clarity and
updating up some of the formatting, so please give me some time to do so.
Jon
On Thu, Dec 1, 2016 at 9:33 AM Dima Kovalyov <Di...@sstech.us>
wrote:
> Hello,
>
> Who can provide me with an access to add new page in this Installation
> space below?
> https://cwiki.apache.org/confluence/display/METRON/Installation
> I want to add up-to-date article on how-to install Metron using Ambari
> mpack.
> Please advise.
>
> My username is "dima" registered with electronic mail dimdroll hosted by
> gmail.
>
> Right now I have it posted here:
> https://goo.gl/HWGwpj
> But instead of maintaining it there it will be convenient for me to have
> it in confluence instead of gdocs as I am using confluence in our internal
> project as well, so I can basically copy formatted doc between each other.
>
> Thank you.
>
> - Dima
>
> On 11/30/2016 04:57 PM, Zeolla@GMail.com<ma...@GMail.com> wrote:
> Probably the best long term place for this would be here<
> https://cwiki.apache.org/confluence/display/METRON/Installation>,
> potentially with a new mpack article (to differentiate from the ambari
> install which is already there, but dated). I can help with migrating it
> to the wiki - I will run through your documentation later today and request
> edit permissions if I find anything worth updating.
>
> Would you be OK if I put your doc in the wiki at some point?
>
> Jon
>
> On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <Dima.Kovalyov@sstech.us
> <ma...@sstech.us>> wrote:
> As multiple users asked I have published document for installing Metron on
> Centos 6 using Ambari 2.4 with HDP 2.5:
> https://goo.gl/HWGwpj
> It may look ugly as this is direct export from internal confluence with
> cosmetic changes and our internal details removed. Let me know if there is
> a better place to publish it.
>
> Feel free to request edit permissions, I would like the document to be
> up-to-date, simplified and corrected if there are any errors. The goal for
> this document is to provide installation guidance in a manner that Linux
> administrator with basic knowledge could perform installation without
> opening any other links or searching for commands, step by step.
> Thank you.
>
> p.s. feel free to let me know if you will encounter any errors as I've
> seen them bunch.
>
> - Dima
>
>
>
> On 11/30/2016 04:45 PM, Zeolla@GMail.com<ma...@GMail.com> wrote:
> I also have written up a method to retrieve and setup the required
> packages in my script<
> https://github.com/JonZeolla/Development/blob/master/Bash/setupMetron.sh>
> which could probably be cleaned up and incorporated.
>
> Jon
>
> On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <nick@nickallen.org<mailto:
> nick@nickallen.org>> wrote:
> In regards to your comment about a "full package version dump" there is at
> least a start at `metron-deployment/scripts/platform-info.sh`. If we can
> enhance that, let's do it.
>
> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <shoggi@gmail.com<mailto:
> shoggi@gmail.com>> wrote:
> Hi
>
> Just to wrap this up. I've tried to deploy it on HDP 2.5 but I got the
> exact same problem. What I did was removing all components for 2.4.3.0 and
> then created a new cluster based on 2.5. I left it at that as the most
> likely issue could be with my base install. May that be wrong version of
> something or even a missing dev package.
>
> As my work also coincided with the announcement by Casey I went ahead and
> installed 0.3.0 as a dev installation instead. That worked great as the
> dev-full-platform guide is very complete.
>
> As a general feedback, in order to get more people interested and not
> having noobs like me come up with redundant questions - At this stage also
> sorry for not providing a meaningful subject, I simply was too excited :)
>
> There is much to win with a small extension of the installation guide. I
> don't think it needs much more prose around but a full package version dump
> with every new release would help greatly to get things going. For the
> things which need to be compiled in, the versioning information should be
> enough. I am saying this, as everyone starts with a different base system.
> Some have the chance to start from scratch, some have a system running and
> build something on top and then there are the experienced Metron users, who
> have many base requirements.. they tend to forget. And I believe at the end
> it always fails just because of some very tiny small difference.
>
> If there is a space somewhere available, this could even be a spot, where
> people can say, "hey, I got this running on XYZ and here is the package
> version dump". A simple table with 'Metron version', 'OS release', 'Package
> list file', 'Version Details of every other needed to-be-compiled package',
> is already a big help. This really could supplement the existing guide
> found here:
> https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+an+Ambari-Managed+Cluster
> .
>
> But then, I could be alone and others did not experience the same and
> simply got it to run, I don't know. Anyway, I would be happy to help if
> help is needed.
>
> Cheers
> Shoggi
>
> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <shoggi@gmail.com<mailto:
> shoggi@gmail.com>> wrote:
> Hi Simon
>
> Thank you for your very quick response. I installed it on HDP 2.4.3.0.
>
> I had a 0.2.0 instance running and was very conscious about versioning the
> various components. Somehow I had the understanding of not touching 2.5 as
> of yet. I give that a try.
>
> Thanks heaps !
>
> Shoggi
>
> On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <
> simon@simonellistonball.com<ma...@simonellistonball.com>> wrote:
> Hi Shoggi,
>
> Can I ask which version of HDP you used as a base. The article is a little
> out of date in that is calls for 2.4, which the 0.3.0 release is based on
> 2.5. There was a major version difference in storm between the two, which
> seems likely to have caused the problem you're seeing.
>
> Simon
>
> Sent from my iPad
>
> On 27 Nov 2016, at 13:48, shoggi <sh...@gmail.com>>
> wrote:
>
> Hi all
>
> I followed the installation guide to deploy Metron via Ambari
> https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html
> .
>
> I ended up with all but the Metron Components (Enrichment, Parsing,
> Parsers) to start up. The closest to pinpoint the issue is, when I try to
> start any new parser topology such as with bro:
>
> /usr/metron/0.3.0/bin/start_parser_topology.sh -k x.localdomain:6667 -z
> x.localdomain:2181 -s bro
>
> The error I get is:
>
> Exception in thread "main" java.lang.BootstrapMethodError:
> java.lang.NoClassDefFoundError: org/apache/storm/Config
>
> Or in more details:
>
> Running: /usr/jdk64/jdk1.8.0_77/bin/java -client -Ddaemon.name=
> -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-227/storm
> -Dstorm.log.dir=/var/log/storm
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> -Dstorm.conf.file= -cp
> /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/clojure-1.6.0.jar:/usr/hdp/
> 2.4.3.<http://2.4.3
> .>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/
> 2.4.3.<http://2.4.3
> .>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/
> 2.4.3.<http://2.4.3
> .>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/
> 2.4.3.<http://2.4.3
> .>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/minlog-1.2.jar:/usr/hdp/
> 2.4.3.<http://2.4.3.>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.4.3.0-227/storm/bin
> -Dstorm.jar=/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar
> org.apache.metron.parsers.topology.ParserTopologyCLI -k x.localdomain:6667
> -z x.localdomain:2181 -s bro
> Exception in thread "main" java.lang.BootstrapMethodError:
> java.lang.NoClassDefFoundError: org/apache/storm/Config
> at
> org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<init>(ParserTopologyCLI.java:199)
> at
> org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<clinit>(ParserTopologyCLI.java:47)
> at
> org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:266)
> Caused by: java.lang.NoClassDefFoundError: org/apache/storm/Config
> ... 3 more
> Caused by: java.lang.ClassNotFoundException: org.apache.storm.Config
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 3 more
>
> Must be something wrong with my config, possibly I have overlooked
> something but can't put my finger on it. Any help is appreciated.
>
> Many thanks
> Shoggi
>
>
>
>
>
> --
> Nick Allen <ni...@nickallen.org>>
> --
>
> Jon
>
> Sent from my mobile device
>
> --
>
> Jon
>
> Sent from my mobile device
>
> --
Jon
Sent from my mobile device
Re: Confluence write access to a space
Posted by James Sirota <js...@apache.org>.
Thanks for the document. Please sign up for the wiki and I will give you access
01.12.2016, 07:33, "Dima Kovalyov" <di...@sstech.us>:
> Hello,
>
> Who can provide me with an access to add new page in this Installation space below?
> https://cwiki.apache.org/confluence/display/METRON/Installation
> I want to add up-to-date article on how-to install Metron using Ambari mpack.
> Please advise.
>
> My username is "dima" registered with electronic mail dimdroll hosted by gmail.
>
> Right now I have it posted here:
> https://goo.gl/HWGwpj
> But instead of maintaining it there it will be convenient for me to have it in confluence instead of gdocs as I am using confluence in our internal project as well, so I can basically copy formatted doc between each other.
>
> Thank you.
>
> - Dima
>
> On 11/30/2016 04:57 PM, Zeolla@GMail.com<ma...@GMail.com> wrote:
> Probably the best long term place for this would be here<https://cwiki.apache.org/confluence/display/METRON/Installation>, potentially with a new mpack article (to differentiate from the ambari install which is already there, but dated). I can help with migrating it to the wiki - I will run through your documentation later today and request edit permissions if I find anything worth updating.
>
> Would you be OK if I put your doc in the wiki at some point?
>
> Jon
>
> On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <Di...@sstech.us>> wrote:
> As multiple users asked I have published document for installing Metron on Centos 6 using Ambari 2.4 with HDP 2.5:
> https://goo.gl/HWGwpj
> It may look ugly as this is direct export from internal confluence with cosmetic changes and our internal details removed. Let me know if there is a better place to publish it.
>
> Feel free to request edit permissions, I would like the document to be up-to-date, simplified and corrected if there are any errors. The goal for this document is to provide installation guidance in a manner that Linux administrator with basic knowledge could perform installation without opening any other links or searching for commands, step by step.
> Thank you.
>
> p.s. feel free to let me know if you will encounter any errors as I've seen them bunch.
>
> - Dima
>
> On 11/30/2016 04:45 PM, Zeolla@GMail.com<ma...@GMail.com> wrote:
> I also have written up a method to retrieve and setup the required packages in my script<https://github.com/JonZeolla/Development/blob/master/Bash/setupMetron.sh> which could probably be cleaned up and incorporated.
>
> Jon
>
> On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <ni...@nickallen.org>> wrote:
> In regards to your comment about a "full package version dump" there is at least a start at `metron-deployment/scripts/platform-info.sh`. If we can enhance that, let's do it.
>
> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <sh...@gmail.com>> wrote:
> Hi
>
> Just to wrap this up. I've tried to deploy it on HDP 2.5 but I got the exact same problem. What I did was removing all components for 2.4.3.0 and then created a new cluster based on 2.5. I left it at that as the most likely issue could be with my base install. May that be wrong version of something or even a missing dev package.
>
> As my work also coincided with the announcement by Casey I went ahead and installed 0.3.0 as a dev installation instead. That worked great as the dev-full-platform guide is very complete.
>
> As a general feedback, in order to get more people interested and not having noobs like me come up with redundant questions - At this stage also sorry for not providing a meaningful subject, I simply was too excited :)
>
> There is much to win with a small extension of the installation guide. I don't think it needs much more prose around but a full package version dump with every new release would help greatly to get things going. For the things which need to be compiled in, the versioning information should be enough. I am saying this, as everyone starts with a different base system. Some have the chance to start from scratch, some have a system running and build something on top and then there are the experienced Metron users, who have many base requirements.. they tend to forget. And I believe at the end
> it always fails just because of some very tiny small difference.
>
> If there is a space somewhere available, this could even be a spot, where people can say, "hey, I got this running on XYZ and here is the package version dump". A simple table with 'Metron version', 'OS release', 'Package list file', 'Version Details of every other needed to-be-compiled package', is already a big help. This really could supplement the existing guide found here: https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+an+Ambari-Managed+Cluster.
>
> But then, I could be alone and others did not experience the same and simply got it to run, I don't know. Anyway, I would be happy to help if help is needed.
>
> Cheers
> Shoggi
>
> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <sh...@gmail.com>> wrote:
> Hi Simon
>
> Thank you for your very quick response. I installed it on HDP 2.4.3.0.
>
> I had a 0.2.0 instance running and was very conscious about versioning the various components. Somehow I had the understanding of not touching 2.5 as of yet. I give that a try.
>
> Thanks heaps !
>
> Shoggi
>
> On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <si...@simonellistonball.com>> wrote:
> Hi Shoggi,
>
> Can I ask which version of HDP you used as a base. The article is a little out of date in that is calls for 2.4, which the 0.3.0 release is based on 2.5. There was a major version difference in storm between the two, which seems likely to have caused the problem you're seeing.
>
> Simon
>
> Sent from my iPad
>
> On 27 Nov 2016, at 13:48, shoggi <sh...@gmail.com>> wrote:
>
> Hi all
>
> I followed the installation guide to deploy Metron via Ambari https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html.
>
> I ended up with all but the Metron Components (Enrichment, Parsing, Parsers) to start up. The closest to pinpoint the issue is, when I try to start any new parser topology such as with bro:
>
> /usr/metron/0.3.0/bin/start_parser_topology.sh -k x.localdomain:6667 -z x.localdomain:2181 -s bro
>
> The error I get is:
>
> Exception in thread "main" java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: org/apache/storm/Config
>
> Or in more details:
>
> Running: /usr/jdk64/jdk1.8.0_77/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-227/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.4.3.0-227/storm/bin -Dstorm.jar=/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar org.apache.metron.parsers.topology.ParserTopologyCLI -k x.localdomain:6667 -z x.localdomain:2181 -s bro
> Exception in thread "main" java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: org/apache/storm/Config
> at org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<init>(ParserTopologyCLI.java:199)
> at org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<clinit>(ParserTopologyCLI.java:47)
> at org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:266)
> Caused by: java.lang.NoClassDefFoundError: org/apache/storm/Config
> ... 3 more
> Caused by: java.lang.ClassNotFoundException: org.apache.storm.Config
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 3 more
>
> Must be something wrong with my config, possibly I have overlooked something but can't put my finger on it. Any help is appreciated.
>
> Many thanks
> Shoggi
>
> --
> Nick Allen <ni...@nickallen.org>>
> --
>
> Jon
>
> Sent from my mobile device
>
> --
>
> Jon
>
> Sent from my mobile device
-------------------
Thank you,
James Sirota
PPMC- Apache Metron (Incubating)
jsirota AT apache DOT org