You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@htrace.apache.org by Michael Wall <mj...@gmail.com> on 2015/02/19 21:26:34 UTC
HTrace and Go
Hi,
I am interested in learning more about HTrace and helping. Hopefully this
is the correct list, I didn't see a users alias.
Specifically, I use Accumulo everyday and am investigating how HTrace can
supplement or replace Accumulo's built-in tracing.
I just checked out the code and tried to build it. To my surprise, I need
to install Golang. What version should I be using? Is there somewhere I
can read about why this was chosen and which modules/functionality Go will
be used for?
Thanks
Mike Wall
Re: HTrace and Go
Posted by "Colin P. McCabe" <cm...@apache.org>.
On Sun, Mar 1, 2015 at 8:58 AM, Michael Wall <mj...@gmail.com> wrote:
> Thanks Colin, that helps a lot.
>
> Couple more.
>
> 1) In htrace-flume, htrace-hbase, and htrace-zipkin, I don't see any
> mention of starting up the htraced daemon. Do these collectors use htraced
> currently? Trying to wrap my head around how Go will do that.
No, htraced is only used by htrace-htraced. htrace-flume just sends
spans to apache flume, htrace-zipkin just sends spans to zipkin, and
the local file span receive just writes them to a local file.
htrace-hbase sends spans to hbase, etc.
The flume and local file receivers are really just ways to dump spans
to a disk. They don't support any querying functionality.
zipkin is a full system of its own, which has a lot of similarities to
htrace. The data model is a little bit different than ours, and it is
non-apache, but it was one of the earliest ways for people to
visualize htrace spans.
> 2) Are the REST endpoints documented anywhere? I found
> https://issues.apache.org/jira/browse/HTRACE-105. From htraced/rest.go, I
> found the following
> 278: r.Handle("/server/info", &serverInfoHandler{lg:
> rsv.lg}).Methods("GET")
> 282: r.Handle("/writeSpans", writeSpansH).Methods("POST")
> 285: r.Handle("/query", queryH).Methods("GET")
> Anything else currently planned? Will
> https://issues.apache.org/jira/browse/HTRACE-118 need one?
Eventually we should have a document describing the REST endpoints.
The API hasn't really stabilized yet, and the most important consumers
are the htrace tool and the web UI. Eventually there will be external
users, but we shouldn't mark the API as stable until the internal
users are working well.
HTRACE-118 is a change to the java client, not to the REST endpoints.
> 3) There is a web UI discussed in htrace-hbase. Will that be replaced by
> the htraced UI? The spans.png and timelines.png in that directory are
> really what I am looking for.
I am really focusing on the htrace-htraced receiver right now, to get
that stable and working. I think eventually we might think about how
to use the web UI on other receivers, but we should get it working
well on one first.
> 4) Is there a way to correlate traces after the fact? Say I have an
> ingest pipeline that processes data then writes it to HDFS. Then another
> process picks it up from HDFS and makes it available for query. I'd like
> to take spans from both parts and tie them together so I can trace
> something all the way through. Any thoughts on that?
Sure. Correlating traces is the whole goal of htraced! That's why it
brings the traces from across the cluster together. There will be a
bunch of ways to load traces into htraced... the simplest one is just
to use htrace-htraced.
cheers,
Colin
>
> Thanks again
>
> Mike
>
> On Sat, Feb 28, 2015 at 11:45 PM, Colin P. McCabe <cm...@apache.org>
> wrote:
>
>> Thanks for this Mike! It should be helpful. I've been planning on
>> doing a blog post about getting htraced set up, but there are a few
>> bugs right now that I'm trying to fix first.
>>
>> Good writeup. One note is that I think most Linux distros newer than
>> CentOS 6 will be able to do "yum install leveldb-devel" or similar
>> (maybe even CentOS 7?) Also, brew is a nice option on MacOS if you
>> are already using that.
>>
>> One other note is that we automatically install the "godep" program.
>> So it should not be necessary to install that manually. The
>> BUILDING.txt is wrong here (it says that godep needs to be installed
>> by hand.) I have filed HTRACE-126 to correct this part of
>> BUILDING.txt.
>>
>> > 1) Is REST the only interface for getting span info in and out of
>> htraced?
>>
>> Yes. I wrote some code to do a protobuf interface, but I haven't
>> posted it yet. I think we should stabilize the other stuff before
>> going for the extra performance that a non-REST interface would give.
>>
>> > 2) How is HTracedRESTReciever in htrace-htraced/ used or how will it be
>> used? I didn't see it anywhere in the Java clients.
>>
>> Span Receivers are basically "sinks" where spans can go. Client code,
>> such as Hadoop or HBase, doesn't need to care about which span
>> receiver the spans are going to. But you will need to put the
>> htrace-htraced jar on your CLASSPATH to use it.
>>
>> > 3) Is it expected that there will be one htraced process running. I am
>> thinking of it like the collector in
>> > Zipkin, http://twitter.github.io/zipkin/Architecture.html. Is that
>> correct? Is there a similar architecture
>> > diagram for HTrace to help me see the big picture? Happy to help make
>> one once I figure it out.
>>
>> Yes, that's right. There is currently one htraced per cluster. Maybe
>> later if load becomes a problem we'll shard it. You are correct that
>> it is similar to the collector in zipkin.
>>
>> I am going to make some architecture slides for my talk at ApacheCon
>> US, I will post those as soon as they exist (I'll also do a short
>> writeup on setting up)
>>
>> Thanks for checking out htrace!
>>
>> Colin
>>
>> On Sat, Feb 28, 2015 at 8:04 PM, Michael Wall <mj...@gmail.com> wrote:
>> > Thanks for the replies. Not sure if this is useful or not, but here are
>> > some notes on how I got htraced running. I missed the
>> > htrace-core/src/go/BUILDING.txt file before, which mentioned these items
>> > already.
>> >
>> > - Install Go
>> > Download 1.3.1 from https://golang.org/dl/
>> > Unzip somewhere, like /local/opt/go
>> > export GOROOT=/local/opt/go
>> > export GOPATH="${GOROOT}/packages"
>> > add $GOROOT/bin and $GOPATH/bin to $PATH
>> > Seems like GOROOT may no longer be needed for newer versions of Go
>> >
>> > - Install godep
>> > Run
>> > go get github.com/tools/godep
>> > Puts it in GOPATH
>> >
>> > - Install leveldb
>> > Download 1.13.1 from
>> >
>> https://code.google.com/p/leveldb/downloads/detail?name=leveldb-1.13.0.tar.gz
>> > extract, cd into leveldb directory and run make
>> > as root from leveldb directory setup lib and include for compiler
>> > -- Linux CentOS 6.5
>> > cp --preserve=links libleveldb.* /usr/lib64
>> > cp -r include/leveldb /usr/include/
>> > ldconfig
>> > -- Mac 10.10
>> > xcode-select --install # to make sure /usr/local/include is included
>> > cp -a libleveldb.* /usr/local/lib
>> > cp -r include/leveldb /usr/local/include/
>> >
>> > - Install maven
>> > Already had 3.0.5 installed
>> >
>> > - Get HTrace
>> > git clone http://git-wip-us.apache.org/repos/asf/incubator-htrace.git
>> > mvn clean package
>> >
>> > I looked the code a little and still have some questions.
>> >
>> > 1) Is REST the only interface for getting span info in and out of
>> htraced?
>> > 2) How is HTracedRESTReciever in htrace-htraced/ used or how will it be
>> > used? I didn't see it anywhere in the Java clients.
>> > 3) Is it expected that there will be one htraced process running. I am
>> > thinking of it like the collector in Zipkin,
>> > http://twitter.github.io/zipkin/Architecture.html. Is that correct? Is
>> > there a similar architecture diagram for HTrace to help me see the big
>> > picture? Happy to help make one once I figure it out.
>> >
>> > Thanks in advance for the help. Once I understand more, I'll look at the
>> > open Jira's if that is the best way to get involved.
>> >
>> > Mike
>> >
>> >
>> > On Thu, Feb 19, 2015 at 9:40 PM, Colin McCabe <cm...@alumni.cmu.edu>
>> > wrote:
>> >
>> >> Hi Michael,
>> >>
>> >> It's great that you're looking at the project!
>> >>
>> >> The build requirements are in BUILDING.txt, which says "Requires go
>> >> version 1.3.1 or 1.4." In practice, other versions of go will
>> >> probably work, we just haven't tested them. If you are using any
>> >> modern version of Linux, you should be able to use "apt-get install
>> >> golang" or "zypper install golang", etc. etc. Please do let us know
>> >> if there are any difficulties building, though... we want this to be
>> >> very accessible.
>> >>
>> >> As Nick commented, you don't need golang to build the Java client
>> >> library. You might find it helpful to use the "LocalFileSpanReceiver"
>> >> which just writes trace spans to a local file. The go code is used in
>> >> htraced, which is a daemon which aggregates the spans all in one
>> >> place.
>> >>
>> >> I'm doing some testing now myself with HTrace+Hadoop+HBase and I'll
>> >> try to send out my notes about how I set things up. It will probably
>> >> take a week to do that though since there is other stuff on my plate.
>> >> But stay tuned.
>> >>
>> >> re: compiling without golang. We are planning on moving the htraced
>> >> stuff into the htace-htraced subproject rather than htrace-core, where
>> >> it is now. Once that move happens you should be able to just compile
>> >> the maven subproject you want. I'm not sure if we need a maven
>> >> profile or not since basically the profile would be equivalent to just
>> >> not compiling that subproject.
>> >>
>> >> best,
>> >> Colin
>> >>
>> >> On Thu, Feb 19, 2015 at 12:26 PM, Michael Wall <mj...@gmail.com>
>> wrote:
>> >> > Hi,
>> >> >
>> >> > I am interested in learning more about HTrace and helping. Hopefully
>> >> this
>> >> > is the correct list, I didn't see a users alias.
>> >> >
>> >> > Specifically, I use Accumulo everyday and am investigating how HTrace
>> can
>> >> > supplement or replace Accumulo's built-in tracing.
>> >> >
>> >> > I just checked out the code and tried to build it. To my surprise, I
>> >> need
>> >> > to install Golang. What version should I be using? Is there
>> somewhere I
>> >> > can read about why this was chosen and which modules/functionality Go
>> >> will
>> >> > be used for?
>> >> >
>> >> > Thanks
>> >> >
>> >> > Mike Wall
>> >>
>>
Re: HTrace and Go
Posted by Michael Wall <mj...@gmail.com>.
Thanks Colin, that helps a lot.
Couple more.
1) In htrace-flume, htrace-hbase, and htrace-zipkin, I don't see any
mention of starting up the htraced daemon. Do these collectors use htraced
currently? Trying to wrap my head around how Go will do that.
2) Are the REST endpoints documented anywhere? I found
https://issues.apache.org/jira/browse/HTRACE-105. From htraced/rest.go, I
found the following
278: r.Handle("/server/info", &serverInfoHandler{lg:
rsv.lg}).Methods("GET")
282: r.Handle("/writeSpans", writeSpansH).Methods("POST")
285: r.Handle("/query", queryH).Methods("GET")
Anything else currently planned? Will
https://issues.apache.org/jira/browse/HTRACE-118 need one?
3) There is a web UI discussed in htrace-hbase. Will that be replaced by
the htraced UI? The spans.png and timelines.png in that directory are
really what I am looking for.
4) Is there a way to correlate traces after the fact? Say I have an
ingest pipeline that processes data then writes it to HDFS. Then another
process picks it up from HDFS and makes it available for query. I'd like
to take spans from both parts and tie them together so I can trace
something all the way through. Any thoughts on that?
Thanks again
Mike
On Sat, Feb 28, 2015 at 11:45 PM, Colin P. McCabe <cm...@apache.org>
wrote:
> Thanks for this Mike! It should be helpful. I've been planning on
> doing a blog post about getting htraced set up, but there are a few
> bugs right now that I'm trying to fix first.
>
> Good writeup. One note is that I think most Linux distros newer than
> CentOS 6 will be able to do "yum install leveldb-devel" or similar
> (maybe even CentOS 7?) Also, brew is a nice option on MacOS if you
> are already using that.
>
> One other note is that we automatically install the "godep" program.
> So it should not be necessary to install that manually. The
> BUILDING.txt is wrong here (it says that godep needs to be installed
> by hand.) I have filed HTRACE-126 to correct this part of
> BUILDING.txt.
>
> > 1) Is REST the only interface for getting span info in and out of
> htraced?
>
> Yes. I wrote some code to do a protobuf interface, but I haven't
> posted it yet. I think we should stabilize the other stuff before
> going for the extra performance that a non-REST interface would give.
>
> > 2) How is HTracedRESTReciever in htrace-htraced/ used or how will it be
> used? I didn't see it anywhere in the Java clients.
>
> Span Receivers are basically "sinks" where spans can go. Client code,
> such as Hadoop or HBase, doesn't need to care about which span
> receiver the spans are going to. But you will need to put the
> htrace-htraced jar on your CLASSPATH to use it.
>
> > 3) Is it expected that there will be one htraced process running. I am
> thinking of it like the collector in
> > Zipkin, http://twitter.github.io/zipkin/Architecture.html. Is that
> correct? Is there a similar architecture
> > diagram for HTrace to help me see the big picture? Happy to help make
> one once I figure it out.
>
> Yes, that's right. There is currently one htraced per cluster. Maybe
> later if load becomes a problem we'll shard it. You are correct that
> it is similar to the collector in zipkin.
>
> I am going to make some architecture slides for my talk at ApacheCon
> US, I will post those as soon as they exist (I'll also do a short
> writeup on setting up)
>
> Thanks for checking out htrace!
>
> Colin
>
> On Sat, Feb 28, 2015 at 8:04 PM, Michael Wall <mj...@gmail.com> wrote:
> > Thanks for the replies. Not sure if this is useful or not, but here are
> > some notes on how I got htraced running. I missed the
> > htrace-core/src/go/BUILDING.txt file before, which mentioned these items
> > already.
> >
> > - Install Go
> > Download 1.3.1 from https://golang.org/dl/
> > Unzip somewhere, like /local/opt/go
> > export GOROOT=/local/opt/go
> > export GOPATH="${GOROOT}/packages"
> > add $GOROOT/bin and $GOPATH/bin to $PATH
> > Seems like GOROOT may no longer be needed for newer versions of Go
> >
> > - Install godep
> > Run
> > go get github.com/tools/godep
> > Puts it in GOPATH
> >
> > - Install leveldb
> > Download 1.13.1 from
> >
> https://code.google.com/p/leveldb/downloads/detail?name=leveldb-1.13.0.tar.gz
> > extract, cd into leveldb directory and run make
> > as root from leveldb directory setup lib and include for compiler
> > -- Linux CentOS 6.5
> > cp --preserve=links libleveldb.* /usr/lib64
> > cp -r include/leveldb /usr/include/
> > ldconfig
> > -- Mac 10.10
> > xcode-select --install # to make sure /usr/local/include is included
> > cp -a libleveldb.* /usr/local/lib
> > cp -r include/leveldb /usr/local/include/
> >
> > - Install maven
> > Already had 3.0.5 installed
> >
> > - Get HTrace
> > git clone http://git-wip-us.apache.org/repos/asf/incubator-htrace.git
> > mvn clean package
> >
> > I looked the code a little and still have some questions.
> >
> > 1) Is REST the only interface for getting span info in and out of
> htraced?
> > 2) How is HTracedRESTReciever in htrace-htraced/ used or how will it be
> > used? I didn't see it anywhere in the Java clients.
> > 3) Is it expected that there will be one htraced process running. I am
> > thinking of it like the collector in Zipkin,
> > http://twitter.github.io/zipkin/Architecture.html. Is that correct? Is
> > there a similar architecture diagram for HTrace to help me see the big
> > picture? Happy to help make one once I figure it out.
> >
> > Thanks in advance for the help. Once I understand more, I'll look at the
> > open Jira's if that is the best way to get involved.
> >
> > Mike
> >
> >
> > On Thu, Feb 19, 2015 at 9:40 PM, Colin McCabe <cm...@alumni.cmu.edu>
> > wrote:
> >
> >> Hi Michael,
> >>
> >> It's great that you're looking at the project!
> >>
> >> The build requirements are in BUILDING.txt, which says "Requires go
> >> version 1.3.1 or 1.4." In practice, other versions of go will
> >> probably work, we just haven't tested them. If you are using any
> >> modern version of Linux, you should be able to use "apt-get install
> >> golang" or "zypper install golang", etc. etc. Please do let us know
> >> if there are any difficulties building, though... we want this to be
> >> very accessible.
> >>
> >> As Nick commented, you don't need golang to build the Java client
> >> library. You might find it helpful to use the "LocalFileSpanReceiver"
> >> which just writes trace spans to a local file. The go code is used in
> >> htraced, which is a daemon which aggregates the spans all in one
> >> place.
> >>
> >> I'm doing some testing now myself with HTrace+Hadoop+HBase and I'll
> >> try to send out my notes about how I set things up. It will probably
> >> take a week to do that though since there is other stuff on my plate.
> >> But stay tuned.
> >>
> >> re: compiling without golang. We are planning on moving the htraced
> >> stuff into the htace-htraced subproject rather than htrace-core, where
> >> it is now. Once that move happens you should be able to just compile
> >> the maven subproject you want. I'm not sure if we need a maven
> >> profile or not since basically the profile would be equivalent to just
> >> not compiling that subproject.
> >>
> >> best,
> >> Colin
> >>
> >> On Thu, Feb 19, 2015 at 12:26 PM, Michael Wall <mj...@gmail.com>
> wrote:
> >> > Hi,
> >> >
> >> > I am interested in learning more about HTrace and helping. Hopefully
> >> this
> >> > is the correct list, I didn't see a users alias.
> >> >
> >> > Specifically, I use Accumulo everyday and am investigating how HTrace
> can
> >> > supplement or replace Accumulo's built-in tracing.
> >> >
> >> > I just checked out the code and tried to build it. To my surprise, I
> >> need
> >> > to install Golang. What version should I be using? Is there
> somewhere I
> >> > can read about why this was chosen and which modules/functionality Go
> >> will
> >> > be used for?
> >> >
> >> > Thanks
> >> >
> >> > Mike Wall
> >>
>
Re: HTrace and Go
Posted by "Colin P. McCabe" <cm...@apache.org>.
Thanks for this Mike! It should be helpful. I've been planning on
doing a blog post about getting htraced set up, but there are a few
bugs right now that I'm trying to fix first.
Good writeup. One note is that I think most Linux distros newer than
CentOS 6 will be able to do "yum install leveldb-devel" or similar
(maybe even CentOS 7?) Also, brew is a nice option on MacOS if you
are already using that.
One other note is that we automatically install the "godep" program.
So it should not be necessary to install that manually. The
BUILDING.txt is wrong here (it says that godep needs to be installed
by hand.) I have filed HTRACE-126 to correct this part of
BUILDING.txt.
> 1) Is REST the only interface for getting span info in and out of htraced?
Yes. I wrote some code to do a protobuf interface, but I haven't
posted it yet. I think we should stabilize the other stuff before
going for the extra performance that a non-REST interface would give.
> 2) How is HTracedRESTReciever in htrace-htraced/ used or how will it be used? I didn't see it anywhere in the Java clients.
Span Receivers are basically "sinks" where spans can go. Client code,
such as Hadoop or HBase, doesn't need to care about which span
receiver the spans are going to. But you will need to put the
htrace-htraced jar on your CLASSPATH to use it.
> 3) Is it expected that there will be one htraced process running. I am thinking of it like the collector in
> Zipkin, http://twitter.github.io/zipkin/Architecture.html. Is that correct? Is there a similar architecture
> diagram for HTrace to help me see the big picture? Happy to help make one once I figure it out.
Yes, that's right. There is currently one htraced per cluster. Maybe
later if load becomes a problem we'll shard it. You are correct that
it is similar to the collector in zipkin.
I am going to make some architecture slides for my talk at ApacheCon
US, I will post those as soon as they exist (I'll also do a short
writeup on setting up)
Thanks for checking out htrace!
Colin
On Sat, Feb 28, 2015 at 8:04 PM, Michael Wall <mj...@gmail.com> wrote:
> Thanks for the replies. Not sure if this is useful or not, but here are
> some notes on how I got htraced running. I missed the
> htrace-core/src/go/BUILDING.txt file before, which mentioned these items
> already.
>
> - Install Go
> Download 1.3.1 from https://golang.org/dl/
> Unzip somewhere, like /local/opt/go
> export GOROOT=/local/opt/go
> export GOPATH="${GOROOT}/packages"
> add $GOROOT/bin and $GOPATH/bin to $PATH
> Seems like GOROOT may no longer be needed for newer versions of Go
>
> - Install godep
> Run
> go get github.com/tools/godep
> Puts it in GOPATH
>
> - Install leveldb
> Download 1.13.1 from
> https://code.google.com/p/leveldb/downloads/detail?name=leveldb-1.13.0.tar.gz
> extract, cd into leveldb directory and run make
> as root from leveldb directory setup lib and include for compiler
> -- Linux CentOS 6.5
> cp --preserve=links libleveldb.* /usr/lib64
> cp -r include/leveldb /usr/include/
> ldconfig
> -- Mac 10.10
> xcode-select --install # to make sure /usr/local/include is included
> cp -a libleveldb.* /usr/local/lib
> cp -r include/leveldb /usr/local/include/
>
> - Install maven
> Already had 3.0.5 installed
>
> - Get HTrace
> git clone http://git-wip-us.apache.org/repos/asf/incubator-htrace.git
> mvn clean package
>
> I looked the code a little and still have some questions.
>
> 1) Is REST the only interface for getting span info in and out of htraced?
> 2) How is HTracedRESTReciever in htrace-htraced/ used or how will it be
> used? I didn't see it anywhere in the Java clients.
> 3) Is it expected that there will be one htraced process running. I am
> thinking of it like the collector in Zipkin,
> http://twitter.github.io/zipkin/Architecture.html. Is that correct? Is
> there a similar architecture diagram for HTrace to help me see the big
> picture? Happy to help make one once I figure it out.
>
> Thanks in advance for the help. Once I understand more, I'll look at the
> open Jira's if that is the best way to get involved.
>
> Mike
>
>
> On Thu, Feb 19, 2015 at 9:40 PM, Colin McCabe <cm...@alumni.cmu.edu>
> wrote:
>
>> Hi Michael,
>>
>> It's great that you're looking at the project!
>>
>> The build requirements are in BUILDING.txt, which says "Requires go
>> version 1.3.1 or 1.4." In practice, other versions of go will
>> probably work, we just haven't tested them. If you are using any
>> modern version of Linux, you should be able to use "apt-get install
>> golang" or "zypper install golang", etc. etc. Please do let us know
>> if there are any difficulties building, though... we want this to be
>> very accessible.
>>
>> As Nick commented, you don't need golang to build the Java client
>> library. You might find it helpful to use the "LocalFileSpanReceiver"
>> which just writes trace spans to a local file. The go code is used in
>> htraced, which is a daemon which aggregates the spans all in one
>> place.
>>
>> I'm doing some testing now myself with HTrace+Hadoop+HBase and I'll
>> try to send out my notes about how I set things up. It will probably
>> take a week to do that though since there is other stuff on my plate.
>> But stay tuned.
>>
>> re: compiling without golang. We are planning on moving the htraced
>> stuff into the htace-htraced subproject rather than htrace-core, where
>> it is now. Once that move happens you should be able to just compile
>> the maven subproject you want. I'm not sure if we need a maven
>> profile or not since basically the profile would be equivalent to just
>> not compiling that subproject.
>>
>> best,
>> Colin
>>
>> On Thu, Feb 19, 2015 at 12:26 PM, Michael Wall <mj...@gmail.com> wrote:
>> > Hi,
>> >
>> > I am interested in learning more about HTrace and helping. Hopefully
>> this
>> > is the correct list, I didn't see a users alias.
>> >
>> > Specifically, I use Accumulo everyday and am investigating how HTrace can
>> > supplement or replace Accumulo's built-in tracing.
>> >
>> > I just checked out the code and tried to build it. To my surprise, I
>> need
>> > to install Golang. What version should I be using? Is there somewhere I
>> > can read about why this was chosen and which modules/functionality Go
>> will
>> > be used for?
>> >
>> > Thanks
>> >
>> > Mike Wall
>>
Re: HTrace and Go
Posted by Michael Wall <mj...@gmail.com>.
Thanks for the replies. Not sure if this is useful or not, but here are
some notes on how I got htraced running. I missed the
htrace-core/src/go/BUILDING.txt file before, which mentioned these items
already.
- Install Go
Download 1.3.1 from https://golang.org/dl/
Unzip somewhere, like /local/opt/go
export GOROOT=/local/opt/go
export GOPATH="${GOROOT}/packages"
add $GOROOT/bin and $GOPATH/bin to $PATH
Seems like GOROOT may no longer be needed for newer versions of Go
- Install godep
Run
go get github.com/tools/godep
Puts it in GOPATH
- Install leveldb
Download 1.13.1 from
https://code.google.com/p/leveldb/downloads/detail?name=leveldb-1.13.0.tar.gz
extract, cd into leveldb directory and run make
as root from leveldb directory setup lib and include for compiler
-- Linux CentOS 6.5
cp --preserve=links libleveldb.* /usr/lib64
cp -r include/leveldb /usr/include/
ldconfig
-- Mac 10.10
xcode-select --install # to make sure /usr/local/include is included
cp -a libleveldb.* /usr/local/lib
cp -r include/leveldb /usr/local/include/
- Install maven
Already had 3.0.5 installed
- Get HTrace
git clone http://git-wip-us.apache.org/repos/asf/incubator-htrace.git
mvn clean package
I looked the code a little and still have some questions.
1) Is REST the only interface for getting span info in and out of htraced?
2) How is HTracedRESTReciever in htrace-htraced/ used or how will it be
used? I didn't see it anywhere in the Java clients.
3) Is it expected that there will be one htraced process running. I am
thinking of it like the collector in Zipkin,
http://twitter.github.io/zipkin/Architecture.html. Is that correct? Is
there a similar architecture diagram for HTrace to help me see the big
picture? Happy to help make one once I figure it out.
Thanks in advance for the help. Once I understand more, I'll look at the
open Jira's if that is the best way to get involved.
Mike
On Thu, Feb 19, 2015 at 9:40 PM, Colin McCabe <cm...@alumni.cmu.edu>
wrote:
> Hi Michael,
>
> It's great that you're looking at the project!
>
> The build requirements are in BUILDING.txt, which says "Requires go
> version 1.3.1 or 1.4." In practice, other versions of go will
> probably work, we just haven't tested them. If you are using any
> modern version of Linux, you should be able to use "apt-get install
> golang" or "zypper install golang", etc. etc. Please do let us know
> if there are any difficulties building, though... we want this to be
> very accessible.
>
> As Nick commented, you don't need golang to build the Java client
> library. You might find it helpful to use the "LocalFileSpanReceiver"
> which just writes trace spans to a local file. The go code is used in
> htraced, which is a daemon which aggregates the spans all in one
> place.
>
> I'm doing some testing now myself with HTrace+Hadoop+HBase and I'll
> try to send out my notes about how I set things up. It will probably
> take a week to do that though since there is other stuff on my plate.
> But stay tuned.
>
> re: compiling without golang. We are planning on moving the htraced
> stuff into the htace-htraced subproject rather than htrace-core, where
> it is now. Once that move happens you should be able to just compile
> the maven subproject you want. I'm not sure if we need a maven
> profile or not since basically the profile would be equivalent to just
> not compiling that subproject.
>
> best,
> Colin
>
> On Thu, Feb 19, 2015 at 12:26 PM, Michael Wall <mj...@gmail.com> wrote:
> > Hi,
> >
> > I am interested in learning more about HTrace and helping. Hopefully
> this
> > is the correct list, I didn't see a users alias.
> >
> > Specifically, I use Accumulo everyday and am investigating how HTrace can
> > supplement or replace Accumulo's built-in tracing.
> >
> > I just checked out the code and tried to build it. To my surprise, I
> need
> > to install Golang. What version should I be using? Is there somewhere I
> > can read about why this was chosen and which modules/functionality Go
> will
> > be used for?
> >
> > Thanks
> >
> > Mike Wall
>
Re: HTrace and Go
Posted by Colin McCabe <cm...@alumni.cmu.edu>.
Hi Michael,
It's great that you're looking at the project!
The build requirements are in BUILDING.txt, which says "Requires go
version 1.3.1 or 1.4." In practice, other versions of go will
probably work, we just haven't tested them. If you are using any
modern version of Linux, you should be able to use "apt-get install
golang" or "zypper install golang", etc. etc. Please do let us know
if there are any difficulties building, though... we want this to be
very accessible.
As Nick commented, you don't need golang to build the Java client
library. You might find it helpful to use the "LocalFileSpanReceiver"
which just writes trace spans to a local file. The go code is used in
htraced, which is a daemon which aggregates the spans all in one
place.
I'm doing some testing now myself with HTrace+Hadoop+HBase and I'll
try to send out my notes about how I set things up. It will probably
take a week to do that though since there is other stuff on my plate.
But stay tuned.
re: compiling without golang. We are planning on moving the htraced
stuff into the htace-htraced subproject rather than htrace-core, where
it is now. Once that move happens you should be able to just compile
the maven subproject you want. I'm not sure if we need a maven
profile or not since basically the profile would be equivalent to just
not compiling that subproject.
best,
Colin
On Thu, Feb 19, 2015 at 12:26 PM, Michael Wall <mj...@gmail.com> wrote:
> Hi,
>
> I am interested in learning more about HTrace and helping. Hopefully this
> is the correct list, I didn't see a users alias.
>
> Specifically, I use Accumulo everyday and am investigating how HTrace can
> supplement or replace Accumulo's built-in tracing.
>
> I just checked out the code and tried to build it. To my surprise, I need
> to install Golang. What version should I be using? Is there somewhere I
> can read about why this was chosen and which modules/functionality Go will
> be used for?
>
> Thanks
>
> Mike Wall
Re: HTrace and Go
Posted by Elliott Clark <ec...@apache.org>.
Yeah we should make working on just the Java code possible. Something like
hbase has with -Pskip<<ModuleName>>
On Thu, Feb 19, 2015 at 2:18 PM, Andrew Purtell <ap...@apache.org> wrote:
> Might be useful followup to document how to build just the Java client? I
> also found it necessary to install golang to build HTrace.
>
> On Thu, Feb 19, 2015 at 1:56 PM, Nick Dimiduk <nd...@gmail.com> wrote:
>
> > Hi Michael,
> >
> > This is indeed the appropriate list. I believe there is some work in
> > progress to see Accumulo using HTrace. I'm sure Billie can comment
> further
> > on this effort. Re: golang, we are using this to implement a
> span-receiver
> > daemon, the htraced module. Go is not required for using our Java client
> > library. I believe we support both go 1.3 and 1.4.
> >
> > Welcome!
> > -n
> >
> > On Thu, Feb 19, 2015 at 12:26 PM, Michael Wall <mj...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I am interested in learning more about HTrace and helping. Hopefully
> > this
> > > is the correct list, I didn't see a users alias.
> > >
> > > Specifically, I use Accumulo everyday and am investigating how HTrace
> can
> > > supplement or replace Accumulo's built-in tracing.
> > >
> > > I just checked out the code and tried to build it. To my surprise, I
> > need
> > > to install Golang. What version should I be using? Is there
> somewhere I
> > > can read about why this was chosen and which modules/functionality Go
> > will
> > > be used for?
> > >
> > > Thanks
> > >
> > > Mike Wall
> > >
> >
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: HTrace and Go
Posted by Andrew Purtell <ap...@apache.org>.
Might be useful followup to document how to build just the Java client? I
also found it necessary to install golang to build HTrace.
On Thu, Feb 19, 2015 at 1:56 PM, Nick Dimiduk <nd...@gmail.com> wrote:
> Hi Michael,
>
> This is indeed the appropriate list. I believe there is some work in
> progress to see Accumulo using HTrace. I'm sure Billie can comment further
> on this effort. Re: golang, we are using this to implement a span-receiver
> daemon, the htraced module. Go is not required for using our Java client
> library. I believe we support both go 1.3 and 1.4.
>
> Welcome!
> -n
>
> On Thu, Feb 19, 2015 at 12:26 PM, Michael Wall <mj...@gmail.com> wrote:
>
> > Hi,
> >
> > I am interested in learning more about HTrace and helping. Hopefully
> this
> > is the correct list, I didn't see a users alias.
> >
> > Specifically, I use Accumulo everyday and am investigating how HTrace can
> > supplement or replace Accumulo's built-in tracing.
> >
> > I just checked out the code and tried to build it. To my surprise, I
> need
> > to install Golang. What version should I be using? Is there somewhere I
> > can read about why this was chosen and which modules/functionality Go
> will
> > be used for?
> >
> > Thanks
> >
> > Mike Wall
> >
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: HTrace and Go
Posted by Nick Dimiduk <nd...@gmail.com>.
Hi Michael,
This is indeed the appropriate list. I believe there is some work in
progress to see Accumulo using HTrace. I'm sure Billie can comment further
on this effort. Re: golang, we are using this to implement a span-receiver
daemon, the htraced module. Go is not required for using our Java client
library. I believe we support both go 1.3 and 1.4.
Welcome!
-n
On Thu, Feb 19, 2015 at 12:26 PM, Michael Wall <mj...@gmail.com> wrote:
> Hi,
>
> I am interested in learning more about HTrace and helping. Hopefully this
> is the correct list, I didn't see a users alias.
>
> Specifically, I use Accumulo everyday and am investigating how HTrace can
> supplement or replace Accumulo's built-in tracing.
>
> I just checked out the code and tried to build it. To my surprise, I need
> to install Golang. What version should I be using? Is there somewhere I
> can read about why this was chosen and which modules/functionality Go will
> be used for?
>
> Thanks
>
> Mike Wall
>