Re: How to deal with htrace conversion of values to base64

[Raam Rosh-Hai <ra...@findhotel.net>]

Seems like HTraceToZipkinConverter.java tags all binary annotations as the
BYTES thrift data type although a string is being sent
https://github.com/apache/incubator-htrace/blob/master/htrace-zipkin/src/main/java/org/apache/htrace/zipkin/HTraceToZipkinConverter.java#L156
.

A string is always sent since this is the signature
of TraceScope.addKVAnnotation: public void addKVAnnotation(String key,
String value)

I opened issue HTRACE-393 and this is the pull request:
https://github.com/apache/incubator-htrace/pull/13

On 31 December 2016 at 13:11, Raam Rosh-Hai <ra...@findhotel.net> wrote:

> Thank you Adrian, I will look into it and update the issue with a specific
> test case.
>
> On Sat, 31 Dec 2016 at 3:02 AM Adrian Cole <ad...@gmail.com>
> wrote:
>
>> ok well update your issue here, then?
>>
>>
>>
>> https://github.com/openzipkin/zipkin/issues/1455
>>
>>
>>
>> There's a test ScribeSpanConsumerTest which covers some base64 related
>>
>> concerns. It is unfortunate that base64 is a requirement as indeed
>>
>> there are edge cases.
>>
>>
>>
>> https://github.com/openzipkin/zipkin/blob/master/zipkin-
>> collector/scribe/src/test/java/zipkin/collector/scribe/
>> ScribeSpanConsumerTest.java#L152
>>
>>
>>
>> -A
>>
>>
>>
>> On Fri, Dec 30, 2016 at 10:37 PM, Raam Rosh-Hai <ra...@findhotel.net>
>> wrote:
>>
>> > Hey Adrian,
>>
>> >
>>
>> > After poking around, and changing org.apache.htrace.impl.
>> ScribeTransport
>>
>> > which didn't seem to solve this issue I am starting to think it's on the
>>
>> > zipkin side seeing that most of the data is displayed correctly other
>> then
>>
>> > the KV annotations. further more the annotations themselves are being
>> sent
>>
>> > to zipkin in a binary format and only the thrift object is base64
>> encoded.
>>
>> > I am trying to find that place that encodes the annotations to base64
>> but
>>
>> > no luck up until now in htrace
>>
>> >
>>
>> > On 23 December 2016 at 10:21, Adrian Cole <ad...@gmail.com>
>> wrote:
>>
>> >
>>
>> >> My guess is that this has to do with url encoding. Can you patch
>>
>> >> org.apache.htrace.impl.ScribeTransport to use
>>
>> >> encodeBase64URLSafeString instead of encodeBase64String?
>>
>> >>
>>
>> >> that might answer it..
>>
>> >>
>>
>> >> On Fri, Dec 23, 2016 at 5:04 PM, Raam Rosh-Hai <ra...@findhotel.net>
>> wrote:
>>
>> >> > Hi St.Ack,
>>
>> >> >
>>
>> >> > Thank you for your reply, I am using "org.apache.htrace" %
>>
>> >> "htrace-core4" %
>>
>> >> > "4.1.0-incubating",
>>
>> >> > "org.apache.htrace" % "htrace-zipkin" % "4.1.0-incubating" in scala.
>>
>> >> > I was looking at an older version of htrace (non incubating master on
>>
>> >> > github) and now I see your are no longer doing that.
>>
>> >> >
>>
>> >> > What I am getting in zipkin UI is a malformed base64 string. where
>> the
>>
>> >> `/`
>>
>> >> > were converted to `_` after debugging the zipkin receiver it seems
>> like
>>
>> >> the
>>
>> >> > spans are sent correctly, maybe you have an idea what can go wrong?
>>
>> >> >
>>
>> >> > On 22 December 2016 at 18:47, Stack <st...@duboce.net> wrote:
>>
>> >> >
>>
>> >> >> Show us where in the code this is happening Raam Rosh-Hai and tell
>> us
>>
>> >> what
>>
>> >> >> version of htrace you are using. Thanks.
>>
>> >> >> St.Ack
>>
>> >> >>
>>
>> >> >> On Thu, Dec 22, 2016 at 3:41 AM, Raam Rosh-Hai <ra...@findhotel.net>
>>
>> >> wrote:
>>
>> >> >>
>>
>> >> >> >         I am saving a simple string value and it seems like the
>> trace
>>
>> >> >> > pipkin connector is converting all values to base64 I then get the
>>
>> >> base64
>>
>> >> >> > values in the zipkin fronted, any suggestions?
>>
>> >> >> >
>>
>> >> >> > Thanks,
>>
>> >> >> > Raam
>>
>> >> >> >
>>
>> >> >>
>>
>> >>
>>
>>