You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by mygityf <gi...@git.apache.org> on 2016/02/14 11:17:53 UTC
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
GitHub user mygityf opened a pull request:
https://github.com/apache/thrift/pull/864
THRIFT-3631 Lua- JSON protocol implement for thrift-lua library.
THRIFT-3631 Lua-This is JSON protocol implement for thrift-lua library.
TJsonProtocol.lua is new source of lua for thrift JSON protocol.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mygityf/thrift json-protocol-lua-library
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/thrift/pull/864.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #864
----
commit 12244b870418fe8feb7051c942d3cc361b0ac028
Author: Wang Yaofu <vo...@sina.cn>
Date: 2016-02-14T10:15:45Z
THRIFT-3631 JSON protocol implement for thrift-lua library.
THRIFT-3631 Lua-This is JSON protocol implement for thrift-lua library.
TJsonProtocol.lua is new source of lua for thrift JSON protocol.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by nsuke <gi...@git.apache.org>.
Github user nsuke commented on a diff in the pull request:
https://github.com/apache/thrift/pull/864#discussion_r53193197
--- Diff: lib/lua/TJsonProtocol.lua ---
@@ -0,0 +1,725 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one
+-- or more contributor license agreements. See the NOTICE file
+-- distributed with this work for additional information
+-- regarding copyright ownership. The ASF licenses this file
+-- to you under the Apache License, Version 2.0 (the
+-- "License"), you may not use this file except in compliance
+-- with the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing,
+-- software distributed under the License is distributed on an
+-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+-- KIND, either express or implied. See the License for the
+-- specific language governing permissions and limitations
+-- under the License.
+--
+
+require 'TProtocol'
+require 'libluabpack'
+require 'libluabitwise'
+
+TJSONProtocol = __TObject.new(TProtocolBase, {
+ __type = 'TJSONProtocol',
+ THRIFT_JSON_PROTOCOL_VERSION = 1,
+ jsonContext = {},
+ jsonContextVal = {first = true, colon = true, ttype = 2, null = true},
+ jsonContextIndex = 1,
+ hasReadByte = ""
+})
+
+TTypeToString = {}
+TTypeToString[TType.BOOL] = "tf"
+TTypeToString[TType.BYTE] = "i8"
+TTypeToString[TType.I16] = "i16"
+TTypeToString[TType.I32] = "i32"
+TTypeToString[TType.I64] = "i64"
+TTypeToString[TType.DOUBLE] = "dbl"
+TTypeToString[TType.STRING] = "str"
+TTypeToString[TType.STRUCT] = "rec"
+TTypeToString[TType.LIST] = "lst"
+TTypeToString[TType.SET] = "set"
+TTypeToString[TType.MAP] = "map"
+
+StringToTType = {
+ tf = TType.BOOL,
+ i8 = TType.BYTE,
+ i16 = TType.I16,
+ i32 = TType.I32,
+ i64 = TType.I64,
+ dbl = TType.DOUBLE,
+ str = TType.STRING,
+ rec = TType.STRUCT,
+ map = TType.MAP,
+ set = TType.SET,
+ lst = TType.LIST
+}
+
+JSONNode = {
+ ObjectBegin = '{',
+ ObjectEnd = '}',
+ ArrayBegin = '[',
+ ArrayEnd = ']',
+ PairSeparator = ':',
+ ElemSeparator = ',',
+ Backslash = '\\',
+ StringDelimiter = '"',
+ ZeroChar = '0',
+ EscapeChar = 'u',
+ Nan = 'NaN',
+ Infinity = 'Infinity',
+ NegativeInfinity = '-Infinity',
+ EscapeChars = "\"\\bfnrt",
+ EscapePrefix = "\\u00"
+}
+
+EscapeCharVals = {
+ '"', '\\', '\b', '\f', '\n', '\r', '\t'
+}
+
+JSONCharTable = {
+ --0 1 2 3 4 5 6 7 8 9 A B C D E F
+ 0, 0, 0, 0, 0, 0, 0, 0, 98,116,110, 0,102,114, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 1, 1,34, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
+}
+
+-- character table string
+local b='ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
+
+-- encoding
+function base64_encode(data)
+ return ((data:gsub('.', function(x)
+ local r,b='',x:byte()
+ for i=8,1,-1 do r=r..(b%2^i-b%2^(i-1)>0 and '1' or '0') end
+ return r;
+ end)..'0000'):gsub('%d%d%d?%d?%d?%d?', function(x)
+ if (#x < 6) then return '' end
+ local c=0
+ for i=1,6 do c=c+(x:sub(i,i)=='1' and 2^(6-i) or 0) end
+ return b:sub(c+1,c+1)
+ end)..({ '', '==', '=' })[#data%3+1])
+end
+
+-- decoding
+function base64_decode(data)
+ data = string.gsub(data, '[^'..b..'=]', '')
+ return (data:gsub('.', function(x)
+ if (x == '=') then return '' end
+ local r,f='',(b:find(x)-1)
+ for i=6,1,-1 do r=r..(f%2^i-f%2^(i-1)>0 and '1' or '0') end
+ return r;
+ end):gsub('%d%d%d?%d?%d?%d?%d?%d?', function(x)
+ if (#x ~= 8) then return '' end
+ local c=0
+ for i=1,8 do c=c+(x:sub(i,i)=='1' and 2^(8-i) or 0) end
+ return string.char(c)
+ end))
+end
+
+function TJSONProtocol:resetContext()
+ self.jsonContext = {}
+ self.jsonContextVal = {first = true, colon = true, ttype = 2, null = true}
+ self.jsonContextIndex = 1
+end
+
+function TJSONProtocol:contextPush(context)
+ self.jsonContextIndex = self.jsonContextIndex + 1
+ self.jsonContext[self.jsonContextIndex] = self.jsonContextVal
+ self.jsonContextVal = context
+end
+
+function TJSONProtocol:contextPop()
+ self.jsonContextVal = self.jsonContext[self.jsonContextIndex]
+ self.jsonContextIndex = self.jsonContextIndex - 1
+end
+
+function TJSONProtocol:escapeNum()
+ if self.jsonContextVal.ttype == 1 then
+ return self.jsonContextVal.colon
+ else
+ return false
+ end
+end
+
+function TJSONProtocol:writeElemSeparator()
+ if self.jsonContextVal.null then
+ return
+ end
+ if self.jsonContextVal.first then
+ self.jsonContextVal.first = false
+ else
+ if self.jsonContextVal.ttype == 1 then
+ if self.jsonContextVal.colon then
+ self.trans:write(JSONNode.PairSeparator)
+ self.jsonContextVal.colon = false
+ else
+ self.trans:write(JSONNode.ElemSeparator)
+ self.jsonContextVal.colon = true
+ end
+ else
+ self.trans:write(JSONNode.ElemSeparator)
+ end
+ end
+end
+
+function TJSONProtocol:hexChar(val)
+ val = libluabitwise.band(val, 0x0f)
+ if val < 10 then
+ return val + 48
+ else
+ return val + 87
+ end
+end
+
+function TJSONProtocol:writeJSONEscapeChar(ch)
+ self.trans:write(JSONNode.EscapePrefix)
+ local outCh = hexChar(libluabitwise.shiftr(ch, 4))
+ local buff = libluabpack.bpack('c', outCh)
+ self.trans:write(buff)
+ outCh = hexChar(ch)
+ buff = libluabpack.bpack('c', outCh)
+ self.trans:write(buff)
+end
+
+function TJSONProtocol:writeJSONChar(byte)
+ ch = string.byte(byte)
+ if ch >= 0x30 then
+ if ch == JSONNode.Backslash then
+ self.trans:write(JSONNode.Backslash)
+ self.trans:write(JSONNode.Backslash)
+ else
+ self.trans:write(byte)
+ end
+ else
+ local outCh = JSONCharTable[ch+1]
+ if outCh == 1 then
+ self.trans:write(byte)
+ elseif outCh > 1 then
+ self.trans:write(JSONNode.Backslash)
+ local buff = libluabpack.bpack('c', outCh)
+ self.trans:write(buff)
+ else
+ self:writeJSONEscapeChar(ch)
+ end
+ end
+end
+
+function TJSONProtocol:writeJSONString(str)
+ self:writeElemSeparator()
+ self.trans:write(JSONNode.StringDelimiter)
+ -- TODO
--- End diff --
Can you elaborate on this ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by mygityf <gi...@git.apache.org>.
Github user mygityf commented on the pull request:
https://github.com/apache/thrift/pull/864#issuecomment-185525091
@nsuke issue of i8(byte) and i32 values are encoded like 1.0 instead of 1.
It is reproduced in receiver side or sender side or both side?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by nsuke <gi...@git.apache.org>.
Github user nsuke commented on the pull request:
https://github.com/apache/thrift/pull/864#issuecomment-185259237
I've tested this on my local env but i8(byte) and i32 values are encoded like 1.0 instead of 1, and naturally the receiver side fails to handle this.
I'm using Lua 5.3.2.
Would be nice if you add a workaround to force them to be integer strings.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by mygityf <gi...@git.apache.org>.
Github user mygityf commented on the pull request:
https://github.com/apache/thrift/pull/864#issuecomment-183865602
@nsuke
I move THRIFT-3631 to new branch and keep only the THRIFT-3631 commit in this pull request.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by nsuke <gi...@git.apache.org>.
Github user nsuke commented on a diff in the pull request:
https://github.com/apache/thrift/pull/864#discussion_r53268905
--- Diff: lib/lua/TJsonProtocol.lua ---
@@ -0,0 +1,725 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one
+-- or more contributor license agreements. See the NOTICE file
+-- distributed with this work for additional information
+-- regarding copyright ownership. The ASF licenses this file
+-- to you under the Apache License, Version 2.0 (the
+-- "License"), you may not use this file except in compliance
+-- with the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing,
+-- software distributed under the License is distributed on an
+-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+-- KIND, either express or implied. See the License for the
+-- specific language governing permissions and limitations
+-- under the License.
+--
+
+require 'TProtocol'
+require 'libluabpack'
+require 'libluabitwise'
+
+TJSONProtocol = __TObject.new(TProtocolBase, {
+ __type = 'TJSONProtocol',
+ THRIFT_JSON_PROTOCOL_VERSION = 1,
+ jsonContext = {},
+ jsonContextVal = {first = true, colon = true, ttype = 2, null = true},
+ jsonContextIndex = 1,
+ hasReadByte = ""
+})
+
+TTypeToString = {}
+TTypeToString[TType.BOOL] = "tf"
+TTypeToString[TType.BYTE] = "i8"
+TTypeToString[TType.I16] = "i16"
+TTypeToString[TType.I32] = "i32"
+TTypeToString[TType.I64] = "i64"
+TTypeToString[TType.DOUBLE] = "dbl"
+TTypeToString[TType.STRING] = "str"
+TTypeToString[TType.STRUCT] = "rec"
+TTypeToString[TType.LIST] = "lst"
+TTypeToString[TType.SET] = "set"
+TTypeToString[TType.MAP] = "map"
+
+StringToTType = {
+ tf = TType.BOOL,
+ i8 = TType.BYTE,
+ i16 = TType.I16,
+ i32 = TType.I32,
+ i64 = TType.I64,
+ dbl = TType.DOUBLE,
+ str = TType.STRING,
+ rec = TType.STRUCT,
+ map = TType.MAP,
+ set = TType.SET,
+ lst = TType.LIST
+}
+
+JSONNode = {
+ ObjectBegin = '{',
+ ObjectEnd = '}',
+ ArrayBegin = '[',
+ ArrayEnd = ']',
+ PairSeparator = ':',
+ ElemSeparator = ',',
+ Backslash = '\\',
+ StringDelimiter = '"',
+ ZeroChar = '0',
+ EscapeChar = 'u',
+ Nan = 'NaN',
+ Infinity = 'Infinity',
+ NegativeInfinity = '-Infinity',
+ EscapeChars = "\"\\bfnrt",
+ EscapePrefix = "\\u00"
+}
+
+EscapeCharVals = {
+ '"', '\\', '\b', '\f', '\n', '\r', '\t'
+}
+
+JSONCharTable = {
+ --0 1 2 3 4 5 6 7 8 9 A B C D E F
+ 0, 0, 0, 0, 0, 0, 0, 0, 98,116,110, 0,102,114, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 1, 1,34, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
+}
+
+-- character table string
+local b='ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
+
+-- encoding
+function base64_encode(data)
+ return ((data:gsub('.', function(x)
+ local r,b='',x:byte()
+ for i=8,1,-1 do r=r..(b%2^i-b%2^(i-1)>0 and '1' or '0') end
+ return r;
+ end)..'0000'):gsub('%d%d%d?%d?%d?%d?', function(x)
+ if (#x < 6) then return '' end
+ local c=0
+ for i=1,6 do c=c+(x:sub(i,i)=='1' and 2^(6-i) or 0) end
+ return b:sub(c+1,c+1)
+ end)..({ '', '==', '=' })[#data%3+1])
+end
+
+-- decoding
+function base64_decode(data)
+ data = string.gsub(data, '[^'..b..'=]', '')
+ return (data:gsub('.', function(x)
+ if (x == '=') then return '' end
+ local r,f='',(b:find(x)-1)
+ for i=6,1,-1 do r=r..(f%2^i-f%2^(i-1)>0 and '1' or '0') end
+ return r;
+ end):gsub('%d%d%d?%d?%d?%d?%d?%d?', function(x)
+ if (#x ~= 8) then return '' end
+ local c=0
+ for i=1,8 do c=c+(x:sub(i,i)=='1' and 2^(8-i) or 0) end
+ return string.char(c)
+ end))
+end
+
+function TJSONProtocol:resetContext()
+ self.jsonContext = {}
+ self.jsonContextVal = {first = true, colon = true, ttype = 2, null = true}
+ self.jsonContextIndex = 1
+end
+
+function TJSONProtocol:contextPush(context)
+ self.jsonContextIndex = self.jsonContextIndex + 1
+ self.jsonContext[self.jsonContextIndex] = self.jsonContextVal
+ self.jsonContextVal = context
+end
+
+function TJSONProtocol:contextPop()
+ self.jsonContextVal = self.jsonContext[self.jsonContextIndex]
+ self.jsonContextIndex = self.jsonContextIndex - 1
+end
+
+function TJSONProtocol:escapeNum()
+ if self.jsonContextVal.ttype == 1 then
+ return self.jsonContextVal.colon
+ else
+ return false
+ end
+end
+
+function TJSONProtocol:writeElemSeparator()
+ if self.jsonContextVal.null then
+ return
+ end
+ if self.jsonContextVal.first then
+ self.jsonContextVal.first = false
+ else
+ if self.jsonContextVal.ttype == 1 then
+ if self.jsonContextVal.colon then
+ self.trans:write(JSONNode.PairSeparator)
+ self.jsonContextVal.colon = false
+ else
+ self.trans:write(JSONNode.ElemSeparator)
+ self.jsonContextVal.colon = true
+ end
+ else
+ self.trans:write(JSONNode.ElemSeparator)
+ end
+ end
+end
+
+function TJSONProtocol:hexChar(val)
+ val = libluabitwise.band(val, 0x0f)
+ if val < 10 then
+ return val + 48
+ else
+ return val + 87
+ end
+end
+
+function TJSONProtocol:writeJSONEscapeChar(ch)
+ self.trans:write(JSONNode.EscapePrefix)
+ local outCh = hexChar(libluabitwise.shiftr(ch, 4))
+ local buff = libluabpack.bpack('c', outCh)
+ self.trans:write(buff)
+ outCh = hexChar(ch)
+ buff = libluabpack.bpack('c', outCh)
+ self.trans:write(buff)
+end
+
+function TJSONProtocol:writeJSONChar(byte)
+ ch = string.byte(byte)
+ if ch >= 0x30 then
+ if ch == JSONNode.Backslash then
+ self.trans:write(JSONNode.Backslash)
+ self.trans:write(JSONNode.Backslash)
+ else
+ self.trans:write(byte)
+ end
+ else
+ local outCh = JSONCharTable[ch+1]
+ if outCh == 1 then
+ self.trans:write(byte)
+ elseif outCh > 1 then
+ self.trans:write(JSONNode.Backslash)
+ local buff = libluabpack.bpack('c', outCh)
+ self.trans:write(buff)
+ else
+ self:writeJSONEscapeChar(ch)
+ end
+ end
+end
+
+function TJSONProtocol:writeJSONString(str)
+ self:writeElemSeparator()
+ self.trans:write(JSONNode.StringDelimiter)
+ -- TODO
--- End diff --
thanks. will include this info to the comment.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by nsuke <gi...@git.apache.org>.
Github user nsuke commented on the pull request:
https://github.com/apache/thrift/pull/864#issuecomment-185531813
sender side.
used lua client against cpp / py server and extracted the json in the packet with tcpdump.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by mygityf <gi...@git.apache.org>.
Github user mygityf closed the pull request at:
https://github.com/apache/thrift/pull/864
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by mygityf <gi...@git.apache.org>.
Github user mygityf commented on the pull request:
https://github.com/apache/thrift/pull/864#issuecomment-185568709
@nsuke Integer is convert to string without dot. can you try again in your env to check whether it work well or not, thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by nsuke <gi...@git.apache.org>.
Github user nsuke commented on the pull request:
https://github.com/apache/thrift/pull/864#issuecomment-183876908
@mygityf the tests passed against all but C++ server.
https://travis-ci.org/apache/thrift/jobs/109138968#L8071
Could you look into the failures ?
You can run the specific part by
test/test.py --regex "cpp.*lua.*json"
and can find log in `test/log/unexpected_failures.log`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by mygityf <gi...@git.apache.org>.
Github user mygityf commented on a diff in the pull request:
https://github.com/apache/thrift/pull/864#discussion_r53265228
--- Diff: lib/lua/TJsonProtocol.lua ---
@@ -0,0 +1,725 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one
+-- or more contributor license agreements. See the NOTICE file
+-- distributed with this work for additional information
+-- regarding copyright ownership. The ASF licenses this file
+-- to you under the Apache License, Version 2.0 (the
+-- "License"), you may not use this file except in compliance
+-- with the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing,
+-- software distributed under the License is distributed on an
+-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+-- KIND, either express or implied. See the License for the
+-- specific language governing permissions and limitations
+-- under the License.
+--
+
+require 'TProtocol'
+require 'libluabpack'
+require 'libluabitwise'
+
+TJSONProtocol = __TObject.new(TProtocolBase, {
+ __type = 'TJSONProtocol',
+ THRIFT_JSON_PROTOCOL_VERSION = 1,
+ jsonContext = {},
+ jsonContextVal = {first = true, colon = true, ttype = 2, null = true},
+ jsonContextIndex = 1,
+ hasReadByte = ""
+})
+
+TTypeToString = {}
+TTypeToString[TType.BOOL] = "tf"
+TTypeToString[TType.BYTE] = "i8"
+TTypeToString[TType.I16] = "i16"
+TTypeToString[TType.I32] = "i32"
+TTypeToString[TType.I64] = "i64"
+TTypeToString[TType.DOUBLE] = "dbl"
+TTypeToString[TType.STRING] = "str"
+TTypeToString[TType.STRUCT] = "rec"
+TTypeToString[TType.LIST] = "lst"
+TTypeToString[TType.SET] = "set"
+TTypeToString[TType.MAP] = "map"
+
+StringToTType = {
+ tf = TType.BOOL,
+ i8 = TType.BYTE,
+ i16 = TType.I16,
+ i32 = TType.I32,
+ i64 = TType.I64,
+ dbl = TType.DOUBLE,
+ str = TType.STRING,
+ rec = TType.STRUCT,
+ map = TType.MAP,
+ set = TType.SET,
+ lst = TType.LIST
+}
+
+JSONNode = {
+ ObjectBegin = '{',
+ ObjectEnd = '}',
+ ArrayBegin = '[',
+ ArrayEnd = ']',
+ PairSeparator = ':',
+ ElemSeparator = ',',
+ Backslash = '\\',
+ StringDelimiter = '"',
+ ZeroChar = '0',
+ EscapeChar = 'u',
+ Nan = 'NaN',
+ Infinity = 'Infinity',
+ NegativeInfinity = '-Infinity',
+ EscapeChars = "\"\\bfnrt",
+ EscapePrefix = "\\u00"
+}
+
+EscapeCharVals = {
+ '"', '\\', '\b', '\f', '\n', '\r', '\t'
+}
+
+JSONCharTable = {
+ --0 1 2 3 4 5 6 7 8 9 A B C D E F
+ 0, 0, 0, 0, 0, 0, 0, 0, 98,116,110, 0,102,114, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 1, 1,34, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
+}
+
+-- character table string
+local b='ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
+
+-- encoding
+function base64_encode(data)
+ return ((data:gsub('.', function(x)
+ local r,b='',x:byte()
+ for i=8,1,-1 do r=r..(b%2^i-b%2^(i-1)>0 and '1' or '0') end
+ return r;
+ end)..'0000'):gsub('%d%d%d?%d?%d?%d?', function(x)
+ if (#x < 6) then return '' end
+ local c=0
+ for i=1,6 do c=c+(x:sub(i,i)=='1' and 2^(6-i) or 0) end
+ return b:sub(c+1,c+1)
+ end)..({ '', '==', '=' })[#data%3+1])
+end
+
+-- decoding
+function base64_decode(data)
+ data = string.gsub(data, '[^'..b..'=]', '')
+ return (data:gsub('.', function(x)
+ if (x == '=') then return '' end
+ local r,f='',(b:find(x)-1)
+ for i=6,1,-1 do r=r..(f%2^i-f%2^(i-1)>0 and '1' or '0') end
+ return r;
+ end):gsub('%d%d%d?%d?%d?%d?%d?%d?', function(x)
+ if (#x ~= 8) then return '' end
+ local c=0
+ for i=1,8 do c=c+(x:sub(i,i)=='1' and 2^(8-i) or 0) end
+ return string.char(c)
+ end))
+end
+
+function TJSONProtocol:resetContext()
+ self.jsonContext = {}
+ self.jsonContextVal = {first = true, colon = true, ttype = 2, null = true}
+ self.jsonContextIndex = 1
+end
+
+function TJSONProtocol:contextPush(context)
+ self.jsonContextIndex = self.jsonContextIndex + 1
+ self.jsonContext[self.jsonContextIndex] = self.jsonContextVal
+ self.jsonContextVal = context
+end
+
+function TJSONProtocol:contextPop()
+ self.jsonContextVal = self.jsonContext[self.jsonContextIndex]
+ self.jsonContextIndex = self.jsonContextIndex - 1
+end
+
+function TJSONProtocol:escapeNum()
+ if self.jsonContextVal.ttype == 1 then
+ return self.jsonContextVal.colon
+ else
+ return false
+ end
+end
+
+function TJSONProtocol:writeElemSeparator()
+ if self.jsonContextVal.null then
+ return
+ end
+ if self.jsonContextVal.first then
+ self.jsonContextVal.first = false
+ else
+ if self.jsonContextVal.ttype == 1 then
+ if self.jsonContextVal.colon then
+ self.trans:write(JSONNode.PairSeparator)
+ self.jsonContextVal.colon = false
+ else
+ self.trans:write(JSONNode.ElemSeparator)
+ self.jsonContextVal.colon = true
+ end
+ else
+ self.trans:write(JSONNode.ElemSeparator)
+ end
+ end
+end
+
+function TJSONProtocol:hexChar(val)
+ val = libluabitwise.band(val, 0x0f)
+ if val < 10 then
+ return val + 48
+ else
+ return val + 87
+ end
+end
+
+function TJSONProtocol:writeJSONEscapeChar(ch)
+ self.trans:write(JSONNode.EscapePrefix)
+ local outCh = hexChar(libluabitwise.shiftr(ch, 4))
+ local buff = libluabpack.bpack('c', outCh)
+ self.trans:write(buff)
+ outCh = hexChar(ch)
+ buff = libluabpack.bpack('c', outCh)
+ self.trans:write(buff)
+end
+
+function TJSONProtocol:writeJSONChar(byte)
+ ch = string.byte(byte)
+ if ch >= 0x30 then
+ if ch == JSONNode.Backslash then
+ self.trans:write(JSONNode.Backslash)
+ self.trans:write(JSONNode.Backslash)
+ else
+ self.trans:write(byte)
+ end
+ else
+ local outCh = JSONCharTable[ch+1]
+ if outCh == 1 then
+ self.trans:write(byte)
+ elseif outCh > 1 then
+ self.trans:write(JSONNode.Backslash)
+ local buff = libluabpack.bpack('c', outCh)
+ self.trans:write(buff)
+ else
+ self:writeJSONEscapeChar(ch)
+ end
+ end
+end
+
+function TJSONProtocol:writeJSONString(str)
+ self:writeElemSeparator()
+ self.trans:write(JSONNode.StringDelimiter)
+ -- TODO
--- End diff --
It can be escaped by being preceded with a backslash,
like " to \", \t to \\t, \r to \\r, \f to \\f, \n to \\n , \b to \\b.
Otherwise, character " will break JSON format.
e.g:
To be sent string is "Begin:This is Send\"String\b\t\n\r\f:End."
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by mygityf <gi...@git.apache.org>.
Github user mygityf commented on the pull request:
https://github.com/apache/thrift/pull/864#issuecomment-184034288
The precision is 15 bits after dot of format casting from double to string in Thrift-cpp-library- json-protocol.
But the precision is 16 bits after dot in Thrift-lua-library-json-protocol.
So, test with C++ server is break.
@nsuke can we modify the precision to 16 bits for C++ server?
The solution:
To Change code 'str.precision(std::numeric_limits<double>::digits10 + 1);' to 'str.precision(std::numeric_limits<double>::digits10 + 2);'
in function doubeToString file TJSONProtocol.cpp at line 524,
e.g:
before:
In C++ TJsonProtocol.cpp:
double a = 1.12345678906666663;
string astr = doubleToString(a);
double b = stringToDouble(astr);
the result as below:
a = 1.1234567890666667
astr = "1.123456789066667"
b = 1.1234567890666669
after changing:
the result as below:
a = 1.1234567890666667
astr = "1.1234567890666667"
b = 1.1234567890666667
This result is expected.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by mygityf <gi...@git.apache.org>.
Github user mygityf commented on the pull request:
https://github.com/apache/thrift/pull/864#issuecomment-184037495
The solution to the test failure:
pull request for C++ thrift https://github.com/apache/thrift/pull/870.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by nsuke <gi...@git.apache.org>.
Github user nsuke commented on the pull request:
https://github.com/apache/thrift/pull/864#issuecomment-185778986
Nice, the last commit fixed the problem. I guess it's a Lua 5.3 issue.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] thrift pull request: THRIFT-3631 Lua- JSON protocol implement for ...
Posted by nsuke <gi...@git.apache.org>.
Github user nsuke commented on the pull request:
https://github.com/apache/thrift/pull/864#issuecomment-185796578
please close this PR as it's merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---