You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@dubbo.apache.org by GitBox <gi...@apache.org> on 2020/05/14 13:49:44 UTC
[GitHub] [dubbo-go-hessian2] zonghaishang opened a new pull request #188: Perf decode string
zonghaishang opened a new pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188
<!-- Thanks for sending a pull request!
-->
**What this PR does**:
**Which issue(s) this PR fixes**:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #
**Special notes for your reviewer**:
**Does this PR introduce a user-facing change?**:
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".
-->
```release-note
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] codecov-io edited a comment on pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#issuecomment-628651665
# [Codecov](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=h1) Report
> Merging [#188](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=desc) into [master](https://codecov.io/gh/apache/dubbo-go-hessian2/commit/bb036e6fcafdb3d897fa47615a253f716c8b0d24&el=desc) will **decrease** coverage by `1.08%`.
> The diff coverage is `63.96%`.
[](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #188 +/- ##
==========================================
- Coverage 67.63% 66.54% -1.09%
==========================================
Files 22 22
Lines 2688 2765 +77
==========================================
+ Hits 1818 1840 +22
- Misses 665 713 +48
- Partials 205 212 +7
```
| [Impacted Files](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [decode.go](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188/diff?src=pr&el=tree#diff-ZGVjb2RlLmdv) | `66.66% <38.46%> (-3.53%)` | :arrow_down: |
| [string.go](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188/diff?src=pr&el=tree#diff-c3RyaW5nLmdv) | `58.20% <67.34%> (-9.93%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=footer). Last update [bb036e6...3a72da5](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r426121880
##########
File path: string.go
##########
@@ -343,36 +342,139 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
+ remain, cap := len(bytesBuf)-offset, charLen<<2
+ if remain < cap {
+ grow := len(bytesBuf) + cap
+ bs := make([]byte, grow)
+ copy(bs, bytesBuf)
+ bytesBuf = bs
+ }
default:
return s, perrors.New("expect string tag")
}
}
- r, charLen, byteLen, err = decodeUcs4Rune(d.reader)
+ if chunkLen > 0 {
+ nread, err := d.next(bytesBuf[offset : offset+chunkLen])
+ if err != nil {
+ if err == io.EOF {
+ break
+ }
+ return s, perrors.WithStack(err)
+ }
+
+ // quickly detect the actual number of bytes
+ prev := offset
+ for i, len := offset, offset+nread; i < len; chunkLen-- {
+ ch := bytesBuf[i]
+ if ch < 0x80 {
+ offset++
+ i++
+ } else if (ch & 0xe0) == 0xc0 {
+ offset += 2
+ i += 2
+ } else if (ch & 0xf0) == 0xe0 {
+ offset += 3
+ i += 3
+ } else {
+ return s, perrors.Errorf("bad utf-8 encoding, offset=%d\n", offset)
+ }
+ }
+
+ if remain := offset - prev - nread; remain > 0 {
+ if remain == 1 {
+ ch, err := d.readByte()
+ if err != nil {
+ return s, perrors.WithStack(err)
+ }
+ bytesBuf[offset-1] = ch
+ } else {
+ var err error
+ if buffed := d.Buffered(); buffed < remain {
+ // trigger fill data if required
+ copy(bytesBuf[offset-remain:offset], d.peek(remain))
+ _, err = d.reader.Discard(remain)
+ } else {
+ // copy remaining bytes.
+ _, err = d.next(bytesBuf[offset-remain : offset])
+ }
+
+ if err != nil {
+ return s, perrors.WithStack(err)
+ }
+ }
+ }
+
+ // the expected length string has been processed.
+ if chunkLen <= 0 {
+ if last {
+ b := bytesBuf[:offset]
+ return *(*string)(unsafe.Pointer(&b)), nil
+ }
+
+ // we need to detect next chunk
+ continue
+ }
+ }
+
+ // decode byte
+ ch, err := d.readByte()
Review comment:
两个目的：
1. 触发buffer fill data.
2. 前面只能保证读取chunkLen字节数, 并且读取整个char的utf-8编码字节(保证通过char的字节不被拆分)，但是可能并没有处理所有chunkLen个数的字符。后续的read可以处理完所有chunkLen字符。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r429919470
##########
File path: string_test.go
##########
@@ -158,11 +158,25 @@ func TestStringEncode(t *testing.T) {
testJavaDecode(t, "argString_65536", s65560[:65536])
}
+func BenchmarkDecodeStringOptimized(t *testing.B) {
+ e := NewEncoder()
+ e.Encode(testString)
+ buf := e.buffer
+
+ d := NewDecoder(buf)
+
+ for i := 0; i < t.N; i++ {
+ d.DecodeValue()
+ d.Reset(buf)
+ }
+}
+
func TestStringEmoji(t *testing.T) {
// see: test_hessian/src/main/java/test/TestString.java
s0 := "emoji🤣"
s0 += ",max" + string(rune(0x10FFFF))
- testDecodeFramework(t, "customReplyStringEmoji", s0)
+ // todo 这里正确拿到hessian解码字节数组，但是构造string的时候，不是rune，emoji表情符号显示有点问题，修改assert？？？
Review comment:
Support emoji decoding, performance reduced by 4%， :-(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r431580487
##########
File path: string_test.go
##########
@@ -158,11 +158,25 @@ func TestStringEncode(t *testing.T) {
testJavaDecode(t, "argString_65536", s65560[:65536])
}
+func BenchmarkDecodeStringOptimized(t *testing.B) {
+ e := NewEncoder()
+ e.Encode(testString)
+ buf := e.buffer
+
+ d := NewDecoder(buf)
+
+ for i := 0; i < t.N; i++ {
+ d.DecodeValue()
+ d.Reset(buf)
+ }
+}
+
func TestStringEmoji(t *testing.T) {
// see: test_hessian/src/main/java/test/TestString.java
s0 := "emoji🤣"
s0 += ",max" + string(rune(0x10FFFF))
+ // todo 这里正确拿到hessian解码字节数组，但是构造string的时候，不是rune，emoji表情符号显示有点问题，修改assert？？？
Review comment:
ok, 我单独提个pr
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r426134794
##########
File path: string.go
##########
@@ -343,36 +342,139 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
+ remain, cap := len(bytesBuf)-offset, charLen<<2
+ if remain < cap {
+ grow := len(bytesBuf) + cap
+ bs := make([]byte, grow)
+ copy(bs, bytesBuf)
+ bytesBuf = bs
+ }
default:
return s, perrors.New("expect string tag")
}
}
- r, charLen, byteLen, err = decodeUcs4Rune(d.reader)
+ if chunkLen > 0 {
+ nread, err := d.next(bytesBuf[offset : offset+chunkLen])
+ if err != nil {
+ if err == io.EOF {
+ break
+ }
+ return s, perrors.WithStack(err)
+ }
+
+ // quickly detect the actual number of bytes
+ prev := offset
+ for i, len := offset, offset+nread; i < len; chunkLen-- {
+ ch := bytesBuf[i]
+ if ch < 0x80 {
+ offset++
+ i++
+ } else if (ch & 0xe0) == 0xc0 {
+ offset += 2
+ i += 2
+ } else if (ch & 0xf0) == 0xe0 {
+ offset += 3
+ i += 3
+ } else {
+ return s, perrors.Errorf("bad utf-8 encoding, offset=%d\n", offset)
+ }
+ }
+
+ if remain := offset - prev - nread; remain > 0 {
+ if remain == 1 {
+ ch, err := d.readByte()
+ if err != nil {
+ return s, perrors.WithStack(err)
+ }
+ bytesBuf[offset-1] = ch
+ } else {
+ var err error
+ if buffed := d.Buffered(); buffed < remain {
+ // trigger fill data if required
+ copy(bytesBuf[offset-remain:offset], d.peek(remain))
+ _, err = d.reader.Discard(remain)
+ } else {
+ // copy remaining bytes.
+ _, err = d.next(bytesBuf[offset-remain : offset])
+ }
+
+ if err != nil {
+ return s, perrors.WithStack(err)
+ }
+ }
+ }
+
+ // the expected length string has been processed.
+ if chunkLen <= 0 {
+ if last {
Review comment:
removed already.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#issuecomment-629810114
> in fact, when decoding you can't know how many bytes you should read, until you analysis all bytes. A possible improvement may be you can first read a chunk length of bytes first, and then loop analysis and read more until reach the length of chunk. This may be useful to improve performance for most situation.
The current optimization already includes this idea.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] codecov-io commented on pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#issuecomment-628651665
# [Codecov](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=h1) Report
> Merging [#188](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=desc) into [master](https://codecov.io/gh/apache/dubbo-go-hessian2/commit/bb036e6fcafdb3d897fa47615a253f716c8b0d24&el=desc) will **decrease** coverage by `1.17%`.
> The diff coverage is `61.68%`.
[](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #188 +/- ##
==========================================
- Coverage 67.63% 66.46% -1.18%
==========================================
Files 22 22
Lines 2688 2761 +73
==========================================
+ Hits 1818 1835 +17
- Misses 665 715 +50
- Partials 205 211 +6
```
| [Impacted Files](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [decode.go](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188/diff?src=pr&el=tree#diff-ZGVjb2RlLmdv) | `67.82% <45.45%> (-2.37%)` | :arrow_down: |
| [string.go](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188/diff?src=pr&el=tree#diff-c3RyaW5nLmdv) | `56.76% <63.54%> (-11.38%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=footer). Last update [bb036e6...f3584d9](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] AlexStocks commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
AlexStocks commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r425656418
##########
File path: string_test.go
##########
@@ -158,11 +158,25 @@ func TestStringEncode(t *testing.T) {
testJavaDecode(t, "argString_65536", s65560[:65536])
}
+func BenchmarkDecodeStringOptimized(t *testing.B) {
+ e := NewEncoder()
+ e.Encode(testString)
+ buf := e.buffer
+
+ d := NewDecoder(buf)
+
+ for i := 0; i < t.N; i++ {
+ d.DecodeValue()
+ d.Reset(buf)
+ }
+}
+
func TestStringEmoji(t *testing.T) {
// see: test_hessian/src/main/java/test/TestString.java
s0 := "emoji🤣"
s0 += ",max" + string(rune(0x10FFFF))
- testDecodeFramework(t, "customReplyStringEmoji", s0)
+ // todo 这里正确拿到hessian解码字节数组，但是构造string的时候，不是rune，emoji表情符号显示有点问题，修改assert？？？
Review comment:
u can modify the assert.
##########
File path: decode.go
##########
@@ -50,15 +50,54 @@ func NewDecoder(b []byte) *Decoder {
return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), typeRefs: &TypeRefs{records: map[string]bool{}}}
}
+// NewDecoder generate a decoder instance
+func NewDecoderSize(b []byte, size int) *Decoder {
+ return &Decoder{reader: bufio.NewReaderSize(bytes.NewReader(b), size), typeRefs: &TypeRefs{records: map[string]bool{}}}
+}
+
// NewDecoder generate a decoder instance with skip
func NewDecoderWithSkip(b []byte) *Decoder {
return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), typeRefs: &TypeRefs{records: map[string]bool{}}, isSkip: true}
}
+// NewCheapDecoderWithSkip generate a decoder instance with skip,
+// only for cache pool, before decode Reset should be called.
+// For example, with pooling use, will effectively improve performance
+//
+// var hessianPool = &sync.Pool{
+// New: func() interface{} {
+// return hessian.NewCheapDecoderWithSkip([]byte{})
+// },
+// }
+//
+// decoder := hessianPool.Get().(*hessian.Decoder)
+// fill decode data
+// decoder.Reset(data[:])
+// decode anything ...
+// hessianPool.Put(decoder)
+func NewCheapDecoderWithSkip(b []byte) *Decoder {
+ return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), isSkip: true}
+}
+
/////////////////////////////////////////
// utilities
/////////////////////////////////////////
+func (d *Decoder) Reset(b []byte) *Decoder {
+ // reuse reader buf, avoid allocate
+ d.reader.Reset(bytes.NewReader(b))
+ d.typeRefs = &TypeRefs{records: map[string]bool{}}
+
+ if d.refs != nil {
Review comment:
the two following lines is enough. u do not need write two if clauses.
```
d.refs = nil
d.classInfoList = nil
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r425651126
##########
File path: decode.go
##########
@@ -50,15 +50,50 @@ func NewDecoder(b []byte) *Decoder {
return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), typeRefs: &TypeRefs{records: map[string]bool{}}}
}
+// NewDecoder generate a decoder instance
+func NewDecoderSize(b []byte, size int) *Decoder {
+ return &Decoder{reader: bufio.NewReaderSize(bytes.NewReader(b), size), typeRefs: &TypeRefs{records: map[string]bool{}}}
+}
+
// NewDecoder generate a decoder instance with skip
func NewDecoderWithSkip(b []byte) *Decoder {
return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), typeRefs: &TypeRefs{records: map[string]bool{}}, isSkip: true}
}
+// NewCheapDecoderWithSkip generate a decoder instance with skip,
+// only for cache pool, before decode Reset should be called.
+// For example, with pooling use, will effectively improve performance
+//
+// var hessianPool = &sync.Pool{
+// New: func() interface{} {
+// return hessian.NewCheapDecoderWithSkip([]byte{})
+// },
+// }
+//
+// decoder := hessianPool.Get().(*hessian.Decoder)
+// fill decode data
+// decoder.Reset(data[:])
+// decode anything ...
+// hessianPool.Put(decoder)
+func NewCheapDecoderWithSkip(b []byte) *Decoder {
+ return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), isSkip: true}
+}
+
/////////////////////////////////////////
// utilities
/////////////////////////////////////////
+func (d *Decoder) Reset(b []byte) *Decoder {
+ // reuse reader buf, avoid allocate
+ d.reader.Reset(bytes.NewReader(b))
+ d.typeRefs = &TypeRefs{records: map[string]bool{}}
+
+ if d.refs != nil || d.classInfoList != nil {
+ d.refs, d.classInfoList = nil, nil
Review comment:
ok, fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang merged pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang merged pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] wongoo commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
wongoo commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r426111970
##########
File path: decode.go
##########
@@ -50,15 +50,54 @@ func NewDecoder(b []byte) *Decoder {
return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), typeRefs: &TypeRefs{records: map[string]bool{}}}
}
+// NewDecoder generate a decoder instance
+func NewDecoderSize(b []byte, size int) *Decoder {
+ return &Decoder{reader: bufio.NewReaderSize(bytes.NewReader(b), size), typeRefs: &TypeRefs{records: map[string]bool{}}}
+}
+
// NewDecoder generate a decoder instance with skip
func NewDecoderWithSkip(b []byte) *Decoder {
return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), typeRefs: &TypeRefs{records: map[string]bool{}}, isSkip: true}
}
+// NewCheapDecoderWithSkip generate a decoder instance with skip,
+// only for cache pool, before decode Reset should be called.
+// For example, with pooling use, will effectively improve performance
+//
+// var hessianPool = &sync.Pool{
+// New: func() interface{} {
+// return hessian.NewCheapDecoderWithSkip([]byte{})
+// },
+// }
+//
+// decoder := hessianPool.Get().(*hessian.Decoder)
+// fill decode data
+// decoder.Reset(data[:])
+// decode anything ...
+// hessianPool.Put(decoder)
+func NewCheapDecoderWithSkip(b []byte) *Decoder {
+ return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), isSkip: true}
+}
+
/////////////////////////////////////////
// utilities
/////////////////////////////////////////
+func (d *Decoder) Reset(b []byte) *Decoder {
+ // reuse reader buf, avoid allocate
+ d.reader.Reset(bytes.NewReader(b))
+ d.typeRefs = &TypeRefs{records: map[string]bool{}}
+
+ if d.refs != nil {
Review comment:
agree with @AlexStocks , just reset them
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#issuecomment-633945909
@wongoo please take a look.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#issuecomment-629608318
I'll explore the byte code for the go emoji first, and there may be a way to fix it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] AlexStocks commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
AlexStocks commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r430105887
##########
File path: string_test.go
##########
@@ -158,11 +158,25 @@ func TestStringEncode(t *testing.T) {
testJavaDecode(t, "argString_65536", s65560[:65536])
}
+func BenchmarkDecodeStringOptimized(t *testing.B) {
+ e := NewEncoder()
+ e.Encode(testString)
+ buf := e.buffer
+
+ d := NewDecoder(buf)
+
+ for i := 0; i < t.N; i++ {
+ d.DecodeValue()
+ d.Reset(buf)
+ }
+}
+
func TestStringEmoji(t *testing.T) {
// see: test_hessian/src/main/java/test/TestString.java
s0 := "emoji🤣"
s0 += ",max" + string(rune(0x10FFFF))
- testDecodeFramework(t, "customReplyStringEmoji", s0)
+ // todo 这里正确拿到hessian解码字节数组，但是构造string的时候，不是rune，emoji表情符号显示有点问题，修改assert？？？
Review comment:
but we should support it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] gaoxinge commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
gaoxinge commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r425556871
##########
File path: string.go
##########
@@ -343,36 +342,137 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
+
+ remain, cap := len(bytesBuf)-offset, charLen<<2
+ if remain < cap {
+ grow := len(bytesBuf) + cap
+ bs := make([]byte, grow)
+ copy(bs, bytesBuf)
+ bytesBuf = bs
+ }
default:
return s, perrors.New("expect string tag")
}
}
- r, charLen, byteLen, err = decodeUcs4Rune(d.reader)
+ if buffed := d.Buffered(); chunkLen > 0 {
+ nread, err := d.next(bytesBuf[offset : offset+chunkLen])
Review comment:
我的理解：从`chunkLen`的[初始化](https://github.com/apache/dubbo-go-hessian2/blob/b716f807bbf6ac2b4236b291c87c22c53b38b5e9/string.go#L322)可以看出`chunkLen`是表示字符串的长度，不是表示字符串在对应编码下byte的长度。byte的长度应该比`chunkLen`大才对。所以使用`chunkLen`作为bytesBuf一次读取byte的长度会不会有问题？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] gaoxinge commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
gaoxinge commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r425556871
##########
File path: string.go
##########
@@ -343,36 +342,137 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
+
+ remain, cap := len(bytesBuf)-offset, charLen<<2
+ if remain < cap {
+ grow := len(bytesBuf) + cap
+ bs := make([]byte, grow)
+ copy(bs, bytesBuf)
+ bytesBuf = bs
+ }
default:
return s, perrors.New("expect string tag")
}
}
- r, charLen, byteLen, err = decodeUcs4Rune(d.reader)
+ if buffed := d.Buffered(); chunkLen > 0 {
+ nread, err := d.next(bytesBuf[offset : offset+chunkLen])
Review comment:
我的理解：从`chunkLen`的[初始化](https://github.com/apache/dubbo-go-hessian2/blob/b716f807bbf6ac2b4236b291c87c22c53b38b5e9/string.go#L322)可以看出`chunkLen`是表示字符串的长度，不是表示字符串在对应编码下byte的长度。byte的长度应该比`chunkLen`大才对。这样不会有问题么？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r431580487
##########
File path: string_test.go
##########
@@ -158,11 +158,25 @@ func TestStringEncode(t *testing.T) {
testJavaDecode(t, "argString_65536", s65560[:65536])
}
+func BenchmarkDecodeStringOptimized(t *testing.B) {
+ e := NewEncoder()
+ e.Encode(testString)
+ buf := e.buffer
+
+ d := NewDecoder(buf)
+
+ for i := 0; i < t.N; i++ {
+ d.DecodeValue()
+ d.Reset(buf)
+ }
+}
+
func TestStringEmoji(t *testing.T) {
// see: test_hessian/src/main/java/test/TestString.java
s0 := "emoji🤣"
s0 += ",max" + string(rune(0x10FFFF))
+ // todo 这里正确拿到hessian解码字节数组，但是构造string的时候，不是rune，emoji表情符号显示有点问题，修改assert？？？
Review comment:
ok, I just submitted pr to fix it: https://github.com/apache/dubbo-go-hessian2/pull/194
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r426134764
##########
File path: string.go
##########
@@ -343,36 +342,139 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
+ remain, cap := len(bytesBuf)-offset, charLen<<2
+ if remain < cap {
+ grow := len(bytesBuf) + cap
+ bs := make([]byte, grow)
+ copy(bs, bytesBuf)
+ bytesBuf = bs
+ }
default:
return s, perrors.New("expect string tag")
}
}
- r, charLen, byteLen, err = decodeUcs4Rune(d.reader)
+ if chunkLen > 0 {
+ nread, err := d.next(bytesBuf[offset : offset+chunkLen])
+ if err != nil {
+ if err == io.EOF {
+ break
+ }
+ return s, perrors.WithStack(err)
+ }
+
+ // quickly detect the actual number of bytes
+ prev := offset
+ for i, len := offset, offset+nread; i < len; chunkLen-- {
Review comment:
optimized.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r425562815
##########
File path: decode_test.go
##########
@@ -126,3 +127,16 @@ func testDecodeFrameworkFunc(t *testing.T, method string, expected func(interfac
}
expected(r)
}
+
+func BenchmarkDecodeStringOptimized(t *testing.B) {
Review comment:
ok, fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] AlexStocks commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
AlexStocks commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r431581000
##########
File path: string_test.go
##########
@@ -158,11 +158,25 @@ func TestStringEncode(t *testing.T) {
testJavaDecode(t, "argString_65536", s65560[:65536])
}
+func BenchmarkDecodeStringOptimized(t *testing.B) {
+ e := NewEncoder()
+ e.Encode(testString)
+ buf := e.buffer
+
+ d := NewDecoder(buf)
+
+ for i := 0; i < t.N; i++ {
+ d.DecodeValue()
+ d.Reset(buf)
+ }
+}
+
func TestStringEmoji(t *testing.T) {
// see: test_hessian/src/main/java/test/TestString.java
s0 := "emoji🤣"
s0 += ",max" + string(rune(0x10FFFF))
+ // todo 这里正确拿到hessian解码字节数组，但是构造string的时候，不是rune，emoji表情符号显示有点问题，修改assert？？？
Review comment:
@wongoo are there any other problems? If not, we can reserve this comment in my opinion. @zonghaishang , u can translate this comment into english.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] codecov-io edited a comment on pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#issuecomment-628651665
# [Codecov](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=h1) Report
> Merging [#188](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=desc) into [master](https://codecov.io/gh/apache/dubbo-go-hessian2/commit/bb036e6fcafdb3d897fa47615a253f716c8b0d24&el=desc) will **decrease** coverage by `1.12%`.
> The diff coverage is `62.96%`.
[](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #188 +/- ##
==========================================
- Coverage 67.63% 66.50% -1.13%
==========================================
Files 22 22
Lines 2688 2762 +74
==========================================
+ Hits 1818 1837 +19
- Misses 665 713 +48
- Partials 205 212 +7
```
| [Impacted Files](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [decode.go](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188/diff?src=pr&el=tree#diff-ZGVjb2RlLmdv) | `66.66% <38.46%> (-3.53%)` | :arrow_down: |
| [string.go](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188/diff?src=pr&el=tree#diff-c3RyaW5nLmdv) | `57.73% <66.31%> (-10.41%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=footer). Last update [bb036e6...fa6429e](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] codecov-commenter commented on pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#issuecomment-633552994
# [Codecov](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=h1) Report
> Merging [#188](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=desc) into [master](https://codecov.io/gh/apache/dubbo-go-hessian2/commit/bb036e6fcafdb3d897fa47615a253f716c8b0d24&el=desc) will **decrease** coverage by `1.62%`.
> The diff coverage is `56.38%`.
[](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #188 +/- ##
==========================================
- Coverage 67.63% 66.01% -1.63%
==========================================
Files 22 22
Lines 2688 2845 +157
==========================================
+ Hits 1818 1878 +60
- Misses 665 749 +84
- Partials 205 218 +13
```
| [Impacted Files](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [decode.go](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188/diff?src=pr&el=tree#diff-ZGVjb2RlLmdv) | `66.66% <38.46%> (-3.53%)` | :arrow_down: |
| [string.go](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188/diff?src=pr&el=tree#diff-c3RyaW5nLmdv) | `55.36% <57.71%> (-12.78%)` | :arrow_down: |
| [request.go](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188/diff?src=pr&el=tree#diff-cmVxdWVzdC5nbw==) | `60.57% <0.00%> (+0.57%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=footer). Last update [bb036e6...39b42f7](https://codecov.io/gh/apache/dubbo-go-hessian2/pull/188?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#issuecomment-635048407
2 votes support, merge to master now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r425561321
##########
File path: string.go
##########
@@ -343,36 +342,137 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
+
+ remain, cap := len(bytesBuf)-offset, charLen<<2
+ if remain < cap {
+ grow := len(bytesBuf) + cap
+ bs := make([]byte, grow)
+ copy(bs, bytesBuf)
+ bytesBuf = bs
+ }
default:
return s, perrors.New("expect string tag")
}
}
- r, charLen, byteLen, err = decodeUcs4Rune(d.reader)
+ if buffed := d.Buffered(); chunkLen > 0 {
+ nread, err := d.next(bytesBuf[offset : offset+chunkLen])
Review comment:
不会有问题，chunkLen是char的长度，优化后的版本会检查后续剩余字节。这里只不过优先一次性读取chunkLen字节长度而已。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] AlexStocks commented on pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
AlexStocks commented on pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#issuecomment-629604592
> > this PR will let the hessian only support at most three bytes length utf8, while four bytes mathematical symbols, and emoji will not supprted. even more the max length of a utf8 character can be six. And the worst is the length of string in hessian definition is the length of A 16-bit unicode character string encoded in UTF-8.
> > ref
> >
> > * https://en.wikipedia.org/wiki/UTF-8
> > * http://hessian.caucho.com/doc/hessian-serialization.html##string
> > * #131
> > @wongoo
> > Any Suggestions for fixing it? I've tried parsing 4-byte utf-8 before, but Java doesn't write 4-byte utf-8
maybe u can set an issue for dubbo hessian2.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] wongoo commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
wongoo commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r431576922
##########
File path: string_test.go
##########
@@ -158,11 +158,25 @@ func TestStringEncode(t *testing.T) {
testJavaDecode(t, "argString_65536", s65560[:65536])
}
+func BenchmarkDecodeStringOptimized(t *testing.B) {
+ e := NewEncoder()
+ e.Encode(testString)
+ buf := e.buffer
+
+ d := NewDecoder(buf)
+
+ for i := 0; i < t.N; i++ {
+ d.DecodeValue()
+ d.Reset(buf)
+ }
+}
+
func TestStringEmoji(t *testing.T) {
// see: test_hessian/src/main/java/test/TestString.java
s0 := "emoji🤣"
s0 += ",max" + string(rune(0x10FFFF))
+ // todo 这里正确拿到hessian解码字节数组，但是构造string的时候，不是rune，emoji表情符号显示有点问题，修改assert？？？
Review comment:
remove the comment
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] AlexStocks commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
AlexStocks commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r425566178
##########
File path: decode.go
##########
@@ -50,15 +50,50 @@ func NewDecoder(b []byte) *Decoder {
return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), typeRefs: &TypeRefs{records: map[string]bool{}}}
}
+// NewDecoder generate a decoder instance
+func NewDecoderSize(b []byte, size int) *Decoder {
+ return &Decoder{reader: bufio.NewReaderSize(bytes.NewReader(b), size), typeRefs: &TypeRefs{records: map[string]bool{}}}
+}
+
// NewDecoder generate a decoder instance with skip
func NewDecoderWithSkip(b []byte) *Decoder {
return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), typeRefs: &TypeRefs{records: map[string]bool{}}, isSkip: true}
}
+// NewCheapDecoderWithSkip generate a decoder instance with skip,
+// only for cache pool, before decode Reset should be called.
+// For example, with pooling use, will effectively improve performance
+//
+// var hessianPool = &sync.Pool{
+// New: func() interface{} {
+// return hessian.NewCheapDecoderWithSkip([]byte{})
+// },
+// }
+//
+// decoder := hessianPool.Get().(*hessian.Decoder)
+// fill decode data
+// decoder.Reset(data[:])
+// decode anything ...
+// hessianPool.Put(decoder)
+func NewCheapDecoderWithSkip(b []byte) *Decoder {
+ return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), isSkip: true}
+}
+
/////////////////////////////////////////
// utilities
/////////////////////////////////////////
+func (d *Decoder) Reset(b []byte) *Decoder {
+ // reuse reader buf, avoid allocate
+ d.reader.Reset(bytes.NewReader(b))
+ d.typeRefs = &TypeRefs{records: map[string]bool{}}
+
+ if d.refs != nil || d.classInfoList != nil {
+ d.refs, d.classInfoList = nil, nil
Review comment:
pls split it into two lines. we do not need such not easily understandable codes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] wongoo commented on pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
wongoo commented on pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#issuecomment-629615382
in fact, when decoding you can't know how many bytes you should read, until you analysis all bytes. A possible improvement may be you can first read a chunk length of bytes first, and then loop analysis and read more until reach the length of chunk. This may be useful to improve performance for most situation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] AlexStocks commented on pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
AlexStocks commented on pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#issuecomment-629609232
> I'll explore the byte code for the go emoji first, and there may be a way to fix it
glad to hear that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] gaoxinge commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
gaoxinge commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r425836986
##########
File path: string.go
##########
@@ -343,36 +342,139 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
+ remain, cap := len(bytesBuf)-offset, charLen<<2
+ if remain < cap {
+ grow := len(bytesBuf) + cap
+ bs := make([]byte, grow)
+ copy(bs, bytesBuf)
+ bytesBuf = bs
+ }
default:
return s, perrors.New("expect string tag")
}
}
- r, charLen, byteLen, err = decodeUcs4Rune(d.reader)
+ if chunkLen > 0 {
+ nread, err := d.next(bytesBuf[offset : offset+chunkLen])
+ if err != nil {
+ if err == io.EOF {
+ break
+ }
+ return s, perrors.WithStack(err)
+ }
+
+ // quickly detect the actual number of bytes
+ prev := offset
+ for i, len := offset, offset+nread; i < len; chunkLen-- {
Review comment:
看起来`offset`和`i`的作用差不多。可以把`i`移除掉么？
##########
File path: string.go
##########
@@ -343,36 +342,139 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
Review comment:
直接把`chunkLen`设置成0就行了，因为外面的判断是`chunkLen<=0`.
##########
File path: string.go
##########
@@ -343,36 +342,139 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+ if charLen < 0 {
Review comment:
`charLen`应该不会小于0吧？
##########
File path: string.go
##########
@@ -343,36 +342,139 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
+ remain, cap := len(bytesBuf)-offset, charLen<<2
+ if remain < cap {
+ grow := len(bytesBuf) + cap
+ bs := make([]byte, grow)
+ copy(bs, bytesBuf)
+ bytesBuf = bs
+ }
default:
return s, perrors.New("expect string tag")
}
}
- r, charLen, byteLen, err = decodeUcs4Rune(d.reader)
+ if chunkLen > 0 {
+ nread, err := d.next(bytesBuf[offset : offset+chunkLen])
+ if err != nil {
+ if err == io.EOF {
+ break
+ }
+ return s, perrors.WithStack(err)
+ }
+
+ // quickly detect the actual number of bytes
+ prev := offset
+ for i, len := offset, offset+nread; i < len; chunkLen-- {
+ ch := bytesBuf[i]
+ if ch < 0x80 {
+ offset++
+ i++
+ } else if (ch & 0xe0) == 0xc0 {
+ offset += 2
+ i += 2
+ } else if (ch & 0xf0) == 0xe0 {
+ offset += 3
+ i += 3
+ } else {
+ return s, perrors.Errorf("bad utf-8 encoding, offset=%d\n", offset)
+ }
+ }
+
+ if remain := offset - prev - nread; remain > 0 {
+ if remain == 1 {
+ ch, err := d.readByte()
+ if err != nil {
+ return s, perrors.WithStack(err)
+ }
+ bytesBuf[offset-1] = ch
+ } else {
+ var err error
+ if buffed := d.Buffered(); buffed < remain {
+ // trigger fill data if required
+ copy(bytesBuf[offset-remain:offset], d.peek(remain))
+ _, err = d.reader.Discard(remain)
+ } else {
+ // copy remaining bytes.
+ _, err = d.next(bytesBuf[offset-remain : offset])
+ }
+
+ if err != nil {
+ return s, perrors.WithStack(err)
+ }
+ }
+ }
+
+ // the expected length string has been processed.
+ if chunkLen <= 0 {
+ if last {
+ b := bytesBuf[:offset]
+ return *(*string)(unsafe.Pointer(&b)), nil
+ }
+
+ // we need to detect next chunk
+ continue
+ }
+ }
+
+ // decode byte
+ ch, err := d.readByte()
Review comment:
为啥最后要decode一下？上面的循环解析cover不掉什么corner case么？能解释一下么？
##########
File path: string.go
##########
@@ -343,36 +342,139 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
+ remain, cap := len(bytesBuf)-offset, charLen<<2
+ if remain < cap {
+ grow := len(bytesBuf) + cap
+ bs := make([]byte, grow)
+ copy(bs, bytesBuf)
+ bytesBuf = bs
+ }
default:
return s, perrors.New("expect string tag")
}
}
- r, charLen, byteLen, err = decodeUcs4Rune(d.reader)
+ if chunkLen > 0 {
+ nread, err := d.next(bytesBuf[offset : offset+chunkLen])
+ if err != nil {
+ if err == io.EOF {
+ break
+ }
+ return s, perrors.WithStack(err)
+ }
+
+ // quickly detect the actual number of bytes
+ prev := offset
+ for i, len := offset, offset+nread; i < len; chunkLen-- {
+ ch := bytesBuf[i]
+ if ch < 0x80 {
+ offset++
+ i++
+ } else if (ch & 0xe0) == 0xc0 {
+ offset += 2
+ i += 2
+ } else if (ch & 0xf0) == 0xe0 {
+ offset += 3
+ i += 3
+ } else {
+ return s, perrors.Errorf("bad utf-8 encoding, offset=%d\n", offset)
+ }
+ }
+
+ if remain := offset - prev - nread; remain > 0 {
+ if remain == 1 {
+ ch, err := d.readByte()
+ if err != nil {
+ return s, perrors.WithStack(err)
+ }
+ bytesBuf[offset-1] = ch
+ } else {
+ var err error
+ if buffed := d.Buffered(); buffed < remain {
+ // trigger fill data if required
+ copy(bytesBuf[offset-remain:offset], d.peek(remain))
+ _, err = d.reader.Discard(remain)
+ } else {
+ // copy remaining bytes.
+ _, err = d.next(bytesBuf[offset-remain : offset])
+ }
+
+ if err != nil {
+ return s, perrors.WithStack(err)
+ }
+ }
+ }
+
+ // the expected length string has been processed.
+ if chunkLen <= 0 {
+ if last {
Review comment:
这个`if`和`if`里面的内容可以去掉。因为去掉之后，下面`continue`也会跳到相同的[代码](https://github.com/apache/dubbo-go-hessian2/blob/84321abc9af13ff63470f685395e0502c706199b/string.go#L328)。
##########
File path: string.go
##########
@@ -343,36 +342,137 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
+
+ remain, cap := len(bytesBuf)-offset, charLen<<2
+ if remain < cap {
+ grow := len(bytesBuf) + cap
+ bs := make([]byte, grow)
+ copy(bs, bytesBuf)
+ bytesBuf = bs
+ }
default:
return s, perrors.New("expect string tag")
}
}
- r, charLen, byteLen, err = decodeUcs4Rune(d.reader)
+ if buffed := d.Buffered(); chunkLen > 0 {
+ nread, err := d.next(bytesBuf[offset : offset+chunkLen])
Review comment:
嗯，我懂了。
##########
File path: string.go
##########
@@ -343,36 +342,139 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
Review comment:
直接写`chunkLen=charLen`就行了。因为从上面的分析看，此时的`chunkLen`一定等于0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] gaoxinge commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
gaoxinge commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r425556871
##########
File path: string.go
##########
@@ -343,36 +342,137 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
+
+ remain, cap := len(bytesBuf)-offset, charLen<<2
+ if remain < cap {
+ grow := len(bytesBuf) + cap
+ bs := make([]byte, grow)
+ copy(bs, bytesBuf)
+ bytesBuf = bs
+ }
default:
return s, perrors.New("expect string tag")
}
}
- r, charLen, byteLen, err = decodeUcs4Rune(d.reader)
+ if buffed := d.Buffered(); chunkLen > 0 {
+ nread, err := d.next(bytesBuf[offset : offset+chunkLen])
Review comment:
我的理解：`chunkLen`是表示字符串的长度，不是表示字符串在对应编码下byte的长度。byte的长度应该比`chunkLen`大才对。这样不会有问题么？
##########
File path: decode_test.go
##########
@@ -126,3 +127,16 @@ func testDecodeFrameworkFunc(t *testing.T, method string, expected func(interfac
}
expected(r)
}
+
+func BenchmarkDecodeStringOptimized(t *testing.B) {
Review comment:
Place decode string benchmark to `string_test.go`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r425665213
##########
File path: decode.go
##########
@@ -50,15 +50,54 @@ func NewDecoder(b []byte) *Decoder {
return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), typeRefs: &TypeRefs{records: map[string]bool{}}}
}
+// NewDecoder generate a decoder instance
+func NewDecoderSize(b []byte, size int) *Decoder {
+ return &Decoder{reader: bufio.NewReaderSize(bytes.NewReader(b), size), typeRefs: &TypeRefs{records: map[string]bool{}}}
+}
+
// NewDecoder generate a decoder instance with skip
func NewDecoderWithSkip(b []byte) *Decoder {
return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), typeRefs: &TypeRefs{records: map[string]bool{}}, isSkip: true}
}
+// NewCheapDecoderWithSkip generate a decoder instance with skip,
+// only for cache pool, before decode Reset should be called.
+// For example, with pooling use, will effectively improve performance
+//
+// var hessianPool = &sync.Pool{
+// New: func() interface{} {
+// return hessian.NewCheapDecoderWithSkip([]byte{})
+// },
+// }
+//
+// decoder := hessianPool.Get().(*hessian.Decoder)
+// fill decode data
+// decoder.Reset(data[:])
+// decode anything ...
+// hessianPool.Put(decoder)
+func NewCheapDecoderWithSkip(b []byte) *Decoder {
+ return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), isSkip: true}
+}
+
/////////////////////////////////////////
// utilities
/////////////////////////////////////////
+func (d *Decoder) Reset(b []byte) *Decoder {
+ // reuse reader buf, avoid allocate
+ d.reader.Reset(bytes.NewReader(b))
+ d.typeRefs = &TypeRefs{records: map[string]bool{}}
+
+ if d.refs != nil {
Review comment:
if d.refs == nill or d.classInfoList == nil already, do nothing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] gaoxinge commented on a change in pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
gaoxinge commented on a change in pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#discussion_r425865759
##########
File path: string.go
##########
@@ -343,36 +342,137 @@ func (d *Decoder) decString(flag int32) (string, error) {
last = true
}
- l, err := d.getStringLength(b)
+ charLen, err = d.getStringLength(b)
if err != nil {
return s, perrors.WithStack(err)
}
- charTotal += l
- bs := make([]rune, charTotal)
- copy(bs, runeData)
- runeData = bs
+ if chunkLen < 0 {
+ chunkLen = 0
+ }
+
+ if charLen < 0 {
+ charLen = 0
+ }
+
+ chunkLen += charLen
+
+ remain, cap := len(bytesBuf)-offset, charLen<<2
+ if remain < cap {
+ grow := len(bytesBuf) + cap
+ bs := make([]byte, grow)
+ copy(bs, bytesBuf)
+ bytesBuf = bs
+ }
default:
return s, perrors.New("expect string tag")
}
}
- r, charLen, byteLen, err = decodeUcs4Rune(d.reader)
+ if buffed := d.Buffered(); chunkLen > 0 {
+ nread, err := d.next(bytesBuf[offset : offset+chunkLen])
Review comment:
嗯，我懂了。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go-hessian2] zonghaishang commented on pull request #188: Perf decode string
Posted by GitBox <gi...@apache.org>.
zonghaishang commented on pull request #188:
URL: https://github.com/apache/dubbo-go-hessian2/pull/188#issuecomment-629604271
> this PR will let the hessian only support at most three bytes length utf8, while four bytes mathematical symbols, and emoji will not supprted. even more the max length of a utf8 character can be six. And the worst is the length of string in hessian definition is the length of A 16-bit unicode character string encoded in UTF-8.
> ref
>
> * https://en.wikipedia.org/wiki/UTF-8
> * http://hessian.caucho.com/doc/hessian-serialization.html##string
> * #131
@wongoo
Any Suggestions for fixing it? I've tried parsing 4-byte utf-8 before, but Java doesn't write 4-byte utf-8
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org