You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by qwemicheal <gi...@git.apache.org> on 2017/06/15 09:23:07 UTC

[GitHub] zeppelin pull request #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when s...

GitHub user qwemicheal opened a pull request:

    https://github.com/apache/zeppelin/pull/2412

    [ZEPPELIN-2641] Change encoding to UTF-8 when sending request to Livy

    ### What is this PR for?
    Change encoding of the request sent from Zeppelin to Livy to UTF-8. In this way, Zeppelin can support many more language than using ISO-8895-1 by default.
    
    
    ### What type of PR is it?
    Bug Fix 
    
    
    ### What is the Jira issue?
    [ZEPPELIN-2641](https://issues.apache.org/jira/browse/ZEPPELIN-2641)
    
    ### How should this be tested?
    Build from source.
    Open a Livy note book.
    Run some simple print command with Chinese or Korean, see whether the return can show the character correctly
    
    ### Screenshots (if appropriate)
    before 
    ![image](https://user-images.githubusercontent.com/14201792/27174528-11d45216-51ef-11e7-8f46-2f2e8347a3de.png)
    
    after
    ![image](https://user-images.githubusercontent.com/14201792/27174517-08cdba04-51ef-11e7-989c-88e516b2d265.png)
    
    ### Questions:
    * Does the licenses files need update? No
    * Is there breaking changes for older versions? No
    * Does this needs documentation? No


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/qwemicheal/zeppelin utf-8

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/zeppelin/pull/2412.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2412
    
----
commit f689a7e26dd79112b3669fe9a6c987384a219256
Author: 汪赫扬 <qw...@gmail.com>
Date:   2017-06-15T09:08:32Z

    add utf-8 encoding

commit 8f144bcda8d170746762572d37cb504b9353f0b4
Author: 汪赫扬 <qw...@gmail.com>
Date:   2017-06-15T09:09:37Z

    add unit test

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by big-loser <gi...@git.apache.org>.
Github user big-loser commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    @felixcheung 
    yes  。Livy is already the wrong character when returning information
    ![image](https://user-images.githubusercontent.com/10458694/37758793-739893d4-2dec-11e8-9224-7c1c969853c9.png)
    So I think it is a problem with the livy server and fix it. It worked
    
    @zjffdu 
    I am sure the branch used is 0.8 


---

[GitHub] zeppelin pull request #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when s...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/zeppelin/pull/2412


---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by qwemicheal <gi...@git.apache.org>.
Github user qwemicheal commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    @felixcheung It didn't fail. But when using byte to encode to  UTF-8 byte, CI log can show Chinese correctly.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by qwemicheal <gi...@git.apache.org>.
Github user qwemicheal commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    @zjffdu @felixcheung  After putting Chinese "你你你你你你好" directly inside test code, I got the following log in Travis CI test. May be use UTF8 byte to create inexplicit Chinese string  would be better? 
    
    DEBUG [2017-06-19 05:17:11,258] ({main} BaseLivyInterpreter.java[callRestAPI]:522) - Call rest api in http://testing-gce-d7875447-863b-4719-82c2-fb072d36b0b2.c.travis-ci-prod-5.internal:8998/sessions/3/statements, method: POST, jsonData: {
      "code": "print(\"ä½ ä½ ä½ ä½ ä½ ä½ å¥½\")"
    }
    DEBUG [2017-06-19 05:17:11,372] ({main} BaseLivyInterpreter.java[callRestAPI]:557) - Get response, StatusCode: 201, responseBody: {"id":1,"state":"running","output":null}
    DEBUG [2017-06-19 05:17:12,373] ({main} BaseLivyInterpreter.java[callRestAPI]:522) - Call rest api in http://testing-gce-d7875447-863b-4719-82c2-fb072d36b0b2.c.travis-ci-prod-5.internal:8998/sessions/3/statements/1, method: GET, jsonData: 
    DEBUG [2017-06-19 05:17:12,379] ({main} BaseLivyInterpreter.java[callRestAPI]:557) - Get response, StatusCode: 200, responseBody: {"id":1,"state":"available","output":{"status":"ok","execution_count":1,"data":{"text/plain":"ä½ ä½ ä½ ä½ ä½ ä½ å¥½"}}}


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by qwemicheal <gi...@git.apache.org>.
Github user qwemicheal commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    @zjffdu Update the PR to what state?  


---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    did it fail? livy seems to be responding with 2xx


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    @big-loser It works for me. I am using zeppelin master and livy 0.5, python3
    ![screen shot 2018-03-06 at 9 23 40 am](https://user-images.githubusercontent.com/164491/37009053-18b00840-2120-11e8-891a-29a28cd73d70.png)



---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by big-loser <gi...@git.apache.org>.
Github user big-loser commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    @felixcheung @zjffdu 



---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by qwemicheal <gi...@git.apache.org>.
Github user qwemicheal commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    @jasper-k 
    Zeppelin 0.7.2 was released on  Jun 12 and this pull got merged to master on Sep 28. Even Zeppelin  0.7.3 doesn't contain this pull since it was released on  Sep 21. So I think you'd have to rebuild from master.


---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    @qwemicheal sorry for late response, would you mind to update the PR ?


---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    Should this be fixed in Livy?
    
    



---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by big-loser <gi...@git.apache.org>.
Github user big-loser commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    sorry for the late reply


---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by jasper-k <gi...@git.apache.org>.
Github user jasper-k commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    Hi,
    @felixcheung @zjffdu @qwemicheal 
    
    I am on Zeppelin 0.7.2 (HDP-2.6.2.0-205) and I tried to port this path on my build. I replaced the jar at /zeppelin/interpreter/livy/zeppelin-livy-<version>.jar and restarted zeppelin.
    
    Still I can't reproduce the fix as this is the output I get:
    
    <img width="177" alt="screen shot 2017-12-06 at 2 13 20 pm" src="https://user-images.githubusercontent.com/6223443/33663304-b2561e08-da8f-11e7-8ba4-88e2d0c05e6d.png">
    
    So, is there any other thing I need to grab/rebuild from +Zeppelin 0.7.2 ?


---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    Sorry, I thought there was code conflict. I will merge it soon


---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by big-loser <gi...@git.apache.org>.
Github user big-loser commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    @zjffdu 
    I solved this problem and it is not actually a Zeppelin issue.
    The main problem is that the contentType in the livy service http header has no charset option.
    
    Livy file location:
    `Vim server/src/main/scala/org/apache/livy/server/SessionServlet.scala +67`
    
    `before() {
             // contentType = "application/json"
             contentType = "application/json;charset=UTF-8"
      }`
    
    rebuild livy server


---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    @big-loser Recently there's one ticket [ZEPPELIN-3099](https://issues.apache.org/jira/browse/ZEPPELIN-3099) fixed for German characters , could you try the latest branch-0.8 which has this fix. 


---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by big-loser <gi...@git.apache.org>.
Github user big-loser commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    @zjffdu 
    Thanks for your reply, our only difference is the python version. I checked again to confirm.


---

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

Posted by big-loser <gi...@git.apache.org>.
Github user big-loser commented on the issue:

    https://github.com/apache/zeppelin/pull/2412
  
    @qwemicheal @zjffdu 
    
    Today I clone the main branch and build it successfully.
    But I use the livy interpreter, still can not normally display or output Chinese.
    
    How do I make Chinese normal display?
    
    ![test](https://user-images.githubusercontent.com/10458694/36977851-91760864-20bd-11e8-8e89-300ec8b7893a.PNG)



---