You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by GitBox <gi...@apache.org> on 2021/02/02 17:53:34 UTC

[GitHub] [zeppelin] prabhjyotsingh opened a new pull request #4041: ZEPPELIN-5231: Livy Interpreter doesn't support Japanese Character - Encoding Issue

prabhjyotsingh opened a new pull request #4041:
URL: https://github.com/apache/zeppelin/pull/4041


   ### What is this PR for?
   Livy Interpreter doesn't support Japanese Character - Encoding Issue
   
   
   ### What type of PR is it?
   [Bug Fix]
   
   
   ### What is the Jira issue?
   * https://issues.apache.org/jira/browse/ZEPPELIN-5231
   
   ### How should this be tested?
   On a paragraph use below to generate data set:
   ```
   %livy.scala
   spark.sql("CREATE TABLE  test ( id int, name String, destination String)")
   spark.sql("INSERT INTO test VALUES ('1', 'みんく', 'みんく')")
   spark.sql("INSERT INTO test VALUES ('2', 'シュレヤン', 'シュレヤン')")
   spark.sql("INSERT INTO test VALUES ('3', 'サイ', 'サイ')")
   spark.sql("INSERT INTO test VALUES ('4', 'ツェッペリン', 'ツェッペリン')")
   spark.sql("INSERT INTO test VALUES ('5', 'テスト', 'テスト')")
   spark.sql("INSERT INTO test VALUES ('6', 'チャラン', 'チャラン')")
   spark.sql("select * from test").show()
   ```
   
   Then in a separate paragraph use this to verify:
   ```
   %livy.sql
   select * from test
   ```
   
   You would notice that on toggling (true/false) between the newly introduced `tableWithUTFCharacter` the same 2nd paragraph works.
   
   
   ### Screenshots (if appropriate)
   N/A
   
   ### Questions:
   * Does the licenses files need update?N/A
   * Is there breaking changes for older versions?N/A
   * Does this needs documentation?N/A
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] zjffdu commented on pull request #4041: ZEPPELIN-5231: Livy Interpreter doesn't support Japanese Character - Encoding Issue

Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4041:
URL: https://github.com/apache/zeppelin/pull/4041#issuecomment-775284261


   LGTM @prabhjyotsingh Is the CI fail related ?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] prabhjyotsingh commented on pull request #4041: ZEPPELIN-5231: Livy Interpreter doesn't support Japanese Character - Encoding Issue

Posted by GitBox <gi...@apache.org>.
prabhjyotsingh commented on pull request #4041:
URL: https://github.com/apache/zeppelin/pull/4041#issuecomment-774399506


   @zjffdu can you help review this.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] prabhjyotsingh commented on pull request #4041: ZEPPELIN-5231: Livy Interpreter doesn't support Japanese Character - Encoding Issue

Posted by GitBox <gi...@apache.org>.
prabhjyotsingh commented on pull request #4041:
URL: https://github.com/apache/zeppelin/pull/4041#issuecomment-775636206


   @zjffdu  Yes, I've looked at the failures it looks unrelated. It fails for `Neo4j interpreter` and one of `test-selenium-with-spark`. 
   
   Thank you for the review, will merge this if no more discussion.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] prabhjyotsingh commented on pull request #4041: ZEPPELIN-5231: Livy Interpreter doesn't support Japanese Character - Encoding Issue

Posted by GitBox <gi...@apache.org>.
prabhjyotsingh commented on pull request #4041:
URL: https://github.com/apache/zeppelin/pull/4041#issuecomment-773149708


   @zjffdu  sure that makes sense, have just added the same.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] asfgit closed pull request #4041: ZEPPELIN-5231: Livy Interpreter doesn't support Japanese Character - Encoding Issue

Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #4041:
URL: https://github.com/apache/zeppelin/pull/4041


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] prabhjyotsingh commented on pull request #4041: ZEPPELIN-5231: Livy Interpreter doesn't support Japanese Character - Encoding Issue

Posted by GitBox <gi...@apache.org>.
prabhjyotsingh commented on pull request #4041:
URL: https://github.com/apache/zeppelin/pull/4041#issuecomment-773149708


   @zjffdu  sure that makes sense, have just added the same.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] prabhjyotsingh commented on pull request #4041: ZEPPELIN-5231: Livy Interpreter doesn't support Japanese Character - Encoding Issue

Posted by GitBox <gi...@apache.org>.
prabhjyotsingh commented on pull request #4041:
URL: https://github.com/apache/zeppelin/pull/4041#issuecomment-771843607


   @zjffdu  can you help review this. 
   
   What I have done in this PR is added a new `tableWithUTFCharacter` in the interpreter setting, so that the old mechanism can still work, was not too sure if this JSON conversion will have any performance effect.
   Please let me know if there is a better way to do the same.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] zjffdu commented on pull request #4041: ZEPPELIN-5231: Livy Interpreter doesn't support Japanese Character - Encoding Issue

Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4041:
URL: https://github.com/apache/zeppelin/pull/4041#issuecomment-772180458


   @prabhjyotsingh Is it possible to add UT ?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] prabhjyotsingh commented on pull request #4041: ZEPPELIN-5231: Livy Interpreter doesn't support Japanese Character - Encoding Issue

Posted by GitBox <gi...@apache.org>.
prabhjyotsingh commented on pull request #4041:
URL: https://github.com/apache/zeppelin/pull/4041#issuecomment-771843607


   @zjffdu  can you help review this. 
   
   What I have done in this PR is added a new `tableWithUTFCharacter` in the interpreter setting, so that the old mechanism can still work, was not too sure if this JSON conversion will have any performance effect.
   Please let me know if there is a better way to do the same.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org