You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by GitBox <gi...@apache.org> on 2021/01/26 08:29:02 UTC

[GitHub] [kylin] liusheng opened a new pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

liusheng opened a new pull request #1558:
URL: https://github.com/apache/kylin/pull/1558


   ## Proposed changes
   
   This change modify the Travis CI configurations to enable the Kylin testing on ARM64 server.
   
   ## Types of changes
   
   What types of changes does your code introduce to Kylin?
   _Put an `x` in the boxes that apply_
   
   - [ ] Bugfix (non-breaking change which fixes an issue)
   - [x] New feature (non-breaking change which adds functionality)
   - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
   - [ ] Documentation Update (if none of the other choices apply)
   
   ## Checklist
   
   _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._
   
   - [x] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail
   - [x] Commit messages in my PR start with the related jira ID, like "KYLIN-0000 Make Kylin project open-source"
   - [x] Compiling and unit tests pass locally with my changes
   - [x] I have added tests that prove my fix is effective or that my feature works
   - [x] If this change need a document change, I will prepare another pr against the `document` branch
   - [x] Any dependent changes have been merged
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc...
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] coveralls edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-767505134


   ## Pull Request Test Coverage Report for [Build 6970](https://coveralls.io/builds/37922258)
   
   * **0** of **0**   changed or added relevant lines in **0** files are covered.
   * **8** unchanged lines in **4** files lost coverage.
   * Overall coverage decreased (**-0.002%**) to **28.174%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-----|--------------|--: |
   | [tool/src/main/java/org/apache/kylin/tool/query/ProbabilityGenerator.java](https://coveralls.io/builds/37922258/source?filename=tool%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Ftool%2Fquery%2FProbabilityGenerator.java#L44) | 1 | 81.58% |
   | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/37922258/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% |
   | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/37922258/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L193) | 2 | 79.07% |
   | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/37922258/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L449) | 3 | 78.42% |
   <!-- | **Total:** | **8** |  | -->
   
   |  Totals | [![Coverage Status](https://coveralls.io/builds/37922258/badge)](https://coveralls.io/builds/37922258) |
   | :-- | --: |
   | Change from base [Build 6963](https://coveralls.io/builds/37861910): |  -0.002% |
   | Covered Lines: | 26708 |
   | Relevant Lines: | 94798 |
   
   ---
   ##### 💛  - [Coveralls](https://coveralls.io)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] hit-lacus edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
hit-lacus edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-809967113


   Hi, from the issue title [Support building and testing Kylin on ARM64 architecture platform](https://issues.apache.org/jira/browse/KYLIN-4864), is aims to support build and run Kylin on ARM64 architecture platform.
   
   Does this patch ensure user can build new binary and run smoothly on ARM64 platfrom? Could you provided a test report for verification ?
   
   Besides, I was concerned by if this upgrade will break current functions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] codecov-io edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-793472223


   # [Codecov](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=h1) Report
   > Merging [#1558](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=desc) (57546e1) into [master](https://codecov.io/gh/apache/kylin/commit/02500818d85955b1125340eaa19ab88a634a5181?el=desc) (0250081) will **decrease** coverage by `0.00%`.
   > The diff coverage is `18.18%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/kylin/pull/1558/graphs/tree.svg?width=650&height=150&src=pr&token=JawVgbgsVo)](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1558      +/-   ##
   ============================================
   - Coverage     25.42%   25.42%   -0.01%     
   + Complexity     6765     6762       -3     
   ============================================
     Files          1508     1508              
     Lines         93918    93924       +6     
     Branches      13158    13160       +2     
   ============================================
   - Hits          23877    23876       -1     
   - Misses        67662    67667       +5     
   - Partials       2379     2381       +2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.46% <0.00%> (ø)` | `51.00 <0.00> (ø)` | |
   | [...che/kylin/storage/gtrecord/CubeTupleConverter.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-Y29yZS1zdG9yYWdlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdG9yYWdlL2d0cmVjb3JkL0N1YmVUdXBsZUNvbnZlcnRlci5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...e/kylin/rest/controller/StreamingV2Controller.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9TdHJlYW1pbmdWMkNvbnRyb2xsZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...apache/kylin/rest/metrics/QueryMetrics2Facade.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvbWV0cmljcy9RdWVyeU1ldHJpY3MyRmFjYWRlLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [.../apache/kylin/rest/metrics/QueryMetricsFacade.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvbWV0cmljcy9RdWVyeU1ldHJpY3NGYWNhZGUuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...apache/kylin/storage/hbase/HBaseResourceStore.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9IQmFzZVJlc291cmNlU3RvcmUuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...lin/stream/core/query/StreamingTupleConverter.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3F1ZXJ5L1N0cmVhbWluZ1R1cGxlQ29udmVydGVyLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...lin/stream/source/kafka/TimedJsonStreamParser.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c3RyZWFtLXNvdXJjZS1rYWZrYS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RyZWFtL3NvdXJjZS9rYWZrYS9UaW1lZEpzb25TdHJlYW1QYXJzZXIuamF2YQ==) | `67.39% <0.00%> (ø)` | `12.00 <0.00> (ø)` | |
   | [...stream/core/query/MultiThreadsResultCollector.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3F1ZXJ5L011bHRpVGhyZWFkc1Jlc3VsdENvbGxlY3Rvci5qYXZh) | `63.91% <66.66%> (+0.37%)` | `4.00 <0.00> (ø)` | |
   | [...in/java/org/apache/kylin/query/util/QueryUtil.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-cXVlcnkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3F1ZXJ5L3V0aWwvUXVlcnlVdGlsLmphdmE=) | `60.00% <100.00%> (-0.40%)` | `25.00 <1.00> (-1.00)` | |
   | ... and [5 more](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=footer). Last update [448d38a...57546e1](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] codecov-io commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-793472223


   # [Codecov](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=h1) Report
   > Merging [#1558](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=desc) (9145147) into [master](https://codecov.io/gh/apache/kylin/commit/02500818d85955b1125340eaa19ab88a634a5181?el=desc) (0250081) will **decrease** coverage by `0.00%`.
   > The diff coverage is `7.69%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/kylin/pull/1558/graphs/tree.svg?width=650&height=150&src=pr&token=JawVgbgsVo)](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1558      +/-   ##
   ============================================
   - Coverage     25.42%   25.41%   -0.01%     
   + Complexity     6765     6762       -3     
   ============================================
     Files          1508     1508              
     Lines         93918    93923       +5     
     Branches      13158    13160       +2     
   ============================================
   - Hits          23877    23873       -4     
   - Misses        67662    67668       +6     
   - Partials       2379     2382       +3     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.46% <0.00%> (ø)` | `51.00 <0.00> (ø)` | |
   | [...che/kylin/storage/gtrecord/CubeTupleConverter.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-Y29yZS1zdG9yYWdlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdG9yYWdlL2d0cmVjb3JkL0N1YmVUdXBsZUNvbnZlcnRlci5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...e/kylin/rest/controller/StreamingV2Controller.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9TdHJlYW1pbmdWMkNvbnRyb2xsZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...apache/kylin/storage/hbase/HBaseResourceStore.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9IQmFzZVJlc291cmNlU3RvcmUuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...lin/stream/core/query/StreamingTupleConverter.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3F1ZXJ5L1N0cmVhbWluZ1R1cGxlQ29udmVydGVyLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...lin/stream/source/kafka/TimedJsonStreamParser.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c3RyZWFtLXNvdXJjZS1rYWZrYS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RyZWFtL3NvdXJjZS9rYWZrYS9UaW1lZEpzb25TdHJlYW1QYXJzZXIuamF2YQ==) | `67.39% <0.00%> (ø)` | `12.00 <0.00> (ø)` | |
   | [...stream/core/query/MultiThreadsResultCollector.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3F1ZXJ5L011bHRpVGhyZWFkc1Jlc3VsdENvbGxlY3Rvci5qYXZh) | `63.91% <66.66%> (+0.37%)` | `4.00 <0.00> (ø)` | |
   | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `72.97% <0.00%> (-5.41%)` | `6.00% <0.00%> (-1.00%)` | |
   | [.../apache/kylin/tool/query/ProbabilityGenerator.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-dG9vbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vdG9vbC9xdWVyeS9Qcm9iYWJpbGl0eUdlbmVyYXRvci5qYXZh) | `71.05% <0.00%> (-5.27%)` | `11.00% <0.00%> (ø%)` | |
   | [...va/org/apache/kylin/tool/query/QueryGenerator.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-dG9vbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vdG9vbC9xdWVyeS9RdWVyeUdlbmVyYXRvci5qYXZh) | `80.00% <0.00%> (-1.12%)` | `22.00% <0.00%> (-2.00%)` | |
   | ... and [2 more](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=footer). Last update [3f3f114...9145147](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] liusheng commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
liusheng commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-887442592


   Thanks for @martin-g  help and @hit-lacus review!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kylin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-874054281


   @hit-lacus Ping! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kylin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] liusheng commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
liusheng commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-809999405


   Hi @hit-lacus ,
   Thanks for your reply. yes, we are aim to promoting some popular open source projects support running on ARM platform. for Kylin, I have verified the building, and runing all tests(seems also includes integration tests), if needed, I will verified the deployment and basic functionalities of Kylin on ARM64 server ans post testing reports.
   
   For the upgrade of rocksdbjni from version `5.9.2`~`5.18.4`, I wil also try to check if there is side effect,  the Storm, Kafka also have upgraded the rocksdbjni version to `5.18.4` to support multiple platforms.
   
   [1] https://issues.apache.org/jira/browse/STORM-3599
   [2] https://github.com/apache/kafka/pull/8284


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] zhangayqian removed a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
zhangayqian removed a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-799043126


   > Hi @zhangayqian @zzcclp Sorry to bother you, could you please help to take a look this PR ?
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] coveralls edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-767505134


   ## Pull Request Test Coverage Report for [Build 6950](https://coveralls.io/builds/37767674)
   
   * **0** of **0**   changed or added relevant lines in **0** files are covered.
   * **5** unchanged lines in **1** file lost coverage.
   * Overall coverage increased (+**0.002%**) to **27.958%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-----|--------------|--: |
   | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/37767674/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439) | 5 | 77.81% |
   <!-- | **Total:** | **5** |  | -->
   
   |  Totals | [![Coverage Status](https://coveralls.io/builds/37767674/badge)](https://coveralls.io/builds/37767674) |
   | :-- | --: |
   | Change from base [Build 6948](https://coveralls.io/builds/37745441): |  0.002% |
   | Covered Lines: | 26259 |
   | Relevant Lines: | 93923 |
   
   ---
   ##### 💛  - [Coveralls](https://coveralls.io)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] coveralls commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-767505134


   ## Pull Request Test Coverage Report for [Build 6829](https://coveralls.io/builds/36600118)
   
   * **0** of **0**   changed or added relevant lines in **0** files are covered.
   * **12** unchanged lines in **4** files lost coverage.
   * Overall coverage decreased (**-0.01%**) to **27.948%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-----|--------------|--: |
   | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/36600118/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% |
   | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/36600118/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% |
   | [tool/src/main/java/org/apache/kylin/tool/query/ProbabilityGenerator.java](https://coveralls.io/builds/36600118/source?filename=tool%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Ftool%2Fquery%2FProbabilityGenerator.java#L44) | 2 | 78.95% |
   | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/36600118/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L193) | 7 | 76.29% |
   <!-- | **Total:** | **12** |  | -->
   
   |  Totals | [![Coverage Status](https://coveralls.io/builds/36600118/badge)](https://coveralls.io/builds/36600118) |
   | :-- | --: |
   | Change from base [Build 6729](https://coveralls.io/builds/36067968): |  -0.01% |
   | Covered Lines: | 26248 |
   | Relevant Lines: | 93918 |
   
   ---
   ##### 💛  - [Coveralls](https://coveralls.io)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] hit-lacus edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
hit-lacus edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-810043356


   > seems also includes integration tests
   
   Currently,  integration test need a **HDP 2.4** sandbox for Hadoop setup. Please check `http://kylin.apache.org/development/howto_test.html` for the detail of how to run integration test.
   
   Besides, I wish you can test deployment and basic functionalities of Kylin on ARM64 server, and share testing reports to us. Sadly to say, I do not know too much on ARM64 technical and do not have a ARM64 server for verification.
   
   For how to run manual test, please check this link https://cwiki.apache.org/confluence/display/KYLIN/Regression+Testing+for+Kylin+Release .
   
   
   
    
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] liusheng commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
liusheng commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-809944119


   Hi @shaofengshi , yes, I previously made the version upgrade for rocksdbjni in https://github.com/apache/kylin/pull/1557,  now I have merged the small change in this PR,  I just updated the reason of this upgrade in this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] hit-lacus commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
hit-lacus commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-810037184


   One of your commit 's commit message is "Use openjdk11 for ARM64 test", did your patch use openjdk11 for test? I didn't find it.
   
   <img width="318" alt="image" src="https://user-images.githubusercontent.com/14030549/112960632-2e5e4d80-9177-11eb-9691-0c498b6e2af6.png">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] liusheng commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
liusheng commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-799037266


   Hi @zhangayqian @zzcclp Sorry to bother you, could you please help to take a look this PR ?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-958724194


   The changes introduced with this PR have been lost! There is no `arm64` in https://github.com/apache/kylin/blob/main/.travis.yml and its git history.
   
   @hit-lacus Were there any Git filtering operations on the repo ? I see that `master` branch has been renamed to `main` but I doubt this is the reason.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kylin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] hit-lacus commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
hit-lacus commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-810043356


   > seems also includes integration tests
   Currently,  integration test need a **HDP 2.4** sandbox for Hadoop setup. Please check `http://kylin.apache.org/development/howto_test.html`. for how to run integration test.
   
   Besides, I wish you can test deployment and basic functionalities of Kylin on ARM64 server, and share testing reports to us. Sad to say, I do not know too much on ARM64 technical and do not have a ARM64 server for test.
   
   For how to run manual test, please check this link https://cwiki.apache.org/confluence/display/KYLIN/Regression+Testing+for+Kylin+Release .
   
   
   
    
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g commented on a change in pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g commented on a change in pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#discussion_r603887136



##########
File path: .travis.yml
##########
@@ -40,6 +46,7 @@ before_script:
   - echo "MAVEN_OPTS='-Xms1024m -Xmx3072m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m'" > ~/.mavenrc
   - sed -i 's/log4j.logger.org.apache.kylin=INFO/log4j.logger.org.apache.kylin=WARN/g' build/conf/kylin-server-log4j.properties
   - sed -i 's/log4j.logger.org.apache.kylin=INFO/log4j.logger.org.apache.kylin=WARN/g' build/conf/kylin-tools-log4j.properties
+  - if [[ $(uname -m) == 'aarch64' ]];then export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-arm64";fi

Review comment:
       This line could be simplified to:
   `export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-${TRAVIS_CPU_ARCH}"`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] zhangayqian closed pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
zhangayqian closed pull request #1558:
URL: https://github.com/apache/kylin/pull/1558


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] coveralls edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-767505134


   ## Pull Request Test Coverage Report for [Build 6966](https://coveralls.io/builds/37872421)
   
   * **0** of **0**   changed or added relevant lines in **0** files are covered.
   * **4** unchanged lines in **2** files lost coverage.
   * Overall coverage decreased (**-0.004%**) to **28.171%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-----|--------------|--: |
   | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/37872421/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% |
   | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/CheckPointStore.java](https://coveralls.io/builds/37872421/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FCheckPointStore.java#L98) | 2 | 73.74% |
   <!-- | **Total:** | **4** |  | -->
   
   |  Totals | [![Coverage Status](https://coveralls.io/builds/37872421/badge)](https://coveralls.io/builds/37872421) |
   | :-- | --: |
   | Change from base [Build 6963](https://coveralls.io/builds/37861910): |  -0.004% |
   | Covered Lines: | 26706 |
   | Relevant Lines: | 94798 |
   
   ---
   ##### 💛  - [Coveralls](https://coveralls.io)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-865898669


   @hit-lacus I was able to run the Docker image on Linux ARM64 with these changes: https://github.com/apache/kylin/compare/master...martin-g:KYLIN-4864-add-support-for-aarch64-to-Dockerfiles?expand=1
   
   To build and run it I use:
   
   ```
   #!/usr/bin/env bash
   
   
   if test "$1" = "rebuild"; then
     echo -e "\n\n-- Building the Hadoop image ... \n\n"
     docker build --build-arg BASE_IMAGE=arm64v8/centos:7.9.2009 --build-arg ARCH=aarch64 --build-arg JDK_ARCH=aarch64 -t kylin-hadoop:2.7 -f Dockerfile_hadoop .
   
     echo -e "\n\n-- Building the Kylin image ... \n\n"
     docker build --build-arg BASE_IMAGE=kylin-hadoop:2.7 -t kylin:2.7 -f Dockerfile .
   fi
   
   
   echo -e "\n\n-- Starting Kylin ... \n\n"
   docker run --rm -it --name kylin \
     --memory=8G \
     -p 7070:7070 \
     -p 8088:8088 \
     -p 50070:50070 \
     -p 8032:8032 \
     -p 8042:8042 \
     -p 16010:16010 \
     --user root \
     kylin:2.7
   
   ```
   
   I had to upgrade MySQL to 8.x because there was no aarch64 binaries at http://repo.mysql.com/yum/mysql-5.7-community/el/7/
   
   I've also updated Kylin, Livy, JDK and Maven to their latest versions.
   
   I've tested it with the steps described at http://kylin.apache.org/docs/tutorial/kylin_sample.html but it got stuck at 90.91% . The logs just repeat the following again and again:
   
   ```
   ==> logs/kylin.log <==
   2021-06-22 11:12:42,935 INFO  [BadQueryDetector] service.BadQueryDetector:148 : Detect bad query.
   2021-06-22 11:12:45,832 INFO  [FetcherRunner 1102308739-43] threadpool.DefaultFetcherRunner:117 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 0 already succeed, 0 error, 0 discarded, 0 others
   
   ==> logs/streaming_coordinator.log <==
   2021-06-22 11:12:45,832 INFO  [FetcherRunner 1102308739-43] threadpool.DefaultFetcherRunner:117 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 0 already succeed, 0 error, 0 discarded, 0 others
   
   ==> logs/kylin.log <==
   2021-06-22 11:13:15,832 INFO  [FetcherRunner 1102308739-43] threadpool.DefaultFetcherRunner:117 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 0 already succeed, 0 error, 0 discarded, 0 others
   
   ==> logs/streaming_coordinator.log <==
   2021-06-22 11:13:15,832 INFO  [FetcherRunner 1102308739-43] threadpool.DefaultFetcherRunner:117 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 0 already succeed, 0 error, 0 discarded, 0 others
   
   ==> logs/kylin.log <==
   ```
   I did not face any problems related to RocksDB ! Only the unit tests fail with 5.9.2 and pass with 5.18.4!
   
   
   Now I am trying to test with 3.1.3-SNAPSHOT but the build fails at `kylin-it` module due to :
   
   ```
   java.lang.IllegalArgumentException: java.net.UnknownHostException: sandbox.hortonworks.com
   	at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:378)
   	at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310)
   	at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
   	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678)
   	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
   	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
   	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
   	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
   	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
   	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:355)
   	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
   	at org.apache.kylin.common.KylinConfigBase.getHdfsWorkingDirectory(KylinConfigBase.java:284)
   	at org.apache.kylin.dict.ITGlobalDictionaryBuilderTest$SharedBuilderThread.run(ITGlobalDictionaryBuilderTest.java:133)
   ```
   
   I cannot find what build step is supposed to build the `**-bin.tar.gz` for https://github.com/apache/kylin/blob/efedf07043c704cbdfa04ac3af3d73cec866cb76/docker/Dockerfile_dev#L25
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] coveralls edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-767505134


   ## Pull Request Test Coverage Report for [Build 6939](https://coveralls.io/builds/37678693)
   
   * **0** of **0**   changed or added relevant lines in **0** files are covered.
   * **5** unchanged lines in **2** files lost coverage.
   * Overall coverage increased (+**0.001%**) to **27.956%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-----|--------------|--: |
   | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/37678693/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L193) | 2 | 79.07% |
   | [tool/src/main/java/org/apache/kylin/tool/query/ProbabilityGenerator.java](https://coveralls.io/builds/37678693/source?filename=tool%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Ftool%2Fquery%2FProbabilityGenerator.java#L42) | 3 | 76.32% |
   <!-- | **Total:** | **5** |  | -->
   
   |  Totals | [![Coverage Status](https://coveralls.io/builds/37678693/badge)](https://coveralls.io/builds/37678693) |
   | :-- | --: |
   | Change from base [Build 6921](https://coveralls.io/builds/37585214): |  0.001% |
   | Covered Lines: | 26257 |
   | Relevant Lines: | 93921 |
   
   ---
   ##### 💛  - [Coveralls](https://coveralls.io)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] liusheng commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
liusheng commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-803842002


   Hi @hit-lacus @zhangayqian Could you please help to review this PR? Thank you.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] hit-lacus commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
hit-lacus commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-809967113


   Hi, from the issue title [Support building and testing Kylin on ARM64 architecture platform](https://issues.apache.org/jira/browse/KYLIN-4864), is aims to support build and run Kylin on ARM64 architecture platform.
   
   Does this patch ensure user can build new binary and run smoothly on ARM64 platfrom? Could you provided a test report for verification ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] shaofengshi commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
shaofengshi commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-809933148


   hi sheng, I see it includes a version upgrade for rocksdb; it is a must-have for this JIRA? @liusheng 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] coveralls edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-767505134


   ## Pull Request Test Coverage Report for [Build 6940](https://coveralls.io/builds/37679248)
   
   * **0** of **0**   changed or added relevant lines in **0** files are covered.
   * **8** unchanged lines in **2** files lost coverage.
   * Overall coverage decreased (**-0.004%**) to **27.951%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-----|--------------|--: |
   | [tool/src/main/java/org/apache/kylin/tool/query/ProbabilityGenerator.java](https://coveralls.io/builds/37679248/source?filename=tool%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Ftool%2Fquery%2FProbabilityGenerator.java#L42) | 1 | 81.58% |
   | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/37679248/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L193) | 7 | 76.29% |
   <!-- | **Total:** | **8** |  | -->
   
   |  Totals | [![Coverage Status](https://coveralls.io/builds/37679248/badge)](https://coveralls.io/builds/37679248) |
   | :-- | --: |
   | Change from base [Build 6921](https://coveralls.io/builds/37585214): |  -0.004% |
   | Covered Lines: | 26252 |
   | Relevant Lines: | 93921 |
   
   ---
   ##### 💛  - [Coveralls](https://coveralls.io)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-861401928


   @hit-lacus It seems Hortonworks resources are no more available. Do you have a Howto with a replacement of HDP ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] coveralls edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-767505134


   ## Pull Request Test Coverage Report for [Build 6950](https://coveralls.io/builds/37767674)
   
   * **0** of **0**   changed or added relevant lines in **0** files are covered.
   * No unchanged relevant lines lost coverage.
   * Overall coverage increased (+**0.002%**) to **27.958%**
   
   ---
   
   
   
   |  Totals | [![Coverage Status](https://coveralls.io/builds/37767674/badge)](https://coveralls.io/builds/37767674) |
   | :-- | --: |
   | Change from base [Build 6948](https://coveralls.io/builds/37745441): |  0.002% |
   | Covered Lines: | 26259 |
   | Relevant Lines: | 93923 |
   
   ---
   ##### 💛  - [Coveralls](https://coveralls.io)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] coveralls edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-767505134


   ## Pull Request Test Coverage Report for [Build 6970](https://coveralls.io/builds/37922258)
   
   * **0** of **0**   changed or added relevant lines in **0** files are covered.
   * **3** unchanged lines in **2** files lost coverage.
   * Overall coverage decreased (**-0.002%**) to **28.174%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-----|--------------|--: |
   | [tool/src/main/java/org/apache/kylin/tool/query/ProbabilityGenerator.java](https://coveralls.io/builds/37922258/source?filename=tool%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Ftool%2Fquery%2FProbabilityGenerator.java#L44) | 1 | 81.58% |
   | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/37922258/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% |
   <!-- | **Total:** | **3** |  | -->
   
   |  Totals | [![Coverage Status](https://coveralls.io/builds/37922258/badge)](https://coveralls.io/builds/37922258) |
   | :-- | --: |
   | Change from base [Build 6963](https://coveralls.io/builds/37861910): |  -0.002% |
   | Covered Lines: | 26708 |
   | Relevant Lines: | 94798 |
   
   ---
   ##### 💛  - [Coveralls](https://coveralls.io)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] coveralls edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-767505134


   ## Pull Request Test Coverage Report for [Build 6966](https://coveralls.io/builds/37872421)
   
   * **0** of **0**   changed or added relevant lines in **0** files are covered.
   * No unchanged relevant lines lost coverage.
   * Overall coverage decreased (**-0.004%**) to **28.171%**
   
   ---
   
   
   
   |  Totals | [![Coverage Status](https://coveralls.io/builds/37872421/badge)](https://coveralls.io/builds/37872421) |
   | :-- | --: |
   | Change from base [Build 6963](https://coveralls.io/builds/37861910): |  -0.004% |
   | Covered Lines: | 26706 |
   | Relevant Lines: | 94798 |
   
   ---
   ##### 💛  - [Coveralls](https://coveralls.io)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] coveralls edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-767505134


   ## Pull Request Test Coverage Report for [Build 6942](https://coveralls.io/builds/37680054)
   
   * **0** of **0**   changed or added relevant lines in **0** files are covered.
   * **7** unchanged lines in **2** files lost coverage.
   * Overall coverage increased (+**0.001%**) to **27.956%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-----|--------------|--: |
   | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/37680054/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L193) | 2 | 79.07% |
   | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/37680054/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439) | 5 | 77.81% |
   <!-- | **Total:** | **7** |  | -->
   
   |  Totals | [![Coverage Status](https://coveralls.io/builds/37680054/badge)](https://coveralls.io/builds/37680054) |
   | :-- | --: |
   | Change from base [Build 6921](https://coveralls.io/builds/37585214): |  0.001% |
   | Covered Lines: | 26257 |
   | Relevant Lines: | 93921 |
   
   ---
   ##### 💛  - [Coveralls](https://coveralls.io)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] codecov-io edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-793472223


   # [Codecov](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=h1) Report
   > Merging [#1558](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=desc) (57546e1) into [master](https://codecov.io/gh/apache/kylin/commit/02500818d85955b1125340eaa19ab88a634a5181?el=desc) (0250081) will **decrease** coverage by `0.01%`.
   > The diff coverage is `18.18%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/kylin/pull/1558/graphs/tree.svg?width=650&height=150&src=pr&token=JawVgbgsVo)](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1558      +/-   ##
   ============================================
   - Coverage     25.42%   25.40%   -0.02%     
   + Complexity     6765     6761       -4     
   ============================================
     Files          1508     1508              
     Lines         93918    93924       +6     
     Branches      13158    13160       +2     
   ============================================
   - Hits          23877    23866      -11     
   - Misses        67662    67674      +12     
   - Partials       2379     2384       +5     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.46% <0.00%> (ø)` | `51.00 <0.00> (ø)` | |
   | [...che/kylin/storage/gtrecord/CubeTupleConverter.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-Y29yZS1zdG9yYWdlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdG9yYWdlL2d0cmVjb3JkL0N1YmVUdXBsZUNvbnZlcnRlci5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...e/kylin/rest/controller/StreamingV2Controller.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9TdHJlYW1pbmdWMkNvbnRyb2xsZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...apache/kylin/rest/metrics/QueryMetrics2Facade.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvbWV0cmljcy9RdWVyeU1ldHJpY3MyRmFjYWRlLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [.../apache/kylin/rest/metrics/QueryMetricsFacade.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvbWV0cmljcy9RdWVyeU1ldHJpY3NGYWNhZGUuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...apache/kylin/storage/hbase/HBaseResourceStore.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9IQmFzZVJlc291cmNlU3RvcmUuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...lin/stream/core/query/StreamingTupleConverter.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3F1ZXJ5L1N0cmVhbWluZ1R1cGxlQ29udmVydGVyLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...lin/stream/source/kafka/TimedJsonStreamParser.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c3RyZWFtLXNvdXJjZS1rYWZrYS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RyZWFtL3NvdXJjZS9rYWZrYS9UaW1lZEpzb25TdHJlYW1QYXJzZXIuamF2YQ==) | `67.39% <0.00%> (ø)` | `12.00 <0.00> (ø)` | |
   | [...stream/core/query/MultiThreadsResultCollector.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3F1ZXJ5L011bHRpVGhyZWFkc1Jlc3VsdENvbGxlY3Rvci5qYXZh) | `63.91% <66.66%> (+0.37%)` | `4.00 <0.00> (ø)` | |
   | [...in/java/org/apache/kylin/query/util/QueryUtil.java](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree#diff-cXVlcnkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3F1ZXJ5L3V0aWwvUXVlcnlVdGlsLmphdmE=) | `60.00% <100.00%> (-0.40%)` | `25.00 <1.00> (-1.00)` | |
   | ... and [5 more](https://codecov.io/gh/apache/kylin/pull/1558/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=footer). Last update [448d38a...57546e1](https://codecov.io/gh/apache/kylin/pull/1558?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-865972469


   I've created `apache-kylin-3.1.3-SNAPSHOT-bin.tar.gz` by using `./build/*` and `./tool/target/kylin-tool-3.1.3-SNAPSHOT.jar`. But when starting Kylin it fails with `Please set kylin.env.hdfs-working-dir in kylin.properties`. Neither `Dockerfile_dev` nor `Dockerfile` sets `kylin.env.hdfs-working-dir`. What should be its recommended value ?
   
   I've added `kylin.env.hdfs-working-dir=hdfs://localhost:9000/kylin/work` to `conf/kylin.properties` but it still fails the same way. I've doubled checked by `docker exec .. bash` and `cat apache-kylin**/conf/kylin.properties` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-958724194


   The changes introduced with this PR have been lost! There is no `arm64` in https://github.com/apache/kylin/blob/main/.travis.yml and its git history.
   
   @hit-lacus Were there any Git filtering operations on the repo ? I see that `master` branch has been renamed to `main` but I doubt this is the reason.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kylin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-958724194


   The changes introduced with this PR have been lost! There is no `arm64` in https://github.com/apache/kylin/blob/main/.travis.yml and its git history.
   
   @hit-lacus Were there any Git filtering operations on the repo ? I see that `master` branch has been renamed to `main` but I doubt this is the reason.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kylin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] liusheng commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
liusheng commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-768100583


   let's wait for https://github.com/apache/kylin/pull/1557 merged firstly


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-865898669


   @hit-lacus I was able to run the Docker image on Linux ARM64 with these changes: https://github.com/apache/kylin/compare/master...martin-g:KYLIN-4864-add-support-for-aarch64-to-Dockerfiles?expand=1
   
   I had to upgrade MySQL to 8.x because there was no aarch64 binaries at http://repo.mysql.com/yum/mysql-5.7-community/el/7/
   
   I've also updated Kylin, Livy, JDK and Maven to their latest versions.
   
   I've tested it with the steps described at http://kylin.apache.org/docs/tutorial/kylin_sample.html but it got stuck at 90.91% . The logs just repeat the following again and again:
   
   ```
   ==> logs/kylin.log <==
   2021-06-22 11:12:42,935 INFO  [BadQueryDetector] service.BadQueryDetector:148 : Detect bad query.
   2021-06-22 11:12:45,832 INFO  [FetcherRunner 1102308739-43] threadpool.DefaultFetcherRunner:117 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 0 already succeed, 0 error, 0 discarded, 0 others
   
   ==> logs/streaming_coordinator.log <==
   2021-06-22 11:12:45,832 INFO  [FetcherRunner 1102308739-43] threadpool.DefaultFetcherRunner:117 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 0 already succeed, 0 error, 0 discarded, 0 others
   
   ==> logs/kylin.log <==
   2021-06-22 11:13:15,832 INFO  [FetcherRunner 1102308739-43] threadpool.DefaultFetcherRunner:117 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 0 already succeed, 0 error, 0 discarded, 0 others
   
   ==> logs/streaming_coordinator.log <==
   2021-06-22 11:13:15,832 INFO  [FetcherRunner 1102308739-43] threadpool.DefaultFetcherRunner:117 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 0 already succeed, 0 error, 0 discarded, 0 others
   
   ==> logs/kylin.log <==
   ```
   I did not face any problems related to RocksDB ! Only the unit tests fail with 5.9.2 and pass with 5.18.4!
   
   
   Now I am trying to test with 3.1.3-SNAPSHOT but the build fails at `kylin-it` module due to :
   
   ```
   java.lang.IllegalArgumentException: java.net.UnknownHostException: sandbox.hortonworks.com
   	at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:378)
   	at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310)
   	at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
   	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678)
   	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
   	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
   	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
   	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
   	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
   	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:355)
   	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
   	at org.apache.kylin.common.KylinConfigBase.getHdfsWorkingDirectory(KylinConfigBase.java:284)
   	at org.apache.kylin.dict.ITGlobalDictionaryBuilderTest$SharedBuilderThread.run(ITGlobalDictionaryBuilderTest.java:133)
   ```
   
   I cannot find what build step is supposed to build the `**-bin.tar.gz` for https://github.com/apache/kylin/blob/efedf07043c704cbdfa04ac3af3d73cec866cb76/docker/Dockerfile_dev#L25
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] zhangayqian commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
zhangayqian commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-799043126


   > Hi @zhangayqian @zzcclp Sorry to bother you, could you please help to take a look this PR ?
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] liusheng edited a comment on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
liusheng edited a comment on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-809999405


   Hi @hit-lacus ,
   Thanks for your reply. yes, we are aim to promoting some popular open source projects support running on ARM platform. for Kylin, I have verified the building, and runing all tests(seems also includes integration tests), if needed, I will verified the deployment and basic functionalities of Kylin on ARM64 server ans post testing reports.
   
   For the upgrade of rocksdbjni from version `5.9.2`~`5.18.4`, I wil also try to check if there is side effect,  rocksdb multi-arch support and version upgrade has already be performed on Kafka/Storm/Flink[1][2][3].
   
   The Rocksdb adds Arm64 support [4] since version 6.4.6, and also backports all Arm64
   related commits to 5.18.4 and release a all platforms support version.
   
   So, from multi-arch support view, the better rocksdb version is the version since
   v6.4.6, or 5.X version is v5.18.4.
   
   [1] https://issues.apache.org/jira/browse/STORM-3599
   [2] https://github.com/apache/kafka/pull/8284
   [3] https://issues.apache.org/jira/browse/FLINK-13598
   [4] https://github.com/facebook/rocksdb/pull/6250  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] liusheng commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
liusheng commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-802437528


   Hi, could any can help to review this ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] liusheng commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
liusheng commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-797261106


   Hi @hit-lacus  It looks the ARM CI job in  Travis works OK (see above check status), all the tests can pass in ARM64, could you please help review this ? Thank you.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] hit-lacus merged pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
hit-lacus merged pull request #1558:
URL: https://github.com/apache/kylin/pull/1558


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kylin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-859564066


   I will do the testing and report back !


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-865898669


   @hit-lacus I was able to run the Docker image on Linux ARM64 with these changes: https://github.com/apache/kylin/compare/master...martin-g:KYLIN-4864-add-support-for-aarch64-to-Dockerfiles?expand=1
   
   I had to upgrade MySQL to 8.x because there was no aarch64 binaries at http://repo.mysql.com/yum/mysql-5.7-community/el/7/
   
   I've also updated Kylin, Livy, JDK and Maven to their latest versions.
   
   I've tested it with the steps described at http://kylin.apache.org/docs/tutorial/kylin_sample.html but it got stuck at 90.91% . The logs just repeat the following again and again:
   
   ```
   ==> logs/kylin.log <==
   2021-06-22 11:12:42,935 INFO  [BadQueryDetector] service.BadQueryDetector:148 : Detect bad query.
   2021-06-22 11:12:45,832 INFO  [FetcherRunner 1102308739-43] threadpool.DefaultFetcherRunner:117 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 0 already succeed, 0 error, 0 discarded, 0 others
   
   ==> logs/streaming_coordinator.log <==
   2021-06-22 11:12:45,832 INFO  [FetcherRunner 1102308739-43] threadpool.DefaultFetcherRunner:117 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 0 already succeed, 0 error, 0 discarded, 0 others
   
   ==> logs/kylin.log <==
   2021-06-22 11:13:15,832 INFO  [FetcherRunner 1102308739-43] threadpool.DefaultFetcherRunner:117 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 0 already succeed, 0 error, 0 discarded, 0 others
   
   ==> logs/streaming_coordinator.log <==
   2021-06-22 11:13:15,832 INFO  [FetcherRunner 1102308739-43] threadpool.DefaultFetcherRunner:117 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 0 already succeed, 0 error, 0 discarded, 0 others
   
   ==> logs/kylin.log <==
   ```
   
   Now I am trying to test with 3.1.3-SNAPSHOT but the build fails at `kylin-it` module due to :
   
   ```
   java.lang.IllegalArgumentException: java.net.UnknownHostException: sandbox.hortonworks.com
   	at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:378)
   	at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310)
   	at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
   	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678)
   	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
   	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
   	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
   	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
   	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
   	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:355)
   	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
   	at org.apache.kylin.common.KylinConfigBase.getHdfsWorkingDirectory(KylinConfigBase.java:284)
   	at org.apache.kylin.dict.ITGlobalDictionaryBuilderTest$SharedBuilderThread.run(ITGlobalDictionaryBuilderTest.java:133)
   ```
   
   I cannot find what build step is supposed to build the `**-bin.tar.gz` for https://github.com/apache/kylin/blob/efedf07043c704cbdfa04ac3af3d73cec866cb76/docker/Dockerfile_dev#L25
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-874054281


   @hit-lacus Ping! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kylin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kylin] martin-g commented on pull request #1558: [KYLIN-4864] Enable running Kylin tests on ARM64 platform

Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1558:
URL: https://github.com/apache/kylin/pull/1558#issuecomment-958724194


   The changes introduced with this PR have been lost! There is no `arm64` in https://github.com/apache/kylin/blob/main/.travis.yml and its git history.
   
   @hit-lacus Were there any Git filtering operations on the repo ? I see that `master` branch has been renamed to `main` but I doubt this is the reason.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kylin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org