[jira] [Commented] (STORM-3401) There is no ARM CI for Storm

["Yikun Jiang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16882763#comment-16882763 ] 

Yikun Jiang commented on STORM-3401:
------------------------------------

Sync the ARM64 support status:
 There are some problems and progress in the arm support for storm project.

*1. mvn install -DskipTests*

The storm can be built/packaged succesfuly in ARM64 env. See more in [1]

*2.* *mvn install* failed due to storm-server mvn test failed

The storm need a "rocksdbjni" with aarch64 support, the rocksdbjni support issue is tracked in [3].

I guess we also need a Storm issue in jira?

*3. mvn test after using the rocksdbjni-xxx.jar*

After using local compiled rocksdbjni, there are only one test case failed. [4], and the reason is still rocksdb related, the detail steps as below:
   *1) Build the rocksdbjni in local*
    git clone [https://github.com/facebook/rocksdb.git]; cd rocksdb
    export DEBUG_LEVEL=0
    make -j8 rocksdbjava
    We can get the `rocksdb/java/target/rocksdbjni-6.2.0-linux64.jar`
   *2) Hack the pom.xml to use the local jar*

    <dependency>
        <groupId>org.rocksdb</groupId>
        <artifactId>rocksdbjni</artifactId>
        <version>${rocksdb-version}</version>
     + <scope>system</scope>
     + <systemPath>/home/zuul/rocksdb/java/target/rocksdbjni-6.2.0-linux64.jar</systemPath>
    </dependency>

*3) mvn install in storm-server again,* there are only one test case failed, the detail log can be found in [4]

[1] [https://github.com/theopenlab/storm/pull/2]
 [2] [https://pastebin.com/xSrPfxg9] storm-server mvn install in arm64 
 [3] [https://github.com/facebook/rocksdb/issues/5559] No arm64/aarch64 support for rocksdbjni-xxx.jar
 [4] [https://pastebin.com/YFk4fBuY] storm-server mvn test after rocksdb local hack

 

So, it seems we should let rocksdbjni support the aarch64 first before we enable the arm64 of storm projects.
 Do you have any other ideas or suggestions on it?

> There is no ARM CI for Storm
> ----------------------------
>
>                 Key: STORM-3401
>                 URL: https://issues.apache.org/jira/browse/STORM-3401
>             Project: Apache Storm
>          Issue Type: Improvement
>          Components: build
>            Reporter: Yikun Jiang
>            Priority: Major
>
> Now the CI of storm (in github) is handled by travis-ci. While the test is running under x86 ARCH, the arm ARCH is missing. This leads an problem that we don't have a way to test every pull request that if it'll break the storm deployment on arm or not.
> We should add a CI system that support ARM ARCH. Using it, storm can officially support arm release in the future. Here I'd like to introduce OpenLab to the community. [OpenLab|https://openlabtesting.org/] is a open source CI system that can test any open source software on either x86 or arm ARCH, it's mainly used by github projects. Now some [projects|https://github.com/theopenlab/openlab-zuul-jobs/blob/master/zuul.d/jobs.yaml] has integrated it already. Such as containerd (a graduated CNCF project, the arm build will be triggerd in every PR, [https://github.com/containerd/containerd/pulls]), terraform and so on.
> OpenLab uses the open source CI software [Zuul |https://github.com/openstack-infra/zuul] for CI system. Zuul is used by OpenStack community as well. integrating with OpneLab is quite easy using its github app. All config info is open source as well.
> If apache storm community has interested with it, I can help for the integration.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)