You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "liusheng (Code Review)" <ge...@cloudera.org> on 2019/12/10 08:04:27 UTC

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

liusheng has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14873


Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................

[KUDU-3007] Add patch for crcutil to support aarch64 platform

This change is one of steps to make Kudu support ARM/aarch64
platform, Added a patch for the crcutil thirdparty depdency to
support building on aarch64 server.

Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
---
M thirdparty/download-thirdparty.sh
A thirdparty/patches/crcutil-support-on-aarch64.patch
2 files changed, 97 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/73/14873/1
-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 1
Gerrit-Owner: liusheng <li...@gmail.com>

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14873 )

Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/14873/1/thirdparty/patches/crcutil-support-on-aarch64.patch
File thirdparty/patches/crcutil-support-on-aarch64.patch:

http://gerrit.cloudera.org:8080/#/c/14873/1/thirdparty/patches/crcutil-support-on-aarch64.patch@7
PS1, Line 7: +if [[ "$(uname -p)" == "aarch64" ]]; then
           : +  crcutil_flags="-DCRCUTIL_USE_MM_CRC32=1 -Wall -march=armv8-a -Icode -Iexamples -Itests -O3"
           : +fi
           : +
> not only -march=armv8-a needed, the flags of '-msse2', '-mcrc32' and '-msse
Right, so you could do something like this:

  crcutil_flags="-DCRCUTIL_USE_MM_CRC32=1 -Wall -Icode -Iexamples -Itests"
  crcutil_flags="${crcutil_flags} -O3"
  if [[ "$(uname -p)" == "aarch64" ]]; then
    crcutil_flags="${crcutil_flags} -march=arvm8-a"
  elif [[ "$IS_CLANG" = "0" && "$(${CXX} -dumpversion)" > "4.4.9" ]]; then
    crcutil_flags="${crcutil_flags} -msse2 -mcrc32"
  elif [[ "$IS_CLANG" = "1" ]]; then
    crcutil_flags="${crcutil_flags} -msse2 -msse4.2"
  fi



-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 3
Gerrit-Owner: liusheng <li...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: liusheng <li...@gmail.com>
Gerrit-Comment-Date: Tue, 10 Dec 2019 18:27:44 +0000
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has abandoned this change. ( http://gerrit.cloudera.org:8080/14873 )

Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................


Abandoned

Superceded by https://gerrit.cloudera.org/c/14939/
-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 6
Gerrit-Owner: liusheng <li...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: liusheng <li...@gmail.com>

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

Posted by "liusheng (Code Review)" <ge...@cloudera.org>.
liusheng has posted comments on this change. ( http://gerrit.cloudera.org:8080/14873 )

Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................


Patch Set 6:

> > Can this be merged ? I just clicked the "Rebase" and "+2" lost :)
 > 
 > We could, except that at the same time that you've been doing this
 > work, we've been working to publish a real crcutil repo (rather
 > than a fork in my own github account). That repo will include this
 > patch in it so that Kudu needn't maintain it downstream.
 > 
 > Once the repo is made public, I'll publish a patch to update Kudu's
 > crcutil dependency to a tarball including those patches. Which
 > means we won't actually need this particular patch, as it'll be
 > available in "upstream" crcutil.

That sounds great. it is better to support aarch64 in upstream crcutil . thank you :)


-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 6
Gerrit-Owner: liusheng <li...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: liusheng <li...@gmail.com>
Gerrit-Comment-Date: Thu, 19 Dec 2019 02:58:49 +0000
Gerrit-HasComments: No

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

Posted by "liusheng (Code Review)" <ge...@cloudera.org>.
liusheng has posted comments on this change. ( http://gerrit.cloudera.org:8080/14873 )

Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................


Patch Set 1:

(4 comments)

Hi Adar Dembo, thank you for your review.

http://gerrit.cloudera.org:8080/#/c/14873/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/14873/1//COMMIT_MSG@10
PS1, Line 10: depdency
> dependency
thank you, done


http://gerrit.cloudera.org:8080/#/c/14873/1/thirdparty/download-thirdparty.sh
File thirdparty/download-thirdparty.sh:

http://gerrit.cloudera.org:8080/#/c/14873/1/thirdparty/download-thirdparty.sh@300
PS1, Line 300: CRCUTIL_PATCHLEVEL=1
> Need to change this to 2.
thank you, done


http://gerrit.cloudera.org:8080/#/c/14873/1/thirdparty/patches/crcutil-support-on-aarch64.patch
File thirdparty/patches/crcutil-support-on-aarch64.patch:

http://gerrit.cloudera.org:8080/#/c/14873/1/thirdparty/patches/crcutil-support-on-aarch64.patch@7
PS1, Line 7: +if [[ "$(uname -p)" == "aarch64" ]]; then
           : +  crcutil_flags="-DCRCUTIL_USE_MM_CRC32=1 -Wall -march=armv8-a -Icode -Iexamples -Itests -O3"
           : +fi
           : +
> Could you incorporate this into the if/else above so that you needn't redef
not only -march=armv8-a needed, the flags of '-msse2', '-mcrc32' and '-msse4.2' are not supportted on aarch64 server.


http://gerrit.cloudera.org:8080/#/c/14873/1/thirdparty/patches/crcutil-support-on-aarch64.patch@92
PS1, Line 92: +#if CRCUTIL_USE_MM_CRC32 && (HAVE_I386 || HAVE_AMD64 || HAVE_AARCH64)
> There's a corresponding #endif whose comment needs to be updated.
oh, yes, thank you :)



-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 1
Gerrit-Owner: liusheng <li...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: liusheng <li...@gmail.com>
Gerrit-Comment-Date: Tue, 10 Dec 2019 10:26:43 +0000
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14873 )

Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................


Patch Set 5: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 5
Gerrit-Owner: liusheng <li...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: liusheng <li...@gmail.com>
Gerrit-Comment-Date: Wed, 11 Dec 2019 04:59:47 +0000
Gerrit-HasComments: No

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

Posted by "liusheng (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14873

to look at the new patch set (#2).

Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................

[KUDU-3007] Add patch for crcutil to support aarch64 platform

This change is one of steps to make Kudu support ARM/aarch64
platform, Added a patch for the crcutil thirdparty depdency to
support building on aarch64 server.

Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
---
M thirdparty/download-thirdparty.sh
A thirdparty/patches/crcutil-support-on-aarch64.patch
2 files changed, 107 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/73/14873/2
-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 2
Gerrit-Owner: liusheng <li...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: liusheng <li...@gmail.com>

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

Posted by "liusheng (Code Review)" <ge...@cloudera.org>.
liusheng has posted comments on this change. ( http://gerrit.cloudera.org:8080/14873 )

Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................


Patch Set 6:

Hi Adar,
Can this be merged ? I just clicked the "Rebase" and "+2" lost :)


-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 6
Gerrit-Owner: liusheng <li...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: liusheng <li...@gmail.com>
Gerrit-Comment-Date: Wed, 18 Dec 2019 01:24:14 +0000
Gerrit-HasComments: No

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

Posted by "liusheng (Code Review)" <ge...@cloudera.org>.
liusheng has posted comments on this change. ( http://gerrit.cloudera.org:8080/14873 )

Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................


Patch Set 3:

> (1 comment)

OK, thank you, will update :)


-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 3
Gerrit-Owner: liusheng <li...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: liusheng <li...@gmail.com>
Gerrit-Comment-Date: Wed, 11 Dec 2019 01:03:52 +0000
Gerrit-HasComments: No

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

Posted by "liusheng (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14873

to look at the new patch set (#3).

Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................

[KUDU-3007] Add patch for crcutil to support aarch64 platform

This change is one of steps to make Kudu support ARM/aarch64
platform, Added a patch for the crcutil thirdparty dependency to
support building on aarch64 server.

Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
---
M thirdparty/download-thirdparty.sh
A thirdparty/patches/crcutil-support-on-aarch64.patch
2 files changed, 107 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/73/14873/3
-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 3
Gerrit-Owner: liusheng <li...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: liusheng <li...@gmail.com>

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14873 )

Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................


Patch Set 1:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/14873/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/14873/1//COMMIT_MSG@10
PS1, Line 10: depdency
dependency


http://gerrit.cloudera.org:8080/#/c/14873/1/thirdparty/download-thirdparty.sh
File thirdparty/download-thirdparty.sh:

http://gerrit.cloudera.org:8080/#/c/14873/1/thirdparty/download-thirdparty.sh@300
PS1, Line 300: CRCUTIL_PATCHLEVEL=1
Need to change this to 2.


http://gerrit.cloudera.org:8080/#/c/14873/1/thirdparty/patches/crcutil-support-on-aarch64.patch
File thirdparty/patches/crcutil-support-on-aarch64.patch:

http://gerrit.cloudera.org:8080/#/c/14873/1/thirdparty/patches/crcutil-support-on-aarch64.patch@7
PS1, Line 7: +if [[ "$(uname -p)" == "aarch64" ]]; then
           : +  crcutil_flags="-DCRCUTIL_USE_MM_CRC32=1 -Wall -march=armv8-a -Icode -Iexamples -Itests -O3"
           : +fi
           : +
Could you incorporate this into the if/else above so that you needn't redefine the entirety of crcutil_flags? What you really want here is just to add -march=armv8-a, right?


http://gerrit.cloudera.org:8080/#/c/14873/1/thirdparty/patches/crcutil-support-on-aarch64.patch@92
PS1, Line 92: +#if CRCUTIL_USE_MM_CRC32 && (HAVE_I386 || HAVE_AMD64 || HAVE_AARCH64)
There's a corresponding #endif whose comment needs to be updated.



-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 1
Gerrit-Owner: liusheng <li...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 10 Dec 2019 08:47:04 +0000
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

Posted by "liusheng (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14873

to look at the new patch set (#4).

Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................

[KUDU-3007] Add patch for crcutil to support aarch64 platform

This change is one of steps to make Kudu support ARM/aarch64
platform, Added a patch for the crcutil thirdparty dependency to
support building on aarch64 server.

Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
---
M thirdparty/download-thirdparty.sh
A thirdparty/patches/crcutil-support-on-aarch64.patch
2 files changed, 115 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/73/14873/4
-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 4
Gerrit-Owner: liusheng <li...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: liusheng <li...@gmail.com>

[kudu-CR] [KUDU-3007] Add patch for crcutil to support aarch64 platform

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14873 )

Change subject: [KUDU-3007] Add patch for crcutil to support aarch64 platform
......................................................................


Patch Set 6:

> Can this be merged ? I just clicked the "Rebase" and "+2" lost :)

We could, except that at the same time that you've been doing this work, we've been working to publish a real crcutil repo (rather than a fork in my own github account). That repo will include this patch in it so that Kudu needn't maintain it downstream.

Once the repo is made public, I'll publish a patch to update Kudu's crcutil dependency to a tarball including those patches. Which means we won't actually need this particular patch, as it'll be available in "upstream" crcutil.


-- 
To view, visit http://gerrit.cloudera.org:8080/14873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3a95504b52407d3e213ff76bf9ebcd960ae4fde3
Gerrit-Change-Number: 14873
Gerrit-PatchSet: 6
Gerrit-Owner: liusheng <li...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: liusheng <li...@gmail.com>
Gerrit-Comment-Date: Wed, 18 Dec 2019 01:39:27 +0000
Gerrit-HasComments: No