You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by bryanpkc <gi...@git.apache.org> on 2015/02/17 19:22:02 UTC
[GitHub] couchdb-snappy pull request: Support big-endian builds
GitHub user bryanpkc opened a pull request:
https://github.com/apache/couchdb-snappy/pull/2
Support big-endian builds
Google snappy's configure script defines WORDS_BIGENDIAN in config.h if it detects that the build machine is big-endian. rebar should do the same when building the C++ code for this NIF. This patch allows CouchDB to be built and run on big-endian Linux systems, e.g. ppc64, s390x, etc.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bryanpkc/couchdb-snappy big-endian
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/couchdb-snappy/pull/2.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2
----
commit 6bc08a598cdd3fac17666c5226a1283cbce983db
Author: Bryan Chan <br...@gmail.com>
Date: 2015-02-17T14:41:27Z
Support big-endian builds
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74927424
All good. At least it doesn't breaks anything (:
+1 from me with the hope that we'll have big-endian host to build the couch there. Also might worth to check the other NIFs for such compatibility.
@bryanpkc please squash all your commits, remove trailing space from there:
```/home/kxepal/projects/couchdb/asf/couchdb-snappy/.git/rebase-apply/patch:25: trailing whitespace.
<<0,1>> ->
```
and let's wait for a while to see what others think about.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by janl <gi...@git.apache.org>.
Github user janl commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74868043
@bryanpkc nice one! Is there a way to add a test case for this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-snappy/pull/2#discussion_r24924159
--- Diff: rebar.config.script ---
@@ -0,0 +1,14 @@
+HandleBigEndianPort = fun
+ F([H|T]) when element(1, H) == port_env ->
--- End diff --
Much better and, what's important, more friendly for reading by humans (:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by bryanpkc <gi...@git.apache.org>.
Github user bryanpkc commented on a diff in the pull request:
https://github.com/apache/couchdb-snappy/pull/2#discussion_r24929628
--- Diff: rebar.config.script ---
@@ -0,0 +1,13 @@
+case <<1:16/native>> of
+ <<1,0>> ->
+ CONFIG;
+ <<0,1>> ->
+ CXXFLAGS = [ { "CXXFLAGS", "$CXXFLAGS -DWORDS_BIGENDIAN" } ],
+ P = lists:keyfind( port_env, 1, CONFIG ),
+ case P of
+ false ->
+ CONFIG ++ [ { port_env, CXXFLAGS } ];
--- End diff --
@kxepal Thanks for the feedback. All done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74894929
theoretically...yes, need to try.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74887216
@janl with qemu it's possible to emulate different endian. Raspberry PI IIRC is a bi-endian, but little one is preferred.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by bryanpkc <gi...@git.apache.org>.
Github user bryanpkc commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-79343104
@janl @kxepal Just a gentle reminder about this pull request. It has already been merged upstream (snappy-erlang-nif). Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by janl <gi...@git.apache.org>.
Github user janl commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74886312
@bryanpkc ongoing: whenever we make a release, run `make test` and report here or on the dev@couchdb.apache.org mailing list :)
Ideally we’d have a big endian CI box. I haven’t looked at this at all, but would it be possible to emulate a big endian system on a little endian host using some sort of virtualisation?
If not, would a Raspberry Pi do? We could hook one into our Jenkins :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-snappy/pull/2#discussion_r24924456
--- Diff: rebar.config.script ---
@@ -0,0 +1,13 @@
+case <<1:16/native>> of
+ <<1,0>> ->
+ CONFIG;
+ <<0,1>> ->
+ CXXFLAGS = [ { "CXXFLAGS", "$CXXFLAGS -DWORDS_BIGENDIAN" } ],
+ P = lists:keyfind( port_env, 1, CONFIG ),
+ case P of
+ false ->
+ CONFIG ++ [ { port_env, CXXFLAGS } ];
--- End diff --
Another notice I forgot to add, we don't uses extra spaces between brackets. E.g. this one would be written as `CONFIG ++ [{port_env, CXXFLAGS}];`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74889419
@bryanpkc quick googling showed me this one: https://people.debian.org/~aurel32/qemu/ || http://www.aurel32.net/info/debian_mips_qemu.php
Much likely there are the others in the wild.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74884731
@janl me too, especially such that we cannot test even manually during regular build process. I believe @bryanpkc made these changes because he is able to test it IRL, If he would help us with testing (and even packaging) for big endian hosts that would be awesome.
This situation is the same as our Windows support where there are no actual tests, but group of people who cares about compatibility with that system. And that's still extremely helpful.
So, @bryanpkc how do you feel about such sort of help? (:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by janl <gi...@git.apache.org>.
Github user janl commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74919967
just a note that I think the CI test discussion here is **not** blocking merging this in
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74870033
Hi @bryanpkc and welcome! (:
@janl I think the only way to test this is to actually try to built it on big-endian hosts.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by bryanpkc <gi...@git.apache.org>.
Github user bryanpkc commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74990294
All commits squashed, trailing space removed. Thanks for the review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by bryanpkc <gi...@git.apache.org>.
Github user bryanpkc commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74885302
@janl @kxepal I can build and run "make check" on s390x (and potentially ppc64 if I can borrow the machines). I could learn how to package too. What is the amount of on-going work involved?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by bryanpkc <gi...@git.apache.org>.
Github user bryanpkc commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74888615
@janl OK, that doesn't sound too bad. I could certainly help with testing that way for now. I am checking if we could help with CI on our hardware, but I cannot promise anything at the moment.
@kxepal, I haven't tried it but I read that the Raspberry Pi is bi-endian, like a few other ARM processors that I have used. The question is whether there is a big-endian OS image for it. Maybe qemu on the Raspberry Pi is the way to go.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by janl <gi...@git.apache.org>.
Github user janl commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74883487
@kxepal yeah I imagine, and we probably don’t want to re-shuffle the code so you can inject endianness from the tests. Maybe we can have a test that only runs on BE hosts? I just feel bad accepting code without any tests :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by janl <gi...@git.apache.org>.
Github user janl commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-74892552
we have some CI hardware, would qemu running there would help?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/couchdb-snappy/pull/2
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
Re: [GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by Jan Lehnardt <ja...@apache.org>.
Heya Bryan,
thanks for your contribution! :)
You already followed the regular process! The one thing that’s missing now is waiting for someone to do a review. Sending regular nudges always is a good strategy, and bringing some patience :)
Best
Jan
--
> On 18 Feb 2015, at 14:51, Bryan Chan <br...@ca.ibm.com> wrote:
>
>
> Hello CouchDB developers,
>
> I am new to the list; please be gentle. :-)
>
> I created a pull request on GitHub yesterday (see below) for a relatively
> simple patch that would allow CouchDB to be built correctly on big-endian
> platforms. What is the usual process for getting this kind of change
> approved and committed?
>
> FWIW I have also sent a pull request for the same change to the upstream
> project, snappy-erlang-nif.
>
> Thanks,
>
> Bryan
> --
> Bryan Chan
> bryan.chan@ca.ibm.com
>
>
>
> bryanpkc <gi...@git.apache.org> wrote on 2015-02-17 01:22:02 PM:
>
>> From: bryanpkc <gi...@git.apache.org>
>> To: dev@couchdb.apache.org
>> Date: 2015-02-17 01:22 PM
>> Subject: [GitHub] couchdb-snappy pull request: Support big-endian builds
>>
>> GitHub user bryanpkc opened a pull request:
>>
>> https://github.com/apache/couchdb-snappy/pull/2
>>
>> Support big-endian builds
>>
>> Google snappy's configure script defines WORDS_BIGENDIAN in
>> config.h if it detects that the build machine is big-endian. rebar
>> should do the same when building the C++ code for this NIF. This
>> patch allows CouchDB to be built and run on big-endian Linux
>> systems, e.g. ppc64, s390x, etc.
>>
>> You can merge this pull request into a Git repository by running:
>>
>> $ git pull https://github.com/bryanpkc/couchdb-snappy big-endian
>>
>> Alternatively you can review and apply these changes as the patch at:
>>
>> https://github.com/apache/couchdb-snappy/pull/2.patch
>>
>> To close this pull request, make a commit to your master/trunk branch
>> with (at least) the following in the commit message:
>>
>> This closes #2
>>
>> ----
>> commit 6bc08a598cdd3fac17666c5226a1283cbce983db
>> Author: Bryan Chan <br...@gmail.com>
>> Date: 2015-02-17T14:41:27Z
>>
>> Support big-endian builds
>>
>> ----
>>
>>
>> ---
>> If your project is set up for it, you can reply to this email and have
> your
>> reply appear on GitHub as well. If your project does not have this
> feature
>> enabled and wishes so, or if the feature is enabled but not working,
> please
>> contact infrastructure at infrastructure@apache.org or file a JIRA ticket
>> with INFRA.
>> ---
Re: [GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by Bryan Chan <br...@ca.ibm.com>.
Hello CouchDB developers,
I am new to the list; please be gentle. :-)
I created a pull request on GitHub yesterday (see below) for a relatively
simple patch that would allow CouchDB to be built correctly on big-endian
platforms. What is the usual process for getting this kind of change
approved and committed?
FWIW I have also sent a pull request for the same change to the upstream
project, snappy-erlang-nif.
Thanks,
Bryan
--
Bryan Chan
bryan.chan@ca.ibm.com
bryanpkc <gi...@git.apache.org> wrote on 2015-02-17 01:22:02 PM:
> From: bryanpkc <gi...@git.apache.org>
> To: dev@couchdb.apache.org
> Date: 2015-02-17 01:22 PM
> Subject: [GitHub] couchdb-snappy pull request: Support big-endian builds
>
> GitHub user bryanpkc opened a pull request:
>
> https://github.com/apache/couchdb-snappy/pull/2
>
> Support big-endian builds
>
> Google snappy's configure script defines WORDS_BIGENDIAN in
> config.h if it detects that the build machine is big-endian. rebar
> should do the same when building the C++ code for this NIF. This
> patch allows CouchDB to be built and run on big-endian Linux
> systems, e.g. ppc64, s390x, etc.
>
> You can merge this pull request into a Git repository by running:
>
> $ git pull https://github.com/bryanpkc/couchdb-snappy big-endian
>
> Alternatively you can review and apply these changes as the patch at:
>
> https://github.com/apache/couchdb-snappy/pull/2.patch
>
> To close this pull request, make a commit to your master/trunk branch
> with (at least) the following in the commit message:
>
> This closes #2
>
> ----
> commit 6bc08a598cdd3fac17666c5226a1283cbce983db
> Author: Bryan Chan <br...@gmail.com>
> Date: 2015-02-17T14:41:27Z
>
> Support big-endian builds
>
> ----
>
>
> ---
> If your project is set up for it, you can reply to this email and have
your
> reply appear on GitHub as well. If your project does not have this
feature
> enabled and wishes so, or if the feature is enabled but not working,
please
> contact infrastructure at infrastructure@apache.org or file a JIRA ticket
> with INFRA.
> ---
>
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by bryanpkc <gi...@git.apache.org>.
Github user bryanpkc commented on a diff in the pull request:
https://github.com/apache/couchdb-snappy/pull/2#discussion_r24923370
--- Diff: rebar.config.script ---
@@ -0,0 +1,14 @@
+HandleBigEndianPort = fun
+ F([H|T]) when element(1, H) == port_env ->
--- End diff --
@kxepal I am fairly new to Erlang so I probably wasn't writing the smartest code. I have rewritten the script to use lists:keyfind and avoid the fun. Does it look better now?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by bryanpkc <gi...@git.apache.org>.
Github user bryanpkc commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-75082325
@janl I have briefly looked at Travis CI; it seems to support only Ubuntu as a build environment. Will it work with Debian? There isn't a Ubuntu build for s390x yet, AFAIK.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-snappy/pull/2#discussion_r24924320
--- Diff: rebar.config.script ---
@@ -0,0 +1,13 @@
+case <<1:16/native>> of
+ <<1,0>> ->
+ CONFIG;
+ <<0,1>> ->
+ CXXFLAGS = [ { "CXXFLAGS", "$CXXFLAGS -DWORDS_BIGENDIAN" } ],
+ P = lists:keyfind( port_env, 1, CONFIG ),
+ case P of
--- End diff --
you can actually avoid temp variable `P` by using `list:keyfiend` expression directly in case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by janl <gi...@git.apache.org>.
Github user janl commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-75084052
@bryanpkc I don’t think Travis is gonna help with running anything but their stock environment, but we have a Jenkins setup that we can use.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-snappy/pull/2#discussion_r24903186
--- Diff: rebar.config.script ---
@@ -0,0 +1,14 @@
+HandleBigEndianPort = fun
+ F([H|T]) when element(1, H) == port_env ->
--- End diff --
Couldn't the search of `port_env` be replaced by `lists:keyfind(port_env, 1, CONFIG)`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-snappy pull request: Support big-endian builds
Posted by janl <gi...@git.apache.org>.
Github user janl commented on the pull request:
https://github.com/apache/couchdb-snappy/pull/2#issuecomment-81632205
merged!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---