You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tuscany.apache.org by Jean-Sebastien Delfino <js...@apache.org> on 2011/08/06 07:55:55 UTC
Fun with LevelDB, LLVM and OpenCL
Hi all,
I'm on vacation for a few days and I'm going to have a little bit of
free time to hack around Tuscany C++. I've been thinking about the
following:
a) Integrate LevelDB [1] as a key-value store component. I've been
experimenting with LevelDB and I think it'll be a useful complement to
the TinyCDB based store (best for constant data) and the PostgreSQL
based store (best for flexible query).
b) Use the LLVM based Clang compiler [2] as an alternative to GCC,
useful on MacOS as it'll enable building without anything extra on top
of Xcode 4.1.
c) More challenging and interesting too, analyze a composite and
transform it into a set of OpenCL [3][4][5][6] kernel functions
running on a mix of CPUs and GPUs (giving you a lot of cheap computing
power to run your SCA compositions). In particular convert a graph of
synchronous service invocations into chained asynchronous tasks and
determine the optimum partition of the composite into OpenCL kernels,
work groups and work items while minimizing the data transfers between
OpenCL kernels.
I only have a few days to bootstrap these projects and will probably
spend most of my energy on the OpenCL one as I want to try a few
innovative ideas (including a functional programming monad based
approach to model the interactions around composites and help the sync
to async transformation, and another idea to have the component
implementations do meta-programming and return their computations as
kernel programs instead of the actual computation results) so if
anybody is interested in working with me on this stuff ping me on
gtalk jsdelfino, the #tuscany IRC channel or even better post a
message here...
[1] http://code.google.com/p/leveldb/
[2] http://clang.llvm.org/
[3] http://www.khronos.org/opencl/
[4] http://developer.apple.com/library/mac/#documentation/Performance/Conceptual/OpenCL_MacProgGuide/Introduction/Introduction.html
[5] http://developer.nvidia.com/opencl
[6] http://developer.amd.com/zones/OpenCLZone/Pages/default.aspx
--
Jean-Sebastien
Re: Fun with LevelDB, LLVM and OpenCL
Posted by Jean-Sebastien Delfino <js...@apache.org>.
On Wed, Aug 10, 2011 at 6:45 AM, dsh <da...@googlemail.com> wrote:
> Hi Jean-Sebastien,
>
> On Wed, Aug 10, 2011 at 6:45 AM, Jean-Sebastien Delfino
> <js...@apache.org> wrote:
>>
>> That's what I meant when I mentioned 'convert usages of __thread to
>> Posix thread TLS calls' in my previous post [1].
>>
>> __thread works for me with GCC on Mac OS X (using the GCC build from
>> the HPC project [2]), but causes a compile error with Clang/LLVM. It's
>> listed as a limitation of Clang on Mac OS X in the Clang user manual
>> (look for __thread on this page [3]).
>>
>
> Does "Works for me" mean it compiles for you just fine AND during
> runtime you are getting the expected behavior or does it just mean the
> code compiles just fine if using GCC?
>
> I am asking cause I googled a bit last night and from what I found I
> had the feeling thread local isn't really supported on OS X during
> runtime.
>
Hi Daniel!
__thread compiles and behaves as expected for me on OS X with GCC 4.6
(the version referenced in the macos/macos-install script).
I've added some test logic to kernel/parallel-test.cpp to help verify
that (see SVN r1156460).
Parallel-test built with GCC works, showing that __thread is correctly
implemented by GCC on OS X.
After commenting __thread on parallel-test.cpp:line 134, parallel-test
breaks with an assertion error, showing that the test is actually
effective at testing thread local storage.
I'm hoping that this additional test code will help you when you
implement thread local storage over the pthread API. In the meantime
you don't have to worry about it if you --disable-threads :)
--
Jean-Sebastien
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Hi Jean-Sebastien,
On Wed, Aug 10, 2011 at 6:45 AM, Jean-Sebastien Delfino
<js...@apache.org> wrote:
>
> That's what I meant when I mentioned 'convert usages of __thread to
> Posix thread TLS calls' in my previous post [1].
>
> __thread works for me with GCC on Mac OS X (using the GCC build from
> the HPC project [2]), but causes a compile error with Clang/LLVM. It's
> listed as a limitation of Clang on Mac OS X in the Clang user manual
> (look for __thread on this page [3]).
>
Does "Works for me" mean it compiles for you just fine AND during
runtime you are getting the expected behavior or does it just mean the
code compiles just fine if using GCC?
I am asking cause I googled a bit last night and from what I found I
had the feeling thread local isn't really supported on OS X during
runtime.
> Are you getting a compile error with GCC too? or just with Clang?
I forced myself to not install GCC on OS X if possible and thus the
compile errors I am getting are those reported by Clang.
>
> For now you could just run configure without --enable-threads, then
> later try to convert the few references to __thread to Posix thread
> TLS calls (i.e. pthread_key_create, pthread_getspecific,
> pthread_setspecific, pthread_key_delete).
>
Yep I am not using --enable-threads or I set it explicitly to
--disable-threads to avoid the compile error for the time being. I
will try to look into your Posix thread TLS calls suggestion today (I
as well have to get APR V2 and back out the symlinks I set so far).
Many thanks and best wishes!
Cheers
Daniel
Re: Fun with LevelDB, LLVM and OpenCL
Posted by Jean-Sebastien Delfino <js...@apache.org>.
On Tue, Aug 9, 2011 at 5:25 PM, dsh <da...@googlemail.com> wrote:
> Looks like OS X does not support thread local storage. I.e. __thread
> causes an compile error and thus one would not need to use
> --enable-threads or would need to disable them explicitely using
> --disable-threads. Is this a known issue on OS X?
>
That's what I meant when I mentioned 'convert usages of __thread to
Posix thread TLS calls' in my previous post [1].
__thread works for me with GCC on Mac OS X (using the GCC build from
the HPC project [2]), but causes a compile error with Clang/LLVM. It's
listed as a limitation of Clang on Mac OS X in the Clang user manual
(look for __thread on this page [3]).
Are you getting a compile error with GCC too? or just with Clang?
For now you could just run configure without --enable-threads, then
later try to convert the few references to __thread to Posix thread
TLS calls (i.e. pthread_key_create, pthread_getspecific,
pthread_setspecific, pthread_key_delete).
[1] http://marc.info/?l=tuscany-dev&m=131265451321095&w=2
[2] http://hpc.sourceforge.net/
[3] http://clang.llvm.org/docs/UsersManual.html
--
Jean-Sebastien
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Looks like OS X does not support thread local storage. I.e. __thread
causes an compile error and thus one would not need to use
--enable-threads or would need to disable them explicitely using
--disable-threads. Is this a known issue on OS X?
Cheers
Daniel
On Wed, Aug 10, 2011 at 12:53 AM, dsh <da...@googlemail.com> wrote:
> Hi Jean-Sebastien Delfino,
>
> some things I noticed during the configure process (I am using MacPorts):
>
> * I think it would help to point out in the INSTALL file that mozjs
> can be downloaded at -> ftp://ftp.mozilla.org/pub/mozilla.org/js/
> * On OSX Lion xulrunner won't install from MacPorts and thus one would
> end up without a proper mozjs installation
> * If compiling mozjs from the prio mentioned URL, it ends up to be
> installed as /opt/local/lib/libmozjs185.1.0.0.dylib which causes the
> configure process to fail
> * I thus symlinked /opt/local/lib/libmozjs185.1.0.0.dylib to
> /opt/local/lib/libmozjs.dylib
> * The configure process is looking for -lapr-2 instead of -lapr-1 (I
> have APR 1.4.5_1 installed)
> * I thus had to symlink /opt/local/lib/libapr-1.0.dylib to
> /opt/local/lib/libapr-2.dylib
>
> If you like I could try to update the INSTALL document accordingly and
> attach the patch to a JIRA. I am doing the dependency setup in
> parallel on FreeBSD 8.2 to see whether I would run into similar issues
> (for instance the one of having a -lapr-1 installed instead of the
> expected -lapr-2). We could then decide whether we want to improve the
> auto* tools infrastructure or not (or even migrate to CMake).
>
> PS: I am planning to create a FreeBSD port for Apache Tuscany (native)
> cause I feel having to setup all of these dependencies is kind of
> daunting.
>
> Cheers
> Daniel
>
> On Mon, Aug 8, 2011 at 7:05 PM, Jean-Sebastien Delfino
> <js...@apache.org> wrote:
>> On Sun, Aug 7, 2011 at 6:35 PM, dsh <da...@googlemail.com> wrote:
>>> Just a little update:
>>>
>>> OSX Lion with Xcode 4.1:
>>>
>>> * i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc.
>>> build 5658) (LLVM build 2335.15.00)
>>> * Apple clang version 2.1
>>>
>>> OSX Lion with Xcode 4.2:
>>>
>>> * i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1
>>> * Apple clang version 3.0
>>>
>>> OSX Snow Leopard with Xcode 4.0.2:
>>>
>>> * i686-apple-darwin10-llvm-gcc-4.2 (GCC) 4.2.1
>>> * Apple clang version 2.0 based on LLVM 2.9svn
>>>
>>> PS: There exists an Apple Xcode version 4.2 for OSX Snow Leopard which
>>> we could use as a fallback if 4.0.2 turns out to be unusable
>>> PPS: I really love these virtualized OSX envs I setup recently because
>>> they allow to have all those different setups and configurations :)
>>>
>>> Cheers
>>> Daniel
>>>
>>
>> OK, here's what I'm using on OS X Lion:
>>
>> Xcode Version 4.1 Build 4B110
>>
>> macbook-air:~$ g++ --version
>> i686-apple-darwin11-llvm-g++-4.2 (GCC) 4.2.1 (Based on Apple Inc.
>> build 5658) (LLVM build 2335.15.00)
>>
>> macbook-air:~$ clang++ --version
>> Apple clang version 2.1 (tags/Apple/clang-163.7.1) (based on LLVM 3.0svn)
>> Target: x86_64-apple-darwin11.0.0
>> Thread model: posix
>>
>> --
>> Jean-Sebastien
>>
>
Re: Fun with LevelDB, LLVM and OpenCL
Posted by Jean-Sebastien Delfino <js...@apache.org>.
On Wed, Aug 10, 2011 at 9:17 AM, dsh <da...@googlemail.com> wrote:
> Hi Jean-Sebastien,
>
> On Wed, Aug 10, 2011 at 5:26 AM, Jean-Sebastien Delfino
> <js...@apache.org> wrote:
>>
>> Thanks for the link to js 185, I'll try it on Ubuntu, Redhat and Mac
>> OS X too, happy to switch to it if it works on these systems, but I'd
>> like to avoid installing it in the default system directories (hoping
>> there's a way to specify the install dir with --prefix).
>>
>
> The below link as well contains information on how to build mozjs:
>
> http://www.mongodb.org/display/DOCS/Building+Spider+Monkey
>
> Cheers
> Daniel
>
Hi Daniel,
Thanks for the link to js-1.8.5 and the build info.
SVN revision r1156555 now uses SpiderMonkey 1.8.5.
The INSTALL doc, macos-install and ubuntu-install* scripts have been
updated accordingly (the build instructions from the MongoDB doc
caused compile errors for me, but configure --prefix=... in the js/src
directory worked fine).
The js and json modules have been ported to the SpiderMonkey 1.8.5
API. A few JSON test files have been adapted to minor JSON formatting
changes in 1.8.5, and all the test cases run successfully for me with
make check, on Mac OS X, Ubuntu and Redhat.
That's with GCC... hoping they'll work as well with Clang...
--
Jean-Sebastien
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Hi Jean-Sebastien,
On Wed, Aug 10, 2011 at 5:26 AM, Jean-Sebastien Delfino
<js...@apache.org> wrote:
>
> Thanks for the link to js 185, I'll try it on Ubuntu, Redhat and Mac
> OS X too, happy to switch to it if it works on these systems, but I'd
> like to avoid installing it in the default system directories (hoping
> there's a way to specify the install dir with --prefix).
>
The below link as well contains information on how to build mozjs:
http://www.mongodb.org/display/DOCS/Building+Spider+Monkey
Cheers
Daniel
Re: Fun with LevelDB, LLVM and OpenCL
Posted by Jean-Sebastien Delfino <js...@apache.org>.
On Mon, Sep 5, 2011 at 10:24 PM, dsh <da...@googlemail.com> wrote:
> Have to look at it during the weekend again or so. Didn't have much
> time recently cause I had to focus on other stuff. I think I will
> disable each broken dependency in the FreeBSD port and thus build
> Tuscany only with the proposed set of minimal dependency for now.
>
> Cheers
> Daniel
Sounds good!
--
Jean-Sebastien
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Have to look at it during the weekend again or so. Didn't have much
time recently cause I had to focus on other stuff. I think I will
disable each broken dependency in the FreeBSD port and thus build
Tuscany only with the proposed set of minimal dependency for now.
Cheers
Daniel
On Tue, Sep 6, 2011 at 12:16 AM, Jean-Sebastien Delfino
<js...@apache.org> wrote:
> On Sun, Aug 14, 2011 at 2:28 PM, Jean-Sebastien Delfino
> <js...@apache.org> wrote:
>> On Thu, Aug 11, 2011 at 8:14 PM, dsh <da...@googlemail.com> wrote:
>> ...
>>>
>>> My attempt to provide an overview what works on FreeBSD (8.2) and what not:
>>>
>>> * autoconf, automake, libtool, doxygen, gcc can be installed from the
>>> FreeBSD ports collection
>>> * at the time there's only Apache HTTPD 2.2 available in the FreeBSD
>>> ports collection. I started creating an additional Apache HTTPD 2.3
>>> port [0] but did not finish it yet
>>> * APR 2 - I sent an problem report [1] including a new port for the
>>> most recent APR 2 version
>>> * OpenSSL can be used from the FreeBSD base system
>>> * pcre-8.12 can be used from the FreeBSD ports collection
>>> * expat-2.0.1_1 can be used from the FreeBSD ports collection
>>> * memcached-1.4.5 can be installed from the FreeBSD ports collection
>>> * libevent-1.4.14b_2 can be installed from the FreeBSD ports collection
>>> * libxml2-2.7.8_1 can be installed from the FreeBSD ports collection
>>> * curl-7.21.3_2 can be installed from the FreeBSD ports collection
>>> (including SSL support)
>>> * I am currently in the process of upgrading the existing FreeBSD
>>> SpiderMonkey 1.7.x port [2], [3]
>>> * tinycdb-0.77 is available from the FreeBSD ports collection
>>> * ap20-mod_security-2.5.13 is available from the FreeBSD ports
>>> collection (not sure whether this one is too old)
>>> * I started to create an Apache Axis2/C port for FreeBSD but hit an
>>> issue which may be a bug [4]
>>> * I started to create an Apache Qpid port for FreeBSD put hit an issue
>>> which may be a bug [5], [6]
>>> * ap20-mod_auth_openid-0.5 is available from the FreeBSD ports
>>> collection including support for libopkele-2.0.2_1
>>> * liboauth-0.9.4 is available from the FreeBSD ports collection
>>> * I think Apache Vysper can be installed relatively easy from the
>>> binary distribution and thus I decided to not create a separate
>>> FreeBSD port
>>> * Didn't create a FreeBSD port for Libstrophe yet cause I don't know
>>> how to pull sources directly from Github while building a FreeBSD port
>>> * postgresql-client-8.4.8 is available from the FreeBSD ports collection
>>> * scribe-2.2_3 is available from the FreeBSD ports collection
>>> * thrift-0.4.0,1 is available from the FreeBSD ports collection
>>> * I created and submitted a new FreeBSD port for Apache Libcloud using a PR [7]
>>>
>>> [0] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=284897+0+current/freebsd-ports
>>> [1] http://www.freebsd.org/cgi/query-pr.cgi?pr=159658
>>> [2] http://people.apache.org/~dsh/projects/fbsd-ports/spidermonkey185/
>>> [3] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=343488+0+current/freebsd-ports
>>> [4] http://mail-archives.apache.org/mod_mbox/axis-c-user/201108.mbox/%3CCAC0wh9aKzrVC3-qzHg3THMfHNVi20KZERB0zP94hq_X+d_FiFQ@mail.gmail.com%3E
>>> [5] https://issues.apache.org/jira/browse/QPID-2549
>>> [6] http://mail-archives.apache.org/mod_mbox/qpid-dev/201108.mbox/%3CCAC0wh9ZjuK0Ns_2LnAbM=oUfhcG2gaVSB99+pKPuYbRtr6KppA@mail.gmail.com%3E
>>> [7] http://www.freebsd.org/cgi/query-pr.cgi?pr=159703
>>>
>>
>> Wow, that's great progress! ... with a pretty long list of dependencies :)
>>
>> If there's too many issues with the dependencies it should be easy to
>> scale down to the short list I mentioned earlier [9].
>>
> ...
>
> So I guess it's going to take a while to figure how to build the
> complete list of dependencies with Clang/LLVM. :)
>
> In the meantime the few changes in SVN revision r1164964 should allow
> a subset of Tuscany to be built with Clang/LLVM.
>
> The macos/macos-install script can be used to build the Tuscany
> runtime, a subset of its components and their dependencies on a clean
> Mac OS X 10.6.7 without requiring anything else than Xcode 4.1.
> The dependencies are built with the Xcode 4.1 GCC compiler. Tuscany is
> built with the Xcode 4.2 Clang/LLVM compiler.
>
> Hope this helps.
> --
> Jean-Sebastien
>
Re: Fun with LevelDB, LLVM and OpenCL
Posted by Jean-Sebastien Delfino <js...@apache.org>.
On Sun, Aug 14, 2011 at 2:28 PM, Jean-Sebastien Delfino
<js...@apache.org> wrote:
> On Thu, Aug 11, 2011 at 8:14 PM, dsh <da...@googlemail.com> wrote:
> ...
>>
>> My attempt to provide an overview what works on FreeBSD (8.2) and what not:
>>
>> * autoconf, automake, libtool, doxygen, gcc can be installed from the
>> FreeBSD ports collection
>> * at the time there's only Apache HTTPD 2.2 available in the FreeBSD
>> ports collection. I started creating an additional Apache HTTPD 2.3
>> port [0] but did not finish it yet
>> * APR 2 - I sent an problem report [1] including a new port for the
>> most recent APR 2 version
>> * OpenSSL can be used from the FreeBSD base system
>> * pcre-8.12 can be used from the FreeBSD ports collection
>> * expat-2.0.1_1 can be used from the FreeBSD ports collection
>> * memcached-1.4.5 can be installed from the FreeBSD ports collection
>> * libevent-1.4.14b_2 can be installed from the FreeBSD ports collection
>> * libxml2-2.7.8_1 can be installed from the FreeBSD ports collection
>> * curl-7.21.3_2 can be installed from the FreeBSD ports collection
>> (including SSL support)
>> * I am currently in the process of upgrading the existing FreeBSD
>> SpiderMonkey 1.7.x port [2], [3]
>> * tinycdb-0.77 is available from the FreeBSD ports collection
>> * ap20-mod_security-2.5.13 is available from the FreeBSD ports
>> collection (not sure whether this one is too old)
>> * I started to create an Apache Axis2/C port for FreeBSD but hit an
>> issue which may be a bug [4]
>> * I started to create an Apache Qpid port for FreeBSD put hit an issue
>> which may be a bug [5], [6]
>> * ap20-mod_auth_openid-0.5 is available from the FreeBSD ports
>> collection including support for libopkele-2.0.2_1
>> * liboauth-0.9.4 is available from the FreeBSD ports collection
>> * I think Apache Vysper can be installed relatively easy from the
>> binary distribution and thus I decided to not create a separate
>> FreeBSD port
>> * Didn't create a FreeBSD port for Libstrophe yet cause I don't know
>> how to pull sources directly from Github while building a FreeBSD port
>> * postgresql-client-8.4.8 is available from the FreeBSD ports collection
>> * scribe-2.2_3 is available from the FreeBSD ports collection
>> * thrift-0.4.0,1 is available from the FreeBSD ports collection
>> * I created and submitted a new FreeBSD port for Apache Libcloud using a PR [7]
>>
>> [0] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=284897+0+current/freebsd-ports
>> [1] http://www.freebsd.org/cgi/query-pr.cgi?pr=159658
>> [2] http://people.apache.org/~dsh/projects/fbsd-ports/spidermonkey185/
>> [3] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=343488+0+current/freebsd-ports
>> [4] http://mail-archives.apache.org/mod_mbox/axis-c-user/201108.mbox/%3CCAC0wh9aKzrVC3-qzHg3THMfHNVi20KZERB0zP94hq_X+d_FiFQ@mail.gmail.com%3E
>> [5] https://issues.apache.org/jira/browse/QPID-2549
>> [6] http://mail-archives.apache.org/mod_mbox/qpid-dev/201108.mbox/%3CCAC0wh9ZjuK0Ns_2LnAbM=oUfhcG2gaVSB99+pKPuYbRtr6KppA@mail.gmail.com%3E
>> [7] http://www.freebsd.org/cgi/query-pr.cgi?pr=159703
>>
>
> Wow, that's great progress! ... with a pretty long list of dependencies :)
>
> If there's too many issues with the dependencies it should be easy to
> scale down to the short list I mentioned earlier [9].
>
...
So I guess it's going to take a while to figure how to build the
complete list of dependencies with Clang/LLVM. :)
In the meantime the few changes in SVN revision r1164964 should allow
a subset of Tuscany to be built with Clang/LLVM.
The macos/macos-install script can be used to build the Tuscany
runtime, a subset of its components and their dependencies on a clean
Mac OS X 10.6.7 without requiring anything else than Xcode 4.1.
The dependencies are built with the Xcode 4.1 GCC compiler. Tuscany is
built with the Xcode 4.2 Clang/LLVM compiler.
Hope this helps.
--
Jean-Sebastien
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Hi,
looks like Qpid experiences a similar thread-local issue on OS X:
https://issues.apache.org/jira/browse/QPID-2206
On Mon, Aug 15, 2011 at 6:30 AM, dsh <da...@googlemail.com> wrote:
> Hi Jean-Sebastien,
>
> the old SpiderMonkey port [10] already contains support for nspr - so
> that end should be covered cause I am using the old port descriptor
> files as a basis for the new 1.8.5 port. That's the type of embedded
> board [11] I am considering. It comes with an FPGA processor that
> accompanies the main CPU that I want to leverage to optimize certain
> operations (maybe that scenario is similar to what you are doing with
> offloading CPU cycles to the GPU).
>
> [11] http://soekris.com/net6501.htm
> [10] http://www.freebsd.org/cgi/ports.cgi?query=^spidermonkey&stype=all&sektion=all
>
> Cheers
> Daniel
>
> On Sun, Aug 14, 2011 at 11:28 PM, Jean-Sebastien Delfino
> <js...@apache.org> wrote:
>> On Thu, Aug 11, 2011 at 8:14 PM, dsh <da...@googlemail.com> wrote:
>> ...
>>>
>>> My attempt to provide an overview what works on FreeBSD (8.2) and what not:
>>>
>>> * autoconf, automake, libtool, doxygen, gcc can be installed from the
>>> FreeBSD ports collection
>>> * at the time there's only Apache HTTPD 2.2 available in the FreeBSD
>>> ports collection. I started creating an additional Apache HTTPD 2.3
>>> port [0] but did not finish it yet
>>> * APR 2 - I sent an problem report [1] including a new port for the
>>> most recent APR 2 version
>>> * OpenSSL can be used from the FreeBSD base system
>>> * pcre-8.12 can be used from the FreeBSD ports collection
>>> * expat-2.0.1_1 can be used from the FreeBSD ports collection
>>> * memcached-1.4.5 can be installed from the FreeBSD ports collection
>>> * libevent-1.4.14b_2 can be installed from the FreeBSD ports collection
>>> * libxml2-2.7.8_1 can be installed from the FreeBSD ports collection
>>> * curl-7.21.3_2 can be installed from the FreeBSD ports collection
>>> (including SSL support)
>>> * I am currently in the process of upgrading the existing FreeBSD
>>> SpiderMonkey 1.7.x port [2], [3]
>>> * tinycdb-0.77 is available from the FreeBSD ports collection
>>> * ap20-mod_security-2.5.13 is available from the FreeBSD ports
>>> collection (not sure whether this one is too old)
>>> * I started to create an Apache Axis2/C port for FreeBSD but hit an
>>> issue which may be a bug [4]
>>> * I started to create an Apache Qpid port for FreeBSD put hit an issue
>>> which may be a bug [5], [6]
>>> * ap20-mod_auth_openid-0.5 is available from the FreeBSD ports
>>> collection including support for libopkele-2.0.2_1
>>> * liboauth-0.9.4 is available from the FreeBSD ports collection
>>> * I think Apache Vysper can be installed relatively easy from the
>>> binary distribution and thus I decided to not create a separate
>>> FreeBSD port
>>> * Didn't create a FreeBSD port for Libstrophe yet cause I don't know
>>> how to pull sources directly from Github while building a FreeBSD port
>>> * postgresql-client-8.4.8 is available from the FreeBSD ports collection
>>> * scribe-2.2_3 is available from the FreeBSD ports collection
>>> * thrift-0.4.0,1 is available from the FreeBSD ports collection
>>> * I created and submitted a new FreeBSD port for Apache Libcloud using a PR [7]
>>>
>>> [0] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=284897+0+current/freebsd-ports
>>> [1] http://www.freebsd.org/cgi/query-pr.cgi?pr=159658
>>> [2] http://people.apache.org/~dsh/projects/fbsd-ports/spidermonkey185/
>>> [3] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=343488+0+current/freebsd-ports
>>> [4] http://mail-archives.apache.org/mod_mbox/axis-c-user/201108.mbox/%3CCAC0wh9aKzrVC3-qzHg3THMfHNVi20KZERB0zP94hq_X+d_FiFQ@mail.gmail.com%3E
>>> [5] https://issues.apache.org/jira/browse/QPID-2549
>>> [6] http://mail-archives.apache.org/mod_mbox/qpid-dev/201108.mbox/%3CCAC0wh9ZjuK0Ns_2LnAbM=oUfhcG2gaVSB99+pKPuYbRtr6KppA@mail.gmail.com%3E
>>> [7] http://www.freebsd.org/cgi/query-pr.cgi?pr=159703
>>>
>>
>> Wow, that's great progress! ... with a pretty long list of dependencies :)
>>
>> If there's too many issues with the dependencies it should be easy to
>> scale down to the short list I mentioned earlier [9].
>>
>> BTW js-1.8.5 also needs nspr-4.8.x to correctly work in multiple
>> threads. I just ran into that today while playing with massive
>> multi-threading in the context of the OpenCL integration.
>>
>> [9] http://marc.info/?l=tuscany-dev&m=131308575106061&w=2
>>
>> ...
>>> I am interested in Apache Tuscany native cause I want to figure out,
>>> whether it would be possible to use OSGi like modules (i.e. provided
>>> by Apache Celix) for SCA components similar to what I did with Java in
>>> a Redbook I wrote last year [8].
>>>
>>> Besides that I want to elaborate whether Apache Celix And Apache
>>> Tuscany (native) can be used on embedded devices such as
>>> router/firewall devices and what the use cases may be and thus my
>>> interest in creating all those various FreeBSD ports.
>>
>> Sounds interesting!
>>
>>> [8] http://www.redbooks.ibm.com/abstracts/sg247911.html?Open
>>>
>> --
>> Jean-Sebastien
>>
>
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Hi Jean-Sebastien,
the old SpiderMonkey port [10] already contains support for nspr - so
that end should be covered cause I am using the old port descriptor
files as a basis for the new 1.8.5 port. That's the type of embedded
board [11] I am considering. It comes with an FPGA processor that
accompanies the main CPU that I want to leverage to optimize certain
operations (maybe that scenario is similar to what you are doing with
offloading CPU cycles to the GPU).
[11] http://soekris.com/net6501.htm
[10] http://www.freebsd.org/cgi/ports.cgi?query=^spidermonkey&stype=all&sektion=all
Cheers
Daniel
On Sun, Aug 14, 2011 at 11:28 PM, Jean-Sebastien Delfino
<js...@apache.org> wrote:
> On Thu, Aug 11, 2011 at 8:14 PM, dsh <da...@googlemail.com> wrote:
> ...
>>
>> My attempt to provide an overview what works on FreeBSD (8.2) and what not:
>>
>> * autoconf, automake, libtool, doxygen, gcc can be installed from the
>> FreeBSD ports collection
>> * at the time there's only Apache HTTPD 2.2 available in the FreeBSD
>> ports collection. I started creating an additional Apache HTTPD 2.3
>> port [0] but did not finish it yet
>> * APR 2 - I sent an problem report [1] including a new port for the
>> most recent APR 2 version
>> * OpenSSL can be used from the FreeBSD base system
>> * pcre-8.12 can be used from the FreeBSD ports collection
>> * expat-2.0.1_1 can be used from the FreeBSD ports collection
>> * memcached-1.4.5 can be installed from the FreeBSD ports collection
>> * libevent-1.4.14b_2 can be installed from the FreeBSD ports collection
>> * libxml2-2.7.8_1 can be installed from the FreeBSD ports collection
>> * curl-7.21.3_2 can be installed from the FreeBSD ports collection
>> (including SSL support)
>> * I am currently in the process of upgrading the existing FreeBSD
>> SpiderMonkey 1.7.x port [2], [3]
>> * tinycdb-0.77 is available from the FreeBSD ports collection
>> * ap20-mod_security-2.5.13 is available from the FreeBSD ports
>> collection (not sure whether this one is too old)
>> * I started to create an Apache Axis2/C port for FreeBSD but hit an
>> issue which may be a bug [4]
>> * I started to create an Apache Qpid port for FreeBSD put hit an issue
>> which may be a bug [5], [6]
>> * ap20-mod_auth_openid-0.5 is available from the FreeBSD ports
>> collection including support for libopkele-2.0.2_1
>> * liboauth-0.9.4 is available from the FreeBSD ports collection
>> * I think Apache Vysper can be installed relatively easy from the
>> binary distribution and thus I decided to not create a separate
>> FreeBSD port
>> * Didn't create a FreeBSD port for Libstrophe yet cause I don't know
>> how to pull sources directly from Github while building a FreeBSD port
>> * postgresql-client-8.4.8 is available from the FreeBSD ports collection
>> * scribe-2.2_3 is available from the FreeBSD ports collection
>> * thrift-0.4.0,1 is available from the FreeBSD ports collection
>> * I created and submitted a new FreeBSD port for Apache Libcloud using a PR [7]
>>
>> [0] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=284897+0+current/freebsd-ports
>> [1] http://www.freebsd.org/cgi/query-pr.cgi?pr=159658
>> [2] http://people.apache.org/~dsh/projects/fbsd-ports/spidermonkey185/
>> [3] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=343488+0+current/freebsd-ports
>> [4] http://mail-archives.apache.org/mod_mbox/axis-c-user/201108.mbox/%3CCAC0wh9aKzrVC3-qzHg3THMfHNVi20KZERB0zP94hq_X+d_FiFQ@mail.gmail.com%3E
>> [5] https://issues.apache.org/jira/browse/QPID-2549
>> [6] http://mail-archives.apache.org/mod_mbox/qpid-dev/201108.mbox/%3CCAC0wh9ZjuK0Ns_2LnAbM=oUfhcG2gaVSB99+pKPuYbRtr6KppA@mail.gmail.com%3E
>> [7] http://www.freebsd.org/cgi/query-pr.cgi?pr=159703
>>
>
> Wow, that's great progress! ... with a pretty long list of dependencies :)
>
> If there's too many issues with the dependencies it should be easy to
> scale down to the short list I mentioned earlier [9].
>
> BTW js-1.8.5 also needs nspr-4.8.x to correctly work in multiple
> threads. I just ran into that today while playing with massive
> multi-threading in the context of the OpenCL integration.
>
> [9] http://marc.info/?l=tuscany-dev&m=131308575106061&w=2
>
> ...
>> I am interested in Apache Tuscany native cause I want to figure out,
>> whether it would be possible to use OSGi like modules (i.e. provided
>> by Apache Celix) for SCA components similar to what I did with Java in
>> a Redbook I wrote last year [8].
>>
>> Besides that I want to elaborate whether Apache Celix And Apache
>> Tuscany (native) can be used on embedded devices such as
>> router/firewall devices and what the use cases may be and thus my
>> interest in creating all those various FreeBSD ports.
>
> Sounds interesting!
>
>> [8] http://www.redbooks.ibm.com/abstracts/sg247911.html?Open
>>
> --
> Jean-Sebastien
>
Re: Fun with LevelDB, LLVM and OpenCL
Posted by Jean-Sebastien Delfino <js...@apache.org>.
On Thu, Aug 11, 2011 at 8:14 PM, dsh <da...@googlemail.com> wrote:
...
>
> My attempt to provide an overview what works on FreeBSD (8.2) and what not:
>
> * autoconf, automake, libtool, doxygen, gcc can be installed from the
> FreeBSD ports collection
> * at the time there's only Apache HTTPD 2.2 available in the FreeBSD
> ports collection. I started creating an additional Apache HTTPD 2.3
> port [0] but did not finish it yet
> * APR 2 - I sent an problem report [1] including a new port for the
> most recent APR 2 version
> * OpenSSL can be used from the FreeBSD base system
> * pcre-8.12 can be used from the FreeBSD ports collection
> * expat-2.0.1_1 can be used from the FreeBSD ports collection
> * memcached-1.4.5 can be installed from the FreeBSD ports collection
> * libevent-1.4.14b_2 can be installed from the FreeBSD ports collection
> * libxml2-2.7.8_1 can be installed from the FreeBSD ports collection
> * curl-7.21.3_2 can be installed from the FreeBSD ports collection
> (including SSL support)
> * I am currently in the process of upgrading the existing FreeBSD
> SpiderMonkey 1.7.x port [2], [3]
> * tinycdb-0.77 is available from the FreeBSD ports collection
> * ap20-mod_security-2.5.13 is available from the FreeBSD ports
> collection (not sure whether this one is too old)
> * I started to create an Apache Axis2/C port for FreeBSD but hit an
> issue which may be a bug [4]
> * I started to create an Apache Qpid port for FreeBSD put hit an issue
> which may be a bug [5], [6]
> * ap20-mod_auth_openid-0.5 is available from the FreeBSD ports
> collection including support for libopkele-2.0.2_1
> * liboauth-0.9.4 is available from the FreeBSD ports collection
> * I think Apache Vysper can be installed relatively easy from the
> binary distribution and thus I decided to not create a separate
> FreeBSD port
> * Didn't create a FreeBSD port for Libstrophe yet cause I don't know
> how to pull sources directly from Github while building a FreeBSD port
> * postgresql-client-8.4.8 is available from the FreeBSD ports collection
> * scribe-2.2_3 is available from the FreeBSD ports collection
> * thrift-0.4.0,1 is available from the FreeBSD ports collection
> * I created and submitted a new FreeBSD port for Apache Libcloud using a PR [7]
>
> [0] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=284897+0+current/freebsd-ports
> [1] http://www.freebsd.org/cgi/query-pr.cgi?pr=159658
> [2] http://people.apache.org/~dsh/projects/fbsd-ports/spidermonkey185/
> [3] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=343488+0+current/freebsd-ports
> [4] http://mail-archives.apache.org/mod_mbox/axis-c-user/201108.mbox/%3CCAC0wh9aKzrVC3-qzHg3THMfHNVi20KZERB0zP94hq_X+d_FiFQ@mail.gmail.com%3E
> [5] https://issues.apache.org/jira/browse/QPID-2549
> [6] http://mail-archives.apache.org/mod_mbox/qpid-dev/201108.mbox/%3CCAC0wh9ZjuK0Ns_2LnAbM=oUfhcG2gaVSB99+pKPuYbRtr6KppA@mail.gmail.com%3E
> [7] http://www.freebsd.org/cgi/query-pr.cgi?pr=159703
>
Wow, that's great progress! ... with a pretty long list of dependencies :)
If there's too many issues with the dependencies it should be easy to
scale down to the short list I mentioned earlier [9].
BTW js-1.8.5 also needs nspr-4.8.x to correctly work in multiple
threads. I just ran into that today while playing with massive
multi-threading in the context of the OpenCL integration.
[9] http://marc.info/?l=tuscany-dev&m=131308575106061&w=2
...
> I am interested in Apache Tuscany native cause I want to figure out,
> whether it would be possible to use OSGi like modules (i.e. provided
> by Apache Celix) for SCA components similar to what I did with Java in
> a Redbook I wrote last year [8].
>
> Besides that I want to elaborate whether Apache Celix And Apache
> Tuscany (native) can be used on embedded devices such as
> router/firewall devices and what the use cases may be and thus my
> interest in creating all those various FreeBSD ports.
Sounds interesting!
> [8] http://www.redbooks.ibm.com/abstracts/sg247911.html?Open
>
--
Jean-Sebastien
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Hi Jean-Sebastien,
On Thu, Aug 11, 2011 at 7:53 PM, Jean-Sebastien Delfino
<js...@apache.org> wrote:
>> thanks for your feedback. You are right as long as Tuscany native
>> hasn't been released as a pre-build package that can be installed on
>> an operating system it doesn't make sense to use libraries as
>> dependencies coming pre-packaged with a particular OS or package
>> management system. Instead I should setup my own staging area
>> containing all the various dependencies. Tho, in the long run I am
>> interested in pushing those dependencies as FreeBSD ports into the
>> FreeBSD ports tree cause there you could have development snapshots of
>> a certain library as a FreeBSD port which would allow to use the
>> dependencies coming with the FreeBSD ports tree instead of manually
>> compiled ones (we did that with DSPAM development snapshots in the
>> past).
>
> That would be really cool!
>
My attempt to provide an overview what works on FreeBSD (8.2) and what not:
* autoconf, automake, libtool, doxygen, gcc can be installed from the
FreeBSD ports collection
* at the time there's only Apache HTTPD 2.2 available in the FreeBSD
ports collection. I started creating an additional Apache HTTPD 2.3
port [0] but did not finish it yet
* APR 2 - I sent an problem report [1] including a new port for the
most recent APR 2 version
* OpenSSL can be used from the FreeBSD base system
* pcre-8.12 can be used from the FreeBSD ports collection
* expat-2.0.1_1 can be used from the FreeBSD ports collection
* memcached-1.4.5 can be installed from the FreeBSD ports collection
* libevent-1.4.14b_2 can be installed from the FreeBSD ports collection
* libxml2-2.7.8_1 can be installed from the FreeBSD ports collection
* curl-7.21.3_2 can be installed from the FreeBSD ports collection
(including SSL support)
* I am currently in the process of upgrading the existing FreeBSD
SpiderMonkey 1.7.x port [2], [3]
* tinycdb-0.77 is available from the FreeBSD ports collection
* ap20-mod_security-2.5.13 is available from the FreeBSD ports
collection (not sure whether this one is too old)
* I started to create an Apache Axis2/C port for FreeBSD but hit an
issue which may be a bug [4]
* I started to create an Apache Qpid port for FreeBSD put hit an issue
which may be a bug [5], [6]
* ap20-mod_auth_openid-0.5 is available from the FreeBSD ports
collection including support for libopkele-2.0.2_1
* liboauth-0.9.4 is available from the FreeBSD ports collection
* I think Apache Vysper can be installed relatively easy from the
binary distribution and thus I decided to not create a separate
FreeBSD port
* Didn't create a FreeBSD port for Libstrophe yet cause I don't know
how to pull sources directly from Github while building a FreeBSD port
* postgresql-client-8.4.8 is available from the FreeBSD ports collection
* scribe-2.2_3 is available from the FreeBSD ports collection
* thrift-0.4.0,1 is available from the FreeBSD ports collection
* I created and submitted a new FreeBSD port for Apache Libcloud using a PR [7]
[0] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=284897+0+current/freebsd-ports
[1] http://www.freebsd.org/cgi/query-pr.cgi?pr=159658
[2] http://people.apache.org/~dsh/projects/fbsd-ports/spidermonkey185/
[3] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=343488+0+current/freebsd-ports
[4] http://mail-archives.apache.org/mod_mbox/axis-c-user/201108.mbox/%3CCAC0wh9aKzrVC3-qzHg3THMfHNVi20KZERB0zP94hq_X+d_FiFQ@mail.gmail.com%3E
[5] https://issues.apache.org/jira/browse/QPID-2549
[6] http://mail-archives.apache.org/mod_mbox/qpid-dev/201108.mbox/%3CCAC0wh9ZjuK0Ns_2LnAbM=oUfhcG2gaVSB99+pKPuYbRtr6KppA@mail.gmail.com%3E
[7] http://www.freebsd.org/cgi/query-pr.cgi?pr=159703
>
> We may be able to do that if people like you start to help out... as I
> only have little free evening / weekend / vacation time for this.
>
I am interested in Apache Tuscany native cause I want to figure out,
whether it would be possible to use OSGi like modules (i.e. provided
by Apache Celix) for SCA components similar to what I did with Java in
a Redbook I wrote last year [8].
Besides that I want to elaborate whether Apache Celix And Apache
Tuscany (native) can be used on embedded devices such as
router/firewall devices and what the use cases may be and thus my
interest in creating all those various FreeBSD ports.
[8] http://www.redbooks.ibm.com/abstracts/sg247911.html?Open
Cheers
Daniel
Re: Fun with LevelDB, LLVM and OpenCL
Posted by Jean-Sebastien Delfino <js...@apache.org>.
On Wed, Aug 10, 2011 at 7:02 AM, dsh <da...@googlemail.com> wrote:
> Hi Jean-Sebastien,
>
> thanks for your feedback. You are right as long as Tuscany native
> hasn't been released as a pre-build package that can be installed on
> an operating system it doesn't make sense to use libraries as
> dependencies coming pre-packaged with a particular OS or package
> management system. Instead I should setup my own staging area
> containing all the various dependencies. Tho, in the long run I am
> interested in pushing those dependencies as FreeBSD ports into the
> FreeBSD ports tree cause there you could have development snapshots of
> a certain library as a FreeBSD port which would allow to use the
> dependencies coming with the FreeBSD ports tree instead of manually
> compiled ones (we did that with DSPAM development snapshots in the
> past).
That would be really cool!
> on another subject - Is there a new Tuscany native release planned
> anytime soon? M3 seems to date back to 2007. I am asking cause at
> least at the time you would plan to ship a new release -- and maybe
> even pre-build packages for certain Linux/Unix distributions such as
> DEBs or RPMs -- you would have to use the packages available on a
> certain OS.
>
Yeah it'd be great to get a release out.
A minimum release (without all the optional component) would only have
dependencies on expat, pcre, apr, httpd, libevent, tinycdb, curl,
libxml2, libmozjs. Now that we've upgraded to mozjs-1.8.5, I think
we'd only need to wait for the next apr release including apr-2 (or
make that dependency optional as well).
A source release could let the developer choose to use dependencies
from system packages when available or to build them from source as
well. I believe that's how apr and httpd are released by Apache for
example, source only and it's up to the packager to decide how to
build them on a particular platform.
We may be able to do that if people like you start to help out... as I
only have little free evening / weekend / vacation time for this.
To release Linux .deb or .rpm packages we'd need recent levels of apr,
httpd (2.3+), and mozjs (1.8.5) as system packages. I believe the
other ones are OK on most Linuxes.
--
Jean-Sebastien
Re: Fun with LevelDB, LLVM and OpenCL
Posted by Giorgio Zoppi <gi...@gmail.com>.
IMHO what you need is a python script who is independent from the OS
which it will cope your dependencies as maven does for building all
the stuff.
2011/8/10 dsh <da...@googlemail.com>:
> Hi Jean-Sebastien,
>
> thanks for your feedback. You are right as long as Tuscany native
> hasn't been released as a pre-build package that can be installed on
> an operating system it doesn't make sense to use libraries as
> dependencies coming pre-packaged with a particular OS or package
> management system. Instead I should setup my own staging area
> containing all the various dependencies. Tho, in the long run I am
> interested in pushing those dependencies as FreeBSD ports into the
> FreeBSD ports tree cause there you could have development snapshots of
> a certain library as a FreeBSD port which would allow to use the
> dependencies coming with the FreeBSD ports tree instead of manually
> compiled ones (we did that with DSPAM development snapshots in the
> past).
>
> on another subject - Is there a new Tuscany native release planned
> anytime soon? M3 seems to date back to 2007. I am asking cause at
> least at the time you would plan to ship a new release -- and maybe
> even pre-build packages for certain Linux/Unix distributions such as
> DEBs or RPMs -- you would have to use the packages available on a
> certain OS.
>
> Cheers
> Daniel
>
> On Wed, Aug 10, 2011 at 5:26 AM, Jean-Sebastien Delfino
> <js...@apache.org> wrote:
>> On Tue, Aug 9, 2011 at 3:53 PM, dsh <da...@googlemail.com> wrote:
>>> Hi Jean-Sebastien Delfino,
>>>
>>> some things I noticed during the configure process (I am using MacPorts):
>>>
>>> * I think it would help to point out in the INSTALL file that mozjs
>>> can be downloaded at -> ftp://ftp.mozilla.org/pub/mozilla.org/js/
>>> * On OSX Lion xulrunner won't install from MacPorts and thus one would
>>> end up without a proper mozjs installation
>>> * If compiling mozjs from the prio mentioned URL, it ends up to be
>>> installed as /opt/local/lib/libmozjs185.1.0.0.dylib which causes the
>>> configure process to fail
>>> * I thus symlinked /opt/local/lib/libmozjs185.1.0.0.dylib to
>>> /opt/local/lib/libmozjs.dylib
>>
>> I usually run configure
>> --with-js-include=$build/tracemonkey-bin/include/js
>> --with-js-lib=$build/tracemonkey-bin/lib
>> with $build pointing to a non-system directory;
>>
>> ... after having built trace monkey like this:
>> curl -OL http://hg.mozilla.org/tracemonkey/archive/e4364736e170.tar.gz
>> mv e4364736e170.tar.gz tracemonkey-e4364736e170.tar.gz
>> tar xzf tracemonkey-e4364736e170.tar.gz
>> cd tracemonkey-e4364736e170/js/src
>> $build/autoconf-2.13-bin/bin/autoconf
>> ./configure --prefix=$build/tracemonkey-bin
>> make
>> make install
>>
>> ... and on Mac OS X this additional step:
>> install_name_tool -id $build/tracemonkey-bin/lib/libmozjs.dylib
>> $build/tracemonkey-bin/lib/libmozjs.dylib
>>
>> I've never had much luck installing libmozjs in the system directories
>> for all kinds of reasons. On Ubuntu and Redhat I had conflicts with
>> already installed incompatible versions, dependencies dragged by
>> xulrunner, and IIRC similar issues on Mac OS X with macports too.
>>
>> Thanks for the link to js 185, I'll try it on Ubuntu, Redhat and Mac
>> OS X too, happy to switch to it if it works on these systems, but I'd
>> like to avoid installing it in the default system directories (hoping
>> there's a way to specify the install dir with --prefix).
>>
>> More generally I try to stay away from using pre-built packaged system
>> dependencies for the following reasons:
>>
>> - most of the times they system dependencies conflict with the
>> versions of the dependencies I want, for example Ubuntu 10.10 ships
>> with Apache httpd 2.2.16 which is really old compared to the 2.3.10
>> version used by Tuscany;
>>
>> - using the system dependencies shipped with the operating system
>> means versions on different systems, different behavior and
>> significant effort to port and test with all these different versions;
>>
>> - if I need to debug or patch them I'll have to recompile them anyway,
>> and if I do that then I'll have to install them in a non-system
>> directory to not mess up my system; After all, that's the beauty of
>> open-source, you can get the source, compile it yourself and install
>> it wherever is most convenient for you...
>>
>> - if I need to install Tuscany and its dependencies on a new machine
>> (an EC2 VM for example), it's much easier to build and install all the
>> dependencies in the same directory as Tuscany, zip that directory up,
>> and transfer that zip to the target VMs without altering its system
>> directories.
>>
>>> * The configure process is looking for -lapr-2 instead of -lapr-1 (I
>>> have APR 1.4.5_1 installed)
>>> * I thus had to symlink /opt/local/lib/libapr-1.0.dylib to
>>> /opt/local/lib/libapr-2.dylib
>>
>> Yes I've been using apr-2 as it's required by httpd's mod-session for
>> crypto. See the related email discussion on the httpd dev list there:
>> http://markmail.org/thread/dbavlf55ywf2a33h
>>
>> I don't think that APR 1.4.5 includes the crypto API required by httpd
>> 2.3.x mod_session_crypto yet, so we're stuck with APR trunk for now,
>> until APR releases that API and httpd ports to that release.
>>
>>>
>>> If you like I could try to update the INSTALL document accordingly and
>>> attach the patch to a JIRA.
>>
>> That would help, but what would help even more , I think, would be to
>> have a script that downloads / checks out the dependencies and builds
>> them end to end on a particular system. The single INSTALL doc may get
>> really big if we start to add to it all the different instructions to
>> build off system dependencies on Ubuntu, Redhat, Mac OS X, FreeBSD
>> etc...
>>
>> There's already a script to build everything on Ubuntu 10.x there:
>> https://svn.apache.org/repos/asf/tuscany/sca-cpp/trunk/ubuntu/ubuntu-install
>>
>> There's another one for Mac OS X Leopard and Lion there:
>> https://svn.apache.org/repos/asf/tuscany/sca-cpp/trunk/macos/macos-install
>>
>> ... which by the way should help you build on MacOS without fighting
>> with any system dependencies. With that script, you should be able to
>> build on a pristine Mac OS system + GCC 4.4+ from Macport or other,
>> without polluting your system directories with any other dependencies
>> at all... as all required dependencies are built from source by the
>> script.
>>
>>> I am doing the dependency setup in
>>> parallel on FreeBSD 8.2 to see whether I would run into similar issues
>>> (for instance the one of having a -lapr-1 installed instead of the
>>> expected -lapr-2). We could then decide whether we want to improve the
>>> auto* tools infrastructure or not (or even migrate to CMake).
>>>
>>> PS: I am planning to create a FreeBSD port for Apache Tuscany (native)
>>> cause I feel having to setup all of these dependencies is kind of
>>> daunting.
>>
>> Cool, it'd be really cool to have another similar build script for
>> FreeBSD too :)
>>
>>> Cheers
>>> Daniel
>>>
>>
>> --
>> Jean-Sebastien
>>
>
--
Quiero ser el rayo de sol que cada día te despierta
para hacerte respirar y vivir en me.
"Favola -Moda".
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Hi Jean-Sebastien,
thanks for your feedback. You are right as long as Tuscany native
hasn't been released as a pre-build package that can be installed on
an operating system it doesn't make sense to use libraries as
dependencies coming pre-packaged with a particular OS or package
management system. Instead I should setup my own staging area
containing all the various dependencies. Tho, in the long run I am
interested in pushing those dependencies as FreeBSD ports into the
FreeBSD ports tree cause there you could have development snapshots of
a certain library as a FreeBSD port which would allow to use the
dependencies coming with the FreeBSD ports tree instead of manually
compiled ones (we did that with DSPAM development snapshots in the
past).
on another subject - Is there a new Tuscany native release planned
anytime soon? M3 seems to date back to 2007. I am asking cause at
least at the time you would plan to ship a new release -- and maybe
even pre-build packages for certain Linux/Unix distributions such as
DEBs or RPMs -- you would have to use the packages available on a
certain OS.
Cheers
Daniel
On Wed, Aug 10, 2011 at 5:26 AM, Jean-Sebastien Delfino
<js...@apache.org> wrote:
> On Tue, Aug 9, 2011 at 3:53 PM, dsh <da...@googlemail.com> wrote:
>> Hi Jean-Sebastien Delfino,
>>
>> some things I noticed during the configure process (I am using MacPorts):
>>
>> * I think it would help to point out in the INSTALL file that mozjs
>> can be downloaded at -> ftp://ftp.mozilla.org/pub/mozilla.org/js/
>> * On OSX Lion xulrunner won't install from MacPorts and thus one would
>> end up without a proper mozjs installation
>> * If compiling mozjs from the prio mentioned URL, it ends up to be
>> installed as /opt/local/lib/libmozjs185.1.0.0.dylib which causes the
>> configure process to fail
>> * I thus symlinked /opt/local/lib/libmozjs185.1.0.0.dylib to
>> /opt/local/lib/libmozjs.dylib
>
> I usually run configure
> --with-js-include=$build/tracemonkey-bin/include/js
> --with-js-lib=$build/tracemonkey-bin/lib
> with $build pointing to a non-system directory;
>
> ... after having built trace monkey like this:
> curl -OL http://hg.mozilla.org/tracemonkey/archive/e4364736e170.tar.gz
> mv e4364736e170.tar.gz tracemonkey-e4364736e170.tar.gz
> tar xzf tracemonkey-e4364736e170.tar.gz
> cd tracemonkey-e4364736e170/js/src
> $build/autoconf-2.13-bin/bin/autoconf
> ./configure --prefix=$build/tracemonkey-bin
> make
> make install
>
> ... and on Mac OS X this additional step:
> install_name_tool -id $build/tracemonkey-bin/lib/libmozjs.dylib
> $build/tracemonkey-bin/lib/libmozjs.dylib
>
> I've never had much luck installing libmozjs in the system directories
> for all kinds of reasons. On Ubuntu and Redhat I had conflicts with
> already installed incompatible versions, dependencies dragged by
> xulrunner, and IIRC similar issues on Mac OS X with macports too.
>
> Thanks for the link to js 185, I'll try it on Ubuntu, Redhat and Mac
> OS X too, happy to switch to it if it works on these systems, but I'd
> like to avoid installing it in the default system directories (hoping
> there's a way to specify the install dir with --prefix).
>
> More generally I try to stay away from using pre-built packaged system
> dependencies for the following reasons:
>
> - most of the times they system dependencies conflict with the
> versions of the dependencies I want, for example Ubuntu 10.10 ships
> with Apache httpd 2.2.16 which is really old compared to the 2.3.10
> version used by Tuscany;
>
> - using the system dependencies shipped with the operating system
> means versions on different systems, different behavior and
> significant effort to port and test with all these different versions;
>
> - if I need to debug or patch them I'll have to recompile them anyway,
> and if I do that then I'll have to install them in a non-system
> directory to not mess up my system; After all, that's the beauty of
> open-source, you can get the source, compile it yourself and install
> it wherever is most convenient for you...
>
> - if I need to install Tuscany and its dependencies on a new machine
> (an EC2 VM for example), it's much easier to build and install all the
> dependencies in the same directory as Tuscany, zip that directory up,
> and transfer that zip to the target VMs without altering its system
> directories.
>
>> * The configure process is looking for -lapr-2 instead of -lapr-1 (I
>> have APR 1.4.5_1 installed)
>> * I thus had to symlink /opt/local/lib/libapr-1.0.dylib to
>> /opt/local/lib/libapr-2.dylib
>
> Yes I've been using apr-2 as it's required by httpd's mod-session for
> crypto. See the related email discussion on the httpd dev list there:
> http://markmail.org/thread/dbavlf55ywf2a33h
>
> I don't think that APR 1.4.5 includes the crypto API required by httpd
> 2.3.x mod_session_crypto yet, so we're stuck with APR trunk for now,
> until APR releases that API and httpd ports to that release.
>
>>
>> If you like I could try to update the INSTALL document accordingly and
>> attach the patch to a JIRA.
>
> That would help, but what would help even more , I think, would be to
> have a script that downloads / checks out the dependencies and builds
> them end to end on a particular system. The single INSTALL doc may get
> really big if we start to add to it all the different instructions to
> build off system dependencies on Ubuntu, Redhat, Mac OS X, FreeBSD
> etc...
>
> There's already a script to build everything on Ubuntu 10.x there:
> https://svn.apache.org/repos/asf/tuscany/sca-cpp/trunk/ubuntu/ubuntu-install
>
> There's another one for Mac OS X Leopard and Lion there:
> https://svn.apache.org/repos/asf/tuscany/sca-cpp/trunk/macos/macos-install
>
> ... which by the way should help you build on MacOS without fighting
> with any system dependencies. With that script, you should be able to
> build on a pristine Mac OS system + GCC 4.4+ from Macport or other,
> without polluting your system directories with any other dependencies
> at all... as all required dependencies are built from source by the
> script.
>
>> I am doing the dependency setup in
>> parallel on FreeBSD 8.2 to see whether I would run into similar issues
>> (for instance the one of having a -lapr-1 installed instead of the
>> expected -lapr-2). We could then decide whether we want to improve the
>> auto* tools infrastructure or not (or even migrate to CMake).
>>
>> PS: I am planning to create a FreeBSD port for Apache Tuscany (native)
>> cause I feel having to setup all of these dependencies is kind of
>> daunting.
>
> Cool, it'd be really cool to have another similar build script for
> FreeBSD too :)
>
>> Cheers
>> Daniel
>>
>
> --
> Jean-Sebastien
>
Re: Fun with LevelDB, LLVM and OpenCL
Posted by Jean-Sebastien Delfino <js...@apache.org>.
On Tue, Aug 9, 2011 at 3:53 PM, dsh <da...@googlemail.com> wrote:
> Hi Jean-Sebastien Delfino,
>
> some things I noticed during the configure process (I am using MacPorts):
>
> * I think it would help to point out in the INSTALL file that mozjs
> can be downloaded at -> ftp://ftp.mozilla.org/pub/mozilla.org/js/
> * On OSX Lion xulrunner won't install from MacPorts and thus one would
> end up without a proper mozjs installation
> * If compiling mozjs from the prio mentioned URL, it ends up to be
> installed as /opt/local/lib/libmozjs185.1.0.0.dylib which causes the
> configure process to fail
> * I thus symlinked /opt/local/lib/libmozjs185.1.0.0.dylib to
> /opt/local/lib/libmozjs.dylib
I usually run configure
--with-js-include=$build/tracemonkey-bin/include/js
--with-js-lib=$build/tracemonkey-bin/lib
with $build pointing to a non-system directory;
... after having built trace monkey like this:
curl -OL http://hg.mozilla.org/tracemonkey/archive/e4364736e170.tar.gz
mv e4364736e170.tar.gz tracemonkey-e4364736e170.tar.gz
tar xzf tracemonkey-e4364736e170.tar.gz
cd tracemonkey-e4364736e170/js/src
$build/autoconf-2.13-bin/bin/autoconf
./configure --prefix=$build/tracemonkey-bin
make
make install
... and on Mac OS X this additional step:
install_name_tool -id $build/tracemonkey-bin/lib/libmozjs.dylib
$build/tracemonkey-bin/lib/libmozjs.dylib
I've never had much luck installing libmozjs in the system directories
for all kinds of reasons. On Ubuntu and Redhat I had conflicts with
already installed incompatible versions, dependencies dragged by
xulrunner, and IIRC similar issues on Mac OS X with macports too.
Thanks for the link to js 185, I'll try it on Ubuntu, Redhat and Mac
OS X too, happy to switch to it if it works on these systems, but I'd
like to avoid installing it in the default system directories (hoping
there's a way to specify the install dir with --prefix).
More generally I try to stay away from using pre-built packaged system
dependencies for the following reasons:
- most of the times they system dependencies conflict with the
versions of the dependencies I want, for example Ubuntu 10.10 ships
with Apache httpd 2.2.16 which is really old compared to the 2.3.10
version used by Tuscany;
- using the system dependencies shipped with the operating system
means versions on different systems, different behavior and
significant effort to port and test with all these different versions;
- if I need to debug or patch them I'll have to recompile them anyway,
and if I do that then I'll have to install them in a non-system
directory to not mess up my system; After all, that's the beauty of
open-source, you can get the source, compile it yourself and install
it wherever is most convenient for you...
- if I need to install Tuscany and its dependencies on a new machine
(an EC2 VM for example), it's much easier to build and install all the
dependencies in the same directory as Tuscany, zip that directory up,
and transfer that zip to the target VMs without altering its system
directories.
> * The configure process is looking for -lapr-2 instead of -lapr-1 (I
> have APR 1.4.5_1 installed)
> * I thus had to symlink /opt/local/lib/libapr-1.0.dylib to
> /opt/local/lib/libapr-2.dylib
Yes I've been using apr-2 as it's required by httpd's mod-session for
crypto. See the related email discussion on the httpd dev list there:
http://markmail.org/thread/dbavlf55ywf2a33h
I don't think that APR 1.4.5 includes the crypto API required by httpd
2.3.x mod_session_crypto yet, so we're stuck with APR trunk for now,
until APR releases that API and httpd ports to that release.
>
> If you like I could try to update the INSTALL document accordingly and
> attach the patch to a JIRA.
That would help, but what would help even more , I think, would be to
have a script that downloads / checks out the dependencies and builds
them end to end on a particular system. The single INSTALL doc may get
really big if we start to add to it all the different instructions to
build off system dependencies on Ubuntu, Redhat, Mac OS X, FreeBSD
etc...
There's already a script to build everything on Ubuntu 10.x there:
https://svn.apache.org/repos/asf/tuscany/sca-cpp/trunk/ubuntu/ubuntu-install
There's another one for Mac OS X Leopard and Lion there:
https://svn.apache.org/repos/asf/tuscany/sca-cpp/trunk/macos/macos-install
... which by the way should help you build on MacOS without fighting
with any system dependencies. With that script, you should be able to
build on a pristine Mac OS system + GCC 4.4+ from Macport or other,
without polluting your system directories with any other dependencies
at all... as all required dependencies are built from source by the
script.
> I am doing the dependency setup in
> parallel on FreeBSD 8.2 to see whether I would run into similar issues
> (for instance the one of having a -lapr-1 installed instead of the
> expected -lapr-2). We could then decide whether we want to improve the
> auto* tools infrastructure or not (or even migrate to CMake).
>
> PS: I am planning to create a FreeBSD port for Apache Tuscany (native)
> cause I feel having to setup all of these dependencies is kind of
> daunting.
Cool, it'd be really cool to have another similar build script for
FreeBSD too :)
> Cheers
> Daniel
>
--
Jean-Sebastien
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Hi Jean-Sebastien Delfino,
some things I noticed during the configure process (I am using MacPorts):
* I think it would help to point out in the INSTALL file that mozjs
can be downloaded at -> ftp://ftp.mozilla.org/pub/mozilla.org/js/
* On OSX Lion xulrunner won't install from MacPorts and thus one would
end up without a proper mozjs installation
* If compiling mozjs from the prio mentioned URL, it ends up to be
installed as /opt/local/lib/libmozjs185.1.0.0.dylib which causes the
configure process to fail
* I thus symlinked /opt/local/lib/libmozjs185.1.0.0.dylib to
/opt/local/lib/libmozjs.dylib
* The configure process is looking for -lapr-2 instead of -lapr-1 (I
have APR 1.4.5_1 installed)
* I thus had to symlink /opt/local/lib/libapr-1.0.dylib to
/opt/local/lib/libapr-2.dylib
If you like I could try to update the INSTALL document accordingly and
attach the patch to a JIRA. I am doing the dependency setup in
parallel on FreeBSD 8.2 to see whether I would run into similar issues
(for instance the one of having a -lapr-1 installed instead of the
expected -lapr-2). We could then decide whether we want to improve the
auto* tools infrastructure or not (or even migrate to CMake).
PS: I am planning to create a FreeBSD port for Apache Tuscany (native)
cause I feel having to setup all of these dependencies is kind of
daunting.
Cheers
Daniel
On Mon, Aug 8, 2011 at 7:05 PM, Jean-Sebastien Delfino
<js...@apache.org> wrote:
> On Sun, Aug 7, 2011 at 6:35 PM, dsh <da...@googlemail.com> wrote:
>> Just a little update:
>>
>> OSX Lion with Xcode 4.1:
>>
>> * i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc.
>> build 5658) (LLVM build 2335.15.00)
>> * Apple clang version 2.1
>>
>> OSX Lion with Xcode 4.2:
>>
>> * i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1
>> * Apple clang version 3.0
>>
>> OSX Snow Leopard with Xcode 4.0.2:
>>
>> * i686-apple-darwin10-llvm-gcc-4.2 (GCC) 4.2.1
>> * Apple clang version 2.0 based on LLVM 2.9svn
>>
>> PS: There exists an Apple Xcode version 4.2 for OSX Snow Leopard which
>> we could use as a fallback if 4.0.2 turns out to be unusable
>> PPS: I really love these virtualized OSX envs I setup recently because
>> they allow to have all those different setups and configurations :)
>>
>> Cheers
>> Daniel
>>
>
> OK, here's what I'm using on OS X Lion:
>
> Xcode Version 4.1 Build 4B110
>
> macbook-air:~$ g++ --version
> i686-apple-darwin11-llvm-g++-4.2 (GCC) 4.2.1 (Based on Apple Inc.
> build 5658) (LLVM build 2335.15.00)
>
> macbook-air:~$ clang++ --version
> Apple clang version 2.1 (tags/Apple/clang-163.7.1) (based on LLVM 3.0svn)
> Target: x86_64-apple-darwin11.0.0
> Thread model: posix
>
> --
> Jean-Sebastien
>
Re: Fun with LevelDB, LLVM and OpenCL
Posted by Jean-Sebastien Delfino <js...@apache.org>.
On Sun, Aug 7, 2011 at 6:35 PM, dsh <da...@googlemail.com> wrote:
> Just a little update:
>
> OSX Lion with Xcode 4.1:
>
> * i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc.
> build 5658) (LLVM build 2335.15.00)
> * Apple clang version 2.1
>
> OSX Lion with Xcode 4.2:
>
> * i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1
> * Apple clang version 3.0
>
> OSX Snow Leopard with Xcode 4.0.2:
>
> * i686-apple-darwin10-llvm-gcc-4.2 (GCC) 4.2.1
> * Apple clang version 2.0 based on LLVM 2.9svn
>
> PS: There exists an Apple Xcode version 4.2 for OSX Snow Leopard which
> we could use as a fallback if 4.0.2 turns out to be unusable
> PPS: I really love these virtualized OSX envs I setup recently because
> they allow to have all those different setups and configurations :)
>
> Cheers
> Daniel
>
OK, here's what I'm using on OS X Lion:
Xcode Version 4.1 Build 4B110
macbook-air:~$ g++ --version
i686-apple-darwin11-llvm-g++-4.2 (GCC) 4.2.1 (Based on Apple Inc.
build 5658) (LLVM build 2335.15.00)
macbook-air:~$ clang++ --version
Apple clang version 2.1 (tags/Apple/clang-163.7.1) (based on LLVM 3.0svn)
Target: x86_64-apple-darwin11.0.0
Thread model: posix
--
Jean-Sebastien
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Just a little update:
OSX Lion with Xcode 4.1:
* i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc.
build 5658) (LLVM build 2335.15.00)
* Apple clang version 2.1
OSX Lion with Xcode 4.2:
* i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1
* Apple clang version 3.0
OSX Snow Leopard with Xcode 4.0.2:
* i686-apple-darwin10-llvm-gcc-4.2 (GCC) 4.2.1
* Apple clang version 2.0 based on LLVM 2.9svn
PS: There exists an Apple Xcode version 4.2 for OSX Snow Leopard which
we could use as a fallback if 4.0.2 turns out to be unusable
PPS: I really love these virtualized OSX envs I setup recently because
they allow to have all those different setups and configurations :)
Cheers
Daniel
On Sun, Aug 7, 2011 at 1:52 AM, dsh <da...@googlemail.com> wrote:
> On Sun, Aug 7, 2011 at 12:34 AM, Jean-Sebastien Delfino
> <js...@apache.org> wrote:
>>
>> The configure.ac language is a little cryptic so let me know if you
>> need help with that detection + option setup, as I've already done
>> similar things several times.
>>
>
> Ok I'll let you know if I struggle with any of the GNU Build Tools
> files such as the autoconf files. I worked on Gnome, Avahi, wzdftpd,
> PostgreSQL, OpenLDAP and some other autoconf based projects back in
> 2005/2006 or so. Thus I hope it doesn't feel too alienating to me :)
>
> Thanks and best wishes!
>
> Cheers
> Daniel
>
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
On Sun, Aug 7, 2011 at 12:34 AM, Jean-Sebastien Delfino
<js...@apache.org> wrote:
>
> The configure.ac language is a little cryptic so let me know if you
> need help with that detection + option setup, as I've already done
> similar things several times.
>
Ok I'll let you know if I struggle with any of the GNU Build Tools
files such as the autoconf files. I worked on Gnome, Avahi, wzdftpd,
PostgreSQL, OpenLDAP and some other autoconf based projects back in
2005/2006 or so. Thus I hope it doesn't feel too alienating to me :)
Thanks and best wishes!
Cheers
Daniel
Re: Fun with LevelDB, LLVM and OpenCL
Posted by Jean-Sebastien Delfino <js...@apache.org>.
On Sat, Aug 6, 2011 at 2:02 PM, dsh <da...@googlemail.com> wrote:
> Hi Jean-Sebastien,
>
> for which Xcode versions shall we be heading for? 4.1 or 4.2 on Lion
> (I have 4.2 dev preview 5 on one of my Lion systems to develop iOS 5
> apps) and 4.0.2 on snow leopard? I would then check which would be the
> min LLVM/clang version that needs to be supported.
If I remember correctly 4.0.2 didn't have all the required C++0x
support (I didn't research it extensively though), but I'm pretty sure
that 4.1 and 4.2 are based on LLVM 3.0+ and should work. I have Xcode
4.1 on Lion right now, but if you only manage to make it work with
Xcode 4.2 that's OK with me too, I'll just upgrade to 4.2 as well :).
> Shall we try to keep GCC compatibility if possible (e.g. should it be
> possible to build Tuscany on platforms where there's no LLVM
> distribution available)? On FreeBSD for instance it's your own
> decision whether to compile the system's user-land utilities etc.
> either using GCC or LLVM/clang. FreeBSD makes this possible by either
> setting CC to CC=gcc or CC=clang and so on in /etc/make.conf. Maybe we
> could use a similar mechanism...
Yes, I'm interested in keeping compatibility with GCC for some time to
run it on my EC2 and home servers, which don't have LLVM yet... Best
would be to detect availability of LLVM in configure.ac and
automatically point CC and CXX to it when available, and have a
--with-llvm=yes/no configure option to override that if needed (like
with the other configure options listed by configure --help).
The configure.ac language is a little cryptic so let me know if you
need help with that detection + option setup, as I've already done
similar things several times.
>
> Concerning CMake - That has for now low priority to me I would focus
> on LLVM and if that is done on CMake.
>
Sounds good!
--
Jean-Sebastien
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Hi Jean-Sebastien,
for which Xcode versions shall we be heading for? 4.1 or 4.2 on Lion
(I have 4.2 dev preview 5 on one of my Lion systems to develop iOS 5
apps) and 4.0.2 on snow leopard? I would then check which would be the
min LLVM/clang version that needs to be supported.
Shall we try to keep GCC compatibility if possible (e.g. should it be
possible to build Tuscany on platforms where there's no LLVM
distribution available)? On FreeBSD for instance it's your own
decision whether to compile the system's user-land utilities etc.
either using GCC or LLVM/clang. FreeBSD makes this possible by either
setting CC to CC=gcc or CC=clang and so on in /etc/make.conf. Maybe we
could use a similar mechanism...
Concerning CMake - That has for now low priority to me I would focus
on LLVM and if that is done on CMake.
Cheers
Daniel
On Sat, Aug 6, 2011 at 8:14 PM, Jean-Sebastien Delfino
<js...@apache.org> wrote:
> On Sat, Aug 6, 2011 at 2:14 AM, dsh <da...@googlemail.com> wrote:
>> Hi Jean-Sebastien,
>>
>> if you like I could take a look at LLVM/clang. From my point of view
>> it not only integrates more nicely with Xcode but you would as well
>> get rid of dynamically linking against things like libgcc* which
>> sometimes triggers licensing discussions if talking to attorneys.
>>
>> If you think it would make sense I could also have a look at replacing
>> the autotools stuff (automake) by CMake. The latter would generate IDE
>> project setups such as that for Xcode on the fly and Xcode provides
>> the same level of abstraction like you would get it from GNU autotools
>> et al.
>>
>> PS: I am on vacation till Aug. 15th and wanted to do some C/C++ coding
>> anyway so maybe that's a nice match and additional excercise I could
>> have.
>>
>> Cheers
>> Daniel
>
> Hi Daniel,
>
> Cool! I'll be happy to work with you on this!
>
> Porting Tuscany to Clang shouldn't be too hard.
>
> To build on Mac OS X Snow Leopard or Lion, take a look at the INSTALL
> doc and the macos/macos-install script, which builds Tuscany and all
> its dependencies from source. The dependencies can be built with the
> GCC 4.2 shipped with Xcode. Tuscany needs GCC 4.4+ as it requires some
> C++0x features.
>
> To build with CLang/LLVM you'll probably just have to set
> CC=/usr/bin/clang, CXX=/usr/bin/clang++, then add -std=libc++ to the
> Tuscany configure.ac script. Then you'll have to adjust bits of the
> code, in particular to convert usages of __thread to Posix thread TLS
> calls, and perhaps fix up some of the C++0x variadic templates which
> showed compile errors last time I tried. I had tried an earlier
> version of Clang but Xcode 4.1 now ships Clang/LLVM 3.0, which should
> support everything that's needed.
>
> To remove all the dependencies on the GCC libs, the next step would be
> to build all the Tuscany dependencies with Clang as well. Most of
> their builds are Autoconf/Automake based so you could try to just run
> them with the CC and CXX environment variables set to point to clang
> and clang++. That should work for most... and then go from there...
>
> About CMake, I don't know it well, so I'm not sure but my guess is
> that'll be more work as there's 44 Makefile.am configs in the source
> tree, unless you have a way to convert them quasi-automatically to
> CMake equivalent configs...
>
> Have a nice vacation :) For any questions, issues etc look for me on
> gtalk, irc #tuscany or post here on the list.
> --
> Jean-Sebastien
>
Re: Fun with LevelDB, LLVM and OpenCL
Posted by Jean-Sebastien Delfino <js...@apache.org>.
On Sat, Aug 6, 2011 at 2:14 AM, dsh <da...@googlemail.com> wrote:
> Hi Jean-Sebastien,
>
> if you like I could take a look at LLVM/clang. From my point of view
> it not only integrates more nicely with Xcode but you would as well
> get rid of dynamically linking against things like libgcc* which
> sometimes triggers licensing discussions if talking to attorneys.
>
> If you think it would make sense I could also have a look at replacing
> the autotools stuff (automake) by CMake. The latter would generate IDE
> project setups such as that for Xcode on the fly and Xcode provides
> the same level of abstraction like you would get it from GNU autotools
> et al.
>
> PS: I am on vacation till Aug. 15th and wanted to do some C/C++ coding
> anyway so maybe that's a nice match and additional excercise I could
> have.
>
> Cheers
> Daniel
Hi Daniel,
Cool! I'll be happy to work with you on this!
Porting Tuscany to Clang shouldn't be too hard.
To build on Mac OS X Snow Leopard or Lion, take a look at the INSTALL
doc and the macos/macos-install script, which builds Tuscany and all
its dependencies from source. The dependencies can be built with the
GCC 4.2 shipped with Xcode. Tuscany needs GCC 4.4+ as it requires some
C++0x features.
To build with CLang/LLVM you'll probably just have to set
CC=/usr/bin/clang, CXX=/usr/bin/clang++, then add -std=libc++ to the
Tuscany configure.ac script. Then you'll have to adjust bits of the
code, in particular to convert usages of __thread to Posix thread TLS
calls, and perhaps fix up some of the C++0x variadic templates which
showed compile errors last time I tried. I had tried an earlier
version of Clang but Xcode 4.1 now ships Clang/LLVM 3.0, which should
support everything that's needed.
To remove all the dependencies on the GCC libs, the next step would be
to build all the Tuscany dependencies with Clang as well. Most of
their builds are Autoconf/Automake based so you could try to just run
them with the CC and CXX environment variables set to point to clang
and clang++. That should work for most... and then go from there...
About CMake, I don't know it well, so I'm not sure but my guess is
that'll be more work as there's 44 Makefile.am configs in the source
tree, unless you have a way to convert them quasi-automatically to
CMake equivalent configs...
Have a nice vacation :) For any questions, issues etc look for me on
gtalk, irc #tuscany or post here on the list.
--
Jean-Sebastien
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Hi Jean-Sebastien,
if you like I could take a look at LLVM/clang. From my point of view
it not only integrates more nicely with Xcode but you would as well
get rid of dynamically linking against things like libgcc* which
sometimes triggers licensing discussions if talking to attorneys.
If you think it would make sense I could also have a look at replacing
the autotools stuff (automake) by CMake. The latter would generate IDE
project setups such as that for Xcode on the fly and Xcode provides
the same level of abstraction like you would get it from GNU autotools
et al.
PS: I am on vacation till Aug. 15th and wanted to do some C/C++ coding
anyway so maybe that's a nice match and additional excercise I could
have.
Cheers
Daniel
On Sat, Aug 6, 2011 at 7:55 AM, Jean-Sebastien Delfino
<js...@apache.org> wrote:
> Hi all,
>
> I'm on vacation for a few days and I'm going to have a little bit of
> free time to hack around Tuscany C++. I've been thinking about the
> following:
>
> a) Integrate LevelDB [1] as a key-value store component. I've been
> experimenting with LevelDB and I think it'll be a useful complement to
> the TinyCDB based store (best for constant data) and the PostgreSQL
> based store (best for flexible query).
>
> b) Use the LLVM based Clang compiler [2] as an alternative to GCC,
> useful on MacOS as it'll enable building without anything extra on top
> of Xcode 4.1.
>
> c) More challenging and interesting too, analyze a composite and
> transform it into a set of OpenCL [3][4][5][6] kernel functions
> running on a mix of CPUs and GPUs (giving you a lot of cheap computing
> power to run your SCA compositions). In particular convert a graph of
> synchronous service invocations into chained asynchronous tasks and
> determine the optimum partition of the composite into OpenCL kernels,
> work groups and work items while minimizing the data transfers between
> OpenCL kernels.
>
> I only have a few days to bootstrap these projects and will probably
> spend most of my energy on the OpenCL one as I want to try a few
> innovative ideas (including a functional programming monad based
> approach to model the interactions around composites and help the sync
> to async transformation, and another idea to have the component
> implementations do meta-programming and return their computations as
> kernel programs instead of the actual computation results) so if
> anybody is interested in working with me on this stuff ping me on
> gtalk jsdelfino, the #tuscany IRC channel or even better post a
> message here...
>
> [1] http://code.google.com/p/leveldb/
> [2] http://clang.llvm.org/
> [3] http://www.khronos.org/opencl/
> [4] http://developer.apple.com/library/mac/#documentation/Performance/Conceptual/OpenCL_MacProgGuide/Introduction/Introduction.html
> [5] http://developer.nvidia.com/opencl
> [6] http://developer.amd.com/zones/OpenCLZone/Pages/default.aspx
> --
> Jean-Sebastien
>
Re: Fun with LevelDB, LLVM and OpenCL
Posted by dsh <da...@googlemail.com>.
Btw, concerning OpenCL I recently found this IBM research project [1].
It not only abstracts OpenCL but OpenMP, CUDA and FPGA programming
too. So you don't have to deal with low level library programming.
Just thought I share that link with you (I am not recommending using
that project as part of Apache Tuscany)..
[1] https://researcher.ibm.com/researcher/view_project.php?id=122
Cheers
Daniel
On Sat, Aug 6, 2011 at 7:55 AM, Jean-Sebastien Delfino
<js...@apache.org> wrote:
> Hi all,
>
> I'm on vacation for a few days and I'm going to have a little bit of
> free time to hack around Tuscany C++. I've been thinking about the
> following:
>
> a) Integrate LevelDB [1] as a key-value store component. I've been
> experimenting with LevelDB and I think it'll be a useful complement to
> the TinyCDB based store (best for constant data) and the PostgreSQL
> based store (best for flexible query).
>
> b) Use the LLVM based Clang compiler [2] as an alternative to GCC,
> useful on MacOS as it'll enable building without anything extra on top
> of Xcode 4.1.
>
> c) More challenging and interesting too, analyze a composite and
> transform it into a set of OpenCL [3][4][5][6] kernel functions
> running on a mix of CPUs and GPUs (giving you a lot of cheap computing
> power to run your SCA compositions). In particular convert a graph of
> synchronous service invocations into chained asynchronous tasks and
> determine the optimum partition of the composite into OpenCL kernels,
> work groups and work items while minimizing the data transfers between
> OpenCL kernels.
>
> I only have a few days to bootstrap these projects and will probably
> spend most of my energy on the OpenCL one as I want to try a few
> innovative ideas (including a functional programming monad based
> approach to model the interactions around composites and help the sync
> to async transformation, and another idea to have the component
> implementations do meta-programming and return their computations as
> kernel programs instead of the actual computation results) so if
> anybody is interested in working with me on this stuff ping me on
> gtalk jsdelfino, the #tuscany IRC channel or even better post a
> message here...
>
> [1] http://code.google.com/p/leveldb/
> [2] http://clang.llvm.org/
> [3] http://www.khronos.org/opencl/
> [4] http://developer.apple.com/library/mac/#documentation/Performance/Conceptual/OpenCL_MacProgGuide/Introduction/Introduction.html
> [5] http://developer.nvidia.com/opencl
> [6] http://developer.amd.com/zones/OpenCLZone/Pages/default.aspx
> --
> Jean-Sebastien
>