You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucy.apache.org by Sebastián Gurin <se...@gmail.com> on 2017/10/29 19:48:39 UTC
[lucy-dev] Port to JavaScript with emscripten
Hello,
I wonder if somebody tried to compile lucy to JavaScript with
http://emscripten.org (C/C++ to JavaScript compiler). There are some
successful projects like https://github.com/medialize/sass.js/ to
bring new technologies to the JavaScript world from C/C++. IN the case
of lucene-like text indexer / search - this is for sure an area that
JavaScript lacks software. Also I wonder if the clownfish thing won't
make it difficult to perform such translation. So, before I tried, I
would like to know your thoughts.
Thanks! and BTW, lucy is very good ! Keep it up!
Re: [lucy-dev] Port to JavaScript with emscripten
Posted by Sebastián Gurin <se...@gmail.com>.
Aja! Thank you very much. Compiling both things, compiler and runtime
did the trick! This is the final script (BTW would be nice to put
this in the docs since clownfish is not intuitive for newcomers:
FOLDER=/home/sg/test
mkdir -p $FOLDER/prefix
cd $FOLDER
git clone https://git-wip-us.apache.org/repos/asf/lucy-clownfish.git
cd $FOLDER/lucy-clownfish/runtime/c
./configure --prefix=$FOLDER/prefix
make
make install
cd $FOLDER/lucy-clownfish/compiler/c
./configure --prefix=$FOLDER/prefix
make
make install
cd $FOLDER
git clone https://git-wip-us.apache.org/repos/asf/lucy.git
cd $FOLDER/lucy/c
./configure --clownfish-prefix=$FOLDER/prefix --prefix=$FOLDER/prefix
make
# .h files generated $FOLDER/lucy/c/autogen/include/
2017-10-29 21:12 GMT-03:00 Nick Wellnhofer <we...@aevum.de>:
> On Oct 30, 2017, at 00:46 , Sebastián Gurin <se...@gmail.com> wrote:
>>
>> This is what I'm doing:
>
>> cd lucy-clownfish/compiler/c
>
> This should be
>
> cd lucy-clownfish/runtime/c
>
> Nick
>
>
Re: [lucy-dev] Port to JavaScript with emscripten
Posted by Nick Wellnhofer <we...@aevum.de>.
On Oct 30, 2017, at 00:46 , Sebastián Gurin <se...@gmail.com> wrote:
>
> This is what I'm doing:
> cd lucy-clownfish/compiler/c
This should be
cd lucy-clownfish/runtime/c
Nick
Re: [lucy-dev] Port to JavaScript with emscripten
Posted by Sebastián Gurin <se...@gmail.com>.
Thank you Marvin, yes that's was my plan more or less. I just disagree
with "The result wouldn't really be Lucy,". The idea of emscripten is
to be able to translate lucy automatically or semi-automatically if
possible, and the generated JavaScript code should be equivalent to
originals. Nevertheless, I was trying to get the .h files without luck
always getting the error:
/home/sg/test/prefix/bin/cfc --source=../core --source=../test
--include=/home/sg/test/prefix/share/clownfish/include --dest=autogen
--header=cfc_header
Parcel Clownfish v0.6.0 required by Lucy not found
make: *** [Makefile:522: autogen/hierarchy.json] Aborted (core dumped)
This is what I'm doing:
rm -rf /home/sg/test
mkdir -p /home/sg/test/prefix
cd /home/sg/test
git clone https://git-wip-us.apache.org/repos/asf/lucy-clownfish.git
cd lucy-clownfish/compiler/c
./configure --prefix=/home/sg/test/prefix
make
make install
cd /home/sg/test
git clone https://git-wip-us.apache.org/repos/asf/lucy.git
cd lucy/c
./configure --clownfish-prefix=/home/sg/test/prefix
--prefix=/home/sg/test/prefix
make
If you have any idea please help since I'm kind of blocked right now.
Also I tried cloning clownfish v0.6.0 with the following command byt
the same error happens:
git clone https://git-wip-us.apache.org/repos/asf/lucy-clownfish.git#cb4228524b7bef5af229c4f302eb68ce1573fa76
Thanks!
2017-10-29 17:55 GMT-03:00 Marvin Humphrey <ma...@rectangular.com>:
> On Sun, Oct 29, 2017 at 12:48 PM, Sebastián Gurin
> <se...@gmail.com> wrote:
>
>> I wonder if somebody tried to compile lucy to JavaScript with
>> http://emscripten.org (C/C++ to JavaScript compiler).
>
> I am not aware of such an effort.
>
>> Also I wonder if the clownfish thing won't
>> make it difficult to perform such translation. So, before I tried, I
>> would like to know your thoughts.
>
> Clownfish would probably take some effort to adapt for this purpose.
> If I were to take on this task (and I did not want to spend the effort
> to familiarize myself with Clownfish first), I'd treat it like a loose
> port: create an empty directory alongside Lucy and flesh out new .c
> and .h files bit by bit, selecting out pieces to copy or munge, always
> maintaining optimum emscripten target output.
>
> Start by having an Indexer commit a Snapshot. Then add a DocWriter
> which does variable length records of key-value pairs. Next, create an
> IndexReader module which delegates to a DocReader module to fetch
> documents from the store.
>
> You can leave the inverted indexing modules and locking code for
> later, because the fundamental data structures and decomposition
> strategies of Lucy are sound.
>
> The result wouldn't really be Lucy, and wouldn't be immediately
> contributable to Lucy. Instead, it would have the same relation to
> Lucy as Lucy had to the original Lucene when it was ported. As the
> exercise moves on and you become more familiar with Lucy's problem
> domain and its implementation choices, you can decide what direction
> you want to take.
>
> Marvin Humphrey
Re: [lucy-dev] Port to JavaScript with emscripten
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Sun, Oct 29, 2017 at 12:48 PM, Sebastián Gurin
<se...@gmail.com> wrote:
> I wonder if somebody tried to compile lucy to JavaScript with
> http://emscripten.org (C/C++ to JavaScript compiler).
I am not aware of such an effort.
> Also I wonder if the clownfish thing won't
> make it difficult to perform such translation. So, before I tried, I
> would like to know your thoughts.
Clownfish would probably take some effort to adapt for this purpose.
If I were to take on this task (and I did not want to spend the effort
to familiarize myself with Clownfish first), I'd treat it like a loose
port: create an empty directory alongside Lucy and flesh out new .c
and .h files bit by bit, selecting out pieces to copy or munge, always
maintaining optimum emscripten target output.
Start by having an Indexer commit a Snapshot. Then add a DocWriter
which does variable length records of key-value pairs. Next, create an
IndexReader module which delegates to a DocReader module to fetch
documents from the store.
You can leave the inverted indexing modules and locking code for
later, because the fundamental data structures and decomposition
strategies of Lucy are sound.
The result wouldn't really be Lucy, and wouldn't be immediately
contributable to Lucy. Instead, it would have the same relation to
Lucy as Lucy had to the original Lucene when it was ported. As the
exercise moves on and you become more familiar with Lucy's problem
domain and its implementation choices, you can decide what direction
you want to take.
Marvin Humphrey