You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by asingh <am...@gmail.com> on 2017/10/18 18:11:48 UTC
ComputeGrid API in C++
Hi
There was a thread started regarding this some time back.
http://apache-ignite-users.70518.x6.nabble.com/Compute-Grid-API-in-C-Scala-td4456.html
It was mentioned that the C++ Compute API may come in 1.7. Did that happen?
I cant seem to find any C++ code in the ComputeGrid documentation.
I have a problem similar to Arthi's in the thread above but would like to
avoid wrapping the C++ computational code in JNA.
Apologies if this has been asked before. Would be grateful if you can point
me to the relevant thread.
Thanks!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: ComputeGrid API in C++
Posted by Igor Sapego <is...@apache.org>.
Yes, this will do.
Best Regards,
Igor
On Fri, Oct 27, 2017 at 5:49 PM, asingh <am...@gmail.com> wrote:
> Ok, so, i first need to wrap ignite within an application that will
> register
> my user functions and start ignite.
> I launch that application on every machine that I have.
> Then, I can launch another calling application that will call RunASync()
> and
> dispatch tasks to each ignite node and collect their responses.
>
> Is that how it should be?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: ComputeGrid API in C++
Posted by asingh <am...@gmail.com>.
Ok, so, i first need to wrap ignite within an application that will register
my user functions and start ignite.
I launch that application on every machine that I have.
Then, I can launch another calling application that will call RunASync() and
dispatch tasks to each ignite node and collect their responses.
Is that how it should be?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: ComputeGrid API in C++
Posted by Igor Sapego <is...@apache.org>.
Hi,
Yes, you are right, Ignition::Start(cfg, "node1"). You don't need to
set any specific properties, all you need to do is to register your
classes on all nodes on all machines after you start them.
> It just launches node1 within the same process as the application.
That's right. You need to write an application that starts node and
registers your classes.
What is the problem?
Best Regards,
Igor
On Thu, Oct 26, 2017 at 10:26 PM, asingh <am...@gmail.com> wrote:
> Ignite node = Ignition::Start("node1");
>
> The first argument to Start() needs to be an xml file. Are there any
> specific properties that need to be set in the xml for this case?
> If I call Start(cfg, "node1"), it just launches node1 within the same
> process as the application.
>
> In the below diagram, what do I have to do to dispatch C1, C2, C3 to nodes
> 1, 2 and 3, specially when nodes 1,2 and 3 are running on different
> physical
> machines?
>
> Compute Grid <https://files.readme.io/1db96a3-in_memory_compute.png>
>
> <http://apache-ignite-users.70518.x6.nabble.com/file/
> t1405/1db96a3-in_memory_compute.png>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: ComputeGrid API in C++
Posted by asingh <am...@gmail.com>.
Ignite node = Ignition::Start("node1");
The first argument to Start() needs to be an xml file. Are there any
specific properties that need to be set in the xml for this case?
If I call Start(cfg, "node1"), it just launches node1 within the same
process as the application.
In the below diagram, what do I have to do to dispatch C1, C2, C3 to nodes
1, 2 and 3, specially when nodes 1,2 and 3 are running on different physical
machines?
Compute Grid <https://files.readme.io/1db96a3-in_memory_compute.png>
<http://apache-ignite-users.70518.x6.nabble.com/file/t1405/1db96a3-in_memory_compute.png>
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: ComputeGrid API in C++
Posted by Igor Sapego <is...@apache.org>.
Hi,
You don't have to modify platforms/cpp/ignite/src/ignite.cpp.
Why won't you modify your code, where you start the node?
I mean, in *your application code* write something like this:
Ignite node = Ignition::Start("node1");
IgniteBinding binding = node.GetBinding();
binding.RegisterComputeFunc<MyFunc>();
You don't have to modify method Ignition::Start() or any
other method for that.
Best Regards,
Igor
On Thu, Oct 26, 2017 at 5:31 PM, asingh <am...@gmail.com> wrote:
> Hi Igor
>
> Shouldn't be a problem, given that the source code is there. I just want to
> make sure that I am doing it the right way. Currently, I have modified the
> main() method in platforms/cpp/ignite/src/ignite.cpp and added these two
> lines:
>
> IgniteBinding binding = ignite.GetBinding();
> binding.RegisterComputeFunc<PrintWords>();
>
> Also, when would I use IgniteModuleInit()? Is it when the Call() method is
> implemented in a shared library?
>
> Thanks!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: ComputeGrid API in C++
Posted by asingh <am...@gmail.com>.
Hi Igor
Shouldn't be a problem, given that the source code is there. I just want to
make sure that I am doing it the right way. Currently, I have modified the
main() method in platforms/cpp/ignite/src/ignite.cpp and added these two
lines:
IgniteBinding binding = ignite.GetBinding();
binding.RegisterComputeFunc<PrintWords>();
Also, when would I use IgniteModuleInit()? Is it when the Call() method is
implemented in a shared library?
Thanks!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: ComputeGrid API in C++
Posted by Igor Sapego <is...@apache.org>.
You need to have the same code on all C++ nodes to be able
to execute it remotely. In other words, you need to register
classes you want to invoke remotely on all three nodes.
Is that a problem for you?
Best Regards,
Igor
On Wed, Oct 25, 2017 at 6:35 PM, asingh <am...@gmail.com> wrote:
> So, here is what I am trying to do:
> 1. Launch ignite node on one Linux machine
> 2. Launch another ignite node on second Linux machine
> 3. On one of the two machines, run the compute_example containing
> PrintWords() class and have its Call() method invoked via
> compute.RunAsync()
> on all three machines in parallel.
>
> All three ignite nodes use the same xml that I had attached earlier.
>
> I was not able to have the example dispatch the calls to the independent
> ignite nodes, without explicitly registering the PrintWords class in
> ignite's main function.
>
> I am sure there is a better way to do this, but I need some help in
> figuring
> out how!
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: ComputeGrid API in C++
Posted by asingh <am...@gmail.com>.
So, here is what I am trying to do:
1. Launch ignite node on one Linux machine
2. Launch another ignite node on second Linux machine
3. On one of the two machines, run the compute_example containing
PrintWords() class and have its Call() method invoked via compute.RunAsync()
on all three machines in parallel.
All three ignite nodes use the same xml that I had attached earlier.
I was not able to have the example dispatch the calls to the independent
ignite nodes, without explicitly registering the PrintWords class in
ignite's main function.
I am sure there is a better way to do this, but I need some help in figuring
out how!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: ComputeGrid API in C++
Posted by Igor Sapego <is...@apache.org>.
Why have you decided to modify an implementation? Are there
any special requirements which making this necessary? Why
can't you register all the necessary classes in user code?
Best Regards,
Igor
On Mon, Oct 23, 2017 at 5:04 PM, asingh <am...@gmail.com> wrote:
> That was it!
> Thank you so much, Igor.
>
> I just had to add a call to GetBinding() and RegisterComputeFunc() in
> platforms/cpp/ignite/src/ignite.cpp main() method. I'm not used to
> modifying
> an implementation in order to run an example ;)
>
> Although, this means that every time I change the closure, I have to
> rebuild
> ignite as well. Not a big problem but if anyone has a more elegant
> solution,
> I am all ears!
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: ComputeGrid API in C++
Posted by asingh <am...@gmail.com>.
That was it!
Thank you so much, Igor.
I just had to add a call to GetBinding() and RegisterComputeFunc() in
platforms/cpp/ignite/src/ignite.cpp main() method. I'm not used to modifying
an implementation in order to run an example ;)
Although, this means that every time I change the closure, I have to rebuild
ignite as well. Not a big problem but if anyone has a more elegant solution,
I am all ears!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: ComputeGrid API in C++
Posted by Igor Sapego <is...@apache.org>.
Have you registered the same function for all three nodes,
or only for the one you are calling RunAsync() on? You should
register it on every node, as this is a local operation.
Best Regards,
Igor
On Fri, Oct 20, 2017 at 7:06 PM, asingh <am...@gmail.com> wrote:
> That makes sense.
>
> Now I am using ignite.GetBinding() from within the main() function
>
> In the main function, I am populating a vector of strings and calling
> RunAsync() three times.
> The ignite node within the binary is able to process the string but the
> other two ignite nodes report the same IgniteException of job not being
> registered.
>
>
> Looks like the example binary is able to communicate with the two ignite
> server nodes but those two nodes have no idea about the PrintWords' Call
> method.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: ComputeGrid API in C++
Posted by asingh <am...@gmail.com>.
That makes sense.
Now I am using ignite.GetBinding() from within the main() function
In the main function, I am populating a vector of strings and calling
RunAsync() three times.
The ignite node within the binary is able to process the string but the
other two ignite nodes report the same IgniteException of job not being
registered.
Looks like the example binary is able to communicate with the two ignite
server nodes but those two nodes have no idea about the PrintWords' Call
method.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: ComputeGrid API in C++
Posted by Igor Sapego <is...@apache.org>.
Well, yeah you should link with -rdynamic. Actually, IgniteModuleInit
is intended to be used in dynamic libraries, not in binaries. When
linking binary, GCC ordinary does not add unused symbols to dynamic
symbol table, so ignite can not find IgniteModuleInit on start.
Consider using Ignite::GetBinding for binary instead.
Best Regards,
Igor
On Fri, Oct 20, 2017 at 6:10 PM, asingh <am...@gmail.com> wrote:
> That piece of message confused me a little. Am i supposed to
> compile(actually, link) with -rdynamic or not?
> Anyway, so I did set LDFLAGS environment variable to -rdynamic and
> recompiled and relinked under platform/cpp as well as under examples. But
> still no luck.
>
> Also, I am running the ignite that is built under
> $IGNITE_HOME/platforms/cpp/ignite/ignite
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: ComputeGrid API in C++
Posted by asingh <am...@gmail.com>.
That piece of message confused me a little. Am i supposed to
compile(actually, link) with -rdynamic or not?
Anyway, so I did set LDFLAGS environment variable to -rdynamic and
recompiled and relinked under platform/cpp as well as under examples. But
still no luck.
Also, I am running the ignite that is built under
$IGNITE_HOME/platforms/cpp/ignite/ignite
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: ComputeGrid API in C++
Posted by Igor Sapego <is...@apache.org>.
So, did you compile your program with -rdynamic as error text suggests?
Best Regards,
Igor
On Thu, Oct 19, 2017 at 6:09 PM, asingh <am...@gmail.com> wrote:
> Hi Igor
>
> I am running Red Hat Linux 6, with gcc 4.4.7
>
> Thanks!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: ComputeGrid API in C++
Posted by asingh <am...@gmail.com>.
Hi Igor
I am running Red Hat Linux 6, with gcc 4.4.7
Thanks!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: ComputeGrid API in C++
Posted by Igor Sapego <is...@apache.org>.
Hi,
Are you running Linux or Windows? What is the your compiler?
Best Regards,
Igor
On Thu, Oct 19, 2017 at 5:47 PM, asingh <am...@gmail.com> wrote:
> compute-example.xml
> <http://apache-ignite-users.70518.x6.nabble.com/file/
> t1405/compute-example.xml>
>
> I have attached the xml file as well. As you can see from the various
> commented out sections, I have been trying different things to make it
> work.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: ComputeGrid API in C++
Posted by asingh <am...@gmail.com>.
compute-example.xml
<http://apache-ignite-users.70518.x6.nabble.com/file/t1405/compute-example.xml>
I have attached the xml file as well. As you can see from the various
commented out sections, I have been trying different things to make it work.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: ComputeGrid API in C++
Posted by asingh <am...@gmail.com>.
Thanks for the links!
So, now I am trying to run the CPP example.
I have compiled CPP version of ignite and running that on two servers. The
two instances of ignite are able to recognize each other:
[10:20:49] Ignite node started OK (id=638a866d)
[10:20:49] Topology snapshot [ver=1, servers=1, clients=0, CPUs=40,
heap=0.89GB]
[10:20:55] Topology snapshot [ver=2, servers=2, clients=0, CPUs=80,
heap=1.8GB]
But when I try to run the CPP example, I get this message from the example:
An error occurred: org.apache.ignite.IgniteException : C++ compute job is
not registered on the node (did you compile your program without
-rdynamic?). [jobTypeId=160893980], class org.apache.ignite.IgniteException:
C++ compute job is not registered on the node (did you compile your program
without -rdynamic?). [jobTypeId=160893980]
Below is the message is see from one of the ignite/ignite processes:
[10:21:32,877][SEVERE][pub-#97%null%][GridJobWorker] Failed to execute job
[jobId=7a5b2053f51-9a565515-196e-4fe0-807d-b48c58b2126a,
ses=GridJobSessionImpl [ses=GridTaskSessionImpl
[taskName=o.a.i.i.processors.platform.compute.PlatformBalancingSingleClosureTask,
dep=GridDeployment [ts=1508422855449, depMode=SHARED,
clsLdr=sun.misc.Launcher$AppClassLoader@5889949a,
clsLdrId=2c322053f51-0e956c0d-17ae-4662-a762-8c52c4b74524, userVer=0,
loc=true,
sampleClsName=o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionFullMap,
pendingUndeploy=false, undeployed=false, usage=1],
taskClsName=o.a.i.i.processors.platform.compute.PlatformBalancingSingleClosureTask,
sesId=6a5b2053f51-9a565515-196e-4fe0-807d-b48c58b2126a,
startTime=1508422892858, endTime=9223372036854775807,
taskNodeId=9a565515-196e-4fe0-807d-b48c58b2126a,
clsLdr=sun.misc.Launcher$AppClassLoader@5889949a, closed=false, cpSpi=null,
failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=false,
subjId=9a565515-196e-4fe0-807d-b48c58b2126a, mapFut=IgniteFuture
[orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=117005517]], execName=null],
jobId=7a5b2053f51-9a565515-196e-4fe0-807d-b48c58b2126a]]
class org.apache.ignite.IgniteException: C++ compute job is not registered
on the node (did you compile your program without -rdynamic?).
[jobTypeId=160893980]
I modified the example to call RegisterComputeFunc:
IGNITE_EXPORTED_CALL void IgniteModuleInit(ignite::IgniteBindingContext&
context)
{
IgniteBinding binding = context.GetBinding();
binding.RegisterComputeFunc<PrintWords>();
}
Any help would be much appreciated!
Thanks!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: ComputeGrid API in C++
Posted by "slava.koptilin" <sl...@gmail.com>.
Hi,
C++ Compute Grid API is available since Apache Ignite 2.1 version.
High-level information can be found here
https://apacheignite-cpp.readme.io/v2.1/docs/compute-grid
C++ Compute Grid API is defined into <compute.h> header fle.
https://ignite.apache.org/releases/2.2.0/cppdoc/classignite_1_1compute_1_1Compute.html
Thanks!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/