You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by mimmo_c <mi...@gmail.com> on 2017/07/05 15:56:55 UTC
Ignite Information
Hi,
I wanted to know if I did start a node by igniting.sh in client mode with a
fixed configuration I can access that instance via java.
Another thing I would do is to partition the input to the ignite server
nodes. For example I have 2 <string, int> maps and I would like the first
one to go to one server and the second to the server two. Should I set the
server cache to "local"? Can I use affinitykey? The maps are practically the
same, there is no particular distinction between the objects inside.
Thank you
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Information-tp14330.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Information
Posted by vkulichenko <va...@gmail.com>.
Mimmo,
Can you describe the use case? What are you trying to achieve exactly?
Generally, sending a lot of data across the network is something that you
should avoid, of course. So it's hard to tell if your approach is correct or
not without knowing all details.
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Information-tp14330p15087.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Information
Posted by Mimmo Celano <mi...@gmail.com>.
Hi Val,
Thanks for reply.
We are trying to eliminate the reduce step in hadoop processing by sending
each map results in a distribuited cache for minimize time of execution. Do
you think it's a reasonable thing? When we send data with DataStream
overwrite true through more parallel DataStreamer the time increases a lot.
So we would like to split Ignite cluster for manage a set of HashMap, (
Node1 word from "a" to "d", Node2 word from "e" to "f" .... ). The HashMap
are divided efficiently in map job but we aren't able to send this Map to
specific node.
Could you advise us on how to send data to node efficiently?
For an HashMap of 10 millions entries
sizenodebuffer 8192 16 sec ----> 1 Client send to 1 Node Server ||
Overwrite true
sizenodebuffer 8192 42 sec ----> 3 Client send at the same time to 1
Node Server || Overwrite true
sizenodebuffer 8192 12 sec ----> 1 Client send to 1 Node Server ||
Overwrite false
sizenodebuffer 8192 23 sec ----> 3 Client send at the same time to 1
Node Server || Overwrite false
Problem is thah we need to send GB of data, and this approach is too slow.
Thanks
Mimmo
2017-07-17 20:14 GMT+02:00 vkulichenko <va...@gmail.com>:
> I can't reproduce this behavior. Can you create a small project that I can
> run on my side and investigate?
>
> -Val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Ignite-Information-tp14330p15025.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
Re: Ignite Information
Posted by vkulichenko <va...@gmail.com>.
I can't reproduce this behavior. Can you create a small project that I can
run on my side and investigate?
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Information-tp14330p15025.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Information
Posted by Mimmo Celano <mi...@gmail.com>.
Hi thanks for the reply.
We can't filter data.
public class NodeFilter implements IgnitePredicate<ClusterNode> {
private static final long serialVersionUID = 0L;
public NodeFilter() {
System.out.println("Start NodeFilter");
}
@Override public boolean apply(ClusterNode node) {
System.out.println(""+node.attribute("id"));
Map<String, Object> nodeAttrs = node.attributes();
System.out.println("Attr = "+nodeAttrs.get("id").toString());
return true;
}
}
This is the class Filter, which print just information about the node
attributes.
Node attributes are added in xml config file
<bean id="ignite.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
<!-- Explicitly configure TCP discovery SPI to provide list of
initial nodes. -->
<property name="discoverySpi">
<bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<!--
Ignite provides several options for automatic
discovery that can be used
instead os static IP based discovery. For
information on all options refer
to our documentation:
http://apacheignite.readme.io/docs/cluster-config
-->
<!-- Uncomment static IP finder to enable static-based
discovery of initial nodes. -->
<!--<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinde$
<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMultic$
<property name="addresses">
<list>
<!-- In distributed environment, replace
with actual host IP address. -->
<value>127.0.0.1:47500..47509</value>
<value>192.168.30.5:47500..47509</value>
<value>192.168.30.22:47500..47509</value>
<value>192.168.30.99:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
<property name="peerClassLoadingEnabled" value="true"/>
<property name="userAttributes">
<map>
<entry key="id" value="uno"/>
</map>
</property>
</bean>
We set it to cache configuration
Ignition.setClientMode(true);
ignite =
Ignition.getOrStart(Ignition.loadSpringBean("/home/hduser/apache-ignite-2.0.0-src/examples/config/example-cache1.xml",
"ignite.cfg"));
CacheConfiguration<String,Long> cfg2=
Ignition.loadSpringBean("/home/hduser/apache-ignite-2.0.0-src/examples/config/example-cache1.xml",
"cacheconf");
-->cfg2.setNodeFilter(new NodeFilter());
cache = ignite.getOrCreateCache(cfg2);
but when we put data into the cache, the filter is not setted. The
"println" in apply function is not starting, so the function is not called.
The filter returns everytime true, it's just an example to test it.
Are we doing something wrong about setting the filter?
Thanks
2017-07-08 1:51 GMT+02:00 ignite_user2016 <ri...@gmail.com>:
> Thanks .. Val
>
> I will look at it in detail.
>
>
>
> Take Care,
> Rishi
>
> On Jul 7, 2017, at 2:17 PM, vkulichenko [via Apache Ignite Users] <[hidden
> email] <http:///user/SendEmail.jtp?type=node&node=14518&i=0>> wrote:
>
> You can read here about user attributes: https://apacheignite.readme.
> io/docs/cluster#cluster-node-attributes
>
> For the node filter example, refer to JavaDoc: https://github.com/apache/
> ignite/blob/master/modules/core/src/main/java/org/apache/ignite/util/
> AttributeNodeFilter.java
>
> -Val
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-
> Information-tp14330p14509.html
> To start a new topic under Apache Ignite Users, email [hidden email]
> <http:///user/SendEmail.jtp?type=node&node=14518&i=1>
> To unsubscribe from Apache Ignite Users, click here.
> NAML
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
> ------------------------------
> View this message in context: Re: Ignite Information
> <http://apache-ignite-users.70518.x6.nabble.com/Ignite-Information-tp14330p14518.html>
>
> Sent from the Apache Ignite Users mailing list archive
> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>
Re: Ignite Information
Posted by ignite_user2016 <ri...@gmail.com>.
Thanks .. Val
I will look at it in detail.
Take Care,
Rishi
> On Jul 7, 2017, at 2:17 PM, vkulichenko [via Apache Ignite Users] <ml...@n6.nabble.com> wrote:
>
> You can read here about user attributes: https://apacheignite.readme.io/docs/cluster#cluster-node-attributes
>
> For the node filter example, refer to JavaDoc: https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/util/AttributeNodeFilter.java
>
> -Val
>
> If you reply to this email, your message will be added to the discussion below:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-Information-tp14330p14509.html
> To start a new topic under Apache Ignite Users, email ml+s70518n1h85@n6.nabble.com
> To unsubscribe from Apache Ignite Users, click here.
> NAML
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Information-tp14330p14518.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Information
Posted by vkulichenko <va...@gmail.com>.
You can read here about user attributes:
https://apacheignite.readme.io/docs/cluster#cluster-node-attributes
For the node filter example, refer to JavaDoc:
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/util/AttributeNodeFilter.java
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Information-tp14330p14509.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Information
Posted by Mimmo Celano <mi...@gmail.com>.
Thank you,
Sorry but I can not understand how to implement it. Could you please show
me an example of its use? I just started to use Ignite.
Thanks
2017-07-07 0:03 GMT+02:00 vkulichenko <va...@gmail.com>:
> Hi,
>
> See CacheConfiguration#nodeFilter property. It's just a predicate that
> filters out nodes, cache will be deployed only on those for which filter
> returns true. The easiest way would be to set custom attributes via
> IgniteConfiguration#userAttributes and filter out using
> AttributeNodeFilter
> which is provided out of the box.
>
> -Val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Ignite-Information-tp14330p14423.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
Re: Ignite Information
Posted by vkulichenko <va...@gmail.com>.
Hi,
See CacheConfiguration#nodeFilter property. It's just a predicate that
filters out nodes, cache will be deployed only on those for which filter
returns true. The easiest way would be to set custom attributes via
IgniteConfiguration#userAttributes and filter out using AttributeNodeFilter
which is provided out of the box.
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Information-tp14330p14423.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Information
Posted by mimmo_c <mi...@gmail.com>.
Thanks for reply.
How Can I use node filter?
I did not find any information about it.
Thank you.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Information-tp14330p14386.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Information
Posted by Nikolai Tikhonov <nt...@apache.org>.
Hi,
> I wanted to know if I did start a node by igniting.sh in client mode with
a fixed configuration I can access that instance via java.
For getting Ignite API you need have ignite node in the same JVM that
running your application.
> Another thing I would do is to partition the input to the ignite server
nodes.
You can getting it easely by node filter. I think it's the better way.
http://apache-ignite-users.70518.x6.nabble.com/CacheConfiguration-AffinityFunction-or-node-filter-td9207.html
On Wed, Jul 5, 2017 at 6:56 PM, mimmo_c <mi...@gmail.com> wrote:
> Hi,
> I wanted to know if I did start a node by igniting.sh in client mode with a
> fixed configuration I can access that instance via java.
> Another thing I would do is to partition the input to the ignite server
> nodes. For example I have 2 <string, int> maps and I would like the first
> one to go to one server and the second to the server two. Should I set the
> server cache to "local"? Can I use affinitykey? The maps are practically
> the
> same, there is no particular distinction between the objects inside.
> Thank you
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Ignite-Information-tp14330.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>