Using data streamer for local entries update

[djm132 <do...@gmail.com>]

Hi,

Is it safe to parallel update local node entries with data streamer ? Here
is the benchmark code, it seems that addData() much faster than individual
put().


val res = Node.ignite.compute(project.indexNodes).broadcast<Long> {
  var n = 0L
  Node.ignite.dataStreamer<String, IndexRecord>(project.index.name).use { ds
->
    ds.allowOverwrite(true)
    project.index.localEntries(CachePeekMode.PRIMARY).forEach { it ->
      it.value.status = IndexRecordStatus.COMPLETED
      ds.addData(it.value.reqKey, it.value)
      n++
    }
  }

  println("Updated $n records")
  n
}




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Using data streamer for local entries update

[Ilya Kasnacheev <il...@gmail.com>]

Hello!

If no rebalance is going on, then the answer is yes.

Regards,
-- 
Ilya Kasnacheev


пт, 10 янв. 2020 г. в 13:35, djm132 <do...@gmail.com>:

> Is localEntries(CachePeekMode.PRIMARY) guarantees that keys will not
> overlap
> for partitioned cache?
>
> Thanks.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Using data streamer for local entries update

[djm132 <do...@gmail.com>]

Is localEntries(CachePeekMode.PRIMARY) guarantees that keys will not overlap
for partitioned cache?

Thanks.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Using data streamer for local entries update

[Ilya Kasnacheev <il...@gmail.com>]

Hello!

Yes, it should be perfectly safe. Data Streamer does batching, thus the
speed up. Please note that Data Streamer batches may lock keys, which means
you should be careful with doing batch operations in parallel to avoid
deadlock.

Regards,
-- 
Ilya Kasnacheev


пт, 10 янв. 2020 г. в 01:15, djm132 <do...@gmail.com>:

> Backup count = 1 and wal mode = background.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Using data streamer for local entries update

[djm132 <do...@gmail.com>]

Backup count = 1 and wal mode = background.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/