You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@rya.apache.org by "Aaron Mihalik (JIRA)" <ji...@apache.org> on 2017/07/14 21:34:00 UTC
[jira] [Commented] (RYA-308) Loading into MongoDB slows as data
grows
[ https://issues.apache.org/jira/browse/RYA-308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088119#comment-16088119 ]
Aaron Mihalik commented on RYA-308:
-----------------------------------
never mind... after I ran longer, it's clear that I had a bunch of noisy samples at the beginning that really stabilized to 1.7K/sec
Jul 14 2017 16:45:47 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 100000
Jul 14 2017 16:46:40 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 200000
Jul 14 2017 16:47:35 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 300000
Jul 14 2017 16:48:41 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 400000
Jul 14 2017 16:49:32 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 500000
Jul 14 2017 16:50:42 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 600000
Jul 14 2017 16:51:54 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 700000
Jul 14 2017 16:53:09 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 800000
Jul 14 2017 16:54:20 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 900000
Jul 14 2017 16:55:36 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 1000000
Jul 14 2017 16:56:51 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 1100000
Jul 14 2017 16:57:59 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 1200000
Jul 14 2017 16:59:12 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 1300000
Jul 14 2017 17:00:27 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 1400000
Jul 14 2017 17:01:41 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 1500000
Jul 14 2017 17:02:57 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 1600000
Jul 14 2017 17:04:09 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 1700000
Jul 14 2017 17:05:07 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 1800000
Jul 14 2017 17:06:19 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 1900000
Jul 14 2017 17:07:29 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 2000000
Jul 14 2017 17:08:35 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 2100000
Jul 14 2017 17:09:40 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 2200000
Jul 14 2017 17:10:46 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 2300000
Jul 14 2017 17:11:51 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 2400000
Jul 14 2017 17:13:00 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 2500000
Jul 14 2017 17:14:02 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 2600000
Jul 14 2017 17:15:05 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 2700000
Jul 14 2017 17:16:10 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 2800000
Jul 14 2017 17:17:13 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 2900000
Jul 14 2017 17:18:18 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 3000000
Jul 14 2017 17:19:22 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 3100000
Jul 14 2017 17:20:26 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 3200000
Jul 14 2017 17:21:27 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 3300000
Jul 14 2017 17:22:35 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 3400000
Jul 14 2017 17:23:36 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 3500000
Jul 14 2017 17:24:36 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 3600000
Jul 14 2017 17:25:32 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 3700000
Jul 14 2017 17:26:33 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 3800000
Jul 14 2017 17:27:32 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 3900000
Jul 14 2017 17:28:30 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 4000000
Jul 14 2017 17:29:33 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 4100000
Jul 14 2017 17:30:33 INFO [main] (LoadDataFileWithListener.java:60) - Size :: 4200000
> Loading into MongoDB slows as data grows
> ----------------------------------------
>
> Key: RYA-308
> URL: https://issues.apache.org/jira/browse/RYA-308
> Project: Rya
> Issue Type: Improvement
> Reporter: Aaron Mihalik
> Assignee: Andrew Smith
>
> I loaded data into mongo using this example [1]. The data loading got significantly slower as I loaded more and more data. Debug output below
> [1] https://github.com/amihalik/rya-mongo-debugging/blob/master/src/main/java/com/github/amihalik/rya/mongo/debugging/loaddata/LoadDataFileWithListener.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)