You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@lucene.apache.org by Aravinth T <ar...@zohocorp.com> on 2016/04/16 03:29:45 UTC
Combining many field as JSON and made it into a single field -
Performance
I tried to combine multiple different fields (Stored Fields) into a JSON object
and added in to lucene document as single field
I get a better performance while reading the field from the index.
though constructing the JSON requires additional time, It still retrieve the data faster
What is the reason for this ?
I done update reading and indexing the data
while reading i convert them back to single fields
from JSON object
No. of Documents
No.of Fields
Run
With JSON & Compression
With JSON & No Compression
Without JSON &Compression
Without JSON & No Compression
Insert
Read
Update
Insert
Read
Update
Insert
Read
Update
Insert
Read
Update
100
10
One
216
35
38
167
55
59
216
23
65
215
25
57
100
10
Two
-
48
93
-
49
102
35
134
25
101
100
10
Three
-
45
80
49
98
37
117
24
106
1000
10
One
404
144
160
447
109
157
503
148
129
482
111
126
1000
10
Two
-
175
268
-
129
259
176
301
135
295
1000
10
Three
-
177
279
142
263
174
299
135
296
10000
100
One
1763
1209
414
2151
1182
357
1574
335
847
1845
321
834
10000
100
Two
-
1352
811
-
1246
643
378
1151
334
1142
10000
100
Three
-
1318
729
1252
701
369
1088
336
1158
100000
1000
One
81021
50172
39480
77690
46138
38554
79289
20623
89653
93558
19357
108458
100000
1000
Two
-
53523
34523
52027
42957
20125
88771
19357
111962
100000
1000
Three
-
53816
33460
51703
43643
20409
98932
19630
110352
Combining many field as JSON and made it into a single field -
Performance
Posted by Aravinth T <ar...@zohocorp.com>.
I tried to combine multiple different fields (Stored Fields) into a JSON object
and added in to lucene document as single field
I get a better performance while reading the field from the index.
though constructing the JSON requires additional time, It still retrieve the data faster
What is the reason for this ?
I done update reading and indexing the data
while reading i convert them back to single fields
from JSON object
No. of Documents
No.of Fields
Run
With JSON & Compression
With JSON & No Compression
Without JSON &Compression
Without JSON & No Compression
Insert
Read
Update
Insert
Read
Update
Insert
Read
Update
Insert
Read
Update
100
10
One
216
35
38
167
55
59
216
23
65
215
25
57
100
10
Two
-
48
93
-
49
102
35
134
25
101
100
10
Three
-
45
80
49
98
37
117
24
106
1000
10
One
404
144
160
447
109
157
503
148
129
482
111
126
1000
10
Two
-
175
268
-
129
259
176
301
135
295
1000
10
Three
-
177
279
142
263
174
299
135
296
10000
100
One
1763
1209
414
2151
1182
357
1574
335
847
1845
321
834
10000
100
Two
-
1352
811
-
1246
643
378
1151
334
1142
10000
100
Three
-
1318
729
1252
701
369
1088
336
1158
100000
1000
One
81021
50172
39480
77690
46138
38554
79289
20623
89653
93558
19357
108458
100000
1000
Two
-
53523
34523
52027
42957
20125
88771
19357
111962
100000
1000
Three
-
53816
33460
51703
43643
20409
98932
19630
110352