You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Harsh Verma <hv...@tableau.com> on 2018/06/27 18:40:07 UTC
Custom FieldType not getting called while querying to get the field
with custom fieldtype
Hi All,
I am trying to fetch values (docValues to be exact) from external redis during query time using a custom written FieldType which I am loading in my solr instance. I notice that my custom FieldType is being initialized based on the schema when loading the core. However, at query time, my custom code is not called and hence the response do not have the field from redis populated.
Solr version: 7.3.1
Here is my implementation: https://gist.github.com/vermaharsh/042e1cf07070c6d9b3b6cc7eaaf0b49c
Here is my solrconfig.xml: https://gist.github.com/vermaharsh/97d7310b242fd7ba3d8d3a3bda209ac3
Here is my managed-schema: https://gist.github.com/vermaharsh/8a89195377802a6bbcdde9215a2fdaf5
Query that I am making: /solr/redis/select?fl=hits&q=*:*&sort=hits%20asc
Response that I get back: https://gist.github.com/vermaharsh/2f63282b10320c4c35a9f85016fe30c0
Can someone help me identify what am I missing?
Thanks,
Harsh
Copy of discussion on #solr irc channel.
<harsh> I am trying to write a custom solr FieldType which fetches value from external redis instance. I got Solr to load my custom jars and have defined the schema with custom fieldType class. But at query time, I cannot get values in the response.
[08:38] <harsh> I do not see any errors in the logs as well, so cannot tell if something failed
[08:38] <harsh> here is the code for my custom FieldType - https://gist.github.com/vermaharsh/042e1cf07070c6d9b3b6cc7eaaf0b49c
[08:39] <harsh> this is my solrconfig.xml for my custom configset - https://gist.github.com/vermaharsh/97d7310b242fd7ba3d8d3a3bda209ac3
[08:42] <harsh> I basically placed the necessary jars under contrib/redis-field
[08:43] <harsh> and my managed-schema - https://gist.github.com/vermaharsh/8a89195377802a6bbcdde9215a2fdaf5
[08:43] <harsh> I am using solr version 7.3.1
[08:44] <harsh> any idea why I am not getting the value for my custom field type back?
[08:45] <@elyograg> harsh: I'm not familiar with the API to that level. FYI, this should go in #solr -- this channel is for development of Solr itself.
[08:46] <@elyograg> the field is not marked as stored. I wonder if that might tell Solr that it shouldn't be returned at all. (I don't know whether setting stored=true might require something more in your code, though)
[08:46] <harsh> I am using it as docValue, but I can try stored. Though, as you mentioned, not sure if that would need more to be implemented in the code
[08:47] <harsh> I will try #solr channel for the question as well
[08:47] <@elyograg> ah, I didn't scroll to the right enough to see that part. :)
[08:48] <@elyograg> I wonder if you might need useDocValuesAsStored="true".
[08:49] <harsh> I thought that is the default value
[08:51] <harsh> for completeness, this is the query that I am using - /solr/redis/select?fl=hits&q=*:*&sort=hits%20asc
[08:52] <harsh> and response that I got back - https://gist.github.com/vermaharsh/2f63282b10320c4c35a9f85016fe30c0
[08:53] == dataminion [~leahy@c-69-181-118-61.hsd1.ca.comcast.net] has joined #solr-dev
[08:53] <@elyograg> this one's probably going to need to go to the mailing list. with all the pastes you've mentioned here.
[08:57] == dataminion [~leahy@c-69-181-118-61.hsd1.ca.comcast.net] has quit [Ping timeout: 264 seconds]
[08:59] <@elyograg> you may be right about that being default. The code in FieldType.java seems to support that.
[09:00] <@elyograg> if (schemaVersion >= 1.6f) properties |= USE_DOCVALUES_AS_STORED;
[09:02] <harsh> alright, thanks elyograg for looking into it. I will send this to mailing list as well.
RE: Custom FieldType not getting called while querying to get the
field with custom fieldtype
Posted by Harsh Verma <hv...@tableau.com>.
Not sure if I am on the mailing list. Trying again. Also additional info.
I added more logs to my custom field type and now I see that my code is called. But, still I cannot get the value to be populated back in response.
Also, the sorting is not working, even though I see that correct values are fetched from redis.
Here is sample response for query - solr/redis/select?debugQuery=on&fl=*,hits&q=*:*&sort=hits%20desc
Response - https://gist.github.com/vermaharsh/f7f74a65a5403ecec9310ceb81cb674c
Sort order should be - 10, 9, 8, 6, 5, 4, 3, 2, 1, 7; because, corresponding hits value from redis are - 11111, 900, 800, 600, 500, 400, 300, 200, 100, 7.
From: Harsh Verma
Sent: Wednesday, June 27, 2018 11:40 AM
To: dev@lucene.apache.org
Subject: Custom FieldType not getting called while querying to get the field with custom fieldtype
Hi All,
I am trying to fetch values (docValues to be exact) from external redis during query time using a custom written FieldType which I am loading in my solr instance. I notice that my custom FieldType is being initialized based on the schema when loading the core. However, at query time, my custom code is not called and hence the response do not have the field from redis populated.
Solr version: 7.3.1
Here is my implementation: https://gist.github.com/vermaharsh/042e1cf07070c6d9b3b6cc7eaaf0b49c
Here is my solrconfig.xml: https://gist.github.com/vermaharsh/97d7310b242fd7ba3d8d3a3bda209ac3
Here is my managed-schema: https://gist.github.com/vermaharsh/8a89195377802a6bbcdde9215a2fdaf5
Query that I am making: /solr/redis/select?fl=hits&q=*:*&sort=hits%20asc
Response that I get back: https://gist.github.com/vermaharsh/2f63282b10320c4c35a9f85016fe30c0
Can someone help me identify what am I missing?
Thanks,
Harsh
Copy of discussion on #solr irc channel.
<harsh> I am trying to write a custom solr FieldType which fetches value from external redis instance. I got Solr to load my custom jars and have defined the schema with custom fieldType class. But at query time, I cannot get values in the response.
[08:38] <harsh> I do not see any errors in the logs as well, so cannot tell if something failed
[08:38] <harsh> here is the code for my custom FieldType - https://gist.github.com/vermaharsh/042e1cf07070c6d9b3b6cc7eaaf0b49c
[08:39] <harsh> this is my solrconfig.xml for my custom configset - https://gist.github.com/vermaharsh/97d7310b242fd7ba3d8d3a3bda209ac3
[08:42] <harsh> I basically placed the necessary jars under contrib/redis-field
[08:43] <harsh> and my managed-schema - https://gist.github.com/vermaharsh/8a89195377802a6bbcdde9215a2fdaf5
[08:43] <harsh> I am using solr version 7.3.1
[08:44] <harsh> any idea why I am not getting the value for my custom field type back?
[08:45] <@elyograg> harsh: I'm not familiar with the API to that level. FYI, this should go in #solr -- this channel is for development of Solr itself.
[08:46] <@elyograg> the field is not marked as stored. I wonder if that might tell Solr that it shouldn't be returned at all. (I don't know whether setting stored=true might require something more in your code, though)
[08:46] <harsh> I am using it as docValue, but I can try stored. Though, as you mentioned, not sure if that would need more to be implemented in the code
[08:47] <harsh> I will try #solr channel for the question as well
[08:47] <@elyograg> ah, I didn't scroll to the right enough to see that part. :)
[08:48] <@elyograg> I wonder if you might need useDocValuesAsStored="true".
[08:49] <harsh> I thought that is the default value
[08:51] <harsh> for completeness, this is the query that I am using - /solr/redis/select?fl=hits&q=*:*&sort=hits%20asc
[08:52] <harsh> and response that I got back - https://gist.github.com/vermaharsh/2f63282b10320c4c35a9f85016fe30c0
[08:53] == dataminion [~leahy@c-69-181-118-61.hsd1.ca.comcast.net] has joined #solr-dev
[08:53] <@elyograg> this one's probably going to need to go to the mailing list. with all the pastes you've mentioned here.
[08:57] == dataminion [~leahy@c-69-181-118-61.hsd1.ca.comcast.net] has quit [Ping timeout: 264 seconds]
[08:59] <@elyograg> you may be right about that being default. The code in FieldType.java seems to support that.
[09:00] <@elyograg> if (schemaVersion >= 1.6f) properties |= USE_DOCVALUES_AS_STORED;
[09:02] <harsh> alright, thanks elyograg for looking into it. I will send this to mailing list as well.