You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by sefty nindyastuti <se...@gmail.com> on 2020/04/13 04:24:23 UTC
how to get rid of double quotes in solr
I have a problem when indexing log data clusters in solr using logstash and
filebeat. there are double quotes in the solr index results,
how to solve this problem, please help
expect the results of the index that appears in solr as below:
{
"input": "log"
"hostname": "localhost"
"id": "22eddbc9-e60f-29cd-a352-b40154ba1736",
"type": "filebeat"
"ephemeral_id": "1a31d6e0-8ed9-1307-215f-5dfd361364c9"
"version": "7.6.1"
"offset": "2061794 "
"path": " /var/log/hadoop/hdfs/hadoop-hdfs-secondarynamenode-xxxxxx.log "
"host": "localhostxxx",
"message": "2020-04-11 19: 04: 28,575 INFO common.Util
(Util.java:receiveFile(314)) - Combined time for file downloads and fsync
to all disks stores 0.02s. The file download stores 0.02s at 58750.00 KB /
s Synchronous (fsync) write to disk of / hadoop / hdfs / namesecondary /
current / edits_tmp_xxxxxxxx "
{
Re: how to get rid of double quotes in solr
Posted by sefty nindyastuti <se...@gmail.com>.
the picture is the output from logstash, so I use config logstash to
accept input from the file and then output to the solr
Pada tanggal Sen, 13 Apr 2020 19.07, Erick Erickson <er...@gmail.com>
menulis:
> I don’t quite know what you’re asking about. Is that input or intput to
> Solr? Or is it output from logstash?
>
> What are you indexing? Because that doesn't look like data from a solr log.
>
> You might want to review: https://wiki.apache.org/solr/UsingMailingLists
>
> Best,
> Erick
>
> > On Apr 13, 2020, at 12:24 AM, sefty nindyastuti <se...@gmail.com>
> wrote:
> >
> > I have a problem when indexing log data clusters in solr using logstash
> and filebeat. there are double quotes in the solr index results,
> > how to solve this problem, please help
> >
> > expect the results of the index that appears in solr as below:
> >
> > {
> > "input": "log"
> > "hostname": "localhost"
> > "id": "22eddbc9-e60f-29cd-a352-b40154ba1736",
> > "type": "filebeat"
> > "ephemeral_id": "1a31d6e0-8ed9-1307-215f-5dfd361364c9"
> > "version": "7.6.1"
> > "offset": "2061794 "
> > "path": " /var/log/hadoop/hdfs/hadoop-hdfs-secondarynamenode-xxxxxx.log "
> > "host": "localhostxxx",
> > "message": "2020-04-11 19: 04: 28,575 INFO common.Util
> (Util.java:receiveFile(314)) - Combined time for file downloads and fsync
> to all disks stores 0.02s. The file download stores 0.02s at 58750.00 KB /
> s Synchronous (fsync) write to disk of / hadoop / hdfs / namesecondary /
> current / edits_tmp_xxxxxxxx "
> > {
> >
>
>
Re: how to get rid of double quotes in solr
Posted by Paras Lehana <pa...@indiamart.com>.
Hi,
Are you referring to the double quotes in the JSON result?
On Tue, 14 Apr 2020 at 08:29, sefty nindyastuti <se...@gmail.com>
wrote:
> the data that I use is log from hadoop, my problem is hadoop log from
> cluster,
> the schema I use is filebeat --> logstash --> solr, I use logstash config
> to parse the hadoop log, the hadoop log is inputted to the logstash via
> filebeat then output from the logstash indexed to the solr
>
> Pada tanggal Sen, 13 Apr 2020 pukul 19.07 Erick Erickson <
> erickerickson@gmail.com> menulis:
>
> > I don’t quite know what you’re asking about. Is that input or intput to
> > Solr? Or is it output from logstash?
> >
> > What are you indexing? Because that doesn't look like data from a solr
> log.
> >
> > You might want to review: https://wiki.apache.org/solr/UsingMailingLists
> >
> > Best,
> > Erick
> >
> > > On Apr 13, 2020, at 12:24 AM, sefty nindyastuti <se...@gmail.com>
> > wrote:
> > >
> > > I have a problem when indexing log data clusters in solr using logstash
> > and filebeat. there are double quotes in the solr index results,
> > > how to solve this problem, please help
> > >
> > > expect the results of the index that appears in solr as below:
> > >
> > > {
> > > "input": "log"
> > > "hostname": "localhost"
> > > "id": "22eddbc9-e60f-29cd-a352-b40154ba1736",
> > > "type": "filebeat"
> > > "ephemeral_id": "1a31d6e0-8ed9-1307-215f-5dfd361364c9"
> > > "version": "7.6.1"
> > > "offset": "2061794 "
> > > "path": "
> /var/log/hadoop/hdfs/hadoop-hdfs-secondarynamenode-xxxxxx.log "
> > > "host": "localhostxxx",
> > > "message": "2020-04-11 19: 04: 28,575 INFO common.Util
> > (Util.java:receiveFile(314)) - Combined time for file downloads and fsync
> > to all disks stores 0.02s. The file download stores 0.02s at 58750.00 KB
> /
> > s Synchronous (fsync) write to disk of / hadoop / hdfs / namesecondary /
> > current / edits_tmp_xxxxxxxx "
> > > {
> > >
> >
> >
>
--
--
Regards,
*Paras Lehana* [65871]
Development Engineer, *Auto-Suggest*,
IndiaMART InterMESH Ltd,
11th Floor, Tower 2, Assotech Business Cresterra,
Plot No. 22, Sector 135, Noida, Uttar Pradesh, India 201305
Mob.: +91-9560911996
Work: 0120-4056700 | Extn:
*1196*
--
*
*
<https://www.facebook.com/IndiaMART/videos/578196442936091/>
Re: how to get rid of double quotes in solr
Posted by sefty nindyastuti <se...@gmail.com>.
the data that I use is log from hadoop, my problem is hadoop log from
cluster,
the schema I use is filebeat --> logstash --> solr, I use logstash config
to parse the hadoop log, the hadoop log is inputted to the logstash via
filebeat then output from the logstash indexed to the solr
Pada tanggal Sen, 13 Apr 2020 pukul 19.07 Erick Erickson <
erickerickson@gmail.com> menulis:
> I don’t quite know what you’re asking about. Is that input or intput to
> Solr? Or is it output from logstash?
>
> What are you indexing? Because that doesn't look like data from a solr log.
>
> You might want to review: https://wiki.apache.org/solr/UsingMailingLists
>
> Best,
> Erick
>
> > On Apr 13, 2020, at 12:24 AM, sefty nindyastuti <se...@gmail.com>
> wrote:
> >
> > I have a problem when indexing log data clusters in solr using logstash
> and filebeat. there are double quotes in the solr index results,
> > how to solve this problem, please help
> >
> > expect the results of the index that appears in solr as below:
> >
> > {
> > "input": "log"
> > "hostname": "localhost"
> > "id": "22eddbc9-e60f-29cd-a352-b40154ba1736",
> > "type": "filebeat"
> > "ephemeral_id": "1a31d6e0-8ed9-1307-215f-5dfd361364c9"
> > "version": "7.6.1"
> > "offset": "2061794 "
> > "path": " /var/log/hadoop/hdfs/hadoop-hdfs-secondarynamenode-xxxxxx.log "
> > "host": "localhostxxx",
> > "message": "2020-04-11 19: 04: 28,575 INFO common.Util
> (Util.java:receiveFile(314)) - Combined time for file downloads and fsync
> to all disks stores 0.02s. The file download stores 0.02s at 58750.00 KB /
> s Synchronous (fsync) write to disk of / hadoop / hdfs / namesecondary /
> current / edits_tmp_xxxxxxxx "
> > {
> >
>
>
Re: how to get rid of double quotes in solr
Posted by Erick Erickson <er...@gmail.com>.
I don’t quite know what you’re asking about. Is that input or intput to Solr? Or is it output from logstash?
What are you indexing? Because that doesn't look like data from a solr log.
You might want to review: https://wiki.apache.org/solr/UsingMailingLists
Best,
Erick
> On Apr 13, 2020, at 12:24 AM, sefty nindyastuti <se...@gmail.com> wrote:
>
> I have a problem when indexing log data clusters in solr using logstash and filebeat. there are double quotes in the solr index results,
> how to solve this problem, please help
>
> expect the results of the index that appears in solr as below:
>
> {
> "input": "log"
> "hostname": "localhost"
> "id": "22eddbc9-e60f-29cd-a352-b40154ba1736",
> "type": "filebeat"
> "ephemeral_id": "1a31d6e0-8ed9-1307-215f-5dfd361364c9"
> "version": "7.6.1"
> "offset": "2061794 "
> "path": " /var/log/hadoop/hdfs/hadoop-hdfs-secondarynamenode-xxxxxx.log "
> "host": "localhostxxx",
> "message": "2020-04-11 19: 04: 28,575 INFO common.Util (Util.java:receiveFile(314)) - Combined time for file downloads and fsync to all disks stores 0.02s. The file download stores 0.02s at 58750.00 KB / s Synchronous (fsync) write to disk of / hadoop / hdfs / namesecondary / current / edits_tmp_xxxxxxxx "
> {
>