You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Sanjay Ramanathan <sa...@lucidworks.com> on 2014/07/18 03:50:30 UTC

Morphlines: getting the error "unclosed quotation" although handled

Hi,


Firstly, sorry for inundating you with queries every other day.


While trying to push data into Solr from a TSV file using Flume/Morphlines.

I'm getting the error


"Unable to deliver event

Caused by: org.kitesdk.morphline.api.MorphlineRuntimeException: java.lang.IllegalStateException: unexpected end of file, unclosed quotation
"

My morphline.conf file has the following properties:
"
readCSV {
          separator : "\t"
          columns : [parcelid,propertyaddress,streetnumber,streetdirection,streetname,streettype,unitnumber,heatedarea,bedrooms,bathrooms,book,page,instrumentnumber,pid,link,mailingaddress1,mailingaddress2,mailingaddresscity,mailingaddresscountry,mailingaddressstate,mailingaddresszip,buyers,sellers,saleamount,saledescriptioncode,saledescription,averageyearbuilt,dateofsale,propertyusecode,numberofbuildings,propertyusetype]
          ignoreFirstLine : false
          quoteChar : "\""
          commentPrefix : " "
          trim : true
          charset : UTF-8

      }
"
I've tried commentPrefix with "" "#" " "
I've tried trim with true and false.

I'm using the flume 1.5.0. The kite-morphlines jars in the lib folder are 0.12.0 version.

I've attached the TSV file(POCSampleData-2.txt).


I don't think there is anything wrong with the flume.conf file as it pushes data properly into Solr, if the records with quotes are removed.


Thanks in advance.


Sincerely,

Sanjay Ramanathan

Re: Morphlines: getting the error "unclosed quotation" although handled

Posted by Wolfgang Hoschek <wh...@cloudera.com>.
This means that your TSV data file contains invalid data. Every opening quote character needs to eventually be followed by a closing quote character in the data file. Such a closing quote is apparently missing.

Consider fixing your input data, or perhaps try to handle it with readLine + split rather than readCSV, considering that it doesn’t follow CSV format after all.

Wolfgang.

On Jul 18, 2014, at 4:50 AM, Sanjay Ramanathan <sa...@lucidworks.com> wrote:

> Hi,
> 
> Firstly, sorry for inundating you with queries every other day.
> 
> While trying to push data into Solr from a TSV file using Flume/Morphlines.
> I'm getting the error 
> 
> "Unable to deliver event
> Caused by: org.kitesdk.morphline.api.MorphlineRuntimeException: java.lang.IllegalStateException: unexpected end of file, unclosed quotation
> "
> 
> My morphline.conf file has the following properties:
> "
> readCSV {
>           separator : "\t"
>           columns : [parcelid,propertyaddress,streetnumber,streetdirection,streetname,streettype,unitnumber,heatedarea,bedrooms,bathrooms,book,page,instrumentnumber,pid,link,mailingaddress1,mailingaddress2,mailingaddresscity,mailingaddresscountry,mailingaddressstate,mailingaddresszip,buyers,sellers,saleamount,saledescriptioncode,saledescription,averageyearbuilt,dateofsale,propertyusecode,numberofbuildings,propertyusetype]
>           ignoreFirstLine : false
>           quoteChar : "\""
>           commentPrefix : " "
>           trim : true
>           charset : UTF-8
> 
>       }
> "
> I've tried commentPrefix with "" "#" " "
> I've tried trim with true and false. 
> 
> I'm using the flume 1.5.0. The kite-morphlines jars in the lib folder are 0.12.0 version.
> 
> I've attached the TSV file(POCSampleData-2.txt).
> 
> I don't think there is anything wrong with the flume.conf file as it pushes data properly into Solr, if the records with quotes are removed.
> 
> Thanks in advance.
> 
> Sincerely,
> Sanjay Ramanathan
> 
> <POCSampleData-2.txt>