[jira] [Commented] (IMPALA-10447) Missing \n every 1024 or 2048 lines when exporting output from shell to a file

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272223#comment-17272223 ] 

ASF subversion and git services commented on IMPALA-10447:
----------------------------------------------------------

Commit 3b763b5c3235ebdb445ad0a4ae1bf79385e8df02 in impala's branch refs/heads/master from Andrew Sherman
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=3b763b5 ]

IMPALA-10447: Add a newline when exporting shell output to a file.

Impala shell outputs a batch of rows using OutputStream. Inside
OutputStream, output to a file is handled slightly differently from
output that is written to stdout. When writing to stdout we use print()
(which appends a newline) while when writing to a file we use write()
(which adds nothing). This difference was introduced in IMPALA-3343 so
this bug may be a regression introduced then. To ensure that output is
the same in either case we need to add a newline after writing each
batch of rows to a file.

TESTING:
    Added a new test for this case.

Change-Id: I078a06c54e0834bc1f898626afbfff4ded579fa9
Reviewed-on: http://gerrit.cloudera.org:8080/16966
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Missing \n every 1024 or 2048 lines when exporting output from shell to a file
> ------------------------------------------------------------------------------
>
>                 Key: IMPALA-10447
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10447
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Clients
>    Affects Versions: Impala 4.0
>            Reporter: Andrew Sherman
>            Assignee: Andrew Sherman
>            Priority: Major
>
> When Impala shell exports output to a file, for example 
> {code:java}
> impala-shell -B  -V  -q "select * from tpcds.item" -o filex.csv --output_delimiter=';'{code}
> then every 1024 (or maybe 2048) rows a newline is missing.
> I think the problem is here: [https://github.com/apache/impala/blob/9bb7157bf014282c95ab3e233b80d77e00c95b52/shell/shell_output.py#L119]
>  where we now use write instead of print to output the rows.
>  It may be sufficient to add a
> {code:java}
> out_file.write('\n')
> {code}
> here



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org