[jira] [Created] (IMPALA-5623) lag() on STRING cols may hold memory until query end

["Matthew Jacobs (JIRA)" <ji...@apache.org>]

Matthew Jacobs created IMPALA-5623:
--------------------------------------

             Summary: lag() on STRING cols may hold memory until query end
                 Key: IMPALA-5623
                 URL: https://issues.apache.org/jira/browse/IMPALA-5623
             Project: IMPALA
          Issue Type: Bug
          Components: Backend
    Affects Versions: Impala 2.8.0
            Reporter: Matthew Jacobs
            Assignee: Matthew Jacobs


IMPALA-4120 fixed an issue where lead/lag was potentially operating on memory that the UDA didn't own, resulting in potentially wrong results. As part of that fix, lead and lag started allocating memory in Init() which needs to be freed in Serialize() or Finalize(), but only lead was updated to free the memory. This memory is eventually freed when the fragment is torn down, but as a result of not freeing the memory in Serialize or Finalize, the memory may be allocated longer than necessary.

A warning is printed when this happens:
{quote}
[localhost:21000] > select concat(' foo ', lag(string_col,1,NULL) over (partition by bool_col order by id)) from functional.alltypestiny order by id;
Query: select concat(' foo ', lag(string_col,1,NULL) over (partition by bool_col order by id)) from functional.alltypestiny order by id
Query submitted at: 2017-07-06 13:56:24 (Coordinator: http://mj-desktop.ca.cloudera.com:25000)
Query progress can be monitored at: http://mj-desktop.ca.cloudera.com:25000/query_plan?query_id=124dfe18a6cee76a:fafdea4000000000
+----------------------------------------------------------------------------------------+
| concat(' foo ', lag(string_col, 1, null) over (partition by bool_col order by id asc)) |
+----------------------------------------------------------------------------------------+
| NULL                                                                                   |
| NULL                                                                                   |
|  foo 0                                                                                 |
|  foo 1                                                                                 |
|  foo 0                                                                                 |
|  foo 1                                                                                 |
|  foo 0                                                                                 |
|  foo 1                                                                                 |
+----------------------------------------------------------------------------------------+
WARNINGS: UDF WARNING: Memory leaked via FunctionContext::Allocate() or FunctionContext::AllocateLocal()

Fetched 8 row(s) in 0.12s
{quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)