You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hawq.apache.org by "Oleksandr Diachenko (JIRA)" <ji...@apache.org> on 2016/10/13 19:42:20 UTC
[jira] [Resolved] (HAWQ-1084) Sometimes psql crashes with "out of
memory" on \d hcatalog.* command
[ https://issues.apache.org/jira/browse/HAWQ-1084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oleksandr Diachenko resolved HAWQ-1084.
---------------------------------------
Resolution: Fixed
> Sometimes psql crashes with "out of memory" on \d hcatalog.* command
> --------------------------------------------------------------------
>
> Key: HAWQ-1084
> URL: https://issues.apache.org/jira/browse/HAWQ-1084
> Project: Apache HAWQ
> Issue Type: Bug
> Components: PXF
> Reporter: Oleksandr Diachenko
> Assignee: Oleksandr Diachenko
> Fix For: backlog
>
>
> Normally it returns definition of all Hive tables:
> {code}
> PXF Hive Table "default.hive_abc"
> Column | Type
> --------+--------
> t0 | text
> num1 | int4
> d1 | float8
> t1 | text
> PXF Hive Table "default.hive_abc222"
> Column | Type
> --------+--------
> d1 | float8
> t0 | text
> num1 | int4
> t1 | text
> PXF Hive Table "default.hive_avro_table"
> Column | Type
> --------+--------
> t0 | text
> d1 | float8
> num1 | int4
> t1 | text
> PXF Hive Table "default.hive_binary"
> Column | Type
> --------+-------
> b1 | bytea
> PXF Hive Table "default.hive_collections_table"
> Column | Type
> --------+--------
> ut1 | text
> sr1 | text
> m1 | text
> a1 | text
> s1 | text
> f1 | float4
> PXF Hive Table "default.hive_many_partitioned_table"
> Column | Type
> --------+-----------
> s1 | text
> s2 | text
> n1 | int4
> d1 | float8
> dc1 | numeric
> tm | timestamp
> f | float4
> bg | int8
> b | bool
> tn | int2
> sml | int2
> dt | date
> vc1 | varchar
> c1 | bpchar
> bin | bytea
> PXF Hive Table "default.hive_orc_all_types"
> Column | Type
> --------+-----------
> bin | bytea
> c1 | bpchar
> vc1 | varchar
> dt | date
> sml | int2
> tn | int2
> b | bool
> bg | int8
> f | float4
> tm | timestamp
> dc1 | numeric
> d1 | float8
> n1 | int4
> s2 | text
> s1 | text
> PXF Hive Table "default.hive_orc_snappy"
> Column | Type
> --------+--------
> t0 | text
> t1 | text
> num1 | int4
> d1 | float8
> PXF Hive Table "default.hive_orc_table"
> Column | Type
> --------+--------
> d1 | float8
> num1 | int4
> t1 | text
> t0 | text
> PXF Hive Table "default.hive_orc_zlib"
> Column | Type
> --------+--------
> t0 | text
> t1 | text
> num1 | int4
> d1 | float8
> PXF Hive Table "default.hive_parquet_table"
> Column | Type
> --------+--------
> d1 | float8
> num1 | int4
> t1 | text
> t0 | text
> PXF Hive Table "default.hive_partitioned_clustered_sorted_table"
> Column | Type
> --------+--------
> fmt | text
> d1 | float8
> num1 | int4
> t1 | text
> t0 | text
> PXF Hive Table "default.hive_partitioned_clustered_table"
> Column | Type
> --------+--------
> fmt | text
> d1 | float8
> num1 | int4
> t1 | text
> t0 | text
> PXF Hive Table "default.hive_partitioned_skewed_stored_table"
> Column | Type
> --------+--------
> fmt | text
> d1 | float8
> num1 | int4
> t1 | text
> t0 | text
> PXF Hive Table "default.hive_partitioned_skewed_table"
> Column | Type
> --------+--------
> fmt | text
> t0 | text
> d1 | float8
> num1 | int4
> t1 | text
> PXF Hive Table "default.hive_partitioned_table"
> Column | Type
> --------+--------
> t0 | text
> t1 | text
> num1 | int4
> d1 | float8
> fmt | text
> PXF Hive Table "default.hive_rc_table"
> Column | Type
> --------+--------
> t0 | text
> d1 | float8
> num1 | int4
> t1 | text
> PXF Hive Table "default.hive_rc_table_no_serde"
> Column | Type
> --------+--------
> d1 | float8
> num1 | int4
> t1 | text
> t0 | text
> PXF Hive Table "default.hive_sequence_table"
> Column | Type
> --------+--------
> t0 | text
> t1 | text
> num1 | int4
> d1 | float8
> PXF Hive Table "default.hive_small_data"
> Column | Type
> --------+--------
> d1 | float8
> n1 | int4
> s2 | text
> s1 | text
> PXF Hive Table "default.hive_small_data_no_data_file"
> Column | Type
> --------+--------
> d1 | float8
> n1 | int4
> s2 | text
> s1 | text
> PXF Hive Table "default.hive_table"
> Column | Type
> --------+--------
> s1 | text
> n1 | int4
> d1 | float8
> bg | int8
> b | bool
> PXF Hive Table "default.hive_types"
> Column | Type
> --------+-----------
> bin | bytea
> c1 | bpchar
> vc1 | varchar
> dt | date
> sml | int2
> tn | int2
> b | bool
> bg | int8
> f | float4
> tm | timestamp
> dc1 | numeric
> d1 | float8
> n1 | int4
> s2 | text
> s1 | text
> PXF Hive Table "default.hivebinaryrc"
> Column | Type
> --------+-------
> data2 | bytea
> data1 | bytea
> n1 | int4
> s1 | text
> PXF Hive Table "default.many_partitions"
> Column | Type
> --------+------
> date_i | text
> i | int4
> PXF Hive Table "default.rc_hive_types"
> Column | Type
> --------+-----------
> bg | int8
> b | bool
> si | int2
> bin | bytea
> f | float4
> tm | timestamp
> d1 | float8
> n1 | int4
> s1 | text
> key | text
> PXF Hive Table "default.reg_collections"
> Column | Type
> --------+--------
> sr1 | text
> m1 | text
> a1 | text
> f1 | float4
> s1 | text
> PXF Hive Table "default.reg_heterogen"
> Column | Type
> --------+--------
> fmt | text
> d1 | float8
> num1 | int4
> t1 | text
> t0 | text
> {code}
> Sometimes, not consistently it fails with OOM:
> {code}
> pxfautomation=# set client_min_messages=DEBUG1;
> DEBUG1: Dropping in memory mapping OidInMemHeapMapping
> DEBUG1: Dropping in memory mapping OidInMemOnlyMapping
> SET
> pxfautomation=# \d hcatalog.*
> DEBUG1: Message type Q received by from libpq, len = 151
> DEBUG1: Inserted entry for query (sessionid=17, commandcnt=3)
> DEBUG1: SISC READER (shareid=0, slice=-1): exec dependency on slice -1, driver_slice is -1
> DEBUG1: SISC READER (shareid=0, slice=-1): exec dependency on slice -1, driver_slice is -1
> DEBUG1: SISC (shareid=0, slice=-1): No tuplestore yet, initializing tuplestore
> DEBUG1: Loopback interface IP address: [::1]
> DEBUG1: Replaced url: http://[::1]:51200/pxf/v14/Metadata/getMetadata?profile=Hive&pattern=*
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field b1, type bytea
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field s1, type text
> DEBUG1: Parsing field f1, type float4
> DEBUG1: Parsing field a1, type text
> DEBUG1: Parsing field m1, type text
> DEBUG1: Parsing field sr1, type text
> DEBUG1: Parsing field ut1, type text
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field s1, type text
> DEBUG1: Parsing field s2, type text
> DEBUG1: Parsing field n1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsing field dc1, type numeric
> DEBUG1: modifier[0]: 38
> DEBUG1: modifier[1]: 18
> DEBUG1: Parsing field tm, type timestamp
> DEBUG1: Parsing field f, type float4
> DEBUG1: Parsing field bg, type int8
> DEBUG1: Parsing field b, type bool
> DEBUG1: Parsing field tn, type int2
> DEBUG1: Parsing field sml, type int2
> DEBUG1: Parsing field dt, type date
> DEBUG1: Parsing field vc1, type varchar
> DEBUG1: modifier[0]: 5
> DEBUG1: Parsing field c1, type bpchar
> DEBUG1: modifier[0]: 3
> DEBUG1: Parsing field bin, type bytea
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field s1, type text
> DEBUG1: Parsing field s2, type text
> DEBUG1: Parsing field n1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsing field dc1, type numeric
> DEBUG1: modifier[0]: 38
> DEBUG1: modifier[1]: 18
> DEBUG1: Parsing field tm, type timestamp
> DEBUG1: Parsing field f, type float4
> DEBUG1: Parsing field bg, type int8
> DEBUG1: Parsing field b, type bool
> DEBUG1: Parsing field tn, type int2
> DEBUG1: Parsing field sml, type int2
> DEBUG1: Parsing field dt, type date
> DEBUG1: Parsing field vc1, type varchar
> DEBUG1: modifier[0]: 5
> DEBUG1: Parsing field c1, type bpchar
> DEBUG1: modifier[0]: 3
> DEBUG1: Parsing field bin, type bytea
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsing field fmt, type text
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsing field fmt, type text
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsing field fmt, type text
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsing field fmt, type text
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsing field fmt, type text
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field s1, type text
> DEBUG1: Parsing field s2, type text
> DEBUG1: Parsing field n1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field s1, type text
> DEBUG1: Parsing field s2, type text
> DEBUG1: Parsing field n1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field s1, type text
> DEBUG1: Parsing field n1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsing field bg, type int8
> DEBUG1: Parsing field b, type bool
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field s1, type text
> DEBUG1: Parsing field s2, type text
> DEBUG1: Parsing field n1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsing field dc1, type numeric
> DEBUG1: modifier[0]: 38
> DEBUG1: modifier[1]: 18
> DEBUG1: Parsing field tm, type timestamp
> DEBUG1: Parsing field f, type float4
> DEBUG1: Parsing field bg, type int8
> DEBUG1: Parsing field b, type bool
> DEBUG1: Parsing field tn, type int2
> DEBUG1: Parsing field sml, type int2
> DEBUG1: Parsing field dt, type date
> DEBUG1: Parsing field vc1, type varchar
> DEBUG1: modifier[0]: 5
> DEBUG1: Parsing field c1, type bpchar
> DEBUG1: modifier[0]: 3
> DEBUG1: Parsing field bin, type bytea
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field s1, type text
> DEBUG1: Parsing field n1, type int4
> DEBUG1: Parsing field data1, type bytea
> DEBUG1: Parsing field data2, type bytea
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field i, type int4
> DEBUG1: Parsing field date_i, type text
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field key, type text
> DEBUG1: Parsing field s1, type text
> DEBUG1: Parsing field n1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsing field tm, type timestamp
> DEBUG1: Parsing field f, type float4
> DEBUG1: Parsing field bg, type int8
> DEBUG1: Parsing field b, type bool
> DEBUG1: Parsing field si, type int2
> DEBUG1: Parsing field bin, type bytea
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field s1, type text
> DEBUG1: Parsing field f1, type float4
> DEBUG1: Parsing field a1, type text
> DEBUG1: Parsing field m1, type text
> DEBUG1: Parsing field sr1, type text
> DEBUG1: Parsed item , namespace
> DEBUG1: Parsing field t0, type text
> DEBUG1: Parsing field t1, type text
> DEBUG1: Parsing field num1, type int4
> DEBUG1: Parsing field d1, type float8
> DEBUG1: Parsing field fmt, type text
> DEBUG1: SISC (shareid=0, slice=-1): No tuplestore yet, initializing tuplestore
> DEBUG1: Dropping in memory mapping OidInMemHeapMapping
> DEBUG1: Deleted entry for query (sessionid=17, commandcnt=3)
> DEBUG1: Dropping in memory mapping OidInMemOnlyMapping
> out of memory
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)