You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Misha Dmitriev (Code Review)" <ge...@cloudera.org> on 2018/07/20 17:28:41 UTC
[Impala-ASF-CR] IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
Misha Dmitriev has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10982
Change subject: IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
......................................................................
IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
This change switches initialization of HashMaps in IncompleteTable.colsByName_ and
StructType.fieldMap_ from eager to lazy. In this way, we avoid wasting memory
when HashMaps stay empty (even an empty HashMap uses at least 48 bytes in the heap).
This optimization becomes really relevant when a catalog server loads a very large
number (millions) of tables.
Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
---
M fe/src/main/java/org/apache/impala/catalog/StructType.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
2 files changed, 35 insertions(+), 8 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/10982/2
--
To view, visit http://gerrit.cloudera.org:8080/10982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
Gerrit-Change-Number: 10982
Gerrit-PatchSet: 2
Gerrit-Owner: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
[Impala-ASF-CR] IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
Posted by "Vuk Ercegovac (Code Review)" <ge...@cloudera.org>.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10982 )
Change subject: IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
......................................................................
Patch Set 2:
(2 comments)
thx for the changes!
questions:
(1) do you know which one of these contributes to more empty collections (structs or tables) for the case(s) you looked at?
(2) an empty collection for a struct comes up when it has no fields (otherwise there will be no savings). have you seen examples of call-sites where these data structures are created with no fields? perhaps some of those are worth following up (separately).
http://gerrit.cloudera.org:8080/#/c/10982/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/10982/2//COMMIT_MSG@14
PS2, Line 14:
pls add any tests that you ran. for this change, passing existing tests should be fine.
http://gerrit.cloudera.org:8080/#/c/10982/2/fe/src/main/java/org/apache/impala/catalog/Table.java
File fe/src/main/java/org/apache/impala/catalog/Table.java:
http://gerrit.cloudera.org:8080/#/c/10982/2/fe/src/main/java/org/apache/impala/catalog/Table.java@320
PS2, Line 320: {
: return;
: }
nit: single line
--
To view, visit http://gerrit.cloudera.org:8080/10982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
Gerrit-Change-Number: 10982
Gerrit-PatchSet: 2
Gerrit-Owner: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Jul 2018 17:55:58 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
Posted by "Vuk Ercegovac (Code Review)" <ge...@cloudera.org>.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10982 )
Change subject: IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
......................................................................
Patch Set 2:
My question (2) may have been unclear. I'm referring to StructType's HashMap that you're changed to be lazy initialized. In particular, I'm noting that its empty only when the no-arg constructor is called or when the input fields array is empty. Todd's question is basically the same (in StructType): why are so many empty (e.g., no field) structs created? We can look at call-sites.. perhaps we should change how/where these structs are used.
--
To view, visit http://gerrit.cloudera.org:8080/10982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
Gerrit-Change-Number: 10982
Gerrit-PatchSet: 2
Gerrit-Owner: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Jul 2018 18:56:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/10982 )
Change subject: IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
......................................................................
Patch Set 2:
Yep, that makes sense -- lazy initializing the StructType field in Table looks like an easy win here. You're right that changing the treatment of IncompleteTable would be a much deeper change.
Worth adding to the commit message, btw, that the "7.5% of catalog server heap" is in the case where the catalog is mostly made up of IncompleteTables. Or quantify the number of bytes per IncompleteTable that are saved. That way we can pretty easily estimate the absolute heap size savings on an arbitrary catalog just by knowing how many tables they have.
--
To view, visit http://gerrit.cloudera.org:8080/10982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
Gerrit-Change-Number: 10982
Gerrit-PatchSet: 2
Gerrit-Owner: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Jul 2018 20:36:58 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap.
Posted by "Misha Dmitriev (Code Review)" <ge...@cloudera.org>.
Misha Dmitriev has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/10982 )
Change subject: IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap.
......................................................................
IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap.
This change switches initialization from eager to lazy for the following fields
in Table: HashMap colsByName_ and ArrayType type_. In this way, we
avoid wasting memory when HashMaps stay empty (an empty HashMap still uses at
least 48 bytes in the heap), and when a StructType with its own empty HashMap
and ArrayList instances is initialized, but stays unused. I have calculated
ArrayType, StructType, two HashMaps and one ArrayList collectively use up
160 bytes in the heap per a Table. Further analysis can be found in the JIRA.
This optimization becomes really relevant when a catalog server loads a very
large number (millions) of tables, and all/most of them are represented as
IncompleteTable instances.
Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
---
M fe/src/main/java/org/apache/impala/catalog/Table.java
1 file changed, 24 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/10982/4
--
To view, visit http://gerrit.cloudera.org:8080/10982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
Gerrit-Change-Number: 10982
Gerrit-PatchSet: 4
Gerrit-Owner: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
[Impala-ASF-CR] IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
Posted by "Misha Dmitriev (Code Review)" <ge...@cloudera.org>.
Misha Dmitriev has posted comments on this change. ( http://gerrit.cloudera.org:8080/10982 )
Change subject: IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
......................................................................
Patch Set 2:
Thank you for the quick response, Vuk. Answering your questions:
1. The respective IMPALA-7219 contains more details. In the case that I looked at, there was exactly the same number (a little less than 8 million) of both kinds of HashMaps. I.e. ~8M coming from IncompleteTable.colsByName_ and ~8M coming from StructType.fieldMap_. Each empty HashMap takes 48 bytes, so collectively they wasted ~700MB of memory.
2. Let me clarify. There is no such thing as an object (collection) "with no fields". When you call 'new HashMap()', you always create a HashMap object, which internally has fields like 'int size, threshold' etc. The good news is that the most important field, 'HashMap$Entry table[]' is null until the first element is added to this table. But just the above data fields, plus the 12-byte internal header that each object in the JVM heap has, result in the fact that an empty, completely unpopulated HashMap, occupies 48 bytes. This is what I want to get rid of. If all this still seems a little vague, I suggest you take a look at the slides here: https://www.slideshare.net/MikhailMishaDmitriev/java-memory-analysis-problems-and-solutions?trk=v-feed In particular, slide 10 shows the internals and lifecycle of a HashMap.
So your suggestion in (2) may or may not make sense depending what exactly you had in mind, and needs some clarification.
Regarding tests: yes, I ran it through the Impala Jenkins build, and all tests passed: https://jenkins.impala.io/job/pre-review-test/186/
--
To view, visit http://gerrit.cloudera.org:8080/10982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
Gerrit-Change-Number: 10982
Gerrit-PatchSet: 2
Gerrit-Owner: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Jul 2018 18:41:03 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/10982 )
Change subject: IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
......................................................................
Patch Set 2:
Right, it seems like this is just a case of wastage by IncompleteTable objects, which contain the entirety of the 'Table' superclass but in fact use none of it. We could probably get away with a much cheaper implementation of IncompleteTable that has only the "name" field and nothing else.
--
To view, visit http://gerrit.cloudera.org:8080/10982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
Gerrit-Change-Number: 10982
Gerrit-PatchSet: 2
Gerrit-Owner: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Jul 2018 19:01:13 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
Posted by "Misha Dmitriev (Code Review)" <ge...@cloudera.org>.
Misha Dmitriev has posted comments on this change. ( http://gerrit.cloudera.org:8080/10982 )
Change subject: IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
......................................................................
Patch Set 2:
Thank you for your comments/clarifications, guys. I see what you mean now.
I've checked the heap dump more carefully, and now I realize that all the StructType instances (all of which have both of their data fields, fieldMap_ and fields_, empty) come from IncompleteTable's. Here is the relevant excerpt from the jxray heap report:
182,587K (1.2%): org.apache.impala.catalog.StructType: 7790403 / 100% objects
↖org.apache.impala.catalog.ArrayType.itemType_
↖org.apache.impala.catalog.IncompleteTable.type_
↖{java.util.concurrent.ConcurrentHashMap}.values
↖org.apache.impala.catalog.CatalogObjectCache.metadataCache_
↖org.apache.impala.catalog.Db.tableCache_
↖{java.util.concurrent.ConcurrentHashMap}.values
↖java.util.concurrent.atomic.AtomicReference.value
↖org.apache.impala.catalog.CatalogServiceCatalog.dbCache_
↖org.apache.impala.catalog.BuiltinsDb.parentCatalog_
↖Java Static org.apache.impala.catalog.Catalog.builtinsDb_
So, StructType objects themselves take 1.2% of the heap, plus their empty fieldMap_ maps take 2.5% and empty fields_ lists take 1.2%. And looks like if we simply initialize these StructType objects lazily (more precisely, the 'ArrayType type_' field in Table), we will save all the above memory, i.e. considerably more than the current patch saves. Furthermore, the lazy changes in StructType itself can be reverted, which would keep the code more readable.
Makes sense? I think this is the best I can do in Impala code with my current very limited understanding of it. More advanced changes, like making IncompleteTable not inherit from Table, would likely require a much deeper knowledge of how this code works.
--
To view, visit http://gerrit.cloudera.org:8080/10982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
Gerrit-Change-Number: 10982
Gerrit-PatchSet: 2
Gerrit-Owner: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Jul 2018 19:50:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/10982 )
Change subject: IMPALA-7219. 7.5% of Catalog Server heap wasted by empty HashMaps and ArrayLists
......................................................................
Patch Set 2:
(5 comments)
See suggestion about changing IncompleteTable to not inherit from Table -- maybe now with the 'FeTable' interface we could avoid that?
http://gerrit.cloudera.org:8080/#/c/10982/2/fe/src/main/java/org/apache/impala/catalog/StructType.java
File fe/src/main/java/org/apache/impala/catalog/StructType.java:
http://gerrit.cloudera.org:8080/#/c/10982/2/fe/src/main/java/org/apache/impala/catalog/StructType.java@37
PS2, Line 37: public class StructType extends Type {
agreed with Vuk -- not clear to me why we have so many empty structs. I thought StructType is only used for describing the schema of a table, and a table with no fields seems uncommon. Do you have handy the "paths to GC root" for these empty fields?
http://gerrit.cloudera.org:8080/#/c/10982/2/fe/src/main/java/org/apache/impala/catalog/StructType.java@39
PS2, Line 39: // Made non-final so that it can be initialized lazily to save memory when this map is not used.
nit: this comment makes sense when looking at this as a patch, but when someone sees the code later, the "Made non-final" thing isn't super clear.
Perhaps better to just describe the current treatment of the member, like: "Map of field name to field. Null is used to indicate empty to save memory."
http://gerrit.cloudera.org:8080/#/c/10982/2/fe/src/main/java/org/apache/impala/catalog/StructType.java@84
PS2, Line 84: public StructField getField(String fieldName) {
I wonder if we could do even better here and only lazily construct a field map at places where method is called. it seems this is the only usage site of the field map, and best I can tell, this method is only called during query analysis, and never by the catalogd itself. Given that, it doesn't really make sense to keep around these members, when it would be pretty trivial to reconstruct the map as necessary during path resolution. Then we could fully remove all these maps (even the non-empty ones). Would that save significant memory?
Another thought here: it seems like the most common case where we have an empty struct woudl be a reference from IncompleteTable. I'm guessing you were looking at a catalogd dump from a cluster with a lot of tables? Maybe we could change IncompleteTable to not inherit from 'Table' so that it can avoid all of these members entirely?
http://gerrit.cloudera.org:8080/#/c/10982/2/fe/src/main/java/org/apache/impala/catalog/Table.java
File fe/src/main/java/org/apache/impala/catalog/Table.java:
http://gerrit.cloudera.org:8080/#/c/10982/2/fe/src/main/java/org/apache/impala/catalog/Table.java@320
PS2, Line 320: {
: return;
: }
> nit: single line
same as below
http://gerrit.cloudera.org:8080/#/c/10982/2/fe/src/main/java/org/apache/impala/catalog/Table.java@447
PS2, Line 447: public Column getColumn(String name) {
I think it would actually be an error to call getColumn before the colsByName_ map was initted. Such a call would indicate that columns haven't been loaded.
--
To view, visit http://gerrit.cloudera.org:8080/10982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
Gerrit-Change-Number: 10982
Gerrit-PatchSet: 2
Gerrit-Owner: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Jul 2018 18:26:02 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap.
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has abandoned this change. ( http://gerrit.cloudera.org:8080/10982 )
Change subject: IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap.
......................................................................
Abandoned
Appears stale, reopen if you disagree
--
To view, visit http://gerrit.cloudera.org:8080/10982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
Gerrit-Change-Number: 10982
Gerrit-PatchSet: 8
Gerrit-Owner: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
[Impala-ASF-CR] IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap.
Posted by "Misha Dmitriev (Code Review)" <ge...@cloudera.org>.
Misha Dmitriev has posted comments on this change. ( http://gerrit.cloudera.org:8080/10982 )
Change subject: IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap.
......................................................................
Patch Set 4:
Patch set 4 failed twice in the Jenkins build, so I suspect there is some problem in it. The last failure is https://jenkins.impala.io/job/pre-review-test/190/ Unfortunately, I don't see any clues in the main build log, and I don't know how to get to the Java impala catalog logs. Any help would be appreciated.
--
To view, visit http://gerrit.cloudera.org:8080/10982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
Gerrit-Change-Number: 10982
Gerrit-PatchSet: 4
Gerrit-Owner: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Sat, 21 Jul 2018 21:36:22 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap.
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10982 )
Change subject: IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap.
......................................................................
Patch Set 4:
I0721 20:07:39.425611 85266 jni-util.cc:230] org.apache.impala.catalog.TableLoadingException: Failed to load metadata for table: tpch.lineitem
at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1290)
at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1217)
at org.apache.impala.catalog.TableLoader.load(TableLoader.java:80)
at org.apache.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:241)
at org.apache.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:238)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at org.apache.impala.catalog.Table.clearColumns(Table.java:192)
at org.apache.impala.catalog.HdfsTable.loadSchema(HdfsTable.java:1583)
at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1257)
... 8 more
I0721 20:07:39.426174 85266 status.cc:125] TableLoadingException: Failed to load metadata for table: tpch.lineitem
CAUSED BY: NullPointerException: null
Logs from the data loading process are here: https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/2771/artifact/Impala/logs_static/logs/data_loading/
--
To view, visit http://gerrit.cloudera.org:8080/10982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6
Gerrit-Change-Number: 10982
Gerrit-PatchSet: 4
Gerrit-Owner: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Misha Dmitriev <co...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Sun, 22 Jul 2018 00:15:14 +0000
Gerrit-HasComments: No