You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Alex Rodoni (Code Review)" <ge...@cloudera.org> on 2019/06/25 15:42:41 UTC

[Impala-ASF-CR] IMPALA-8341: [DOCS] Describe the settings for remote data caching 

Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13724


Change subject: IMPALA-8341: [DOCS] Describe the settings for remote data caching
......................................................................

IMPALA-8341: [DOCS] Describe the settings for remote data caching

Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
---
M docs/impala.ditamap
A docs/topics/impala_data_cache.xml
2 files changed, 85 insertions(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/13724/1
-- 
To view, visit http://gerrit.cloudera.org:8080/13724
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Gerrit-Change-Number: 13724
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni <ar...@cloudera.com>

[Impala-ASF-CR] IMPALA-8341: [DOCS] Describe the setting for remote data caching

Posted by "Michael Ho (Code Review)" <ge...@cloudera.org>.
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/13724 )

Change subject: IMPALA-8341: [DOCS] Describe the setting for remote data caching
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/13724
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Gerrit-Change-Number: 13724
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Jun 2019 18:45:27 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8341: [DOCS] Describe the setting for remote data caching

Posted by "Alex Rodoni (Code Review)" <ge...@cloudera.org>.
Hello Michael Ho, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13724

to look at the new patch set (#2).

Change subject: IMPALA-8341: [DOCS] Describe the setting for remote data caching
......................................................................

IMPALA-8341: [DOCS] Describe the setting for remote data caching

Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
---
M docs/impala.ditamap
A docs/topics/impala_data_cache.xml
2 files changed, 86 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/13724/2
-- 
To view, visit http://gerrit.cloudera.org:8080/13724
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Gerrit-Change-Number: 13724
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>

[Impala-ASF-CR] IMPALA-8341: [DOCS] Describe the settings for remote data caching

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13724 )

Change subject: IMPALA-8341: [DOCS] Describe the settings for remote data caching
......................................................................


Patch Set 1: Verified+1

Build Successful 

https://jenkins.impala.io/job/gerrit-docs-auto-test/356/ : Doc tests passed.


-- 
To view, visit http://gerrit.cloudera.org:8080/13724
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Gerrit-Change-Number: 13724
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Jun 2019 15:48:27 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8341: [DOCS] Describe the setting for remote data caching

Posted by "Alex Rodoni (Code Review)" <ge...@cloudera.org>.
Hello Michael Ho, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13724

to look at the new patch set (#3).

Change subject: IMPALA-8341: [DOCS] Describe the setting for remote data caching
......................................................................

IMPALA-8341: [DOCS] Describe the setting for remote data caching

Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
---
M docs/impala.ditamap
A docs/topics/impala_data_cache.xml
2 files changed, 87 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/13724/3
-- 
To view, visit http://gerrit.cloudera.org:8080/13724
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Gerrit-Change-Number: 13724
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>

[Impala-ASF-CR] IMPALA-8341: [DOCS] Describe the setting for remote data caching

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13724 )

Change subject: IMPALA-8341: [DOCS] Describe the setting for remote data caching
......................................................................


Patch Set 3:

Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/359/ 

Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317


-- 
To view, visit http://gerrit.cloudera.org:8080/13724
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Gerrit-Change-Number: 13724
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Jun 2019 17:33:02 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8341: [DOCS] Describe the settings for remote data caching

Posted by "Michael Ho (Code Review)" <ge...@cloudera.org>.
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/13724 )

Change subject: IMPALA-8341: [DOCS] Describe the settings for remote data caching
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13724/1/docs/topics/impala_data_cache.xml
File docs/topics/impala_data_cache.xml:

http://gerrit.cloudera.org:8080/#/c/13724/1/docs/topics/impala_data_cache.xml@44
PS1, Line 44: --data_cache_dir
--data_cache_dir and --data_cache_size are options built specifically only for ./bin/start-impala-cluster.py as that script needs to create extra sub-directories for the caching directory.

To specify the caching directory, the user should use the flag: --data_cache=<dir1>,<dir2>,<dir3>:<quota>

With the above configuration, data will be stored in <dir1>, <dir2> and <dir3> respectively. The user needs to make sure those directories exist in the local filesystem to begin with. In addition, the filesystem which the directory resides in must support hole punching. Modern filesystems such as ext4 and xfs support this feature. The cache may consume up to <quota> bytes for each of the directories specified. In other words, with the above configuration, the total cache size can be up to 3 * <quota>.

Please see https://github.com/apache/impala/blob/master/be/src/runtime/io/disk-io-mgr.cc#L58-L63 for the definition of the flag.



-- 
To view, visit http://gerrit.cloudera.org:8080/13724
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Gerrit-Change-Number: 13724
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Jun 2019 17:50:09 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8341: [DOCS] Describe the setting for remote data caching

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13724 )

Change subject: IMPALA-8341: [DOCS] Describe the setting for remote data caching
......................................................................


Patch Set 3: Verified+1

Build Successful 

https://jenkins.impala.io/job/gerrit-docs-auto-test/359/ : Doc tests passed.


-- 
To view, visit http://gerrit.cloudera.org:8080/13724
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Gerrit-Change-Number: 13724
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Jun 2019 17:59:18 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8341: [DOCS] Describe the setting for remote data caching

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13724 )

Change subject: IMPALA-8341: [DOCS] Describe the setting for remote data caching
......................................................................


Patch Set 2:

Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/358/ 

Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317


-- 
To view, visit http://gerrit.cloudera.org:8080/13724
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Gerrit-Change-Number: 13724
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Jun 2019 20:43:56 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8341: [DOCS] Describe the settings for remote data caching

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13724 )

Change subject: IMPALA-8341: [DOCS] Describe the settings for remote data caching
......................................................................


Patch Set 1:

Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/356/ 

Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317


-- 
To view, visit http://gerrit.cloudera.org:8080/13724
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Gerrit-Change-Number: 13724
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Jun 2019 15:42:49 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8341: [DOCS] Describe the setting for remote data caching

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13724 )

Change subject: IMPALA-8341: [DOCS] Describe the setting for remote data caching
......................................................................


Patch Set 2: Verified+1

Build Successful 

https://jenkins.impala.io/job/gerrit-docs-auto-test/358/ : Doc tests passed.


-- 
To view, visit http://gerrit.cloudera.org:8080/13724
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Gerrit-Change-Number: 13724
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Jun 2019 20:57:43 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8341: [DOCS] Describe the setting for remote data caching

Posted by "Alex Rodoni (Code Review)" <ge...@cloudera.org>.
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13724 )

Change subject: IMPALA-8341: [DOCS] Describe the setting for remote data caching
......................................................................

IMPALA-8341: [DOCS] Describe the setting for remote data caching

Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Reviewed-on: http://gerrit.cloudera.org:8080/13724
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Michael Ho <kw...@cloudera.com>
---
M docs/impala.ditamap
A docs/topics/impala_data_cache.xml
2 files changed, 87 insertions(+), 2 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Michael Ho: Looks good to me, approved

-- 
To view, visit http://gerrit.cloudera.org:8080/13724
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7dd958e4de109b46eaf906fe93145799af123b3f
Gerrit-Change-Number: 13724
Gerrit-PatchSet: 4
Gerrit-Owner: Alex Rodoni <ar...@cloudera.com>
Gerrit-Reviewer: Alex Rodoni <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>