You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2022/10/01 17:45:14 UTC
[GitHub] [accumulo-website] cshannon opened a new pull request, #343: Update troubleshooting tools documentation
cshannon opened a new pull request, #343:
URL: https://github.com/apache/accumulo-website/pull/343
This commit adds updates for the new utilities that were added to the acuumulo accumulo admin command. Also add information on rfile-info when reading encrypted files.
This closes #337
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-website] cshannon commented on a diff in pull request #343: Update troubleshooting tools documentation
Posted by GitBox <gi...@apache.org>.
cshannon commented on code in PR #343:
URL: https://github.com/apache/accumulo-website/pull/343#discussion_r1006248257
##########
_docs-2/troubleshooting/tools.md:
##########
@@ -59,6 +59,45 @@ Likewise, `rfile-info` will dump the key-value pairs and show you the contents o
row columnFamily:columnQualifier [visibility] timestamp deleteFlag -> Value
...
+### Encrypted Files
+
+To examine an encrypted rfile the necessary encryption properties must be provided to the utility. To do this
+the `accumulo.properties` file can be copied, the necessary encryption parameters added, and then the properties file can
+be passed to the utility with the `-p` argument.
+
+For example, if using `PerTableCryptoFactory` and the `AESCryptoService`, you would need the following properties in
Review Comment:
Yep you are right that should be `PerTableCryptoServiceFactory`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-website] cshannon commented on pull request #343: Update troubleshooting tools documentation
Posted by GitBox <gi...@apache.org>.
cshannon commented on PR #343:
URL: https://github.com/apache/accumulo-website/pull/343#issuecomment-1264434504
This is ready for review but I made it a draft pull request for now because some of the output depends on https://github.com/apache/accumulo/pull/2992 being merged first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-website] cshannon commented on a diff in pull request #343: Update troubleshooting tools documentation
Posted by GitBox <gi...@apache.org>.
cshannon commented on code in PR #343:
URL: https://github.com/apache/accumulo-website/pull/343#discussion_r985142960
##########
_docs-2/troubleshooting/tools.md:
##########
@@ -59,6 +59,41 @@ Likewise, `rfile-info` will dump the key-value pairs and show you the contents o
row columnFamily:columnQualifier [visibility] timestamp deleteFlag -> Value
...
+### Encrypted Files
+
+To examine an encrypted rfile the necessary encryption properties must be provided to the utility. To do this
+the `accumulo.properties` file can be copied, the necessary encryption parameters added, and then the properties file can
+be passed to the utility with the `-p` argument.
+
+For example, if using `PerTableCryptoFactory` and the `AESCryptoService`, you would need the following properties in
+your accumulo.properties file:
+
+ general.custom.crypto.key.uri=<path-to-key>/data-encryption.key
+ instance.crypto.opts.factory=org.apache.accumulo.core.spi.crypto.PerTableCryptoServiceFactory
+ table.crypto.opts.service=org.apache.accumulo.core.spi.crypto.AESCryptoService
+
+Example output:
+
+ $ accumulo rfile-info /accumulo/tables/1/default_tablet/A0000004.rf -p <path-to-properties>/accumulo.properties
+ 2022-10-01T12:19:32,564 [rfile.PrintInfo] WARN : Attempting to find file across filesystems. Consider providing URI instead of path
+ Reading file: hdfs://localhost:8020/accumulo/tables/1/default_tablet/A0000004.rf
+ Encrypted with Params: ...
+ 2022-10-01T12:19:32,760 [bcfile.CompressionAlgorithm] INFO : Trying to load codec class org.apache.hadoop.io.compress.LzoCodec
+ 2022-10-01T12:19:32,762 [bcfile.CompressionAlgorithm] INFO : Trying to load codec class org.apache.hadoop.io.compress.Lz4Codec
+ 2022-10-01T12:19:32,765 [bcfile.CompressionAlgorithm] INFO : Trying to load codec class org.apache.hadoop.io.compress.ZStandardCodec
+ 2022-10-01T12:19:32,767 [bcfile.CompressionAlgorithm] INFO : Trying to load codec class org.apache.hadoop.io.compress.DefaultCodec
Review Comment:
Good point, I will get rid of that as it is too verbose and there's also already a full example of output earlier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #343: Update troubleshooting tools documentation
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on code in PR #343:
URL: https://github.com/apache/accumulo-website/pull/343#discussion_r999480733
##########
_docs-2/troubleshooting/tools.md:
##########
@@ -59,6 +59,45 @@ Likewise, `rfile-info` will dump the key-value pairs and show you the contents o
row columnFamily:columnQualifier [visibility] timestamp deleteFlag -> Value
...
+### Encrypted Files
+
+To examine an encrypted rfile the necessary encryption properties must be provided to the utility. To do this
+the `accumulo.properties` file can be copied, the necessary encryption parameters added, and then the properties file can
+be passed to the utility with the `-p` argument.
+
+For example, if using `PerTableCryptoFactory` and the `AESCryptoService`, you would need the following properties in
Review Comment:
Should this be PerTableCryptoServiceFactory?
```suggestion
For example, if using `PerTableCryptoServiceFactory` and the `AESCryptoService`, you would need the following properties in
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-website] cshannon commented on pull request #343: Update troubleshooting tools documentation
Posted by GitBox <gi...@apache.org>.
cshannon commented on PR #343:
URL: https://github.com/apache/accumulo-website/pull/343#issuecomment-1264476181
Thanks for the quick feedback @ctubbsii, I should have some time Monday or Tuesday to make updates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #343: Update troubleshooting tools documentation
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on code in PR #343:
URL: https://github.com/apache/accumulo-website/pull/343#discussion_r985141049
##########
_docs-2/troubleshooting/tools.md:
##########
@@ -59,6 +59,41 @@ Likewise, `rfile-info` will dump the key-value pairs and show you the contents o
row columnFamily:columnQualifier [visibility] timestamp deleteFlag -> Value
...
+### Encrypted Files
+
+To examine an encrypted rfile the necessary encryption properties must be provided to the utility. To do this
+the `accumulo.properties` file can be copied, the necessary encryption parameters added, and then the properties file can
+be passed to the utility with the `-p` argument.
+
+For example, if using `PerTableCryptoFactory` and the `AESCryptoService`, you would need the following properties in
+your accumulo.properties file:
+
+ general.custom.crypto.key.uri=<path-to-key>/data-encryption.key
+ instance.crypto.opts.factory=org.apache.accumulo.core.spi.crypto.PerTableCryptoServiceFactory
+ table.crypto.opts.service=org.apache.accumulo.core.spi.crypto.AESCryptoService
+
+Example output:
+
+ $ accumulo rfile-info /accumulo/tables/1/default_tablet/A0000004.rf -p <path-to-properties>/accumulo.properties
+ 2022-10-01T12:19:32,564 [rfile.PrintInfo] WARN : Attempting to find file across filesystems. Consider providing URI instead of path
Review Comment:
I think it's better to use the triple backtick method to create blocks, rather than rely on indentation, which is less obvious, and has more limitations (like no syntax highlighting).
##########
_docs-2/troubleshooting/tools.md:
##########
@@ -59,6 +59,41 @@ Likewise, `rfile-info` will dump the key-value pairs and show you the contents o
row columnFamily:columnQualifier [visibility] timestamp deleteFlag -> Value
...
+### Encrypted Files
+
+To examine an encrypted rfile the necessary encryption properties must be provided to the utility. To do this
+the `accumulo.properties` file can be copied, the necessary encryption parameters added, and then the properties file can
+be passed to the utility with the `-p` argument.
+
+For example, if using `PerTableCryptoFactory` and the `AESCryptoService`, you would need the following properties in
+your accumulo.properties file:
+
+ general.custom.crypto.key.uri=<path-to-key>/data-encryption.key
+ instance.crypto.opts.factory=org.apache.accumulo.core.spi.crypto.PerTableCryptoServiceFactory
+ table.crypto.opts.service=org.apache.accumulo.core.spi.crypto.AESCryptoService
+
+Example output:
+
+ $ accumulo rfile-info /accumulo/tables/1/default_tablet/A0000004.rf -p <path-to-properties>/accumulo.properties
Review Comment:
The example should use the fully qualified URI, rather than just the path, in order to avoid the subsequent warning, and set a better example to users. Something like `hdfs://localhost:9020/accumulo/tables...`
##########
_docs-2/troubleshooting/tools.md:
##########
@@ -59,6 +59,41 @@ Likewise, `rfile-info` will dump the key-value pairs and show you the contents o
row columnFamily:columnQualifier [visibility] timestamp deleteFlag -> Value
...
+### Encrypted Files
+
+To examine an encrypted rfile the necessary encryption properties must be provided to the utility. To do this
+the `accumulo.properties` file can be copied, the necessary encryption parameters added, and then the properties file can
+be passed to the utility with the `-p` argument.
+
+For example, if using `PerTableCryptoFactory` and the `AESCryptoService`, you would need the following properties in
+your accumulo.properties file:
+
+ general.custom.crypto.key.uri=<path-to-key>/data-encryption.key
+ instance.crypto.opts.factory=org.apache.accumulo.core.spi.crypto.PerTableCryptoServiceFactory
+ table.crypto.opts.service=org.apache.accumulo.core.spi.crypto.AESCryptoService
+
+Example output:
+
+ $ accumulo rfile-info /accumulo/tables/1/default_tablet/A0000004.rf -p <path-to-properties>/accumulo.properties
+ 2022-10-01T12:19:32,564 [rfile.PrintInfo] WARN : Attempting to find file across filesystems. Consider providing URI instead of path
+ Reading file: hdfs://localhost:8020/accumulo/tables/1/default_tablet/A0000004.rf
+ Encrypted with Params: ...
+ 2022-10-01T12:19:32,760 [bcfile.CompressionAlgorithm] INFO : Trying to load codec class org.apache.hadoop.io.compress.LzoCodec
+ 2022-10-01T12:19:32,762 [bcfile.CompressionAlgorithm] INFO : Trying to load codec class org.apache.hadoop.io.compress.Lz4Codec
+ 2022-10-01T12:19:32,765 [bcfile.CompressionAlgorithm] INFO : Trying to load codec class org.apache.hadoop.io.compress.ZStandardCodec
+ 2022-10-01T12:19:32,767 [bcfile.CompressionAlgorithm] INFO : Trying to load codec class org.apache.hadoop.io.compress.DefaultCodec
Review Comment:
I'm not sure it's useful to show the user all the logging messages in the output, especially these about compression algorithms. They create noise, and distract from the tool behavior we are trying to document. Also, they depend heavily on the user's class path and configuration, so they may not experience the same output anyway. It'd probably be enough to just have a footnote that there may be log messages from the code that show up in the output, depending on their logging configuration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-website] cshannon commented on pull request #343: Update troubleshooting tools documentation
Posted by GitBox <gi...@apache.org>.
cshannon commented on PR #343:
URL: https://github.com/apache/accumulo-website/pull/343#issuecomment-1279992407
@ctubbsii - I don't think I have any more changes for now for this so it can be reviewed again as I think I addressed all the comments. Let me know if you see anything else.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-website] cshannon merged pull request #343: Update troubleshooting tools documentation
Posted by GitBox <gi...@apache.org>.
cshannon merged PR #343:
URL: https://github.com/apache/accumulo-website/pull/343
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org