You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2021/03/10 21:25:11 UTC
[GitHub] [accumulo] ivakegg opened a new pull request #1964: re #1963: Tracking bulk imported files
ivakegg opened a new pull request #1964:
URL: https://github.com/apache/accumulo/pull/1964
* Added the generation of a JSON encode mappings.txt file in the import directory which contains a the mapping of files.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ivakegg closed pull request #1964: re #1963: Tracking bulk imported files
Posted by GitBox <gi...@apache.org>.
ivakegg closed pull request #1964:
URL: https://github.com/apache/accumulo/pull/1964
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] EdColeman commented on pull request #1964: re #1963: Tracking bulk imported files
Posted by GitBox <gi...@apache.org>.
EdColeman commented on pull request #1964:
URL: https://github.com/apache/accumulo/pull/1964#issuecomment-805799474
Was the necessary info in the log files already, or did you add additional logging?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] dlmarion commented on a change in pull request #1964: re #1963: Tracking bulk imported files
Posted by GitBox <gi...@apache.org>.
dlmarion commented on a change in pull request #1964:
URL: https://github.com/apache/accumulo/pull/1964#discussion_r600492458
##########
File path: server/master/src/main/java/org/apache/accumulo/master/tableOps/BulkImport.java
##########
@@ -266,6 +277,26 @@ public Exception call() throws Exception {
throw ex.get();
}
}
+
Review comment:
Suggest using a serialization framework here to serialize an object to the file. Is GSON or JAXB on the classpath in 1.10?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] milleruntime commented on pull request #1964: re #1963: Tracking bulk imported files
Posted by GitBox <gi...@apache.org>.
milleruntime commented on pull request #1964:
URL: https://github.com/apache/accumulo/pull/1964#issuecomment-796875355
This is an enhancement and 1.10 is only going to have bug fix releases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on pull request #1964: re #1963: Tracking bulk imported files
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on pull request #1964:
URL: https://github.com/apache/accumulo/pull/1964#issuecomment-798563527
I don't have enough knowledge of the bulk import code to understand exactly what this accomplishes, but it seems to me that this is a slight implementation change intended to leave files around to help troubleshoot, and does not have any associated API changes. Is that correct?
Although we're only doing bugfixes on 1.10, I don't necessarily have a problem including a change that helps troubleshooting for further bugfixes, if the change is small and low-risk. However, I did have a few questions about these proposed changes:
1. Does this risk leaving extra files scattered around the filesystem that aren't cleaned up or is this placed in a directory that is already subject to cleanup?
2. Why not use one of the JSON libraries (like Gson) on our class path instead of manually constructing a JSON blob? Is there a performance concern?
3. Does the new behavior of adding a new file risk breaking existing user code around bulk import?
4. How does this change impact the new bulk import code paths and behavior, if it does at all?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ivakegg commented on pull request #1964: re #1963: Tracking bulk imported files
Posted by GitBox <gi...@apache.org>.
ivakegg commented on pull request #1964:
URL: https://github.com/apache/accumulo/pull/1964#issuecomment-805783808
This is not any kind of api change but merely a mechanism by which the client can track where the bulk imported file went. On some or our systems I installed a mechanism to track this information via the logs, so this is not really needed anymore. I am closing this pull request. The original ticket however still stands in which it would be good to facilitate a mechanism to track files through the system from bulk import through compactions and garbage collection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on a change in pull request #1964: re #1963: Tracking bulk imported files
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on a change in pull request #1964:
URL: https://github.com/apache/accumulo/pull/1964#discussion_r601899448
##########
File path: server/master/src/main/java/org/apache/accumulo/master/tableOps/BulkImport.java
##########
@@ -266,6 +277,26 @@ public Exception call() throws Exception {
throw ex.get();
}
}
+
Review comment:
I think they both are.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org